High Payoff in Electronic Government

*Measuring the Return on E-Government Investments*

A Report for the

Intergovernmental Advisory Board

May 2003

Intergovernmental Advisory Board

Federation of Government Information Processing Councils

In cooperation with the

Office of Intergovernmental Solutions

GSA Office of Citizen Services and Communications

U.S. General Services Administration

1800 F Street NW

Washington, DC 20405

[www.gsa.gov/intergov](http://www.gsa.gov/intergov)

High Payoff in Electronic Government

*Measuring the Return on E-Government Investments*

A Report for the

Intergovernmental Advisory Board

Table of Contents

A. Executive
Summary..................................................................................2

B.
Introduction.............................................................................................6

Background........................................................................................
6

The
Challenge....................................................................................10

C. Defining and Measuring High-Payoff E-Gov
Programs....................................10

1.  Financial: Reduced costs of Government Operations/Enhanced Revenue
    Collection...........................................................................................11

2.  Economic
    Development.....................................................................\....15

3.  Reducing Redundancy: Consolidating And Integrating Government
    Systems\....17

4.  Fostering Democratic
    Principles..............................................................22

5.  Improved Service To Citizens And Other
    Constituencies...............................25

```{=html}
<!-- -->
```
D.  Conclusions..........................................................................................\...31

E.  Sources and
    References............................................................................32

Appendices

I. Payoff Categories and Benefits Offered by 75 Nominees for 2001 and
2002 IT Recognition Awards National Association of State Chief
Information Officers............35

II\. Summary of E-Gov Measurement
Systems.....................................................47

III\. Benefits Cited in State E-Gov Programs Nominated for NASCIO
Digital Government
Awards......................................................................................51

IV\. OMB A-11 Section 300 Scoring
Criteria......................................................\...58

For further information, contact Darlene Meskell at 202-501-1092 or
<Darlene.meskell@gsa.gov>

High Payoff in Electronic Government

A. Executive Summary

The Intergovernmental Advisory Board (IAB) undertook this study to
identify prime examples of "high-payoff" electronic government (E-Gov)
programs to share the secrets of their successes, and to determine how
they measure the results achieved from investments made.

**Defining High-Payoff E-Gov Programs**

As a new and powerful concept, E-Gov promises many benefits to its
government sponsors. It can reduce costs of government operations; open
new sources of revenue; attract businesses, tourists and new residents
to the area; make it easy for citizens to do business with the
government; and reinforce the relationship between the citizen and
democratic government. As with any government program, the value of
E-Gov is in the benefits it delivers to the public and the new avenues
it opens to create value. But E-Gov can be costly, and its value to the
public that supports it must be shown.

We found E-Gov programs offer many types of benefits to a government.
They can be distilled into five categories. Any successful E-Gov program
should address at least one of these areas, but the most successful will
probably deliver benefits in multiple ways.

1.  Financial: Reduced costs of government operations/enhanced revenue
    collection

2.  Economic development

3.  Reduced redundancy: Consolidating and integrating government systems

4.  Fostering democratic principles

5.  Improved service to citizens and other constituencies.

**Financial: Reduced Costs of Government Operations/Enhanced Revenue
Collections.** The low-hanging fruit of electronic government are the
programs that automate routine government processes and eliminate
paperwork, printing and mailing costs, check processing, document
storage and retrieval, and the need for personnel to interact with
citizens in person or on the telephone. This type of savings is
available to every jurisdiction that implements an E-Gov program. E-Gov
eliminates the need for staff intervention in online programs that
provide government forms, and those that permit citizens to obtain and
renew licenses and registrations, apply for government grants and
benefits, and get information over the Internet.

E-Gov programs make it easier for the public to pay permits, licenses
and registration fees, fines, and taxes owed to the government. These
programs eliminate the need for check processing, time delays in
handling checks, receipts and other paperwork. Many jurisdictions employ
contractors to create and maintain the E-Gov programs used to process
these payments---some of which incur no cost to the government. The use
of these programs is increasing over time, although there are some
indications that citizens will resist making payments online if the
"convenience fee" assessed to pay for online processing is more than a
nominal \$1 or \$2.

**Economic Development.** E-Gov programs designed to promote economic
development are a boon to businesses, both small and large, thereby
increasing their contribution to the local economy. One of the earliest
and most popular types of E-Gov program reduces the amount of regulatory
paperwork of doing business in a community by permitting online document
searches and filings, tax and wage reporting, employee background
checks, workers' compensation claims, and one-stop regulatory
submissions. Other E-Gov programs foster economic development by
promoting the beauty, benefits, historic and recreational attractions of
the area to potential visitors, businesses and residents.

**Reduced Redundancy: Consolidating and integrating government
systems.** E-Gov programs that integrate systems and databases and
provide one-stop sources of government information enable government to
operate more responsively and more efficiently. Governments that have
already achieved the "low hanging fruit" of automation recognize the
payoff that will come with the next big step toward integration and
transformation of government services. National E-Gov strategies
recognize the benefits of achieving economies of scale and reducing the
number of duplicative systems, stressing the benefits of implementing
modular applications ("build it once, use it often") and a centralized
infrastructure to reduce the national investment in IT. Other benefits
of consolidating and integrating E-Gov systems include the ability to
provide high quality, multi-channel, user-centric services to citizens
and to ensure the security of E-Gov systems,

**Fostering Democratic Principles.** The free flow of information
permitted by the Internet facilitates transparency and accountability in
government. It also increases the accessibility of government at all
levels. Developing countries, especially, value E-Gov for making
government more transparent and more democratic, and for encouraging
citizen education and participation.

**Improved Service to Citizens and Other Constituencies.** Service to
citizens is the primary purpose for E-Gov programs in most countries.
National and state portals make government information readily available
and online services accessible. Benefits delivered by these programs are
defined in terms of convenience, time savings, quality and completeness
of information. In past years, the focus on delivering benefits to
citizens in general or specific constituencies (e.g., vacation planners,
students, seniors) has been so intense that costs were often not
considered.

**Measuring E-Gov Programs**

Regardless of the type of benefits, the best way to measure the
performance of E-Gov programs is relative to the objectives of the
program itself and the public agency that sponsors it. The number of
ways to measure a government program is growing daily. One or more of
these programs are used in most governments including national, state,
county, municipal, and tribal governments. We reviewed a wide range of
processes developed to evaluate E-Gov programs, including:

-   Cost-benefit analysis, net-present-value and internal rate of return

-   Return on Investment

-   Customer Satisfaction

-   Take-up Rates

-   Benchmarking.

As the need for performance measurement and accountability has
increased, many jurisdictions are performing more complex and
multifaceted analyses to determine the relative value of different E-Gov
programs. The United States government requires the development of
convincing business cases for large E-Gov programs, and for all
crosscutting E-Gov initiatives. These business cases require extensive
analysis of the costs of each alternative (including "do nothing") for
satisfying a business need weighed against the total benefits. Costs,
benefits and risks of each alternative are evaluated in the context of
the current situation. Potential funding sources, sponsors,
partnerships, and synergies are considered. Market demand and promotions
are also considered in developing a business case.

Many jurisdictions use Portfolio Analysis to assess the risk inherent in
all of their E-Gov programs viewed as a whole. This kind of analysis
permits the funding of a high-cost program as long as other E-Gov
programs that deliver high benefits relative to their costs offset it.
It is measured by quantifying the aggregate risk relative to expected
returns of an entire portfolio.

Risk Management is a part of all comprehensive analyses. A risk analysis
considers the impact and probability that specific factors will impede
the organization's ability to realize the benefits of an E-Gov program.
These include the risks of cost overruns, of technical obsolescence, or
of becoming misaligned with political priorities. They also include the
risks that program managers will not leverage the technology or that the
target audience will reject a program. Risks of security and privacy
violations must also be assessed.

The balanced scorecard is a conceptual framework for translating an
organization's strategic objectives into a set of performance indicators
distributed among four perspectives: Financial, Customer, Internal
Business Processes, and Learning and Growth. First proposed in a
*Harvard Business Review* article in 1993, it has become a popular
measurement methodology in governments. A scorecard is only appropriate
for organizations with a well-defined strategy with which the
performance indicators can be aligned.

These assessment methodologies have been used around the world, to a
greater or lesser extent in different jurisdictions. Many E-Gov pioneers
agree with the assertion made by *Public Sector CXO* magazine that
"E-Government, like politics, is 'local.'" There is no uniform "cookie
cutter" approach to determining the E-Gov program that will have the
highest payoff for a jurisdiction. Accordingly, there is no single
measurement method that will apply as well for one government or agency
as for another.

**Conclusion**

Any definition of "high payoff" represents value for taxpayers, through
cost savings, economic development, synergies achieved through
integration of government processes, strengthened democratic processes,
and service to citizens and other constituent groups. Many tools can be
used to measure the performance of these programs, with different tools
for different values. In each case, the strategic decision-making
process should determine the appropriate metrics.

In these times of tight budgets, measuring E-Gov benefits is a growing
priority in governments, although the state of the art appears to be in
a fairly primitive stage. Investments in E-Government, like other
government investments, traditionally have not been driven solely, or
even generally, by the prospects for financial return; rather, these
programs have been created to deliver better services to
citizen/business/interest group constituencies. Each case requires a
tailored measurement approach that considers the quality, speed and
comprehensiveness of services to citizens, economic efficiencies,
alignment with government's strategic/political priorities, and the
risks of changing technologies, potential cost overruns and changing
needs.

To maximize the benefits from E-Gov technology and increase the use of
E-Gov programs, governments must market them broadly. Not all E-Gov
programs are welcomed enthusiastically, despite the benefits they
promise to deliver, and gaining full acceptance for E-Gov will require
marketing, information and education campaigns. Citizens must be made
aware that they can interact with their government online and that it is
advantageous to do so. E-Gov managers must continuously assess the
citizens' level of acceptance through preference polling, customer
satisfaction surveys and online trend monitoring.

The important economic value of E-Gov will be the transformational value
of re-engineering crosscutting government processes, and integrating IT
investments into business processes. There are formidable organizational
impediments to this significant change-management objective, however,
and internal factors are more important than competition as indicators
of transformational readiness.

# Introduction

**Background**

Electronic government (E-Gov) is the use of technology, particularly
Web-based Internet applications, to enhance the access to and delivery
of government information and services to citizens, business partners,
employees, agencies, and other entities. E-Gov promises its government
sponsors a powerful tool for improving processes and communicating with
the rest of the world. However, the real value of an E-Gov program is in
the benefits it delivers to the public, and the new avenues it opens to
create value. The value of E-Gov to the public that supports it must be
shown.

E-Gov projects have been seen as essential communications and
operational tools for public administration. Many were initiated without
consideration of return on investment. Yet, as government expands its
E-Gov programs, their sponsors are being asked to make a strong business
case, to demonstrate a return on this investment, as they would project
results from any information technology investment. Government agencies
with tight budgets and an increasingly demanding clientele are
reexamining their spending priorities and inquiring whether E-Gov is
delivering the payoff it has been promising and which programs deliver
the highest payoff.

The purpose of the Intergovernmental Advisory Board (IAB)
(<http://www.gsa.gov/Portal/content/offerings_content.jsp?contentOID=114859&contentType=1004&PMGZ=1&S201=1>)
is to provide advice and guidance to The Federation of Government
Information Processing Councils on emerging IT intergovernmental issues
and challenges they present. In early 2002, the IAB recognized a need to
evaluate and differentiate among E-Gov programs in terms of the outcomes
they deliver. Our objective was to identify the qualities that make an
E-Gov program exceptionally valuable to the government that commissions
it, and how the sponsoring governments measure the "payoff" delivered.
Little definitive work has been done in this area, although many
jurisdictions, with some urgency, are seeking information about the best
practices in measuring high-payoff in E-Gov.

Measuring the benefits being achieved by individual E-Gov investments is
a growing priority in governments. We examined measurement methodologies
in place and in process. We reviewed E-Gov programs at the Federal,
state and local levels and in other countries, seeking appropriate case
studies to illustrate the qualities of a high-payoff program. We found
that the state of the art appears to be in a fairly primitive stage.
This report describes our findings.

##### 

##### The Challenge

The first challenge in determining which programs are delivering the
highest payoff is defining terms: What form should the payoff in
E-Government take? How does one evaluate "payoff" in this sector of
society whose mission extends well beyond the bottom-line profit and
loss indicators of a commercial enterprise? Unlike commercial
enterprises, government is responsible to its constituents to perform a
number of other functions, many of which, by definition, are to provide
services that are not commercially viable. These are not easily
measurable; defining payoff in these areas requires addressing the
purpose of government and the priorities of its leaders.

The term "high payoff" evokes images of windfall gains resulting from
modest investments. Given limited resources to invest, and the prospect
of adding to those resources through high returns on investment, it
makes sense to invest in the projects that offer the highest payoff.

Since many governments are using financial measures to evaluate
performance, it would seem to be a relatively straightforward task to
identify and rank-order those projects that generate benefits greater
than their costs. This is especially the case in the United States,
where increasing importance is placed on achieving quantitative results,
but the trend is spreading to Europe. A report for the E-Government
workgroup of the European Union found the U.S. government to be the
frontrunner in measuring economic return. Furthermore, the report found
signs that "an explicit focus on value to investors as *the* key
criteria is also spreading to governments in Europe... unconditional
support for E-Government is being replaced by a growing demand for
projects to create value and deliver Return on Investment."
(<http://www.e.gov.dk/sitemod/upload/Root/English/Value_Creation_in_eGovernment_projects.pdf>)

Even in the United States, however, the value of E-Gov programs is
increasingly seen as multi-faceted. In a poll of government leaders
responsible for delivering programs or program mission support, *Public
Sector CXO Magazine* found that organization leaders view E-Gov
primarily as a way to improve services to citizens, through enhancements
in mission completion and improved customer satisfaction. Only one-fifth
of these CXOs considered cost-efficiency to be the most important return
on investment. (The magazine's audience includes 14,000 Chief Executive
Officers, Chief Information Officers, Chief Financial Officers and other
"chief officers" and their deputies who work for Federal, state and
local governments.)
(<http://cxoadvisory.com.futuresite.register.com/_wsn/page16.html>)

We found "payoff" defined in many non-financial ways. Public service
functions can have immense value in terms of delivering services to
citizens, law enforcement, public safety and health. That value may or
may not be reflected in financial results. The types and quality of
E-Gov infrastructure can have a major impact on a local economy and the
businesses that drive it. The use of E-Gov programs to consolidate the
back-room processes that support service delivery yields significant
results in both efficiency and effectiveness over the long-term. It also
brings about fundamental realignment of government organizations and
redefines "business as usual." The use of E-Gov can be an important tool
of democratic governance, facilitating the transparent, two-way, open
communication that makes government-of-the-people possible.

Governments have viewed their E-Gov programs as a public service, or a
utility, providing value to citizens and establishing a presence for the
city, state or country in the worldwide Internet. The international
leaders, Canada, Singapore and Australia in particular, are recreating
their government service programs around a Web-enabled framework that
focuses on customer-service goals and systems integration. E-Gov
programs of U.S. jurisdictions at all levels are also predominantly
customer-oriented, but not centrally coordinated as in some other
countries.

The preponderance of recent studies, articles and publications
addressing the valuation of E-Gov programs suggests that government
executives and legislators want to see more accountability from their
investments in E-Gov, in order to demonstrate that taxpayer dollars are
being well-spent. Program managers are expected to justify their E-Gov
expenditures with sound business cases that address the performance
expectations of each individual E-Gov project and how they help achieve
the goals of the program offices that implement them.

These calls for measuring E-Gov performance reflect recent findings of
inadequate justification for E-Gov expenditures that illustrate the
critical need for controlling government spending on E-Gov and other
information technology (IT) investments.

-   The US Office of Management and Budget (OMB), designated 771 of
    1,400 IT projects budgeted for FY 2004 as "at-risk," for not
    demonstrating sufficient potential for success or adequate IT
    security. "At risk" projects will be unable to proceed unless their
    sponsors can present a credible business case.
    (<http://www.cdt.org/egov/handbook/>[)]{.underline}

-   The Performance Institute, in its October 2002 report *Creating a
    Performance-Based Electronic Government*, summarized its survey of
    3,500 Federal IT personnel, finding : "Of the \$48 billion spent on
    information technology in FY 2002, this survey indicates that most
    of those expenditures were not justified by mission-aligned
    performance measures. This practice represents a "high risk"
    business practice that could result in failed IT projects and losses
    to the taxpayer."
    (<http://www.performanceweb.org/research/egovernment.htm>)

-   In Sweden, economist Richard Murray has produced manuals for
    government agencies to use in performing cost-benefit analyses and
    regularly studies the economics of government operations. He
    reported similar findings: "I asked about forty agencies in central
    government to give me cost-benefit calculations, accounts or
    assessments of any kind to help me judge the profitability of IT
    investments. I got no complete calculation and mostly assertions
    that these investments are or will be very profitable." (email
    10/07/02)

-   The Texas State Auditor's Office found IT management to be a major
    risk area for the state. The report found that 48 large projects had
    delivery delays averaging 14 months and total cost overruns of more
    than \$352 million. In addition, 46 of 63 systems assessed were
    found to be vulnerable to disruptions, data tampering, fraud and
    inappropriate disclosure. Only 27% had adequate system protection.
    (<http://www.dir.state.tx.us/TIC/dir_info/dirpubs.htm>)

```{=html}
<!-- -->
```
-   The Texas auditors' findings are particularly notable when compared
    to the results of a 2000 Brown University study, *Assessing
    E-Government: The Internet, Democracy, and Service Delivery by State
    and Federal Governments*, which found that Texas ranked highest of
    all states in terms of the quality of citizen access to information
    and services.
    ([www.brown.edu/Departments/Taubman_Center/polreports/egovtreport00.html](http://www.brown.edu/Departments/Taubman_Center/polreports/egovtreport00.html))

These findings suggest there is a significant performance gap in
E-Government. Public investments in IT, including E-Government, are
failing to deliver commensurate payoff. Not only are they disconnected
from the mission-critical programs they promise to improve; these
systems, across the board, are failing to achieve their own operational
goals. Governments at all levels are reconsidering their E-Gov
strategies and seeking ways to reduce the costs while increasing the
effectiveness of this fundamental arm of the public infrastructure.

This report examines five ways in which governments gain high payoff
from their E-Gov programs, and the methods they use to determine whether
they are achieving the results they want.

B.  **Defining and Measuring High-Payoff E-Gov Programs**

As a new and powerful concept, E-Gov promises many benefits. It can
reduce costs of government operations; open new sources of revenue;
attract businesses, tourists and new residents to an area; make it easy
for citizens to do business with the government; and reinforce the
relationship between the citizen and democratic government. As with any
government program, the real value of E-Gov is in the benefits it
delivers to the public and the new avenues it opens to create value. But
it can be costly, and the value of E-Gov to the public that supports it
must be shown.

We undertook this report to find and describe E-Gov programs that
clearly deliver high payoff to the governments that fund them. It was
not an easy task, not because there are no E-Gov programs in place that
generate significant cost savings, but because there are many ways to
describe the benefits of E-Gov programs, each of which is valuable. We
distilled these into five categories:

1.  Financial savings: reduced costs of government operations/enhanced
    revenue collection

2.  Economic development

3.  Reduced redundancy: consolidating and integrating government systems

4.  Fostering democratic principles

5.  Improved service to citizens and other constituencies.

Any E-Gov program will address at least one of these categories. The
most successful will provide benefits in multiple areas. The nature of
the benefits determines which metrics are in alignment with the
performance objectives of the program itself and those of the public
agency that sponsors it.

Most jurisdictions---national, state, local and tribal---use a
combination of metrics, including financial measures,
customer-satisfaction, and risk assessment. Decisions may be based
largely on political or legislative priorities, or these priorities may
be assigned weights and considered along with other factors.
Increasingly, jurisdictions at all levels are using a mix of measurement
methods that evaluate mission-related indicators as well as
cost-savings.

Single-factor measurements are rarely definitive. Most jurisdictions
recognize that, in E-Gov decision-making, it can be as self-defeating to
fund only those E-Gov projects that show financial savings, regardless
of the service they provide, as to fund only those projects that promote
citizen participation, regardless of their cost.

We reviewed a number of metrics used to evaluate E-Gov programs,
including:

-   Financial measures, such as return-on-investment, cost-benefit
    analysis, including net-present-value and internal-rate-of-return

-   Indicators of public approval and acceptance, such as customer
    satisfaction measures and E-Gov take-up, or adoption, rates

-   Benchmarking

-   Balanced scorecard measures

-   Business cases

-   Portfolio analysis and risk management.

These measures range from the very precise measurement of cash flows to
the comprehensive view of all the possible factors that might have an
impact on the success of a government's E-Gov portfolio. While the
process of taking measurements may not itself define what is high-payoff
in E-Gov programs, government planners who use multi-factor analysis can
build a comprehensive case for using E-Gov to achieve their program
objectives.

This report contains a discussion of highly regarded E-Gov programs in
each of these five categories. Case studies and relevant reports are
cited and examples provided of methods used to measure the effectiveness
of these programs. More information about the programs cited can be
found at the links provided in the text.

1.  **Financial: Reduced Costs of Government Operations/Enhanced Revenue
    Collection**

Many governments, particularly in the United States, define "payoff" in
economic terms---e.g., revenue collected or costs reduced or avoided.
They evaluate their programs in terms of the economic return on
investment. States, with severely limited budgets, are the most tightly
constrained. Many E-Gov managers are under pressure to assess the value
of their programs and their return on investment. Some perform
traditional cost/benefit analyses, and many employ elaborate formulas
and ranking systems to evaluate the effectiveness of their E-Gov
programs. Some are still setting up data collection systems that would
enable them to perform these analyses.

a.  **Reduced costs of government operations**

# The process improvements and streamlining achieved by E-Gov can provide significant savings or cost avoidance. By eliminating paperwork processing, jurisdictions across-the-board are reducing staffing requirements, paperwork, printing and mailing costs, cycle time, check processing, document storage, telephone calls and visits to field offices---efficiencies that can translate directly into overhead cost savings. These are the low-hanging fruit of electronic government, and they are becoming as commonplace as voicemail. 

E-Gov programs that offer these kinds of cost reductions include many of
those nominated for the National Association of State Chief Information
Officers' (NASCIO) Recognition Awards for Outstanding Achievement in the
Field of Information Technology
([www.nascio.org/awards/2002awards/](http://www.nascio.org/awards/2002awards/)
and
[www.nascio.org/awards/2001awards/](http://www.nascio.org/awards/2001awards/)).
For example:

-   MyFlorida.com search engine reduces the number of calls to the
    state's call center for a savings of \$1.5 million a year.

-   The Massachusetts Educator Licensure and Recruitment Initiative
    streamlined the state licensing process to save \$1.6 million a
    year.

-   The Idaho Paperless Online Personnel/Payroll System saves \$430,000
    a year through a 20% reduction in personnel, and saves another
    \$75,000 a year in costs of printing pay stubs.

-   The \$7.3 million overhead cost savings and up to \$4 million in
    forms processing savings, combined, are just a small percentage of
    the \$120 million annual savings the Ohio Bureau of Workers'
    Compensation achieved by automating its workers' compensation claims
    process.

-   The Wisconsin Workers' Compensation Insurers' Web Reports system
    achieves an estimated \$1.5 million by eliminating a half million
    pieces of paper each year---100,000 forms and 300,000 state
    mailings---as well as 20,000 annual hours of staff time.

Some of the most effective E-Gov programs are those that enable
government procurement activities online. They are complex systems that
facilitate government purchasing and eliminate the costs of purchase
orders, reduce the need for printed catalogs and reduce calls to
customer service representatives. Some examples are:

-   The State of California's award-winning CAL-Buy Online Procurement
    System, for instance, projects \$9.7 million a year in cost savings,
    or \$37 per purchase order. CAL-Buy won the NASCIO Recognition Award
    for Digital Government in 2001.
    (<http://www.nascio.org/awards/2001awards/digital_gov.cfm>)

-   The eMaryland Marketplace has eliminated \$100 in costs per purchase
    order.

-   The prime example of an online acquisition program is the Federal
    government's GSA *Advantage!*™, which offers 2.4 million products
    and services to Federal buyers over the Web, eliminating the cost of
    thousands of purchase orders every year.
    ([www.gsaadvantage.gov](http://www.gsaadvantage.gov/)). The success
    of GSA *Advantage!* enabled the GSA Federal Supply Service to close
    six of its eight distribution centers and forward supply points in
    2001.

-   Italy's Consip e-procurement project delivers to the Italian
    Government an annual average saving of 30% on the price of goods and
    services purchased through the site. Savings of 20% on
    administrative costs were estimated to total 1,500 billion lire
    (about \$700 million) for 2001.

E-Gov also offers operational, and, ultimately, economic benefits more
indirectly by improving the processes of government. These include
benefits from maintaining accurate data (without re-keying or checking
for errors) and savings on cash-management costs by reducing the cycle
time of check processing and time-to-market for service improvements.
Other sources of savings include those used by the North Carolina
Security Portal, which saved \$2.2 million by avoiding 550 computer
incidents costing an estimated \$4,000 apiece, and by the State of
Kansas, which saves nearly \$9 million a year in unemployment
compensation it doesn't have to pay to people who find a job quickly
through online listings.
([www.nascio.org/awards/2002awards/](http://www.nascio.org/awards/2002awards/)[)]{.underline}

States use E-Gov programs for issuing and renewing licenses and
registrations, in particular, hunting, fishing and boat licenses,
professional licenses (e.g. for nurses, CPAs, educators, engineers, land
surveyors, barbers and audiologists), business licenses and motor
vehicle/motor carrier credentials. They find administrative cost savings
in offering online licenses and permits for a wide variety of
activities, such as cross-country skiing in Minnesota, viewing wildlife
in Alaska, and moving native plants in Arizona. States that implemented
E-Gov programs for grants management streamlined their processes,
eliminated paperwork, reduced application processing time and saw their
staff costs reduced by as much as 35%. (See Appendix I)

b.  **Enhanced Revenue Collection**

We found several E-Gov systems in place that either bring in revenue to
their sponsors, through convenience fees that cover the costs of
conducting transactions online, cash management savings derived from
prompt payment of fines, fees and taxes, vendor fees that pay for the
cost of online government procurement programs, such as those in
Virginia and North Carolina.

Of particular interest are E-Gov programs that are developed, managed
and maintained by vendors, who recover their costs by assessing fees for
the services they provide. Some of these programs deliver significant
cost savings to governments. Kansas has created a public/private
network, the Information Network of Kansas (INK), which operates the
self-funded Consolidated IT Management Model and manages 215,000 pages
for the state. The network builds service applications and web sites for
state agencies and associations at no cost to the agencies. The state
portal encompasses all state agencies, regents and many association web
sites and services. More than 90% are free to users; the other 10%
collect small transaction fees and the \$75 up-front fees and \$60
annual subscription fees for access to the Kansas *Administration
Regulations* and other documents provided by INK. The INK model costs
the state nothing. Nine years after its creation, Ink's revenues total
more than \$7 million a year. Of that, 80%, or \$5.6 million, is
returned to state agencies. (<http://www.accesskansas.org/>)

ServiceArizona, which was developed, hosted and maintained by IBM,
allows citizens to register their vehicles, at no cost to the state. IBM
recovers its costs by charging a fee for each registration. Since
processing an online request costs about \$4 less than a counter
transaction, the state also saves more than a million dollars a year,
with over 15% of renewals being processed by ServiceArizona.
([www.servicearizona.com](http://www.servicearizona.com/)) However,
there are some indications that the demand for online government is very
price sensitive and that use of online transaction processing systems
goes down if a "convenience fee" of more than \$1 or \$2 is charged for
conducting the transaction over the Internet.

The U.K. Private Funding Initiative (PFI) places the burden of
challenging inefficiency in E-Gov programs on private sector investors
in public/private E-Gov partnerships that use private investment to fund
public projects. Between 1997 and 2002, \$28 billion was made available
through PFI in the U.K. PFI promoters cite advantages in allowing the
private sector to assume the financial risk for public projects: it
requires the government only pay for the achievement of specific
outcomes, forces the costing of risk, and leaves it to the private
partner to challenge and correct inefficiencies in building or
maintaining the programs. While this process was used primarily to
finance large construction projects, with only a 25% total success rate,
successful IT PFI projects include the Ministry of Defense's voice and
data network across the U.K.

Some jurisdictions have explored the possibilities of funding E-Gov
programs by selling government data for commercial use, although there
is significant opposition to the practice. Recognizing these objections,
ServiceArizona runs a disclaimer saying it does not collect or share
data on Web site users.

**Measuring Cost-saving E-Gov Projects.** E-Gov projects that promise
operational efficiencies are perhaps the most straightforward prospects
for measuring "payoff."

Economic measures range from simple cost-benefit analysis to more
complex return-on-investment systems. Here are some examples.

> [Cost-benefit analysis]{.underline}. If the costs and benefits are
> clearly identifiable, this is a simple way to measure the economic
> benefits of a program. For example, the District of Columbia
> determined that its Business Resource Center saves District taxpayers
> \$1.8 million a year, net of operating costs, by eliminating some
> customer service positions. The Center cost \$1 million to design and
> build. The cost-benefit ratio is \$1million/\$1.8 million = .56.
> Ratios less than 1 indicate a positive return, and should be funded.
>
> Some of the benefits of proposed E-Gov programs are difficult to
> substantiate; and cost estimates are not always reliable. These
> analyses are generally prepared by the sponsor of a proposed E-Gov
> program without an independent review, and can often lead to
> inaccurate conclusions. Jurisdictions such as the State of Iowa
> require independent third-party reviews of proposals for IT
> investments above a certain cost level.

[Return on Investment.(ROI)]{.underline} Many states calculate return on
investment to measure economic, as well as intangible benefits. Some of
these are universally difficult to measure---an individual's
productivity, for instance, or the impact of training programs. Others,
such as the value provided to citizen constituencies, are inherent to
the government process itself. The State of Iowa's highly regarded
Return on Investment program, is the most robust ROI measurement system
used by the states. It was recognized by NASCIO as the best state IT
management initiative in 2002.
(<https://www.nascio.org/awards/2002awards/state_management.cfm>) The
Iowa ROI program is a process that has been incorporated into the
State's overall IT management program. All IT programs costing \$100,000
or more must be reviewed by the State's Information Technology Council,
which assigns values to different features of each system under
consideration and determines which programs should be funded each year.
The system attempts to balance the output-related attributes of the
project with other important drivers, such as legislative mandates. (See
Appendix II.) This complex return-on-investment program must be
rigorously enforced, and as such, requires continuous management
attention. The numerous analyses required also may require more
resources to perform than are available during a period of tight
budgets.

Return-on-investment calculations are required for all proposed IT
project plans in Kansas. The ROI metrics are custom tailored for each
project. ROI is filed with the Kansas Information Technology Office as
part of a cost benefit statement, to demonstrate benefits realized
during, and one year after, project completion. In its first year, the
Kansas Consolidated Management Model delivered a 13.7% return on
investment for 23 major projects worth \$75 million.

Other tools used for measuring Return on Investment are Net Present
Value (NPV) and Internal Rate of Return (IRR) measures. See Appendix II
for descriptions.

2.  **Economic Development**

Economic development programs traditionally have focused on attracting
businesses that would build large plants and bring jobs to the
community, perhaps in exchange for tax breaks. This is not the focus of
economic development on the Web. Instead, the emphasis is twofold: on
helping small businesses---the fastest growing economic sector
worldwide, and a favorite constituency for state E-Gov programs---and on
promoting the geographic area to attract tourists and residents.

The cost of complying with laws and regulations cost small firms nearly
half a trillion dollars in 2000, or \$7,000 per employee in firms with
fewer than 20 employees, according to the SBA. Small businesses struggle
to comply with regulations that govern fundamental activities as paying
taxes, hiring and managing employees and obtaining the necessary
licenses and permits.

One of the 24 U.S. Federal E-Gov initiatives, Business Compliance
One-stop, which uses <http://www.businesslaw.gov/>as its platform, helps
businesses by creating a one-stop where they can easily access
information about laws and regulations and learn how to comply. The site
initially will focus on four legal and regulatory areas (environment,
workplace health and safety, employment, and taxes). By providing quick
access to information about laws and regulations, compliance assistance
tools, and the ability to perform online transactions, Business
Compliance One-Stop will offer improved customer service to the business
community. Estimates indicate that businesses will save at least \$275
million annually by searching for information in an organized,
user-friendly manner in one portal.
(<http://www.whitehouse.gov/omb/egov/gtob/compliance.htm>)

Many of the most successful state E-Gov programs also target small
businesses with programs that save money by permitting online document
searches and filings, tax and wage reporting, employee background
checks, renewal of professional fees and licenses, and by reducing time,
transactions and office visits. Some of these applications also save
money for the sponsoring jurisdictions, as well as for local businesses.

Making information and services (e.g., reservations, permits,
scheduling) available online helps bring tourists and businesses and
their spending money into a region. Every state and national government
has a home page; many local governments do as well. Most governments
have established a central portal that consolidates their Web presence
and provides access to their programs and information. Portals are
excellent promotional tools. They can present the city, county, state or
country in a very attractive light for outsiders, and make it easy for
tourists, visitors and residents to make plans and reservations in the
area. Portals can make basic citizen services and business transactions
available at a click, so a citizen can check the school lunch menus,
register his boat, pay a traffic fine, and apply for unemployment
insurance, all online. Portals can be very helpful to citizens and
businesses, but they also can save a lot of taxpayer money. Mass.gov,
the State of Massachusetts portal, for example, is expected to return
\$250 million over five years on an investment of \$180 million.
([www.nascio.org/awards/2001awards/digital_gov.cfm](http://www.nascio.org/awards/2001awards/digital_gov.cfm))

**Singapore.** As with traditional economic development programs, the
government bears the initial costs for the benefit of business. For
example, the Singapore government uses E-Gov as a testbed to allow the
commercial sector to develop products and experience, such as the "radio
frequency I.D." (RFID), which helps increase productivity in many
functions. The RFID uses a tag composed of a memory chip and a copper
antenna that stores a unique identifier and a small amount of
programmable information without battery power and transmit information
to a handheld reader when scanned. The process is used in Singapore to
automate the entire book loan and return process for better library
management and for more efficient ATM maintenance. The country uses this
kind of investments in IT to grow industry, and makes its "information
banks" available to benefit industry.

**Finland.** Finland launched a "portal for companies" in Spring 2002. A
joint effort of seven agencies working in the field of business
development, it offers information on venture capital, research and
development, exports and regulations.
(<http://www.ica-it.org/conf36/docs/Finland.pdf>)

**Measuring the payoff from economic development E-Gov Programs.** While
E-Gov programs designed to foster economic development may deliver
direct financial returns to the government sponsors, the direct results
are generally more beneficial to industry. The Texas System for
Electronic Rate and Form-Filing, for instance, offers the insurance
industry up to \$12 million a year in process savings. The Colorado
Secretary of State Business Center offers businesses annual savings
totaling \$2 million, about 6 times the savings projected for the
government. (<http://www.nascio.org/awards/>)

This is illustrated by the results of a nationwide *E-Government
Benefits Study* conducted by the Australian National Office for the
Information Economy in 2001-2002. The study measured the
benefits-to-cost ratio for 24 government online programs that together
represented an investment of approximately \$109 million. The overall
benefits-to-cost ratio of 92% reflected a negative return on investment.
However, of the three types of programs reviewed, only the programs
directed to business pulled down the results. They scored a 54%
benefits-to-cost ratio compared to 121% for citizen online programs and
128% for internally focused government online programs.
(<http://www.noie.gov.au/publications/NOIE/egovt_benefits/Egov_benefits.pdf>)

Like the other crosscutting Federal E-Gov initiatives, Business
Compliance One-stop has specific performance targets to meet, and is
required to report progress regularly to OMB. The purpose of the
Business Compliance project is to reduce the burden on businesses by
making it easy to find, understand and comply with relevant laws and
regulations at all levels of government.
(<http://businesslaw.gov/index.cfm>) Its metrics include the following:

-   Time savings for business compliance and filing

-   Regulatory agency savings through transition to compliance from
    enforcement through automated processes

-   Number of days reduced for issuing permits and licenses

-   Cycle time to issue permits and licenses

-   Number of visitors/page views

-   Reduction in redundant IT investments.

3.  **Reduced Redundancy: Consolidating and Integrating Government
    Systems**

The national governments with the most advanced E-Gov programs are
turning their attention to the need to integrate E-Gov programs across
agencies at all levels of government. High up-front costs may make
integrating programs appear uneconomical, and organizational barriers to
change present a daunting challenge.

E-Gov programs that integrate systems and databases and provide one-stop
sources of government information enable government to operate more
responsively and more efficiently. Governments that already have
achieved the "low hanging fruit" of automation recognize the payoff that
will come with the next big step toward integration and transformation
of government services.

National E-Gov strategies recognize the benefits of achieving economies
of scale and reducing the number of duplicative systems, stressing the
benefits of implementing modular applications ("build it once, use it
often") and a centralized infrastructure to reduce the national
investment in IT. Other benefits of consolidating and integrating E-Gov
systems include the ability to provide high quality, multi-channel,
user-centric services to citizens and to ensure the security of E-Gov
systems.

Governments are looking to integrate horizontally, *i.e.,* consolidating
IT infrastructure to link similar systems across agencies; vertically,
*i.e.,* consolidating within a line of business at different levels of
government; and consolidating around customers. The objective is to make
government more efficient and more effective. Opportunities abound for
consolidating like programs, eliminating redundant systems and replacing
multiple licensing agreements with enterprise software licenses.

**Finland.** Finland's Ministry of Finance has launched a project in
which 48 agencies work together on E-Gov service strategies. This
approach has been successful in the field of information security, and
other efforts are underway for joint projects to handle paperless
accounting. (<http://www.infosoc.fi/PublicServices.pdf>)

Finland's consolidation efforts are also being extended to the local
level. They provide a public-tenders database that targets Calls for
Tender below a threshold level, particularly in the municipal sector.
Finland's government-owned public procurement company, Hansel, also
operates a complete system for electronic tendering and procurement that
was rated the most advanced in Europe by the European Union Commission.
(<http://www.posti.fi/english/uutiset/31-12-102-14-01-35.html>)

**United States.** The major finding of the U.S. E-Government Task Force
in 2001 was significant overlap and redundancy in the Executive Branch,
with multiple agencies performing each of approximately 35 major
functions of government. The Task Force's *E-Government Strategy* calls
for simplifying government by leveraging technology to create new and
more effective ways of achieving government's objectives. Integration of
processes, particularly cross-agency and across governments, enables the
simplification of government processes and achievement of significant
benefits over the long-term.
(<http://www.whitehouse.gov/omb/inforeg/egovstrategy.pdf>)

A year and a half later, OMB is investing a significant effort to
develop a comprehensive Federal Enterprise Architecture and find
opportunities to achieve synergy through consolidation.
(<http://feapmo.gov/>) Directed centrally by OMB, this effort is being
replicated by Federal agencies, as well as some states and
municipalities. The formulation or implementation of an agency
architecture was named as the primary challenge of Federal Chief
Information Officers in 2002 in a survey conducted for the
Washington-based Association for Federal Information Resources
Management (AFFIRM). By contrast, the category, "Using IT to Improve
Service to Customers/Stakeholders/Citizens," the number-one challenge in
2001, dropped to 9^th^ place in the 2002 survey.
(<http://www.affirm.org/>)

**United Kingdom.** In its *UK Online Annual Report 2002,* the British
Government commits to "a radical reform of public services," citing
E-Government as "a powerful catalyst to bringing about this
transformation." The keystone of this transformation will be the
installation of a common infrastructure across Government.
([www.e-envoy.gov.uk/oee/oee.nsf/sections/reports-annrep-2002-pdf/\$file/annualreport02.pdf](http://www.e-envoy.gov.uk/oee/oee.nsf/sections/reports-annrep-2002-pdf/$file/annualreport02.pdf))

The U.K. is building a central infrastructure designed to host multiple
Government Web sites, known as Delivering on the Promise (DotP). With
the Government Gateway, it will lead the way in delivering a central
common infrastructure, bringing economies of scale benefits to
Government departments through a modular 'build-once, use-many'
architecture. The U.K. expects to realize benefits from this program as
it achieves a critical mass, by leveraging opportunities for content
syndication and cross-site content sharing making the service available
to local authorities and agencies.

**Measuring the payoff from reducing redundancy.** The payoff from
consolidating and integrating systems is in implementing an enterprise
(or whole-of-government) approach to E-Government. Benefits come from
eliminating unnecessary backroom systems, creating an enterprise-wide
infrastructure, re-engineering business processes to achieve synergies
and order-of-magnitude improvements, and from providing simplified and
unified access for users. While the benefits accrue over the long-term,
the costs, which tend to be up-front, can be high. These include the
costs of developing and implementing enterprise architecture, acquiring
new systems to replace redundant systems, and redesigning business
processes, as well as the significant costs of retraining personnel to
work with redesigned processes.

[Return on Investment]{.underline}. Because of the broad impact of these
multi-agency programs, a measurable return on investment should be
expected over time. Consolidation across organizational lines requires
extensive change management efforts and long-term investments. Breaking
down silos can require sustained effort over time, and the dedication of
considerable political capital. And, since economic savings come some
time after implementation and E-Gov programs require up-front
investment, there is often no funding to initiate these programs.

> The State of Iowa realized savings of \$2.4 million in 2001 and \$1.06
> million in 2002 as the result of combining similar or duplicative
> systems that were revealed by the required Return on Investment
> analysis.
> (<https://www.nascio.org/awards/2002awards/state_management.cfm>)

The Washington State Combined Application Program (WASHCAP), an
award-winning Federal demonstration project, brought together the Social
Security Administration, the U.S. Department of Agriculture's food
stamps program, and the Washington State Department of Health and Social
Services to consolidate benefits programs that serve needy citizens in
the state. By automating the process and centralizing caseload
maintenance in WASHCAP, the program simplified the process for benefits
recipients and dramatic savings for taxpayers. Annual savings in
administrative and outreach costs amount to approximately \$6.37 million
per year. This represents a total return of \$31.5 million, over the
span of the five-year demonstration project, on a \$400,000 investment.
(<https://www.nascio.org/awards/2002awards/dg_g_to_g.cfm>)

> Virginia's state government spent \$10 million to develop a base
> mapping program, a single consistent spatial data infrastructure for
> Virginia based on high-resolution digital orthophotography for the
> State's entire land base. The program can significantly reduce the
> overall cost of producing a digital base map for Virginia's counties
> and communities by providing statewide coverage at half the cost that
> counties and cities pay to produce the products individually. The
> potential Return on Investment is over \$10 million. But it will not
> be realized unless the 100+ cities and counties participate in the
> program and, as a result, reduce their own costs.
> ([www.nascio.org/scoring/viles/2002Virginia5.doc](http://www.nascio.org/scoring/viles/2002Virginia5.doc))
>
> [Citizen/Customer Satisfaction]{.underline}. Building on the
> understanding it has developed over years of experience with E-Gov
> programs, the Government of Canada has adopted a broad service vision
> that focuses on client-centric service delivery. This vision requires
> a number of government wide strategies to ensure responsive,
> cost-effective, accessible, trusted and secure service delivery, and
> demands a much higher degree of coordination and integration across
> government. The government is committed to changing the governance
> structure of its Government Online and Service Improvement initiatives
> to achieve this vision. Accordingly, it has developed a performance
> measurement framework that encompasses three main
> outcomes---citizen/client-centered government; better more responsive
> service and capacity for online delivery. Specific indicators have
> been developed for these measures and work is underway to develop
> performance measures for them. These indicators are as follows:

+----------------------------------+-----------------------------------+
| Citizen/client centered          | -   Convenience                   |
| government                       |                                   |
|                                  | -   Accessibility                 |
|                                  |                                   |
|                                  | -   Credibility                   |
+----------------------------------+-----------------------------------+
| Better, more responsive service  | -   Critical mass of services     |
|                                  |                                   |
|                                  | -   Take-up                       |
|                                  |                                   |
|                                  | -   Service transformation        |
|                                  |                                   |
|                                  | -   Citizen/client satisfaction   |
+----------------------------------+-----------------------------------+
| Capacity for online delivery     | -   Security                      |
|                                  |                                   |
|                                  | -   Privacy                       |
|                                  |                                   |
|                                  | -   Efficiency                    |
|                                  |                                   |
|                                  | -   Innovation                    |
+----------------------------------+-----------------------------------+

> (<http://www.iccs-isac.org/eng/cmt-about.htm>)
>
> [Business Cases]{.underline}. Comprehensive business cases encourage
> E-Gov program directors to consider and plan for all the ways the
> program will achieve its desired outcomes. Projects in different
> stages will require different levels of maturity, but the basic
> elements of a business case should be incorporated in project
> planning. These elements include cost, schedule, and customer
> analysis, plan performance metrics, project management and analysis of
> risk and alternatives. Customers and other stakeholders should
> independently validate assumptions used in the analysis. It is
> essential that the business case go beyond simply project-oriented
> financial justification. The business case must address broader
> issues, e.g., political, legislative, value to external and internal
> audiences, and enterprise-wide impact, like some of the States'
> return-on-investment tools.
>
> The U.S. Office of Management and Budget is seeking an order of
> magnitude improvement in government performance by closing performance
> gaps and leveraging e-business techniques. In its budget process, OMB
> requires agencies and crosscutting E-Gov projects to use business
> cases to assure that IT investments generate increased efficiency,
> effectiveness and maximized service.
> (<http://www.whitehouse.gov/omb/circulars/a11/2002/part7.pdf>) Several
> key criteria are applied to determine the viability of a business
> case:

-   Linkage to the agency mission, strategic goals and program
    performance Support of the President's Management Agenda by
    incorporating multiple agencies or governments, by using e-business
    techniques, and by aligning with the Federal Enterprise Architecture
    and the other E-Gov initiatives.

-   Mitigation of risk to the Federal government

-   Compliance with IT security requirements.

OMB scores business cases with a 1-to-5 rating in 10 categories. The
strongest documented business cases score between 41 and 50 points
overall. Business cases scoring between 31 and 40 points still need
strengthening. Those that score 30 points or below require much more
work. The criteria and scoring are shown in Appendix IV.

# OMB requires agencies to map their major IT investments to the Federal Enterprise Architecture's Business Reference Model. Functions that are performed by multiple agencies are now clearly delineated and the opportunities for cross-agency collaboration to improve performance are readily apparent. ([**www.feapmo.gov**](http://www.feapmo.gov/)) 

> [Balanced Scorecard]{.underline}. The balanced scorecard set of
> performance indicators was originally developed to measure private
> industry non-financial performance. It has often been modified for use
> in government agencies by accounting for mission and mandate
> requirements that are essential to public sector entities. Balance
> Scorecard measures are distributed among four perspectives: Financial,
> Customer, Internal Business Processes, and Learning and Growth. Some
> indicators are maintained to measure an organization's progress toward
> achieving its vision; other indicators are maintained to measure the
> long-term drivers of success. Through the balanced scorecard, an
> organization monitors both its current performance (finance, customer
> satisfaction, and business process results) and its efforts to improve
> processes, motivate and educate employees, and enhance information
> systems---its ability to learn and improve.
>
> [Risk Assessment]{.underline}. Risk associated with an E-Gov
> investment may impact performance, impede implementation or drive up
> costs. Risk that is not identified cannot be mitigated and can cause a
> project to fail. The more thought is given up-front to mitigating the
> risks of a project, the more likely it is to succeed. Risk assessments
> are used increasingly as part of the business cases made to support
> E-Gov investment proposals. OMB requires agencies to demonstrate an
> acquisition strategy that uses a strong risk mitigation plan to limit
> the risk to the government. Formal risk assessments are a major
> feature in the IT business case process developed for the U.S. Postal
> Service and for the value measuring model developed for the U.S.
> Social Security Administration and the U.S. General Services
> Administration. The USPS model quantifies four categories of risk:
> business risk, technology risk, schedule risk, and financial risk. The
> SSA/GSA Value Measuring Model considers risk as one of three
> categories for analysis, along with cost and a category that
> represents five "value" determinants. In New Zealand, risk-based
> funding rules for complex projects have been developed. Using
> quantitative risk analysis, each risk is assessed along with its
> impact and probability. Thus, the fiscal impact of a project's risks
> can be made explicit to decision-makers.

4.  **Fostering Democratic Principles**

The Internet offers many opportunities for citizens to interact directly
with their governments on civic issues, opportunities that may
significantly change the functioning of democracy. It exemplifies many
of the qualities that underlie a democratic society, such as the free
flow of information, interactive communication and feedback, community
development and transparency. For E-Gov programs that provide these
qualities, the high-payoff may come in the quality of civic
decision-making that results from increased participation by citizens.

> *"E-democracy involves not only strengthening existing democracy but
> also developing new forms of democracy. We are witnessing the advent
> not only of a new society and a new economy but also of a new
> democracy. We must realize that our old values and standards can be
> preserved only if our democracy is sufficiently flexible to adapt to
> the changes taking place around us."* --Dr. Pauline Poland, Advisor in
> the Netherlands Ministry of the Interior and Kingdom Relations,
> February 8, 2002.

Some countries are actively reaching out to citizens to encourage the
broadest participation in the democratic process. The Netherlands, in
particular, has undertaken many experiments with public consultation and
service delivery in recent years. The Ministry of the Interior and
Kingdom Relations, has published a *Handbook for Electronic Consultation
of Citizens,* provides substantial financial incentives for government
bodies to post information encouraging the involvement of citizens in
the democratic process, and links to all political Web sites through the
government portal. In 2001, the handbook was revised to outline various
technical instruments that can be used to consult citizens on the web,
with the aim of lowering the threshold for citizen-involvement as much
as possible.

In developing countries, in particular, transparency is the
characteristic most valued in E-Gov programs. E-Gov promotes
transparency in government operations by making information freely
available, making transactions highly visible online and deterring
corruption. Over 58% of the participants from 35 countries who were
surveyed at the 2002 State Department E-Gov Conference identified
"transparency and democracy" as the goal of primary importance for E-Gov
programs in their countries, over "supporting local industry," "social
and human capital development," "foreign direct investment" and "export
Promotion."
(<http://www.mcconnellinternational.com/ereadiness/The_Global_E-Government_Outlook.pdf>)

E-Mexico was launched to bridge the digital divide by fostering the
creation of a national technology infrastructure and transitioning
Mexican society into the information age through programs for
E-Government, e-health, e-education and e-economy. It is an initiative
of the President's Office for Innovation in Government, whose objectives
are to produce a world-class federal government around six major
objectives:

> -- Culture of quality in public service
>
> -- Savings and cost reduction
>
> -- Deregulation and administrative simplification
>
> -- Use of information and communication technologies
>
> -- Cultural change and professional development of public service
>
> -- Transparency and fighting corruption.

Transparency is central to the Mexican president's agenda. It is at the
heart of the government's E-Gov outreach program, which is attempting to
bring entire communities online---both geographic communities and
communities of interest through a network of Internet community centers
and electronic forums with dozens of topics and thousands of
participants. The primary goal of the system's developers is to expand
the reach of the E-Government program by encouraging citizens to
participate through local programs, through communities of interest, or
through individual Internet use.
(<http://www.ica-it.org/conf36/docs/Mexico.pdf>)

The Bhoomi project in Karnataka State in India is exemplary in the way
it delivered for its citizens by making government more transparent.
Bhoomi is a computerized system of land records that replaced an
exploitative manual system. About 6.7 million farmers in Karnataka State
own 20 million land holdings. The documents recording these land
holdings are required for land transactions, for obtaining loans and for
other concessions. Under the manual system, these records were
maintained by village accountants, without public scrutiny but with many
reported instances of records manipulation, bribery, and harassment.
With all records now maintained in the public domain, farmers have ready
access to the documents they need and information in the public record.
They can access these records easily through a network of kiosks
installed in 177 locations, where queues are routinely seen. Despite
some concern about higher fees to use the kiosks---15 rupees vs. 2
rupees for the manual system---some 330,000 people have paid the fee.
(<http://www1.worldbank.org/publicsector/egov/bhoomi_cs.htm>[)]{.underline}

**Voting Online.** Although the use of online voting in Brazil's
landmark presidential 2002 presidential election was watched with
interest around the world, there is considerable resistance to voting
over the Internet. A 1999 IBM survey of elected officials in 14 European
countries found that 74.2% believe information technology can enhance
democracy, but only 50.4% supported the introduction of online voting
alongside traditional methods. In the United States, a large majority
(63%) of the American public opposes the idea of allowing people to vote
online for federal offices such as the presidency or Congress, according
to a February 2002 report of the Council for Excellence in Government.
Not only did 51% strongly oppose online voting, but support for it
dropped from 38% to 33% during the previous year.
(<http://www.excelgov.org/displayContent.asp?Keyword=ppp022602>)

The implicit promise that online voting will increase the percentage of
citizens who exercise their franchise ensures that jurisdictions will
continue to experiment with this process. The U.S. Defense Department,
which administers the Federal Voting Assistance Program for American
citizens in the military or overseas, piloted an Internet-based absentee
voting project in the November 2000 general election. This was the first
time binding votes were cast over the Internet for Federal, state and
local offices, including President and members of Congress. The States
of South Carolina, Florida, Texas, and Utah and counties within those
states also participated. The study concluded that further development
is needed before Internet voting can be provided effectively, reliably
and securely on a large scale. The integrity of the online electoral
process has not yet been established and the potential for election
fraud when using the Internet is still a major concern.
(<http://www.calvoter.org/votingtech/VOIReport7-9.pdf>)

The first binding U.S. public-sector election online was the 2000
Democratic Party primary in Arizona. Al Gore beat Bill Bradley 4-to-1 in
this primary, in which twice as many voters participated as in previous
primaries. The system encountered some technical problems, such as
overloaded telephone lines, and older browsers that crashed, but the
major obstacle to online voting---in the short term---may be lawsuits
challenging online voting for unfairly favoring the class of voters that
has easy access to the Internet.
(<http://usinfo.state.gov/topical/rights/democracy/inetvot110701.pdf>)

E-Rulemaking, one of the U.S. Federal E-Gov initiatives, is a one-stop
Federal regulatory clearinghouse that makes it easy for citizens to
participate in Federal rulemaking and the democratic process by
improving access to, and quality of, the rulemaking process. When
citizens submit comments on regulations, Federal agencies must take
their views into account and justify their regulatory actions.
Descriptions and the full text of every proposed and final Federal
regulation currently open for comment can be read online on
[www.regulations.gov](http://www.regulations.gov/), and readers may
submit comments to the Federal agencies responsible for the rulemaking
action. Launched in January 2003, this initiative already has taken
major steps:

-   All agency regulatory docket sites are linked to FirstGov.gov

-   A "Clinger-Cohen" letter was issued by OMB to consolidate redundant
    and siloed Web sites

-   Planning has begun to consolidate existing agency e-docket systems
    at the Department of Transportation, the Food and Drug
    Administration, and the Department of Labor with the Environmental
    Protection Agency's system.

-   Business process reengineering of the rulemaking process has been
    initiated.

The quality of government service delivery can and should contribute to
strengthen democratic citizenship and the bonds of trust between
citizens and their democratic governments.

Measuring E-democracy programs. Most typical E-Gov metrics are not
applicable to this high-payoff category. E-Gov programs designed to meet
civic objectives are generally cost centers. Rather than reducing the
costs of government, these programs represent a commitment to provide
good government without the expectation of a return on investment. Over
the long run, the returns may be high in terms of the civic and economic
health of a community, but the immediate financial impact is likely to
be costly.

Governments around the world are actively promoting citizen feedback
online in order to strengthen democracy and development, according to a
report by the Government Online International Network.
((<http://governments-online.org/documents/e-consultation.pdf>) However,
few countries have developed programs to measure e-consultation
projects. Those that have, have done so mainly on a pilot basis.

During the Organisation for Economic Co-operation and Development (OECD)
meetings on "e-consultation" or democratic processes online,
participants discussed a framework for information, consultation and
public participation in policy-making, and criteria for evaluating (1)
the impact of government efforts to inform, consult and engage citizens
in policy-making; and (2) the process used.
<http://www.olis.oecd.org/olis/2002doc.nsf/43bb6130e5e86e5fc12569fa005d004c/10fb657f44a588d4c1256bec004df3a6/$FILE/JT00130234.PDF>)

For example, the effectiveness of on-line voter registration could be
indicated by the percentage of eligible voters registered; the
effectiveness of online voting by an increase in the percentage of
voters.

5.  **Improved Service to Citizens and Other Constituencies**

The primary goal of E-Gov programs at all levels and in all countries is
to make it easy for citizens to obtain service and information and to
interact with the government. This is the first stated goal in the U.S*.
E-Government Strategy* and it reflects a national perception. Two thirds
(68%) of Americans polled by the Council for Excellence in Government in
a 2000 survey said it should be a priority for government to invest tax
dollars in making more information and services available over the
Internet. The return on those investments will increase as citizens find
they can obtain the government information and services they need
online.

> *....for the public services, the real opportunity is to use
> information technology to help create fundamental improvement in the
> efficiency, convenience and quality of our services. That is why we
> aim to have all government services on-line by 2005, building on best
> practices\... But we recognise that British businesses and citizens
> are not yet using government services online in the numbers that match
> the best in the world. So our new strategy will focus on driving up
> access in key categories in the NHS, education, transport, benefits,
> tax and criminal justice. It will include, for example, services to
> enable drivers to conduct all their dealings with Governments online
> including tax discs, vehicle registration and driving license
> applications.* -- U.K. Prime Minister Tony Blair in his keynote speech
> to e-Summit, November 19, 2002.
> (<http://www.number10.gov.uk/output/Page1734.asp>)

The results of a University of Texas study suggest another reason
governments would find it useful to increase citizen use of E-Government
programs: funding. Its *Cost-Benefit Study of Online Services* in
January 2003 revealed that 55% of the Texans surveyed believe E-Gov
should be considered a "value-added service whose costs should be
shouldered by its users" and that using tax dollars to support
E-Government was unacceptable. Sponsors of E-Gov programs must be
vigilant to ensure that they are "delighting" the public with the
convenience and ease-of-use of online government, or they run the risk
of losing their funding.
([www.dir.state.tx.us/TIC/dir_info/dirpubs.htm](http://www.dir.state.tx.us/TIC/dir_info/dirpubs.htm))

Use of E-Gov is slowly increasing. In 2002, the average use of
government online services had increased by 4% to 30%, according to a
Government Online survey of 28,000 people in 31 countries. The survey
showed that 42% of the population had personally used the Internet in
the previous month. The use of E-Gov ranged from 3% in Hungary to 57% in
Sweden. Major gains since 2001 were in Australia (up 15% to 46%), Turkey
(up 10% to 13%), and the United States (up 9% to 43%). Only Japan showed
a drop in E-Gov usage, from 17% to 13%; Internet use in Japan dropped.
([www.tnsofres.com/gostudy2002/](http://www.tnsofres.com/gostudy2002/))

The survey reported that 2002 take-up rates, i.e., the percentage of
Internet users that use E-Gov, ranged from 16% in Hungary to 82% in the
Faroe Islands. The high-percentage users included Norway and Denmark
(81%), Finland (78%), Estonia (77%), Australia (73%), Canada (70%),
France and Hong Kong (69%) and Singapore (68%). The United States
take-up rate was 62%. The U.K. was near the low at 30%.

**Australia.** The National Office for the Information Economy, reports
that there is ongoing demand for E-Gov services and users believe
significant benefits are available.

The survey, taken in 2001-2002 found the take-up of E-Gov services among
businesses ranged from 54% for small businesses (fewer than 4 employees)
to 86% for businesses with more than 100 employees.
(<http://www.noie.gov.au/publications/NOIE/egovt_benefits/Egov_benefits.pdf>)

# Canada. Sixty four percent of all Internet users visited Government of Canada Web sites, up from 52% in 2001. Although only 22% report the Internet/e-mail as their main method of contact with governments, 40% expect it to be their primary communication channel in two years and 48% expect it to be so in 5 years.

**USA.** Internet usage in the United States is estimated at 71% by the
University of California at Los Angeles (UCLA); 63% of the population
used the Internet in the previous month, Taylor Nelson Sofres reported.
More than a third of these U.S. Internet users visited a federal
government Web site in February 2003, according to Nielsen//NetRatings.
Traffic to those sites increased 26% from December 2002 to February
2003. The Treasury Department experienced the largest growth during this
period: 147%; traffic to NASA sites grew 124%; the Education and State
Departments also grew by 90% or more. (Nielsen//NetRatings, 3/17/03,
<http://www.nielsen-netratings.com/news.jsp>)

**U.K.** The U.K., which is in the process of spending £6 billion on IT
improvements, is focusing on improvements to its very low take-up
rates---only 13% of the population (30 of Internet users) used
government online programs last year, despite the fact that close to 75%
of all government services were available online. To boost usage, the
U.K. has undertaken a multifaceted approach to making computers easy to
use by the entire population. Some of the initiatives outlined by Prime
Minister Blair in his address to the November 2002 e-Summit include:

-   Putting all government services online by 2005, following a strategy
    that will focus on driving up access in key categories in health,
    education, transport, benefits, tax and criminal justice.

-   Encouraging competition in the broadband market to keep the price of
    broadband low and falling.

-   Providing funding to deliver broadband connections to every primary
    and secondary school in the country by 2006.

-   Investing more than £1 billion in broadband connectivity for public
    services, including criminal justice, hospitals and doctors'
    offices.

-   Providing incentives for businesses and individuals to invest in
    information and communications technology, to build skills at every
    level and for businesses to increase their proportion of R&D
    spending.

-   Ensuring access to the Internet for everyone who wants it by
    2005,starting with the establishment of 6,000 online centers
    throughout the country to provide a crucial entry point for those
    unable to afford PCs and connections.

-   Gaining trust in the way data will be used, ensuring that systems
    are secure, and addressing ways public services can authenticate
    each other and their customers.

**Canada.** The success of Canada's E-Government program can be measured
by its designation, for two years in a row, as the leader among 23
nations for online service delivery. The Accenture report *E-Government
leadership---Realizing the Vision*, April 2002, found that "Canada has
achieved its leadership position largely due to its focus on the citizen
in its E-Government programs."
([www.accenture.com](http://www.accenture.com/)[)]{.underline}

Canada's Government-on-Line initiative has resulted in higher levels of
satisfaction and "surprisingly resilient positive perceptions of
government service." The public's overall rating of the quality of the
service or information they receive from their federal government has
risen steadily every year from 59% saying 'good' in 1999 to 67% in 2002.
One valuable benefit of the improvements in E-Gov services offered to
Canadians is the finding that 67% of Internet users surveyed said they
would be comfortable submitting their personal information over the
Internet to the government.

Michelle d'Auray, CIO for the Government of Canada, has an even more
ambitious vision for E-Gov---and for government itself:

> *\"Technology is driving us to rethink how we are organized in
> government to serve individuals and business, in Canada and abroad,
> and how we can be more effective and innovative in achieving the
> public interest faster and more efficiently. The big payoff will come
> when we\'ve moved from \"E-Government\" to create a de facto \"my
> government\" for every citizen and business. This means looking beyond
> service, so that Canadians can participate more actively and
> systematically in the development of the policies that affect their
> daily lives and the future of the country.\"*

**Australia.** Ann Steward, of the Australian National Office for the
Information Economy (NOIE), views E-Gov's most important benefit as
"enabling government to transform itself through technology into a
citizen-centric, high quality integrated services delivery agent." NOIE
surveyed citizens and businesses in 2001/2002 and found that 86% of
E-Gov users felt the overall benefit of E-Gov was either significant or
moderate. However, only 45% were able to quantify actual cost savings
associated with interacting online compared to traditional channels;
only a third of those estimated savings of more than \$25 per
interaction.

The survey found that interacting with government online offered some to
significant improvements in the following areas:

-   Ease of finding information (80%)

-   Service quality (75%)

-   Ability to make decisions (75%)

-   Access to public records (68%).

Over 80% of businesses and nearly 90% of government employees saw either
some or significant improvements in the quality of their
decision-making.
(<http://www.noie.gov.au/publications/NOIE/egovt_benefits/Egov_benefits.pdf>)

The most successful E-Gov programs are tailored to the citizens they
serve, particularly E-Gov programs managed by the more "hands-on"
providers of services in State and local governments. "Cookie-cutter"
approaches are not useful.

> *The biggest benefit E-Government offers government isa way to tailor
> their service delivery mechanism to meet the needs of their customers.
> E-Government makes it possible to deliver transactions and information
> through multiple channels, i.e. Web Sites/Portals, Interactive Voice
> Response, Kiosks, PDA\'s, and text messaging, when and where it is
> convenient for the customer.* -- David Molchany, Chief Information
> Officer, Fairfax County, VA.

**Measuring improved service to citizens.** Most of the E-Gov programs
we reviewed---on the state, federal and international levels---are
primarily directed at providing better service to various
constituencies, such as taxpayers, small businesses, lobbyists, nurses,
hunters, job-seekers, or insurance claims filers. It is easy to see
their effectiveness, in terms of improving transactions between citizens
and government. It is often more difficult to measure their value in
terms of a return on investment.

Some jurisdictions assess the value of their customer service E-Gov
efforts with numerical scores to quantify how much they help achieve the
organization's objectives. Others attempt to quantify the savings to
individuals, in terms of waiting time, or travel expenses. Recognizing
that the function of electronic government is to serve citizens better,
however, many programs find E-Gov programs strengthen their own
effectiveness by offering citizens another---usually better---channel
for interacting with their government. These programs are low-risk,
since they only augment, and do not replace, the existing systems that
permit a citizen to get the same service by mail or in person.

There are many ways to measure the effectiveness of the service provided
to constituent groups by E-Gov programs. According to the Performance
Institute, many Federal agencies use accessibility metrics to measure
improved program performance. These include measuring customer
satisfaction, the percent of audience reached, response time, reduced
constituent cost, returning users, and service availability. The first
two of these measures are widely considered effective measures of the
effectiveness of E-Gov programs.

> [Customer satisfaction surveys]{.underline}. Customer satisfaction
> measures are becoming more common and more standardized. In the United
> States, the American Customer Satisfaction Index (ACSI), a private
> application developed in partnership with the University of Michigan
> School of Business and the American Society for Quality, tracks annual
> trends in customer satisfaction with Federal government agencies. The
> quantitative ACSI survey instrument, which is available for use in
> Federal agencies through a U.S. Treasury franchise, is regularly
> employed to measure customer satisfaction with government services. In
> its 2002 survey, for the first time, the ACSI survey covered several
> government Web sites, whose average score (73.5) is higher than the
> overall ACSI score for private industry (73.1), news and information
> sites (73) and government offline (70.2).
>
> Canada has developed a Common Measurement Tool (CMT) for use in
> measuring client satisfaction. The Common Measurements Tool (CMT)
> provides public organizations with a set of standard questions and
> standard measurement scales for use in surveying their clients. It is
> a comprehensive collection of potential survey questions that an
> organization may select from, to custom design a client satisfaction
> survey that meets their information requirements. The use of standard
> questions allows the organization to benchmark progress over time and,
> since questions are standard, organizations can compare results with
> other organizations within the same business line. To ensure this
> ability to benchmark performance, several core questions are required
> for inclusion in all surveys. Designed to provide client feedback to
> any public organization and ensure that all aspects of client service
> are considered, the CMT is conceived around five key elements: client
> expectations, perceptions of the service experience, satisfaction
> levels, levels of importance, and priorities for service improvements.
> ([www.iccs-isac.org/eng/cmt-about.htm](http://www.iccs-isac.org/eng/cmt-about.htm))
>
> [Benchmarking]{.underline}. The U.K. expanded its exercise
> benchmarking the G7 countries (US, U.K., France, Germany, Italy,
> Japan, and Canada), Sweden and Australia, to measure key E-Government
> indicators within its benchmarking framework. The benchmarking report,
> issued at the e-Summit in November 2002, places the U.K. ahead of the
> other European countries and Japan, and in second place overall behind
> the United States. The report highlighted areas in which the U.K. can
> learn from other countries---Canada's online services and Sweden's
> expansion of PC use, among others.
> (<http://www.e-envoy.gov.uk/oee/oee.nsf/sections/esummit-benchmarking/$file/indexpage.htm>)
>
> [Improving Take-up Rates.]{.underline} There is no "payoff" from an
> E-Gov program that is not used and there is no guarantee that "if you
> build it, they will come." The effectiveness of an E-Gov program is in
> direct proportion to the audience it reaches, and the savings it
> provides. State and local officials have found that E-Gov programs
> must be tailored to provide what their communities need, using the
> methods they prefer.
>
> One way to increase the effectiveness of E-Gov is to increase its use
> by the intended audience. By monitoring usage and repeatedly polling
> citizens and other stakeholders, program managers can learn what steps
> to take to improve service delivery and outreach to citizens and so
> increase usage.
>
> A recently published State of Texas Cost-Benefit Study of Online
> Services found that 11 applications and services offered by
> TexasOnline could save personnel costs of between \$35,207 and \$1.9
> million if they achieved 30% take up. These prospective savings would
> rise proportionately---to as much as \$6.3 million---as the take-up
> rate neared 100%.

"After all, the potential of E-Governance is actualized only when it has
reached to each and every individual and community in its specific
conditions and in a manner that is sustainable," an E-Gov official from
India commented during the State Department's E-Gov Conference in
November 2002.
([http://www.mcconnellinternational.com/ereadiness/The_Global_E-Government_Outlook.pdf](http://www.mcconnellinternational.com/))

#  

# D. Conclusions

Any definition of "high payoff" represents value for taxpayers, through
cost savings, economic development, synergies achieved through
integration of government processes, strengthened democratic processes,
and service to citizens and other constituent groups. Many tools can be
used to measure the performance of these programs, with different tools
for different values. In each case, the strategic decision-making
process should determine the appropriate measurement.

Each case requires a tailored measurement approach that considers the
quality, speed and comprehensiveness of services to citizens, economic
efficiencies, alignment with government's strategic/political
priorities, and the risks of changing technologies, potential cost
overruns and changing needs.

To maximize the benefits from E-Gov technology and increase the use of
E-Gov programs, governments must market them broadly. Not all E-Gov
programs are welcomed enthusiastically, despite the benefits they
promise to deliver, and gaining full acceptance for E-Gov will require
marketing, information and education campaigns. Citizens must be made
aware that they can interact with their government online and that it is
advantageous to do so. E-Gov managers must continuously assess the
citizens' level of acceptance through preference polling, customer
satisfaction surveys and online trend monitoring.

The important economic value of E-Gov will be the transformational value
of re-engineering crosscutting government processes, and integrating IT
investments into business processes. There are formidable organizational
impediments to this significant change-management objective, however. In
the United States, the Office of Management and Budget has taken on the
challenge of transforming government through E-Gov. As stated in the
U.S. *E-Government Strategy,* February 27, 2002

(<http://www.whitehouse.gov/omb/inforeg/egovstrategy.pdf>):

> "...\[T\]he pay-off will not result from automating current processes,
> but rather through the transformation of how the government interacts
> with its citizens and customers. Only through changing how we do
> business internally---that is, streamlining work processes to take
> advantage of modern IT systems---will citizens experience the
> transformation envisioned."

**E. Sources and References**

We reviewed national E-Gov programs around the world and reports from
two major international E-Gov conferences, the 36^th^ Conference of the
International Council for Information Technology in Government
Administration (ICA) (<http://www.ica-it.org/conf36/>) in October 2002,
and the U.S. State Department's Conference on E-Government
Implementation in November 2002, and reviewed materials from the United
Kingdom's e-Summit in the same month. We sought out award-winning E-Gov
programs in all jurisdictions. We interviewed IT officials and reviewed
E-Gov strategy papers issued by the U.S. and other national governments.

We reviewed a number of studies issued on performance measurement in
government. We examined several measurement systems created to calculate
return on investment in E-Gov programs, at the national, Federal and
State levels. We compared and categorized best-in-class State E-Gov
programs that were nominated by their States for IT Recognition Awards
submitted to NASCIO in 2001 and 2002. (See Appendix III)

Following is a list of the documents referenced in this report.

Accenture, *eGovernment Leadership -- Realizing the Vision,* 2002
[(]{.underline}[www.accenture.com](http://www.accenture.com/)

Accenture, *Value Creation in eGovernment projects - An exploratory
analysis conducted for the Danish presidency of the eGovernment
workgroup of the Directors General*, 2002,
(<http://www.e.gov.dk/sitemod/upload/Root/English/Value_Creation_in_eGovernment_projects.pdf>[)]{.underline}

Anexys, LLC, Indiana University-Bloomington Institute for Development
Studies and META Group, Inc., *The Anexys Primer on Measuring ROI in
E-Government,* 2001.

Assirati, Bob, Executive Director -- IT Directorate, UK Office of
Government Commerce, *PFI -- The Cost of Risk, The Price of Success,*
presentation, October 22, 2002

Association for Federal Information Resources Management, *The Federal
Chief Information Officer/Seventh Annual Top Ten Challenges Survey*,
December 2002 ([http://www.affirm.org](http://www.affirm.org/))

Australian National Office for the Information Economy, *E-Government
Benefits Study/Measuring the demand for, and the benefits of,
E-Government*, Preliminary findings, 11/11/02
(<http://www.noie.gov.au/publications/NOIE/egovt_benefits/Egov_benefits.pdf>)

Australian National Office for the Information Economy, *Better
Services, Better Government*, November 2002
(<http://www.noie.gov.au/publications/NOIE/better_services-better_gov/Better_Services-Better_Gov.pdf>)

Blair, Tony, Prime Minister, UK, Keynote speech to e-Summit, November
19, 2002
(<http://www.number10.gov.uk/output/Page1734.asp>[)]{.underline}

Booz Allen Hamilton, *International e-Economy Benchmarking/The World's
Most Effective Policies for the e-Economy,* November 19, 2002
(<http://www.e-envoy.gov.uk/oee/oee.nsf/sections/esummit-benchmarking/$file/indexpage.htm>)

Caldow, Janet, IBM Institute for Electronic Government, *The Quest for
Electronic Government/ A Defining Vision,* July 1999
([www.ieg.ibm.com/thought_leadership/egovvision.pdf](http://www.ieg.ibm.com/thought_leadership/egovvision.pdf))

Canadian Institute for Citizen-Centred Service, *Benchmarking*
([www.iccs-isac.org/eng/bench-ben.htm](http://www.iccs-isac.org/eng/bench-ben.htm))

Canadian Institute for Citizen-Centred Service, *Common Measurements
Tool*
([www.iccs-isac.org/eng/cmt-about.htm](http://www.iccs-isac.org/eng/cmt-about.htm))

CAPAM International Awards Programme 2001-2002, India submission,
([http://www.capam.comnet.mt/Docs/IIA2002/submissions/2%20SILVER%20BHOOMI%20Land%20registry%20India.pdf](http://www.capam.comnet.mt/Docs/IIA2002/submissions/2 SILVER BHOOMI Land registry India.pdf))

Center for Democracy & Technology and InfoDev, *The E-Government
Handbook for Developing Countries,* November 2002
(<http://www.cdt.org/egov/handbook/>).

Judy, Debbie, U.S. Postal Service, *Building IT Business Cases and
Portfolio Investment Analysis,* presentation September 18, 2002

Department of Defense Washington Headquarters Services Federal Voting
Assistance Program, *Voting Over the Internet/Pilot Project Assessment
Report*, June 2001
(<http://www.calvoter.org/votingtech/VOIReport7-9.pdf>)

*Economist* Magazine, *Survey: Government and the Internet,* January 22,
2003
([www.economist.com/displayStory.cfm?Story_id=80746](http://www.economist.com/displayStory.cfm?Story_id=80746))

Finnish Information Society Advisory Board, *Services in the New
Millennium/ Programme of Action to Support Online Government,* 31
December 2001 (<http://www.infosoc.fi/PublicServices.pdf>[)]{.underline}

U.S. Office of Management and Budget, *National Strategy for
Transforming to an Electronic Government,* February 8, 2002
([http://www.whitehouse.gov/omb/egov/](http://www.whitehouse.gov/omb/))

Accenture Government Practice, *Three Radical Changes Promise Practical
Impact*
([http://www.accenture.com/xd/xd.asp?it=enweb&xd=industries\\government\\gove_waves.xml#summary](http://www.accenture.com/xd/xd.asp?it=enweb&xd=industries\government\gove_waves.xml#summary))

ICA Member Country Reports, ICA 36^th^ Conference, Singapore, October
2002 (<http://www.ica-it.org/conf36/index.html>)

Malatest & Associates Ltd., for the Public Sector Service Delivery
Council and the Institute of Public Administration of Canada, *Clients
Speak, A Report on Single-Window Government Services in Canada,* 2002
([www.ipaciapc.ca](http://www.ipaciapc.ca/))

McConnell International, *The Global E-Government Outlook,* 2002
([http://www.mcconnellinternational.com/ereadiness/The_Global_E-Government_Outlook.pdf](http://www.mcconnellinternational.com/))

Miller, Jason, *Government Computer News, "OMB to Agencies*

*Justify that IT Spending,\"* 2/10/03
(<http://www.gcn.com/22_3/news/21095-1.html>)

National Association of State Chief Information Officers (NASCIO),
*Creating Citizen-Centric Digital Government/A Guide for the States v.
2001* (<https://www.nascio.org/hotIssues/dg/introduction.cfm>).

National Association of State Chief Information Officers (NASCIO),
*Resource Funding Pools/An Innovative Way of Funding Digital
Government/Issue Brief,* October 25, 2002

Organization for Economic Co-operation and Development, *OECD
E-Government Project, Key Issues and Findings,* October 2, 2002

Organization for Economic Co-operation and Development, *Summary Record
of the 1^st^ Session of the PUMA Expert Group on Government Relations
with Citizens and Civil Society,* February 8, 2002,
(<http://www.olis.oecd.org/olis/2002doc.nsf/43bb6130e5e86e5fc12569fa005d004c/a4da88b37fe7cc68c1256b5a00380229/$FILE/JT00120520.PDF>)

Organization for Economic Co-operation and Development, *Summary Record
of the 2^nd^ Session of the PUMA Expert Group on Government Relations
with Citizens and Civil Society,* August 7, 2002
(<http://www.olis.oecd.org/olis/2002doc.nsf/43bb6130e5e86e5fc12569fa005d004c/10fb657f44a588d4c1256bec004df3a6/$FILE/JT00130234.PDF>)

Poland, Pauline, Government Online International Network, *Online
Consultation In GOL-IN Countries - Initiatives to foster e-democracy,*
(<http://governments-online.org/documents/e-consultation.pdf>)

*Public Sector CXO* Magazine, *Insights on transformation for government
chief officers*
(<http://cxoadvisory.com.futuresite.register.com/_wsn/page16.html>)

Taylor Nelson Sofres, *Government Online Study 2002*
([www.tnsofres.com/gostudy2002/](http://www.tnsofres.com/gostudy2002/)

TexasOnline Authority, Department of Information Resources,
*Cost-Benefit Study of Online Services,* January 2003.
(<http://www.dir.state.tx.us/TIC/dir_info/dirpubs.htm>)

The Performance Institute, *Creating a Performance-Based Electronic
Government*, October 30, 2002
([www.performanceweb.org/research/egovernment.htm](http://www.performanceweb.org/research/egovernment.htm))

The World Bank Group, *"BHOOMI" Online Delivery of Land Records in
Karnataka, India*
([www1.worldbank.org/publicsector/egov/bhoomi_cs.htm](../www1.worldbank.org/publicsector/egov/bhoomi_cs.htm)[)]{.underline}

U.S. General Accounting Office, *Electronic Government/Success of the
Office of Management and Budget's 25 Initiatives Depends on Effective
Management and Oversight*, March 13, 2003
([www.gao.gov](http://www.gao.gov/))

U.S. Office of Management and Budget, *E-Gov Strategy/Simplified
Delivery of Services to Citizens,* February 27, 2002
(<http://www.whitehouse.gov/omb/inforeg/egovstrategy.pdf>)

U.S. Social Security Administration, U.S. General Services
Administration, Booz Allen Hamilton and Harvard University, *The Value
Measuring Methodology,* August 2002.

UK National Audit Office, *Government on the Web II, Report by the
Comptroller and Auditor General,* April 25, 2002
(<http://www.nao.gov.uk/publications/nao_reports/01-02/0102764.pdf>)

*UK Online/Annual Report 2002*
([www.e-envoy.gov.uk/oee/oee.nsf/sections/reports-annrep-2002-pdf/\$file/annualreport02.pdf](http://www.e-envoy.gov.uk/oee/oee.nsf/sections/reports-annrep-2002-pdf/$file/annualreport02.pdf))

"*[Working](http://www.optimizemag.com/issue/015/gap.htm) Smarter Takes
Work,"* *Optimize* Magazine, January 2003, Issue 15,
([www.optimizemag.com/issue/015/gap.htm](http://www.optimizemag.com/issue/015/gap.htm)*,)*

West, Darrell, Brown University, *Assessing E-Government/The Internet,
Democracy and Service Delivery by State and Federal Governments,*
September 2000,
([www.brown.edu/Departments/Taubman_Center/polreports/egovtreport00.html](http://www.brown.edu/Departments/Taubman_Center/polreports/egovtreport00.html))

Correspondence:

Email from Michelle D'Auray, CIO, Canada 3/24/03

Email from Dave Molchany, CIO, Fairfax County, VA, 3/14/2003

Email from Ian Sears, UK, 9/30/02

Email from Richard Murray, Chief Economist, Sweden, 10/07/02

Appendix I

Payoff Categories and Benefits Offered by

75 Nominees for 2001 and 2002 IT Recognition Awards

National Association of State Chief Information Officers

A. Summary of 22 NASCIO 2001 Recognition Award nominees

+---------------+--------------------+---------------------------------+
| 2001 Project  | 5 Payoff           | Benefits                        |
|               | Categories         |                                 |
+===============+====================+=================================+
| California    | **Financial:**     | \- Efficiency in purchasing     |
|               | Cost savings or    |                                 |
| CAL-Buy       | cash input         | \- Easier for vendors to do     |
| Online        |                    | business with State             |
| Procurement   | **Constituent      |                                 |
| System        | Services:**        | \- Ultimately \$9.7 million     |
|               | Delivery of        | annual process savings @\$37    |
|               | services and info  | per purchase order.             |
|               | to the users       |                                 |
|               |                    |                                 |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| Connecticut   | **Constituent      | \- FOIA requests dropped from   |
| Ethics        | Services:**        | 500 to 50/mo.                   |
| Commission    | Delivery of        |                                 |
| Online        | services and info  | \- \$10,000 savings in          |
| Reporting     | to the users       | printing, etc., costs in 2000   |
| System        |                    |                                 |
|               |                    | \- Reduced staff-time,          |
|               |                    | paperwork                       |
|               |                    |                                 |
|               |                    | (cost \$200,000)                |
+---------------+--------------------+---------------------------------+
| Idaho         | **Constituent      | \- 3 million hits to home page  |
| Official Web  | Services:**        |                                 |
| Site          | Delivery of        | \- Increased ability to         |
| [ac           | services and info  | transact e- business            |
| cessidaho.org | to the users       |                                 |
| ]{.underline} |                    | \- future savings of many       |
|               | **Reduced          | thousands of dollars            |
| UCC           | Redundancy:**      |                                 |
|               | Eliminating and    | (cost to state = 0)             |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| Illinois      | **Financial:**     | \- \$149,500 first year savings |
| E-Filing      | Cost savings or    |                                 |
|               | cash input         | \- better customer service      |
|               |                    |                                 |
|               | **Constituent      | \- more efficient operations    |
|               | Services:**        |                                 |
|               | Delivery of        | \- more resources left over to  |
|               | services and info  | promote voluntary compliance    |
|               | to the users       |                                 |
|               |                    | (cost = \$34,000)               |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| accessIndiana | **Financial:**     | -\$3 million/year in            |
| -- Next       | Cost savings or    | development, support,           |
| Generation    | cash input         | maintenance and marketing saved |
| Portal        |                    | based on public-private         |
|               | **Constituent      | partnership                     |
|               | Services:**        |                                 |
|               | Delivery of        | \- fewer telephone inquiries    |
|               | services and info  |                                 |
|               | to the users       | \- better accuracy              |
|               |                    |                                 |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | systems redundant  |                                 |
|               | systems            |                                 |
+---------------+--------------------+---------------------------------+
| State of Iowa | **Constituent      | \- Consolidates info systems    |
| Resource      | Services:**        | and alleviates overhead costs   |
| Network --    | Delivery of        |                                 |
| Community     | services and info  |                                 |
| Resource      | to the users       |                                 |
| Directory     |                    |                                 |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| Kansas State  | **Constituent      | \- reduced phone calls by +90%  |
| Board of      | Services:**        | in 5 years                      |
| Nursing       | Delivery of        |                                 |
| Online        | services and info  | \- streamlined processes        |
| License       | to the users       |                                 |
| Renewal       |                    | -self funding with user fees    |
+---------------+--------------------+---------------------------------+
| Maine Rapid   | **Constituent      | \- time savings                 |
| Renewal       | Services:**        |                                 |
| Service       | Delivery of        | \- improved customer service    |
|               | services and info  |                                 |
|               | to the users       | \- transaction fees for private |
|               |                    | partner                         |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant          |                                 |
+---------------+--------------------+---------------------------------+
| eMaryland     | **Financial:**     | \- Reduces the cost of purchase |
| Marketplace   | Cost savings or    | orders by \$100.                |
|               | cash input         |                                 |
|               |                    | \- Reduces "maverick" buying    |
|               | **Constituent      |                                 |
|               | Services:**        | \- Increases state purchasing   |
|               | Delivery of        | power                           |
|               | services and info  |                                 |
|               | to the users       |                                 |
+---------------+--------------------+---------------------------------+
| Massachusetts | **Financial:**     | Improved:                       |
| Enterprise    | Cost savings or    |                                 |
| Portal --     | cash input         | \- convenience                  |
| Mass.gov      |                    |                                 |
|               | **Constituent      | \- customer svc                 |
|               | Services:**        |                                 |
|               | Delivery of        | \- efficiency                   |
|               | services and info  |                                 |
|               | to the users       | \- productivity                 |
|               |                    |                                 |
|               |                    | \$250 million total estimate    |
|               |                    |                                 |
|               |                    | (cost \$180 m over 5 years)     |
+---------------+--------------------+---------------------------------+
| Missouri      | **Financial:**     | Not seen as a revenue generator |
| iGrants:      | Cost savings or    |                                 |
| Internet      | cash input         | \- improves productivity        |
| Grant         |                    |                                 |
| Management    | **Constituent      | \- improves availability of     |
| Application   | Services:**        | service                         |
|               | Delivery of        |                                 |
|               | services and info  | \- allows reallocation of       |
|               | to the users       | resources to other projects     |
|               |                    |                                 |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| Nevada On     | **Constituent      | -10-15% increase in corporate   |
| Line          | Services:**        | filings with increased fees of  |
| Corporate     | Delivery of        | \$57,000 in year 1.             |
| Name          | services and info  |                                 |
| Reservation   | to the users       | (cost = \$500)                  |
|               |                    |                                 |
|               | **Economic         | -improved accuracy, turnaround  |
|               | Development:**     | time and better info.           |
|               | Encouraging growth |                                 |
|               | in regional        | \- attracts business to the     |
|               | economic activity  | state/                          |
+---------------+--------------------+---------------------------------+
| New Jersey    | **Constituent      | \- 2.7 million hits per day     |
| Portal:       | Services:**        |                                 |
| Virtual       | Delivery of        | \- bring seamless E-Government  |
| Gateway to    | services and info  | services to NJ citizens         |
| Government    | to the users       |                                 |
| Services      |                    | \- reusable "build it once"     |
|               | **Reduced          | components such as PKI,         |
|               | Redundancy:**      | e-payment, shopping cart        |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| New York      | **Constituent      | \- Government services are      |
| State         | Services:**        | easier and more convenient to   |
| Government    | Delivery of        | access                          |
| Without Walls | services and info  |                                 |
| Initiative    | to the users       | \- eliminating duplication will |
|               |                    | save state money                |
| Portal        | **Reduced          |                                 |
|               | Redundancy:**      | -almost 600 transactions and    |
|               | Eliminating and    | services available online       |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| North         | **Constituent      | \- self-funded e-procurement    |
| Carolina's    | Services:**        | model                           |
| Portal:       | Delivery of        |                                 |
|               | services and info  |                                 |
| <NC           | to the users       |                                 |
| @YourService> |                    |                                 |
|               | **Economic         |                                 |
|               | Development:**     |                                 |
|               | Encouraging growth |                                 |
|               | in regional        |                                 |
|               | economic activity  |                                 |
|               |                    |                                 |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| Ohio Dolphin  | **Financial:**     | \- \$122 million annual savings |
| Project       | Cost savings or    | from process improvements and   |
|               | cash input         | claims cost reductions          |
| Workers comp  |                    |                                 |
|               | **Constituent      | (cost = \$15 million)           |
|               | Services:**        |                                 |
|               | Delivery of        |                                 |
|               | services and info  |                                 |
|               | to the users       |                                 |
+---------------+--------------------+---------------------------------+
| Pennsylvania  | **Financial:**     | \- "friction-free government"   |
| PA Open for   | Cost savings or    |                                 |
| Business      | cash input         | \- eliminate redundant systems  |
|               |                    | through\                        |
|               | **Constituent      | de-fragmenting                  |
|               | Services:**        |                                 |
|               | Delivery of        |                                 |
|               | services and info  |                                 |
|               | to the users       |                                 |
|               |                    |                                 |
|               | **Economic         |                                 |
|               | Development:**     |                                 |
|               | Encouraging growth |                                 |
|               | in regional econ   |                                 |
|               | activity           |                                 |
|               |                    |                                 |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| Texas System  | **Financial:**     | \- Costs to states have been    |
| for           | Cost savings or    | "all but eliminated"            |
| Electronic    | cash input         |                                 |
| Rate and      |                    | -Industry benefits              |
| Form-Filing   | **Constituent      |                                 |
| (SERFF)       | Services:**        | \$150k to \$12 m per year for   |
|               | Delivery of        | SERFF filings                   |
|               | services and info  |                                 |
|               | to the users       |                                 |
+---------------+--------------------+---------------------------------+
| Virginia eVA  | **Financial:**     | Funded through a self-financing |
|               | Cost savings or    | model                           |
|               | cash input         |                                 |
|               |                    | \$1 billion annual business     |
|               | **Constituent      | volume                          |
|               | Services:**        |                                 |
|               | Delivery of        | (start-up cost = \$300k)        |
|               | services and info  |                                 |
|               | to the users       |                                 |
+---------------+--------------------+---------------------------------+
| Transact      | **Financial:**     | \- Long-term savings from       |
| Washington    | Cost savings or    | consolidated infrastructure and |
|               | cash input         | eliminating duplication         |
|               |                    |                                 |
|               | **Constituent      | \- Reduced operating costs from |
|               | Services:**        | more efficient processes        |
|               | Delivery of        |                                 |
|               | services and info  | \- convenience for businesses   |
|               | to the users       | and citizens                    |
|               |                    |                                 |
|               | **Reduced          |                                 |
|               | Redundancy:**      |                                 |
|               | Eliminating and    |                                 |
|               | integrating        |                                 |
|               | redundant systems  |                                 |
+---------------+--------------------+---------------------------------+
| West Virginia | **Financial:**     | \- reduced staff by 2.5         |
| Service at    | Cost savings or    | positions                       |
| the Speed of  | cash input         |                                 |
| Technology    |                    | \- \$4,000 savings              |
|               | **Constituent      |                                 |
|               | Services:**        | \- shared infrastructure        |
|               | Delivery of        |                                 |
|               | services and info  |                                 |
|               | to the users       |                                 |
+---------------+--------------------+---------------------------------+
| Wisconsin.    | **Constituent      | \- more convenient ways to      |
| Gov Portal    | Services:**        | access state and local          |
|               | Delivery of        | government services             |
|               | services and info  |                                 |
|               | to the users       |                                 |
+---------------+--------------------+---------------------------------+

Review of 53 NASCIO 2002 Recognition Award nominees

+---------------+---------------------+--------------------------------+
| 2002 Project  | 5 Payoff Categories | Benefits                       |
+===============+=====================+================================+
| Colorado      | **Constituent       | \- Reduced mailing costs       |
| Integrated    | Services:**         |                                |
| Criminal      | Delivery of         | \- Improved accuracy of info   |
| Justice       | services and info   |                                |
| Information   | to the public       | \- Improved timeliness         |
| System        |                     |                                |
|               | **Reduced           | \- Reduced data entry          |
|               | Redundancy:**       | redundancy                     |
|               | Eliminating and     |                                |
|               | integrating         | \- Reduced paper processing    |
|               | redundant systems   | and interagency contacts       |
+---------------+---------------------+--------------------------------+
| MyFlorida.com | **Financial:** Cost | \$2.2 million a year:          |
|               | savings or cash     |                                |
|               | input               | \- Improved access to online   |
|               |                     | services reduces operating     |
|               | **Constituent       | cost \$700,000 a year          |
|               | Services:**         |                                |
|               | Delivery of         | \- Improved search engine      |
|               | services and info   | reduces call center calls by   |
|               | to the public       | 1%, for \$1.5 million savings  |
|               |                     |                                |
|               | **Reduced           |                                |
|               | Redundancy:**       |                                |
|               | Eliminating and     |                                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Idaho         | **Constituent       | \- Saves printing/ mailing     |
| Attorney      | Services:**         | costs of \$thousands           |
| General's     | Delivery of         |                                |
| No-Call List  | services and info   | \- initial \$420k receipts     |
| Website and   | to the public       | covers \$180k costs plus 2     |
| Purchase and  |                     | years' expenses                |
| Registration  |                     |                                |
| Applications  |                     |                                |
+---------------+---------------------+--------------------------------+
| Indiana       | **Financial:** Cost | \$51,000 a year and increasing |
| Department of | savings or cash     | annually                       |
| Revenue       | input               |                                |
| I-File        |                     |                                |
|               | **Constituent       |                                |
| \(a\)         | Services:**         |                                |
|               | Delivery of         |                                |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Kansas Job    | **Financial:** Cost | \- Individuals using job       |
| Link          | savings or cash     | function need .75 weeks' less  |
|               | input               | of unemployment benefits,      |
| \(b\)         |                     | which saves \$9 million per    |
|               | **Constituent       | year                           |
|               | Services:**         |                                |
|               | Delivery of         | \- Time required to process a  |
|               | services and info   | claim reduced 76%              |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Kentucky IVR  | **Constituent       | \- \$48,000 per year over 5    |
| Deer Harvest  | Services:**         | years                          |
| Program       | Delivery of         |                                |
|               | services and info   | \- Savings of 119 days for     |
|               | to the public       | data collection                |
+---------------+---------------------+--------------------------------+
| Massachusetts | **Financial:** Cost | \- Streamlined licensing cuts  |
| Educator      | savings or cash     | 6 months off process           |
| Licensure and | input               |                                |
| Recruitment   |                     | \- cuts \$1.6 million annual   |
| Initiative    | **Constituent       | costs                          |
|               | Services:**         |                                |
| G2G           | Delivery of         | \- adds \$2.5 million          |
|               | services and info   | licensing revenue              |
| # WINNER      | to the public       |                                |
|               |                     | (cost: \$2.6 million)          |
+---------------+---------------------+--------------------------------+
| Missouri      | **Financial:** Cost | \- potential savings of        |
| Internet      | savings or cash     | \$61,250 per year              |
| On-line       | input               |                                |
| Claims Filing |                     | \- redirected 6600 staff hours |
| System        | **Constituent       | toward customer service        |
|               | Services:**         |                                |
| G2G           | Delivery of         |                                |
|               | services and info   |                                |
| \(b\)         | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Montana       | **Constituent       | \- Crime victims' peace of     |
| Criminal      | Services:**         | mind                           |
| Offenders     | Delivery of         |                                |
| Network --    | services and info   | \- \$54,550 maintenance paid   |
| Online        | to the public       | by other state agencies        |
| Services      |                     |                                |
+---------------+---------------------+--------------------------------+
| Nevada        | **Constituent       | -System availability 24/7      |
| Employment    | Services:**         |                                |
| Insurance     | Delivery of         | \- Data Integrity              |
| Internet      | services and info   |                                |
| Claim         | to the public       | \- System Security             |
| System(b)     |                     |                                |
+---------------+---------------------+--------------------------------+
| New York City | **Constituent       | Improved sanitation in city    |
| Restaurant    | Services:**         | restaurants due to             |
| Inspection,   | Delivery of         | availability of information    |
| Recording and | services and info   | about health code violations   |
| Reporting     | to the public       |                                |
| System        |                     |                                |
+---------------+---------------------+--------------------------------+
| North         | **Constituent       | The portal improved process    |
| Carolina's    | Services:**         | for filing unemployment claims |
| Portal to     | Delivery of         | online and to pay weekly       |
| Automated     | services and info   | claims by direct deposit.      |
| Unemployment  | to the public       |                                |
| Insurance     |                     |                                |
| Services (b)  |                     |                                |
| (v)           |                     |                                |
+---------------+---------------------+--------------------------------+
| Ohio          | **Financial:** Cost | Automating fishing/hunting     |
| Department of | savings or cash     | licenses supports marketing    |
| Natural       | input               | program to attract/retain      |
| Resources     |                     | loyal customers; \$500k in     |
|               | **Constituent       | (incremental?) licensing       |
|               | Services:**         | revenue                        |
|               | Delivery of         |                                |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Pennsylvania  | **Constituent       | Empowers people by sending     |
| Department of | Services:**         | them customized notifications  |
| Environmental | Delivery of         | of environmental permitting    |
| Protection's  | services and info   | actions so that they can       |
| eNOTICE       | to the public       | comment on pending actions.    |
| Service       |                     | Costs = \$20,000.              |
|               | **Fostering         |                                |
|               | Democracy:**        |                                |
|               | Nurturing civic     |                                |
|               | practices that      |                                |
|               | support democratic  |                                |
|               | ideals              |                                |
+---------------+---------------------+--------------------------------+
| Rhode Island  | **Financial:** Cost | \- \$40k (\$0 cost)            |
| Online Boat   | savings or cash     |                                |
| Registration  | input               | \- Less State employee time    |
| Renewal       |                     | needed for data entry,         |
| Service       | **Constituent       | answering calls, processing    |
|               | Services:**         | checks, marketing.             |
|               | Delivery of         |                                |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| South Dakota  | **Financial:** Cost | \- Savings in staff time,      |
| Service       | savings or cash     | postage, storage, check        |
| Direct (Forms | input               | processing.                    |
| Portal)       |                     |                                |
|               | **Constituent       | \- Customer convenience of     |
|               | Services:**         | online forms and pre-populated |
|               | Delivery of         | forms.                         |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Texas State   | **Constituent       | Saves time and trouble in      |
| of Texas      | Services:**         | predetermining whether         |
| Assistance    | Delivery of         | individuals are eligible for   |
| and Referral  | services and info   | government benefits            |
| System G2G    | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Utah          | **Constituent       | \- Up to date information      |
| Community     | Services:**         | available to citizens          |
| Services      | Delivery of         |                                |
| Directory     | services and info   | \- Information consolidated    |
|               | to the public       | and easy to access             |
|               |                     |                                |
|               |                     | \- Database can be leveraged   |
|               |                     | by other state agencies        |
+---------------+---------------------+--------------------------------+
| Virginia VEC  | **Financial:** Cost | \- \$6.5 million savings for   |
| Claim for     | savings or cash     | claimants in saved travel and  |
| Benefits (b)  | input               | time;                          |
|               |                     |                                |
|               | **Constituent       | \- \$821,786 operational       |
|               | Services:**         | savings                        |
|               | Delivery of         |                                |
|               | services and info   | (cost = \$250,000)             |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Florida       | **Financial:** Cost | \- Eliminates the cost of      |
| Juvenile      | savings or cash     | connecting 443 police          |
| Justice       | input               | departments and sheriffs'      |
| Information   |                     | offices of \$2.1 million/year. |
| System        | **Reduced           |                                |
|               | Redundancy:**       | (cost of coding =\$211,200)    |
|               | Eliminating and     |                                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Idaho         | **Financial:** Cost | \- \$430,000 annual savings in |
| Paperless     | savings or cash     | personnel                      |
| Online        | input               |                                |
| Pers          |                     | -20% reduction in personnel    |
| onnel/Payroll |                     |                                |
| System        |                     | -\$75,000 annual savings in    |
|               |                     | printing pay stubs             |
|               |                     |                                |
|               |                     | (cost = \$1.65 million)        |
+---------------+---------------------+--------------------------------+
| Illinois      | **Constituent       | \- Enhancement of interagency  |
| Wireless      | Services:**         | communication                  |
| Information   | Delivery of         |                                |
| Network       | services and info   |                                |
| (             | to the public       |                                |
| I-WIN/ALERTS) |                     |                                |
|               | **Reduced           |                                |
|               | Redundancy:**       |                                |
|               | Eliminating and     |                                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Indiana State | **Constituent       | \- 35% reduction in personnel  |
| Student       | Services:**         | costs in the Grants Division   |
| Assistance    | Delivery of         |                                |
| Commission    | services and info   | \- \$390,000 received in sale  |
| Suite of      | to the public       | to another state               |
| Services      |                     |                                |
+---------------+---------------------+--------------------------------+
| Missouri      | **Constituent       | \- 86% reduction in            |
| E-Grants      | Services:**         | application processing time    |
|               | Delivery of         | using 35% less staff           |
|               | services and info   |                                |
|               | to the public       | \- 360% increase in technical  |
|               | **Reduced           | support to school districts    |
|               | Redundancy:**       |                                |
|               | Eliminating and     | (cost = \$2.5 m)               |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Nevada        | **Reduced           | \- seamless info and services  |
| Multi-County  | Redundancy:**       | at all levels of government    |
| Integrated    | Eliminating and     |                                |
| Justice       | integrating         | \- reduced paper flow          |
| Information   | redundant systems   |                                |
| System        |                     | \- reduced errors              |
+---------------+---------------------+--------------------------------+
| New Jersey    | **Reduced           | \- reduced backup of State     |
| Pre-Inmate    | Redundancy:**       | inmates in county jails which  |
| Management    | Eliminating and     | costs up to \$4.7 m.           |
| System        | integrating         |                                |
|               | redundant systems   | \- streamlined processes and   |
|               |                     | reduced redundancies           |
+---------------+---------------------+--------------------------------+
| New Mexico    | **Constituent       | \- Savings from revised        |
| Web-based     | Services:**         | processes \$236,440/year       |
| Instructional | Delivery of         |                                |
| Materials     | services and info   | (costs \$167,551)              |
| System        | to the public       |                                |
|               | **Reduced           |                                |
|               | Redundancy:**       |                                |
|               | Eliminating and     |                                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| North         | **Financial:** Cost | Savings from avoiding 550      |
| Carolina      | savings or cash     | computer incidents @ \$4k      |
| Security      | input               | =\$2.2 million                 |
| Portal        |                     |                                |
|               | **Constituent       | (cost = \$160k)                |
|               | Services:**         |                                |
|               | Delivery of         |                                |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Pennsylvania  | **Constituent       | \- reduced costs               |
| Justice       | Services:**         |                                |
| Network       | Delivery of         | \- reduced delays              |
|               | services and info   |                                |
|               | to the public       | \- public safety               |
|               |                     |                                |
|               | **Reduced           | \- easy to replicate in other  |
|               | Redundancy:**       | states and federal agencies    |
|               | Eliminating and     |                                |
|               | integrating         | (cost = \$18 million)          |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Texas ARP/ATP | **Constituent       | \$21,338 per grant net savings |
| Online System | Services:**         | (includes cost savings at      |
|               | Delivery of         | colleges)                      |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Virginia Base | **Reduced           | \$10 million avoided costs for |
| Mapping       | Redundancy:**       | counties and cities if paid    |
| Program       | Eliminating and     | for individually               |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Washington    | **Constituent       | Total net savings, including   |
| State         | Services:**         | outreach and admin = \$6.37 m  |
| Combined      | Delivery of         | per year                       |
| Application   | services and info   |                                |
| Program       | to the public       | (cost = \$400k)                |
|               | **Reduced           |                                |
| # WINNER      | Redundancy:**       |                                |
|               | Eliminating and     |                                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| West Virginia | **Reduced           | \- data accuracy               |
| Data          | Redundancy:**       |                                |
| Integration   | Eliminating and     | \- personnel time savings      |
| for Effective | integrating         |                                |
| Reporting and | redundant systems   |                                |
| Efficient     |                     |                                |
| Fiscal        |                     |                                |
| Management    |                     |                                |
+---------------+---------------------+--------------------------------+
| Colorado      | **Financial:** Cost | Annual savings:                |
| Secretary of  | savings or cash     |                                |
| State         | input               | \- \$2 million (\$169.5/       |
| Business      |                     | month) for businesses on UCC   |
| Center        | **Constituent       | filings and documents          |
|               | Services:**         |                                |
| \(d\)         | Delivery of         | \- \$372,000 State savings on  |
|               | services and info   | UCC filings, certificates      |
| \(e\)         | to the public       | (\$31k/month)                  |
+---------------+---------------------+--------------------------------+
| District of   | **Financial:** Cost | \$1.8 million a year savings   |
| Columbia      | savings or cash     | from reduced customer service  |
| Business      | input               | positions. (cost- \$1 million) |
| Resource      |                     |                                |
| Center        | **Constituent       |                                |
|               | Services:**         |                                |
| \(d\)         | Delivery of         |                                |
|               | services and info   |                                |
| # WINNER      | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Florida       | **Constituent       | \- annual admin cost savings   |
| Business and  | Services:**         | of \$40,000 per FTE            |
| Professional  | Delivery of         |                                |
| Regulation    | services and info   | from customer contact center   |
| Single        | to the public       | consolidation; reduced data    |
| Licensing     | **Reduced           | entry; and other personal      |
| Project       | Redundancy:**       | services                       |
|               | Eliminating and     |                                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Idaho Small   | **Constituent       | \- \$7,000 per year            |
| Business      | Services:**         |                                |
| Solutions     | Delivery of         | \- fosters 15-30 new           |
|               | services and info   | businesses                     |
|               | to the public       |                                |
|               | **Reduced           | \- adds 15-90 jobs             |
|               | Redundancy:**       |                                |
|               | Eliminating and     | (cost = \$120k)                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Illinois      | **Constituent       | \- reduced paperwork backlog   |
| E-Batch       | Services:**         |                                |
| License       | Delivery of         | \- user-friendly and timely    |
| Renewal       | services and info   | services to professionals      |
|               | to the public       |                                |
|               | **Reduced           | (cost = \$12,500)              |
|               | Redundancy:**       |                                |
|               | Eliminating and     |                                |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Indiana       | **Constituent       | \- time savings with 24/7      |
| Uniform       | Services:**         | access                         |
| Commercial    | Delivery of         |                                |
| Code          | services and info   | \- \$30,000 salary unnecessary |
| Services:     | to the public       |                                |
| Filings and   |                     | \- accuracy                    |
| Searches (e)  |                     |                                |
+---------------+---------------------+--------------------------------+
| Iowa Single   | **Constituent       | Total -\$264k/year:            |
| Contact       | Services:**         |                                |
| Repository    | Delivery of         | \- savings to employers doing  |
|               | services and info   | background checks @\$100 =     |
|               | to the public       | \$132,000/yr                   |
|               |                     |                                |
|               |                     | \- savings to state government |
|               |                     | = \$132,000/year               |
|               |                     |                                |
|               |                     | (cost = \$277,000)             |
+---------------+---------------------+--------------------------------+
| Kansas Online | **Constituent       | Savings for Sec of State       |
| UCC Filing    | Services:**         | Office @\$..71 per filing =    |
| System        | Delivery of         | \$710,000/year                 |
|               | services and info   |                                |
| \(e\)         | to the public       | Other benefits are in time     |
|               | **Reduced           | saved, convenience, reduced    |
|               | Redundancy:**       | payment processing and greater |
|               | Eliminating and     | accuracy.                      |
|               | integrating         |                                |
|               | redundant systems   |                                |
+---------------+---------------------+--------------------------------+
| Kentucky      | **Financial:** Cost | \- timely and cost-efficient   |
|               | savings or cash     | info updates for travelers to  |
| The Extranet: | input               | Kentucky                       |
| Business      |                     |                                |
| Friendly      | **Economic          | \- \$4.9 million/year state    |
| Control       | Development:**      | tax revenue from website       |
|               | Encouraging growth  | visitors                       |
|               | in regional         |                                |
|               | economic activity   | (cost = \$138,333/year)        |
+---------------+---------------------+--------------------------------+
| Massachusetts | **Financial:** Cost | \- reduces costs of mailing    |
| Webfile for   | savings or cash     | and forms printing, data entry |
| Employers     | input               | and data processing            |
|               |                     |                                |
| \(b\)         | **Constituent       | \- reduces reporting burden on |
|               | Services:**         | employers                      |
|               | Delivery of         |                                |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Montana       | **Financial:** Cost | Public/private alliance that   |
| Business      | savings or cash     | developed and maintained       |
| Entity Search | input               | program saved taxpayers \$60k  |
| -- Online     |                     | in 3 months.                   |
| Service       | **Constituent       |                                |
|               | Services:**         |                                |
|               | Delivery of         |                                |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Nebraska's    | **Financial:** Cost | \$424,164 in Yr 1:             |
| UIConnect     | savings or cash     |                                |
|               | input               | \- \$361,624 for employers in  |
| \(c\)         |                     | tax calculation, report        |
|               | **Constituent       | preparation, bookkeeper time   |
|               | Services:**         |                                |
|               | Delivery of         | \- \$62,540 for government in  |
|               | services and info   | data entry, wage reporting,    |
|               | to the public       | payment processing             |
+---------------+---------------------+--------------------------------+
| North         | **Financial:** Cost | By allowing online renewals,   |
| Carolina      | savings or cash     | registrations and fee          |
| International | input               | payments, offers savings in:   |
| Registration  |                     |                                |
| Plan          | **Constituent       | -agency time                   |
|               | Services:**         |                                |
|               | Delivery of         | \- admin burden                |
|               | services and info   |                                |
|               | to the public       | \- motor carrier operating     |
|               |                     | time and costs                 |
+---------------+---------------------+--------------------------------+
| Ohio Business | **Financial:** Cost | Reduction of:                  |
| Gateway       | savings or cash     |                                |
|               | input               | \- time, data redundancy,      |
| \(d\)         |                     | errors in complying with tax   |
|               | **Constituent       | laws                           |
|               | Services:**         |                                |
|               | Delivery of         | \- data entry                  |
|               | services and info   |                                |
|               | to the public       | \- paperwork                   |
+---------------+---------------------+--------------------------------+
| Pennsylvania  | **Constituent       | \- Reduced lag time in         |
| Open for      | Services:**         | business registrations         |
| Business      | Delivery of         |                                |
| Interactive   | services and info   | \- Average \$500 professional  |
| Registration  | to the public       | services charge per business   |
| Portal        |                     |                                |
|               |                     | \- Available for use by other  |
|               |                     | states                         |
+---------------+---------------------+--------------------------------+
| Texas         | **Financial:** Cost | \- Savings of \$300,000 a year |
| Electronic    | savings or cash     |                                |
| Reporting     | input               | \- Reduction of FTEs from      |
| System for    |                     | 27-20                          |
| Reporting and | **Constituent       |                                |
| Payment of    | Services:**         | \- Mail processing, data       |
| Oil and Gas   | Delivery of         | entry, error correction, time  |
| Royalties     | services and info   | and rework reduced by \>70%.   |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+
| Utah Impound  | **Financial:** Cost | Reduced cost of:               |
| Vehicle       | savings or cash     |                                |
| System        | input               | \- data entry staff            |
|               |                     |                                |
|               | **Constituent       | -decertifying tow companies    |
|               | Services:**         |                                |
|               | Delivery of         | \- locating vehicles           |
|               | services and info   |                                |
|               | to the public       | Citizens get real-time info on |
|               |                     | their vehicles                 |
+---------------+---------------------+--------------------------------+
| Virginia      | **Financial:** Cost | \- Service 24/7                |
| WebCat: A     | savings or cash     |                                |
| Virginia DMV  | input               | \- Transactions at half the    |
| Extranet      |                     | cost of manual                 |
| Application   | **Constituent       |                                |
| for Motor     | Services:**         | \- reduced paperwork and       |
| Carriers      | Delivery of         | in-person visits               |
|               | services and info   |                                |
|               | to the public       | \- reduced time requirements   |
+---------------+---------------------+--------------------------------+
| Washington    | **Financial:** Cost | Annual savings:                |
| Online        | savings or cash     |                                |
| Uniform       | input               | \- \$323k salaries             |
| Commercial    |                     |                                |
| Code Filings  | **Constituent       | \- \$154k data processing      |
| and Searches  | Services:**         |                                |
|               | Delivery of         | \- \$36k microfilm processing  |
| \(e\)         | services and info   |                                |
|               | to the public       | (cost=\$1.05m)                 |
+---------------+---------------------+--------------------------------+
| Wisconsin     | **Financial:** Cost | Annual cost savings of \$1.5   |
| Worker's      | savings or cash     | million @\$10 for each piece   |
| Compensation  | input               | of paper saved by all parties  |
| Insurers'     |                     |                                |
| Internet      | **Constituent       | (costs = \$275k startup +      |
| Reporting     | Services:**         | \$25k/yr)                      |
| System        | Delivery of         |                                |
|               | services and info   |                                |
|               | to the public       |                                |
+---------------+---------------------+--------------------------------+

Appendix II

Summary of E-Gov Measurement Systems

1.  [The Anexys Primer on Measuring ROI in E-Government, by Anexys LLC,
    Indiana University School of Public and Environmental Affairs, and
    Meta-Group (August 2001)]{.underline}

-   Early adopters of new valuation methods will test models that move
    beyond the traditional ROI measurements. By 2004-2005, such efforts
    will drive widespread adoption of comprehensive valuation models in
    large jurisdictions.

-   In today's legislative climate, it's likely that government agencies
    will be increasingly required to conduct ongoing ROI studies.
    Agencies that fail to measure ROI of E-Government initiatives may
    face difficulties in receiving both initial and ongoing funding.

-   ROI Valuation is critical to E-Government success because the cost
    of failure is too high, because E-Government initiatives:

    -   come with substantial expense

    -   can have very long durations

    -   are infrequent occurrences

    -   and have limited irreversibility.

-   While most E-Government initiatives are established to reflect the
    policy direction of the agency, such as citizen convenience,
    economic development, tourism or creation of an educated work force,
    the most common themes are:

    -   Increasing the competitive advantage of the agency over other
        agencies within the same jurisdiction

    -   Creating greater citizen satisfaction

    -   Economic development or reduction of transaction costs by
        providing online services.

-   Carol Kelly of Meta-Group suggests jurisdictions should:

    -   Determine components based on public policy makers' articulated
        agenda, e.g., decentralization, flexibility and personalization

    -   Develop explicit analysis criteria for each scorecard component
        (balanced scorecard method)

    -   Determine measurement points for data collection

    -   Map known existing relations/approval processes

    -   Develop methods to quantify intangible benefits

    -   Ensure that overall valuation model measures stated policy
        requirements

-   Anexys suggests four possible valuation methods:

    1.  Net present Value to measure dollar return on projects

    2.  Benefit Cost Analysis to calculate project costs relative to
        benefits (tangible and intangible) affecting diverse groups.

    3.  Cost Effectiveness Analysis to quantify tangible and intangible
        benefits for specific, standardized populations.

    4.  Portfolio Analysis to quantify aggregate risk relative to
        expect6ed returns of an entire portfolio of initiatives

**ROI Valuation Methodologies for E-Government**

From The Anexys Primer on Measuring ROI in E-Government, 2001

By Anexys, LLCIndiana University-Bloomington---Institute for
Development StudiesMETA Group, Inc.

+-----------+------------+---------------+-------------+-------------+
| *         | **Net      | **Benefit     | **Cost      | **Portfolio |
| *Method** | Present    | Cost          | Ef          | Analysis**  |
|           | Value**    | Analysis**    | fectiveness |             |
|           |            |               | Analysis**  |             |
+-----------+------------+---------------+-------------+-------------+
| ## Wha    | Measures   | Calculates    | Quantifies  | Quantifies  |
| t it does | dollar     | project costs | tangible    | aggregate   |
|           | return on  | relative to   | and         | risk        |
|           | projects   | benefits      | intangible  | relative to |
|           |            | (tangible and | benefits    | expected    |
|           |            | intangible)   | for         | returns of  |
|           |            | affecting     | specific    | an entire   |
|           |            | diverse       | s           | portfolio   |
|           |            | groups        | tandardized | of          |
|           |            |               | populations | i           |
|           |            |               |             | nitiatives. |
+-----------+------------+---------------+-------------+-------------+
| **        | Present    | Net Total     | Total       | Return of   |
| Formula** | Value of   | Benefit/Net   | Cost/Total  | Inve        |
|           | Net Cash   | Total Cost    | Output      | stment/Risk |
|           | Flow --    |               |             | of          |
|           | Net        |               |             | Investment  |
|           | Investment |               |             |             |
+-----------+------------+---------------+-------------+-------------+
| **When to | When cash  | When cash     | When cash   | When        |
| use**     | flows are  | flows are     | flows are   | valuating   |
|           | private    | social and    | private and | an agency's |
|           | and        | benefits are  | benefits    | total risk. |
|           | benefits   | both tangible | are both    | Cash flows  |
|           | tangible   | and           | tangible    | can be      |
|           |            | intangible    | and         | social or   |
|           |            |               | intangible  | private and |
|           |            |               |             | benefits    |
|           |            |               |             | can be      |
|           |            |               |             | tangible or |
|           |            |               |             | intangible  |
|           |            |               |             | but must be |
|           |            |               |             | consistent  |
|           |            |               |             | across the  |
|           |            |               |             | portfolio.  |
+-----------+------------+---------------+-------------+-------------+
| **Basic   | For        | If ratio is   | If ratio is | If ratio is |
| decision  | i          | \>1, there is | \>1, there  | \>1, it     |
| rule**    | ndependent | a positive    | is a        | means there |
|           | projects   | ROI and the   | positive    | is more     |
|           | with NPV   | project       | ROI and the | return      |
|           | greater    | should be     | project     | relative to |
|           | than \$0,  | accepted. If  | should be   | risk and    |
|           | the        | ratio is \<1, | accepted.   | the         |
|           | project    | there is a    | If ratio is | portfolio   |
|           | should be  | negative      | \<1, there  | is within   |
|           | accepted   | return        | is a        | tolerable   |
|           |            |               | negative    | risk        |
|           |            |               | return      | levels. If  |
|           |            |               |             | the ratio   |
|           |            |               |             | is \<1, it  |
|           |            |               |             | means risk  |
|           |            |               |             | is higher   |
|           |            |               |             | relative to |
|           |            |               |             | return and  |
|           |            |               |             | the         |
|           |            |               |             | portfolio   |
|           |            |               |             | exceeds     |
|           |            |               |             | tolerable   |
|           |            |               |             | risk.       |
+-----------+------------+---------------+-------------+-------------+
| **Adv     | Strai      | Flexible.     | Useful at   | Good for    |
| antages** | ghtforward | Allows        | analyzing   | r           |
|           |            | additional    | incremental | isk-adverse |
|           |            | intangible    | benefits,   | deci        |
|           |            | benefits to   | in terms of | sion-makers |
|           |            | be included   | achieving   | who may not |
|           |            |               | specific    | want to     |
|           |            |               | goals, in   | attempt     |
|           |            |               | relation to | multiple    |
|           |            |               | marginal    | h           |
|           |            |               | costs.      | igh-stakes, |
|           |            |               |             | high-risk   |
|           |            |               |             | projects    |
|           |            |               |             | simu        |
|           |            |               |             | ltaneously. |
+-----------+------------+---------------+-------------+-------------+
| **Disadv  | Limited to | Can be        | Technique   | Complex,    |
| antages** | internal   | expensive,    | not         | but could   |
|           | capital    | ti            | directly    | possibly be |
|           | in         | me-consuming. | related to  | the most    |
|           | vestments. | Challenging   | the         | accurate in |
|           | Does not   | to agree on   | outcome;    | terms of    |
|           | allow for  | values of     | measurement | strategic   |
|           | intangible | intangible    | is limited  | management. |
|           | benefits.  | costs         | to the      |             |
|           |            |               | cost-eff    |             |
|           |            |               | ectiveness, |             |
|           |            |               | not         |             |
|           |            |               | outcomes.   |             |
+-----------+------------+---------------+-------------+-------------+

2.  [*Creating a Performance-Based Electronic Government*, a report
    issued October 30, 2002, by The Performance Institute, et
    al.]{.underline}

Key themes:

1.  The administration and OMB are to be commended for their strong
    leadership of E-Gov initiatives, though some improvements are
    needed.

2.  Federal agencies generally fail to use mission-aligned IT
    performance measures to justify, manage and evaluate the success of
    E-Government.

3.  Agencies need to become more creative and willing to "blow up" old
    program structures with technology.

4.  Non-governmental intermediaries are providing greater opportunities
    to borrow rather than build an E-Government solutions

5.  E-Government is increasingly focusing on the citizen again, but not
    all E-Government initiatives are "citizen-centered."

6.  CIOs are assuming an appropriate role of "enabler" of agency
    business processes and are more integrated with the rest of the
    agency's leadership.

7.  More program managers are playing leadership roles in E-Government,
    but more needs to be done to engage all program managers in
    E-Government leadership roles.

8.  Excellent cross-agency coordination is seen in the priority
    E-Government initiatives, but stove-piped systems and processes
    remain an obstacle to an integrated E-Government.

9.  The federal enterprise architecture is recognized as the necessary,
    scaffolding for all agency E-Gov initiatives.

10. Establishing clear priorities is paying off in generating attention
    to and sufficient funding for key E-Gov initiatives.

```{=html}
<!-- -->
```
2.  [*Technology in Government: Riding the Waves of Change*, Accenture
    Institute for Strategic Change, 2002]{.underline}

Accenture interviewed 70 political leaders and top government executives
from a range of government agencies in 10 countries.

Public officials interviewed agreed on five broad areas that are
priorities for improvement in government:

1)  Differentiate services to create more constituent-centric government

2)  Share assets and solutions across organizations and levels of
    government

3)  Establish new funding mechanisms and incentives to achieve
    cross-departmental goals

4)  Improve services by taking a consolidated view of constituents'
    information

5)  Foster prosperity by developing a robust national technology
    infrastructure.

Public officials' efforts to date in these five areas have begun to pay
off in improved management practices and greater efficiency. But they
have not gone far enough.

60% of public officials interviewed are discussing future directions but
not yet taking action...

Accenture foresees 3 dramatic changes enabled by emerging technology
waves that democratic governments should pursue now:

1)  Establishing dynamic connections...between governments and their
    constituents to enable intelligent interactions....The primary
    characteristics include a single, integrated face for government
    interactions; a practical set of channel choices, two-way flows of
    meaningful information, convenient services that are embedded in
    everyday activities; and an agile network of intermediaries for
    service delivery.

2)  Embracing policy speed to market, or radically accelerating the
    processes of policy formulation and implementation in order to
    improve outcomes.

3)  Engaging constituents as integral stakeholders in the everyday
    activities of government. Using new processes and powerful tools,
    citizens and businesses will leave behind the traditional arms
    length relationship with governments to embark on a closer
    collaboration....And constituents will take responsibility for
    paying for the services and resources they use.

# *Value Measuring Methodology: Capturing, Measuring and Evaluating the Value of IT Investments,* U.S. Social Security Administration/U.S. General Services Administration

VMM is a toolkit for capturing, measuring and evaluating the value of
E-Government IT investments. It provides the structure, tools and
techniques for a comprehensive analysis and comparison of value
(benefits), cost and risk. Outcomes include:

-   Measures that are used to define success, identify and analyze
    alternatives, address values, and support desired outcomes.

-   Documentation that provides an audit trail of decision-making and
    assumptions regarding priorities, value, risk and cost.

-   Comparison of alternatives and relationships between value and
    investment cost.

-   Analysis that must be summarized in the preparation of budget
    justification documents such as OMB Exhibit 300.

VMM provides a foundation for decision-making and management of an
initiative. The methodology has the approval of the CIO Council Best
Practices Committee.

Appendix III

**Benefits Cited in**

**State E-Gov Programs Nominated for NASCIO Digital Government Awards**

# Workers Compensation

+--------------+--------------------+---------------------------------+
| State        | Workers'           | Benefits                        |
|              | Compensation       |                                 |
|              | Project            |                                 |
+--------------+--------------------+---------------------------------+
| Ohio Bureau  | The Dolphin        | Over \$120 million annual       |
| of Workers'  | Project            | savings to BWC on a \$15        |
| Compensation | ([www.oh           | million investment.             |
| (BWC)        | iobwc.com](http:// |                                 |
|              | www.ohiobwc.com/)) | Short-term returns to BWC:      |
|              | provides a window  |                                 |
|              | into the state's   | \- Early intervention reduces   |
|              | workers'           | time to report a claim from 25  |
|              | compensation       | to 7 days; lowers claims costs  |
|              | system through     | by \$113.4 million a year       |
|              | 24/7 account       |                                 |
|              | access and         | \- Demographic information      |
|              | electronic request | updates shift costs to          |
|              | and receipt of     | customers of \$500,000 a year   |
|              | services.          |                                 |
|              |                    | \- Processing electronic forms  |
|              |                    | vs. paper saves \$800,000 to    |
|              |                    | \$4 million a year              |
|              |                    |                                 |
|              |                    | \- Overhead cost reductions     |
|              |                    | associated with public data     |
|              |                    | record requests and status      |
|              |                    | inquiries total \$7.3 million a |
|              |                    | year.                           |
|              |                    |                                 |
|              |                    | In addition, shorter medical    |
|              |                    | bill payment cycle benefits     |
|              |                    | providers                       |
+--------------+--------------------+---------------------------------+
| Wisconsin    | The Insurers'      | \$1.5 million estimated annual  |
| Worker's     | Pending Reports    | savings to State, industry and  |
| Compensation | System is a claims | workers from eliminating        |
| Insurers'    | management tool    | 500,000 pieces of paper, vs.    |
| Web Reports  | for insurers and   | \$275,000 start-up and \$25,000 |
|              | claims             | annual operating costs.         |
|              | administrators to  |                                 |
|              | get real-time      | Eliminates:                     |
|              | access to the      |                                 |
|              | status of workers  | \- almost 100,000 forms each    |
|              | compensation       | year                            |
|              | claims.            |                                 |
|              |                    | \- 250,000-300,000 State        |
|              |                    | mailings a year                 |
|              |                    |                                 |
|              |                    | \- 20,000 annual hours of staff |
|              |                    | time for private insurance      |
|              |                    | claims adjusters and State      |
|              |                    | staff.                          |
+--------------+--------------------+---------------------------------+

[Unemployment Compensation]{.underline}

+--------------+--------------------+---------------------------------+
| State        | Unemployment       | Benefits                        |
|              | Insurance Project  |                                 |
+--------------+--------------------+---------------------------------+
| Kansas Job   | The Tax, Appeals   | \- Processing time for          |
| Link         | and Benefits       | unemployment claims dropped 76% |
|              | Self-Service       | from 47 minutes per claim to 11 |
|              | (TABS) System was  | minutes per claim.              |
|              | initiated by the   |                                 |
|              | Kansas Department  | \- By integrating the           |
|              | of Human Resources | unemployment claims and job     |
|              | to improve,        | search applications in a single |
|              | reengineer,        | Web application, the system     |
|              | redesign and       | helps claimants become employed |
|              | restructure the    | sooner than they would have.    |
|              | entire             | Therefore, they receive, on     |
|              | unemployment       | average, several days less in   |
|              | insurance          | unemployment benefits. This     |
|              | infrastructure. It | saves the State trust fund      |
|              | allows unemployed  | nearly \$9 million a year.      |
|              | workers the        |                                 |
|              | ability to file    |                                 |
|              | Unemployment       |                                 |
|              | insurance claims   |                                 |
|              | and link directly  |                                 |
|              | to job             |                                 |
|              | opportunities at   |                                 |
|              | www                |                                 |
|              | .kansasjoblink.com |                                 |
+--------------+--------------------+---------------------------------+
| Missouri     | The Missouri       | \- Saves over \$28,000 in       |
| Department   | On-line Internet   | toll-free telephone costs       |
| of Labor and | Claims Filing      |                                 |
| Industrial   | System developed   | \- Reduces staff hours by       |
| Relations    | by the Office of   | 6,600.                          |
|              | Information        |                                 |
|              | Systems and        | \- Available 24/7, not just     |
|              | Employment         | during business hours           |
|              | Security, permits  |                                 |
|              | automated          | \- Allows claimants to file in  |
|              | unemployment       | privacy.                        |
|              | claims processing  |                                 |
|              | and links to the   |                                 |
|              | State's on-line    |                                 |
|              | employment search  |                                 |
|              | engine.            |                                 |
+--------------+--------------------+---------------------------------+
| Nevada       | The Unemployment   | The State's return on           |
| Department   | Insurance Internet | investment is defined as "the   |
| of           | Claims System is   | seamless delivery of services   |
| Employment,  | linked to the      | at the government level around  |
| Training and | State's job search | the needs of the citizen. As    |
| Re           | functions. It was  | the usage of the site           |
| habilitation | built on a         | increases, therefore, increased |
|              | fast-track basis   | service is provided to the      |
|              | when it became     | citizen."                       |
|              | clear that the     |                                 |
|              | number of          |                                 |
|              | unemployment       |                                 |
|              | claims would       |                                 |
|              | exceed capacity to |                                 |
|              | handle them.       |                                 |
+--------------+--------------------+---------------------------------+

+--------------+--------------------+---------------------------------+
| State        | Unemployment       | Benefits                        |
|              | Insurance Project  |                                 |
+--------------+--------------------+---------------------------------+
| North        | The North Carolina | Savings of \$1.8 million a year |
| Carolina     | Portal to          | on the investment of \$480,000  |
| Employment   | Automated          | in federal grant funds. This    |
| Security     | Unemployment       | includes:                       |
| Commission   | Insurance Services |                                 |
|              | delivers workforce | \- Close to an estimated        |
|              | services to North  | \$950,000 a year in unneeded    |
|              | Carolina's workers | staff time to answer calls and  |
|              | through real time, | process claims, based on:       |
|              | Internet-based     |                                 |
|              | unemployment       | \- \$71,200/month for in-person |
|              | insurance benefit  | help                            |
|              | claims and access  |                                 |
|              | to the State's job | \- \$8,300/month for phone      |
|              | listings.          | assistance.                     |
|              |                    |                                 |
|              |                    | \- Direct deposit of            |
|              |                    | unemployment checks saves over  |
|              |                    | \$850,000 a year on checks,     |
|              |                    | envelopes and postage.          |
|              |                    |                                 |
|              |                    | "As current programs are        |
|              |                    | improved and new ones are       |
|              |                    | added, ESC expects more and     |
|              |                    | more return on its investment,  |
|              |                    | both in the financial sense and |
|              |                    | in its emotional investment to  |
|              |                    | fulfill its mission statement   |
|              |                    | by providing the best customer  |
|              |                    | service possible..."            |
+--------------+--------------------+---------------------------------+
| Virginia     | The Virginia       | VEC estimates the Claim for     |
| Employment   | Employment         | Benefits system will save       |
| Commission   | Commission Claim   | \$7.27 million over its 3 year  |
| (VEC)        | for Benefits       | lifetime, vs. a \$250,000       |
|              | system was         | investment from a U.S. Labor    |
|              | developed to       | Department grant:               |
|              | reduce traffic by  |                                 |
|              | initial claimants  | \- \$6.45 million in travel     |
|              | into VEC's local   | hours in driving time for       |
|              | field offices and  | claimants who filed online      |
|              | eliminate keying   | rather than appearing in person |
|              | of unemployment    | at a field office.              |
|              | insurance          |                                 |
|              | information by VEC | \- \$822,000 in operational     |
|              | personnel.         | time savings for field office   |
|              |                    | personnel who won't have to     |
|              |                    | process claims filed in person. |
+--------------+--------------------+---------------------------------+

[Government-to-Business Programs]{.underline}

+--------------+-------------------------------+-----------------------+
| State        | Business Project              | Benefits              |
+--------------+-------------------------------+-----------------------+
| Connecticut  | Ethics Commission Online      | -FOIA requests        |
|              | Reporting System              | dropped by 90% to     |
|              |                               | 50/month              |
|              |                               |                       |
|              |                               | \- savings in         |
|              |                               | printing costs        |
|              |                               |                       |
|              |                               | \- reduced staff time |
|              |                               | and paperwork         |
+--------------+-------------------------------+-----------------------+
| District of  | Business Resource Center      | \- \$1.8 million a    |
| Columbia     |                               | year from eliminating |
|              |                               | customer service jobs |
+--------------+-------------------------------+-----------------------+
| Illinois     | e-Filing                      | \- \$149,500 year 1   |
| e-Filing     |                               | savings               |
|              |                               |                       |
|              |                               | \- better customer    |
|              |                               | service               |
|              |                               |                       |
|              |                               | \- more efficient     |
|              |                               | operations            |
+--------------+-------------------------------+-----------------------+
| M            | Webfile for Employers         | \- reduces mailing,   |
| assachusetts |                               | printing and data     |
|              |                               | entry costs           |
|              |                               |                       |
|              |                               | \- reduces reporting  |
|              |                               | burden on employers   |
+--------------+-------------------------------+-----------------------+
| Nebraska     | UIConnect                     | Savings in year one\  |
| UIConnect    |                               | - \$361,624 for       |
|              |                               | employers in tax      |
|              |                               | calculation, report   |
|              |                               | preparation,          |
|              |                               | bookkeeper time       |
|              |                               |                       |
|              |                               | \- \$62,540 for the   |
|              |                               | government in data    |
|              |                               | entry, wage           |
|              |                               | reporting, payment    |
|              |                               | processing            |
+--------------+-------------------------------+-----------------------+
| Ohio         | Business Gateway              | Reduced time, data    |
| Business     |                               | redundancy, errors,   |
| Gateway      |                               | data entry, paperwork |
+--------------+-------------------------------+-----------------------+
| Texas System | System for Electronic Rate    | Savings of \$300,000  |
| for          | and Form Filing               | a year                |
| Electronic   |                               |                       |
| Rate and     |                               | \- reduction of 7 FTE |
| Form Filing  |                               |                       |
| (SERFF)      |                               | \- reduced mail       |
|              |                               | processing, data      |
|              |                               | entry, error          |
|              |                               | correction, time and  |
|              |                               | rework by 70%         |
+--------------+-------------------------------+-----------------------+

[Online Acquisition Systems]{.underline}

+-------------+-----------------------+-------------------------------+
| State       | Online Acquisition    | Benefits                      |
|             | System                |                               |
+-------------+-----------------------+-------------------------------+
| California  | CAL-Buy Online        | Process savings of \$37 per   |
|             |                       | purchase order. Ultimately    |
|             |                       | \$9.7 million annual process  |
|             |                       | savings.                      |
+-------------+-----------------------+-------------------------------+
| Maryland    | EMaryland Marketplace | Savings of \$100 per purchase |
|             |                       | order.                        |
|             |                       |                               |
|             |                       | Reduces "maverick" buying     |
|             |                       |                               |
|             |                       | Increases state purchasing    |
|             |                       | power                         |
+-------------+-----------------------+-------------------------------+
| New Mexico  | Web-based             | Savings from revised          |
|             | Instructional         | processes \$236,440/year.     |
|             | Materials System      | Costs \$167,551)              |
+-------------+-----------------------+-------------------------------+
| North       | Nort                  | Self-funding                  |
| Carolina    | hCarolina@YourService |                               |
+-------------+-----------------------+-------------------------------+
| Virginia    | EVA                   | Start up costs of \$300,000   |
|             |                       | plus time of 10 managers and  |
|             |                       | 4 staff                       |
|             |                       |                               |
|             |                       | Self-financing system: AMS    |
|             |                       | doesn't get paid until system |
|             |                       | is used.                      |
+-------------+-----------------------+-------------------------------+

[Online Licensing]{.underline}

+--------------+--------------------+---------------------------------+
| State        | Online Licensing   | Benefits                        |
+--------------+--------------------+---------------------------------+
| Florida      | Business and       | Annual administrative cost      |
|              | Professional       | savings of \$40,000 per FTE;    |
|              | Regulation Single  | for savings of 9 FTE, \$360,000 |
|              | Licensing Project  | annual savings                  |
+--------------+--------------------+---------------------------------+
| Illinois     | E-Batch License    | -   Reduced paperwork backlog   |
|              | Renewal            |                                 |
|              |                    | -   User-friendly and timely    |
|              |                    |     services                    |
+--------------+--------------------+---------------------------------+
| Kansas       | State Board of     | Reduced phone calls by \>90%    |
|              | Nursing Online     |                                 |
|              | License Renewal    |                                 |
+--------------+--------------------+---------------------------------+
| Maine        | Rapid Renewal      | Time savings for citizens and   |
|              | Service            | state employees                 |
+--------------+--------------------+---------------------------------+
| M            | Educator Licensure | Eliminates                      |
| assachusetts | and Recruitment    |                                 |
|              | Initiative         | -   6 months processing time    |
|              |                    |                                 |
|              |                    | -   \$1.6 million annual costs  |
|              |                    |                                 |
|              |                    | Increases licensing revenue by  |
|              |                    | \$2.5 mill.                     |
|              |                    |                                 |
|              |                    | (Cost = \$2.6 million)          |
+--------------+--------------------+---------------------------------+
| North        | International      | Savings in                      |
| Carolina     | Registration Plan  |                                 |
|              |                    | -   agency time                 |
|              |                    |                                 |
|              |                    | -   administrative burden       |
|              |                    |                                 |
|              |                    | -   motor carrier operating     |
|              |                    |     time/costs                  |
+--------------+--------------------+---------------------------------+
| Rhode Island | Online Boat        | \- \$40,000 (cf. no cost)       |
|              | Registration       |                                 |
|              | Renewal Service    | \- Reduces employee time for    |
|              |                    | data entry, answering calls,    |
|              |                    | processing checks, marketing    |
+--------------+--------------------+---------------------------------+
| Virginia     | WebCat: A Virginia | -Transactions at ½ cost of      |
|              | DMV Extranet       | manual                          |
|              | Application for    |                                 |
|              | Motor Carriers     | -   reduced paperwork and       |
|              |                    |     in-person visits            |
|              |                    |                                 |
|              |                    | -   reduced time requirements   |
+--------------+--------------------+---------------------------------+

[Law Enforcement Information Systems]{.underline}

+--------------+--------------------+---------------------------------+
| State        | Law Enforcement    | Benefits                        |
|              | Project            |                                 |
+--------------+--------------------+---------------------------------+
| Colorado     | Integrated         | \- Reduced mailing costs, paper |
|              | Criminal Justice   | processing and interagency      |
|              | Info System        | contacts                        |
|              |                    |                                 |
|              |                    | \- Improved accuracy of info    |
|              |                    |                                 |
|              |                    | \- Improved timeliness          |
+--------------+--------------------+---------------------------------+
| Florida      | Juvenile Justice   | \- Eliminates the \$2.1 million |
|              | Information        | annual cost of connecting 443   |
|              |                    | police and sheriff's offices    |
|              |                    | and \$211,000 cost of coding    |
+--------------+--------------------+---------------------------------+
| Illinois     | Wireless           | -Enhancement of interagency     |
|              | Information        | communication                   |
|              | Network            |                                 |
|              | (I-WIN/ALERTS)     |                                 |
+--------------+--------------------+---------------------------------+
| Montana      | Criminal Offenders | \- Crime victims' peace of mind |
|              | Network---Online   |                                 |
|              | Services           | \- \$55,000 maintenance paid by |
|              |                    | other agencies                  |
+--------------+--------------------+---------------------------------+
| Nevada       | Multi-county       | \- Seamless information and     |
|              | Integrated Justice | services at all levels of       |
|              | Information System | government                      |
|              |                    |                                 |
|              |                    | \- Reduced paper flow           |
|              |                    |                                 |
|              |                    | \- Fewer errors                 |
+--------------+--------------------+---------------------------------+
| New Jersey   | Pre-Inmate         | \- Reduced backup of State      |
|              | Management System  | inmates in county jails which   |
|              |                    | costs up to \$4.7 million       |
|              |                    |                                 |
|              |                    | \- Streamlined processes and    |
|              |                    | reduced redundancies            |
+--------------+--------------------+---------------------------------+
| Pennsylvania | Justice Network    | -   reduced costs               |
|              |                    |                                 |
|              |                    | -   reduced delays              |
|              |                    |                                 |
|              |                    | -   public safety               |
|              |                    |                                 |
|              |                    | -   easy to replicate           |
|              |                    |                                 |
|              |                    | Total cost \$18 million.        |
+--------------+--------------------+---------------------------------+

[Grants Management Programs]{.underline}

+--------------+--------------------+---------------------------------+
| State        | Grants Management  | Benefits                        |
|              | Program            |                                 |
+--------------+--------------------+---------------------------------+
| Indiana      | State Student      | -Reduced staff costs by 35%     |
|              | Assistance         |                                 |
|              | Commission         | \- Sold backend system to       |
|              |                    | another state for \$390,000     |
|              |                    |                                 |
|              |                    | \- Streamlined processes        |
|              |                    |                                 |
|              |                    | \- Improved communications      |
+--------------+--------------------+---------------------------------+
| Missouri     | eGrants            | \- Reduced application          |
|              |                    | processing time by 86%          |
|              |                    |                                 |
|              |                    | \- Reduced staff by 35%         |
|              |                    |                                 |
|              |                    | \- Eliminated 17 forms          |
|              |                    |                                 |
|              |                    | \- Consolidated Federal grant   |
|              |                    | programs                        |
+--------------+--------------------+---------------------------------+
| Missouri     | iGrants: Internet  | \- Reduced staff required for   |
|              | Grant Management   | grant processing by 31          |
|              | Application        |                                 |
|              |                    | \- Improves service             |
|              |                    | availability                    |
|              |                    |                                 |
|              |                    | from 18.5% of resources to      |
|              |                    | 44.4%                           |
|              |                    |                                 |
|              |                    | \- Shortened elapsed days for   |
|              |                    | application                     |
|              |                    |                                 |
|              |                    | from 65 days to 5 days          |
+--------------+--------------------+---------------------------------+
| Texas        | ARP/ATP Online     | \- Increased grant proposals    |
|              | System             | processed from 2,900 to 9,000   |
|              |                    | with no extra staff.            |
|              |                    |                                 |
|              |                    | \- Eliminated 6 temporary staff |
|              |                    |                                 |
|              |                    | \- Saved \$21,338 per grant     |
|              |                    | cycle                           |
+--------------+--------------------+---------------------------------+
| West         | Service at the     | \- Reduced staff needed to      |
| Virginia     | Speed of           | disburse funds by 1.5 FTE and 1 |
|              | Technology         | contractor                      |
|              |                    |                                 |
|              |                    | \- Saved \$40,000               |
+--------------+--------------------+---------------------------------+

**Appendix IV**

**OMB A-11 Section 300 Scoring Criteria**

*Note: OMB has renamed its required "Exhibit 300" as "Capital Asset Plan
and Business Case.\"*

300.10 How will OMB evaluate the business cases in the exhibits 300?

All business cases are scored against a core set of criteria and the
results are provided to the agency via the budget pass-back process.
While one size scoring does not fit all categories, this scoring is
meant to ensure that agency planning and management of assets is
consistent with OMB policy and guidance. For projects other than IT, the
IT specific categories are awarded full points as they are not
applicable. The scoring of a business case is two-fold. The business
case is scored based upon the criteria listed below and then a
programmatic review is done for the project. A business case may be the
strongest possible based upon the criteria listed here and if the
program is deemed ineffective and changes are being made, then there is
no need for the investment represented by the business case scoring.
Business case scoring is as follows:

# Business Case (BC) (composite of all categories) Total Score for Business Case 

Projects scoring 5 and meeting program requirements are automatically
recommended for funding. Projects scoring a 4 and meeting program
requirements, and meeting most of the business case requirements are
recommended for funding and the agency is instructed to continue
improvements in the areas identified as needing work. Projects scoring 3
or below have the opportunity to improve to a 4 or degrade to a 2 rather
easily. Projects scoring a 2 or below are not recommended for funding.

[Score: Category/Total]{.underline} [Definition]{.underline}

5 41--50 Strong documented business case (including all sections as
appropriate).

> 4 31--40 Very few weak points within the BC but still needs
> strengthening.

3 21--30 Much work remains to solidify and quantify BC. BC has the
opportunity to either improve or degrade very quickly.

2 11--20 Significant gaps in the required categories of the BC.

1 1--10 Inadequate in every category of the required BC.

**Supports the President\'s Management Agenda Items (AI) (Multiple
Sections)**

5 This is a collaborative project that includes multiple agencies,
state, local, or tribal governments, uses e-business technologies and
the project is governed by citizen needs. Project also supports the
Federal Business Architecture published by OMB. If project is a steady
state project, then an E-Gov strategy review is underway and includes
all of the necessary elements. If appropriate, this project is fully
aligned with one or more of the President\'s E-Gov initiatives.

4 This is a collaborative project that includes multiple agencies,
state, local, or tribal governments, uses e-business technologies though
work remains to solidify these relationships. Project also supports the
Federal Business Architecture published by OMB though work remains to
solidify the linkage. If project is a steady state project, then an
E-Gov strategy review is underway but needs work in order to strengthen
the analysis. If appropriate, project supports one or more of the
President\'s E-Gov initiatives but is not yet fully aligned.

3 This is not a collaborative project though it could be and much work
remains to strengthen the ties to the President\'s Management Agenda. If
a steady state project and no E-Gov strategy is evident, this project
will have a difficult time securing continued or new funding from OMB.
If appropriate, this project supports one or more of the President\'s
E-Gov initiatives but alignment is not demonstrated.

2.  This is not a collaborative project and it is difficult to ascertain
    support for the AI. If steady state project, no E-Gov strategy was
    performed or is planned.

1 There does not seem to be any link to the AI and no E-Gov strategy.

**Acquisition Strategy (AS) (Part I, Section I.G)**

5 Strong Acquisition Strategy that mitigates risk to the Federal
Government, accommodates Section 508 as needed, and contracts and
statements of work (SOWs) are performance based. Implementation of the
Acquisition Strategy is clearly defined.

4.  Contracts and SOWs are performance based with very few weak points
    that agency is strengthening and implementation of the AS is clearly
    defined.

```{=html}
<!-- -->
```
3.  Much work remains to solidify and quantify the AS.

```{=html}
<!-- -->
```
2.  Some parts of the AS are present but no clear implementation
    strategy.

```{=html}
<!-- -->
```
1.  There is no evidence of an AS.

**Program Management (PM) (Part I, Sections I.D and I.H)**

5 Program is very strong and has resources in place to manage it.

4 Program has some weak points in the area of PM and agency is working
to strengthen PM.

3.  Much work remains in order for PM to manage the risks for this
    project.

```{=html}
<!-- -->
```
2.  There is some understanding of PM for this project but it is very
    rudimentary.

1 There is no evidence of PM.

**Enterprise Architecture (EA) (Part II, Section II.A) for IT Only.**

5 This project is included in the Agency EA and CPIC process. BC
demonstrates business, data, and application, and technology layers of
the EA in relationship to this project.

4.  This project is included in the Agency EA and CPIC process. BC
    demonstrates weaknesses in the business, data, application, and
    technology layers of the EA in relationship to this project.

```{=html}
<!-- -->
```
3.  This project is not included in the Agency EA and CPIC process. BC
    demonstrates a lack of understanding on the layers of the EA
    (business, data, application, and technology).

```{=html}
<!-- -->
```
2.  While the agency has an EA Framework, it is not implemented in the
    agency and does not include this project.

1 There is no evidence of a comprehensive EA in the agency.

**Alternatives Analysis (AA) (Part I, Section I.E)**

5.  AA includes three viable alternatives, alternatives were compared
    consistently, and alternative chosen provides benefits and reasons.

```{=html}
<!-- -->
```
4.  AA includes three viable alternatives, however work needs to
    continue in terms of the alternative chosen and the accompanying
    analysis.

```{=html}
<!-- -->
```
3.  AA includes fewer than three alternatives and overall analysis needs
    strengthening.

```{=html}
<!-- -->
```
2.  AA includes weak AA information overall, significant weaknesses
    exist.

1 There is no evidence that an AA was performed.

**Risk Management (RM) (Part I, Section I.F)**

5.  Risk Assessment was performed for all mandatory elements and risk is
    managed throughout the project.

```{=html}
<!-- -->
```
4.  Risk assessment addresses some of the Risk, but not all that should
    be addressed for this project.

```{=html}
<!-- -->
```
3.  Risk Management is very weak and does not seem to address or manage
    most of the risk associated with the project.

```{=html}
<!-- -->
```
2.  Risk Assessment was performed at the outset of the project but does
    not seem to be part of the program management.

1 There is no evidence of a Risk Assessment Plan or Strategy.

**Performance Goals (PG) (Part I, Section I.C)**

5 Performance Goals are provided for the agency, are linked to the
annual performance plan, the project discusses the agency mission and
strategic goals, and performance measures are provided.

4 Performance Goals are provided for the agency, are linked to the
annual performance plan, the project discusses the agency mission and
strategic goals, and performance measures are provided yet work remains
to strengthen the PG.

3.  Performance Goals exist but linkage to the agency mission and
    strategic goals is weak.

```{=html}
<!-- -->
```
2.  Performance Goals are in their initial stages and are not
    appropriate for the type of project. Much work remains to strengthen
    the PG.

1 There is no evidence of PG for this project.

**Security and Privacy (SE) (Part II, Section II.B)**

5 Security and privacy issues for the project and all questions are
answered, detail is provided about the individual project throughout the
life-cycle to include budgeting for SE.

4 Security and privacy information for the project is provided but there
are weaknesses in the information that need to be corrected.

3 Security and privacy information for the project is provided but fails
to answer the minimum requirements.

2.  Security and privacy information points to an overall Agency
    Security Process with little to detail at this project level.

1 There is no security or privacy information provided for the project.

**Performance Based Management System (PB) (Part I, Section I.H)**

5 Agency will use, or uses an Earned Value Management System (EVMS) that
meets ANSI/EIA Standard 748 and project is earning the value as planned
for costs, schedule, and performance goals.

4 Agency uses the required EVMS, is within the variance levels for two
of the three criteria, and needs work on the third issue.

3.  Agency uses required EVMS but the process within the agency is very
    new and not fully implemented or there are weaknesses for this
    individual project\'s EVMS information.

```{=html}
<!-- -->
```
2.  Agency seems to re-baseline rather than report variances.

1 There is no evidence of PB.

**Life-Cycle Costs Formulation (LC) (Multiple Sections)**

5 Life-cycle costs seems to reflect formulation that includes all of the
required resources and is risk adjusted to accommodate items addressed
in the RM. It appears that the project is planned well enough to come in
on budget.

4.  Life-cycle costs seem to reflect formulation of some of the
    resources and some of the issues as included in the risk adjustment
    strategy but work remains in order to ensure that LC costs are
    accurately portrayed.

```{=html}
<!-- -->
```
3.  Life-cycle costs seem to reflect formulation of the resources but
    are not risk adjusted based upon the risk management plan.

2 Life-cycle costs seem to include some of the resource criteria and are
not risk adjusted.

1 Life-cycle costs do not seem to reflect a planned formulation process.

  ------------------------------------------------------- ---------------
  Scoring Element                                         Score

  Business Case (BC) Total                                

  Supports the President's Management Agenda Items (IA)   

  Acquisition Strategy (AS)                               

  Program Management (PM)                                 

  Enterprise Architecture (EA)                            

  Alternatives Analysis (AA)                              

  Risk Management (RM)                                    

  Performance Goals (PG)                                  

  Security (SE)                                           

  Performance Based Management System (PB)                

  Life Cycle Costs Formulation (LC)                       

  TOTAL SCORE                                             
  ------------------------------------------------------- ---------------