GAO

United States General Accounting Office

Report to the Ranking Democratic

Member, Committee on Veterans’ Affairs,

House of Representatives

April 2000 DISABLED

VETERANS’ CARE

Better Data and More

Accountability Needed

to Adequately Assess

Care

GAO/HEHS-00-57

Page 1 GAO/HEHS-00-57 Disabled Veterans’ Care

Contents

Letter 3

Appendixes Appendix I: VA Definitions of Six Special Disabilities 22

Appendix II: Performance and Access Monitors 23

Appendix III: Comments From the Department of Veterans Affairs 34

Tables Table 1: Change in Number of Special Disability Veterans Served and

Dollars Spent Between Fiscal Years 1996 and 1998 8

Table 2: Change in Number of Special Disability Veterans Served in

Inpatient and Outpatient Settings Between Fiscal Years 1996

and 1998 9

Table 3: Change in FTE Employees and Beds for Spinal Cord

Dysfunction and Blind Rehabilitation Between Fiscal Years

1996 and 1998 10

Abbreviations

BROS blind rehabilitation outpatient specialist

CDR cost distribution report

DSM-IV Diagnostic and Statistical Manual of Mental Disorders, Fourth

Edition

FTE full-time-equivalent

PTSD post-traumatic stress disorder

SCI spinal cord injury

SCI&D spinal cord injury and disorder

SMI serious mental illness

TBI traumatic brain injury

VA Department of Veterans Affairs

VHA Veterans Health Administration

VISN Veterans Integrated Service Network
Page 2 GAO/HEHS-00-57 Disabled Veterans’ Care
Page 3 GAO/HEHS-00-57 Disabled Veterans’ Care

United States General Accounting Office

Washington, D.C. 20548

Health, Education, and

Human Services Division

B-283462 Letr

April 21, 2000

The Honorable Lane Evans

Ranking Democratic Member

Committee on Veterans’ Affairs

House of Representatives

Dear Mr. Evans:

In 1996, the Congress expressed concern that budgetary pressures and

ongoing reorganization within the Department of Veterans Affairs (VA)

health care system could make VA’s specialized programs for disabled

veterans vulnerable to inappropriate cost cutting. Section 104 of the

Veterans Health Care Eligibility Reform Act of 1996 (P.L. 104-262) requires

the Secretary of VA to (1) ensure that VA’s systemwide capacity to provide

specialized treatment and rehabilitative services to veterans with spinal

cord dysfunction, blindness, amputations, or mental illness is not reduced

below October 1996 levels and (2) provide veterans with reasonable access

to needed specialized care and services. VA is required to report to the

House and Senate Committees on Veterans’ Affairs annually from 1997

through 2001 about its systemwide capacity to provide this specialized

care.

Although the legislation directed VA to preserve capacity and to ensure

reasonable access for veterans with special disabilities, it did not define

capacity or access or specify how each was to be measured. After

consultation with stakeholders,1 VA defined capacity as the number of

individual veterans treated within specialized inpatient units and clinics

and the dollars expended for their care. VA included number of beds and

staffing levels as additional measures of capacity for spinal cord

dysfunction and blind rehabilitation. Access was defined as timeliness in

providing services to veterans for their specialized needs. In addition, VA

planned to implement outcome measures within 2 to 3 years to evaluate

program effectiveness, regardless of resources expended, by measuring

treatment results.

1

Stakeholders included members of the Advisory Committee on Prosthetics and Special

Disabilities Programs and the Committee on the Care of Severely Chronically Mentally Ill

Veterans.
B-283462

Page 4 GAO/HEHS-00-57 Disabled Veterans’ Care

This report responds to your request that we review VA’s compliance with

the requirements to maintain capacity and access for veterans with special

disabilities.2 Specifically, we provide the results of our review of (1) the

accuracy of the conclusions in VA’s fiscal year 1998 annual capacity report

and (2) challenges facing VA in managing its special disability programs. In

addition, we assessed whether VA has complied with section 903 of the

Veterans Programs Enhancement Act of 1998 (P.L. 105-368), which directed

the Under Secretary for Health to prescribe, by January 1, 1999, job

performance standards for employees responsible for allocating and

managing special disability program resources.

To develop this information, we met with VA officials responsible for

developing and analyzing information on the disability programs; VA

officials responsible for managing the special disability programs at the

national level; officials at Veterans Integrated Service Networks (VISN) in

Durham, N.C., Atlanta, Ga., and San Francisco, Calif.; officials at facilities

in Durham, N.C., Richmond, Va., Decatur and Augusta, Ga., and Palo Alto

and San Francisco, Calif.; and representatives from veterans’ service

organizations and advisory committees with which VA is required to

consult in responding to the 1996 legislation. We also reviewed relevant VA

and advisory committee reports, policies, manuals, and publications. We

performed our work between October 1998 and January 2000 in

accordance with generally accepted government auditing standards.

Results in Brief VA concluded in its annual report for fiscal year 1998 that it had maintained

its capacity to treat veterans with special disabilities. However, VA’s data

are not sufficient to support that conclusion because of extensive data

problems, such as the use of unreliable proxy measures to identify veterans

with special disabilities. Moreover, VA based its conclusion on national

statistics that indicated more special disability veterans were served with

fewer resources expended in 1998 than in 1996. However, there is

considerable variability among the VISNs, and, in fact, some VISNs

reported serving fewer veterans. In addition, VA attributes reduced

expenditures and the use of fewer resources to efficiency gains; however,

because it lacks outcome measures, VA cannot tell whether it has

maintained, enhanced, or diminished quality of care.

2

We provided preliminary information on the results of our review in a briefing to your staff

on May 24, 1999.
B-283462

Page 5 GAO/HEHS-00-57 Disabled Veterans’ Care

VA faces challenges in maintaining its capacity to serve special disability

populations. In particular, the lack of a single VA headquarters unit

accountable for ensuring compliance with the capacity legislation may

have caused delays in (1) monitoring and investigating locations where

capacity appears to have declined and (2) fully implementing

congressionally mandated performance standards for VA employees

responsible for allocating and managing special disability program

resources. In order to ensure compliance with the capacity legislation, we

are recommending that VA designate a single office to be accountable for

fully implementing the mandate to maintain capacity in special disability

programs.

Background Since 1995, VA has taken significant steps to transform its health care

system from a hospital- and specialist-based system to a preventionoriented, community-based system with primary care as its foundation. To

accomplish this transition, VA moved from a management structure based

on 172 hospitals to one based on VISNs in 22 separate geographic areas.

These VISNs have substantial operational autonomy and are responsible

for making basic budgetary, planning, and operational decisions to meet

the health care needs of veterans living within the 22 geographic areas.

Each VISN oversees between 5 and 11 large hospital facilities, as well as

clinics and other delivery locations.

The Veterans Health Care Eligibility Reform Act of 1996 authorized new

eligibility rules for outpatient treatment that permit VA to provide medical

care in the most appropriate settings. Eligibility reform was intended to

reduce inappropriate admissions and denial of care to many veterans who

were ineligible under the old rules for outpatient treatment. In addition, VA

proposed a plan to operate within the same annual appropriation for VA

health care through 2002.3 As a result, VISN and facility managers had

strong incentives to reengineer delivery systems to offset rising health care

costs.

In this environment of shifting service delivery from inpatient to outpatient

settings, system reorganization, and no-growth budgets, section 104 of the

Veterans Health Care Eligibility Reform Act directed VA to protect services

and resources committed to veterans with spinal cord dysfunction,

3

Nevertheless, VA’s fiscal year 2000 budget appropriation for health care was increased by

$1.7 billion over the fiscal year 1999 budget appropriation.
B-283462

Page 6 GAO/HEHS-00-57 Disabled Veterans’ Care

blindness, amputations, or mental illness. After discussions with

stakeholders, as required by the act, VA added two more special

disabilities—traumatic brain injury and post-traumatic stress disorder

(PTSD)—to this list. For the purpose of the capacity requirement, VA

limited the definition of mental illness to refer to only those veterans with

serious mental illness and created two subcategories: veterans disabled as

a result of a disorder related to substance abuse and homeless veterans

disabled as a result of serious mental illness.

House Report 104-690, which accompanied the 1996 legislation, noted that

the special disability programs constitute a vital core of VA’s mission, tend

to be high-cost efforts, and are unmatched in scope and quality in the

private sector. The six special disabilities were targeted because of their

close association with service-related illnesses and the likelihood of

progressively worsening disability in the absence of specialized treatment

or rehabilitation. VA must carry out the requirements of the legislation in

consultation with the Advisory Committee on Prosthetics and Special

Disabilities Programs and the Committee on the Care of Severely

Chronically Mentally Ill Veterans,4 and VA has done so.

Although the legislation directed VA to preserve capacity and to ensure

reasonable access for veterans with special disabilities, it did not specify

how capacity and access were to be defined or measured. VA defined

capacity as the number of individual veterans treated within specialized

inpatient units and clinics and the dollars expended for the care of these

veterans. VA intends to use outcome measures, when they become

available, to measure the effectiveness of its specialized programs. At the

insistence of veterans’ service organizations, number of beds and staffing

levels were included as additional measures of capacity for spinal cord

dysfunction and blind rehabilitation. VA defined access as timeliness in

providing services to veterans for their specialized needs. Although VA

considered other measures of access, the data necessary to develop these

measures, primarily the number and location of the universe of veterans

with each disability, are not generally available.

4

The members of the Advisory Committee on Prosthetics and Special Disabilities Programs

are from veterans’ service organizations, universities, and private sector health care

providers. In accordance with the Veterans Health Care Eligibility Reform Act, members of

the Committee on the Care of Severely Chronically Mentally Ill Veterans must be employees

of the Veterans Health Administration with expertise in the care of the chronically mentally

ill and must be appointed by VA’s Under Secretary for Health.
B-283462

Page 7 GAO/HEHS-00-57 Disabled Veterans’ Care

VA Has Insufficient

Evidence to Support Its

Conclusion That

Capacity Has Been

Maintained

VA’s fiscal year 1998 annual capacity report concluded that VA’s capacity to

treat special disability groups nationwide had been generally maintained.

This conclusion was based on national statistics that indicated that more

special disability veterans were served in 1998 than in 1996. However,

measurement results varied among VISNs, with some showing a decrease

in veterans served. In our view, the available data are insufficient to

support any conclusions because VA’s workload and resource data are

inaccurate. In addition, reliable outcome measures are not available to

assess whether the quality of care provided to special disability populations

has changed or is satisfactory.

VA Has Claimed That

Serving More at Less Cost

Maintains Capacity

VA has asserted that capacity to treat veterans with special disabilities has

been maintained because the number of veterans treated in special

disability programs increased from fiscal year 1996 to fiscal year 1998 by 8

percent, or 28,141 individuals. However, during that same period, spending

for these programs decreased by 8 percent, or approximately $184 million.

Veterans with a serious mental illness accounted for 81 percent of the

special disability veterans served and 84 percent of the dollars spent in the

special disability categories. The number of veterans with special

disabilities served during the period increased or remained relatively

constant for all conditions except amputations. VA reported the decline in

the number of veterans with amputations as a favorable outcome of

successful efforts to prevent amputations in diabetic patients. VA

expenditures decreased for amputations, serious mental illness, and PTSD.

Analysis of expenditures for two subcategories of serious mental illness

shows a decrease of 29 percent for veterans with a disorder related to

substance abuse and an increase of 23 percent for veterans who were

homeless. Expenditures also increased for spinal cord dysfunction,

traumatic brain injury, and blind rehabilitation. (See table 1.)
B-283462

Page 8 GAO/HEHS-00-57 Disabled Veterans’ Care

Table 1: Change in Number of Special Disability Veterans Served and Dollars Spent Between Fiscal Years 1996 and 1998

a

The total for serious mental illness is more than the sum of the subcategories listed under it because

the category includes, but is not limited to, veterans who are substance-abusing or homeless.

Source: VA’s 1998 capacity report to the Congress: Maintaining Capacity to Provide for the Specialized

Treatment and Rehabilitative Needs of Disabled Veterans (VA, June 1999).

VA’s fiscal year 1998 capacity report was an improvement over its previous

reports because it included for the first time a breakout of veterans served

in both inpatient and outpatient settings. Between 1996 and 1998, the

number of veterans served in inpatient settings increased for blindness,

traumatic brain injury, the homeless subcategory of serious mental illness,

and PTSD. The shift from inpatient to outpatient care was most evident in

the broad category of serious mental illness and its subcategory of

substance abuse. The number of seriously mentally ill veterans treated in

an inpatient setting declined by approximately 19 percent, while the

number of seriously mentally ill veterans with substance abuse disorders

who received inpatient care declined by approximately 41 percent. The

number of special disability veterans treated in outpatient settings

increased for all conditions with the exception of amputations. (See table

2.)

Disability

Veterans served Dollars spent (in thousands)

FY 1996 FY 1998

Change

(percentage) FY 1996 FY 1998

Change

(percentage)

Spinal cord

dysfunction 8,598 9,252 +654 (+8) $199,848 $202,878 +$3,030 (+2)

Blindness 9,726 11,930 +2,204 (+23) 43,855 53,935 +10,080 (+23)

Traumatic brain

injury 176 189 +13 (+7) 4,439 4,906 +467 (+11)

Amputations 4,765 4,549 -216 (-5) 5,953 5,286 -667 (-11)

Serious mental

illnessa 269,009 290,961 +21,952 (+8) 2,080,240 1,900,938 -179,302 (-9)

Substance

abuse 107,074 106,599 -475 (-0.4) 575,902 407,334 -168,568 (-29)

Homeless 24,539 27,201 +2,662 (+11) 75,071 92,614 +17,543 (+23)

PTSD 39,653 43,187 +3,534 (+9) 101,882 84,112 -17,770 (-17)

Total 331,927 360,068 +28,141(+8) $2,436,217 $2,252,055 -$184,162 (-8)
B-283462

Page 9 GAO/HEHS-00-57 Disabled Veterans’ Care

Table 2: Change in Number of Special Disability Veterans Served in Inpatient and

Outpatient Settings Between Fiscal Years 1996 and 1998

a

Data were not reported for both inpatient and outpatient settings.

Source: VA 1998 capacity report (VA, June 1999).

For veterans disabled by spinal cord dysfunction or blindness, capacity

was also measured by staff resources—full-time-equivalent (FTE)

employees—and the number of specialized beds dedicated to veterans with

these disabilities. From 1996 to 1998, VA reported that staffing levels

dropped by 12 percent (267 FTE employees) for spinal cord dysfunction

and increased by 1 percent (3 FTE employees) for blind rehabilitation.

Numbers of beds declined in both areas: 15 percent (180 beds) for spinal

cord dysfunction and 7 percent (15 beds) for blind rehabilitation. (See table

Disability/Setting

Veterans served Percentage

FY 1996 FY 1998 change

Spinal cord dysfunction

Inpatient 5,185 5,117 -1

Outpatient 6,599 7,576 +15

Blindness

Inpatient 1,607 1,976 +23

Outpatient 9,345 11,560 +24

Traumatic brain injurya

Inpatient 176 189 +7

Amputationsa

Outpatient 4,765 4,549 -5

Serious mental illness

Inpatient 117,088 95,068 -19

Outpatient 251,216 278,674 +11

Substance abuse

Inpatient 50,628 30,021 -41

Outpatient 90,916 99,337 +9

Homeless

Inpatient 5,273 7,072 +34

Outpatient 21,913 23,763 +8

PTSD

Inpatient 4,312 4,694 +9

Outpatient 37,768 41,224 +9
B-283462

Page 10 GAO/HEHS-00-57 Disabled Veterans’ Care

3.) Veterans’ service organizations have questioned the accuracy of these

numbers on the basis of surveys they have conducted at VA facilities and

have concluded that capacity reductions have been even greater: 18

percent fewer spinal cord care beds and 32 percent fewer spinal cord care

staff resources. In addition, the Advisory Committee on Prosthetics and

Special Disabilities Programs has questioned whether VA inappropriately

included unstaffed spinal cord and blind rehabilitation beds in the capacity

report. Two of the facilities we visited reported delays in transferring

veterans with acute care needs to a specialized spinal cord injury unit. In

addition, the national average waiting time for admission to an inpatient

blind rehabilitation program increased slightly, from 31.8 weeks in fiscal

year 1996 to 33.4 weeks in fiscal year 1998. Moreover, outreach efforts by

the facility-based Visual Impairment Services Team, which provides

coordinated services to legally blind veterans, continued to increase the

already lengthy waiting lists for blind rehabilitation programs. The delays

in admission to spinal cord care beds in some areas and the lengthy waiting

times for blind rehabilitation indicate that the reduction in bed levels may

be affecting access to these services.

Table 3: Change in FTE Employees and Beds for Spinal Cord Dysfunction and Blind

Rehabilitation Between Fiscal Years 1996 and 1998

Source: VA 1998 capacity report (VA, June 1999).

Some VISNs Reported

Fewer Special Disability

Veterans Served

While VA stated that it had maintained capacity nationally, some VISNs

appeared to be maintaining workloads and expenditures for special

disability populations, while others showed declines in veterans served and

expenditures. For example, only two VISNs maintained or increased their

workloads for all six disabilities. Another two VISNs served fewer veterans

in at least four of the six special disability groups. VA’s Committee on the

Disability/Measure FY 1996 FY 1998 Percentage change

Spinal cord

dysfunction

FTE employees 2,175.8 1,909.3 -12

Beds 1,209 1,029 -15

Blindness

FTE employees 414.5 417.3 +1

Beds 228 213 -7
B-283462

Page 11 GAO/HEHS-00-57 Disabled Veterans’ Care

Care of Severely Chronically Mentally Ill Veterans stated in its response to

the capacity report that because the measure of maintenance of clinical

effort (dollars expended) had actually decreased, VA should scrutinize

those VISNs with the largest reductions in capacity. The committee further

stated that many VISNs showed substantial increases in the numbers of

veterans treated (up to 17 percent) with relatively constant expenditures,

while other networks showed decreases in veterans treated and decreases

in funds expended. VA provided no data to show that decreased demand

for services accounted for the decrease in veterans treated.

VA’s Data Are Inadequate for

Capacity Analysis

The information management systems currently used by VA are not precise

enough to capture the information necessary to accurately calculate

workload and expenditure statistics for the special disability populations.

As a result, VA developed a complex process using eight different

databases to compile VISN and national workload and resource data for the

six disabling conditions. This process used inpatient diagnostic

information, when available, and a set of proxy measures to infer a likely

condition when diagnostic information was not available. For example,

because VA’s outpatient care database does not currently include

diagnostic information,5 VA identified additional patients as belonging to a

special disability group on the basis of information regarding the number of

visits to specific clinics. Thus, veterans who visited certain

psychiatric/mental health clinics at least six times were counted as

disabled by serious mental illness. Because of the lack of diagnostic

information and the sometimes inappropriate proxies used, we are not

confident that the workload figures and subsequent expenditure amounts

are accurate. Stakeholders have voiced similar concerns. For example, the

cochair of the Committee on the Care of Severely Chronically Mentally Ill

Veterans stated during testimony in 1998 that “the currently available data

is inadequate to comprehensively and reliably monitor the Veterans Health

Administration’s efforts to maintain capacity for these disabling

conditions.”

5

VA has begun implementation of a Decision Support System that will capture diagnostic

information for outpatient clinic visits and will provide cost accounting information.
B-283462

Page 12 GAO/HEHS-00-57 Disabled Veterans’ Care

Our visits to selected field locations helped confirm the validity of our

concerns about the accuracy of VA’s workload and resource data. We were

unable to validate workload and resource data contained in the capacity

report using information maintained at VA facilities because data were not

routinely available under the definitions developed for the disabling

conditions (see app. I).6 Several clinicians told us that the definitions used

in the capacity report had no clinical basis when it came to treating

patients. For example, as defined by VA’s capacity report, seriously

mentally ill veterans represented about 81 percent of the universe of

veterans with special disabilities in fiscal year 1998. Yet the seriously

mentally ill category would not be tracked at the facility level because it

does not represent a meaningful grouping of patients who would receive

similar medical care.

Our site visits also found that despite 3 years of requirements to report on

capacity, management staff at VA facilities generally did not know the

definitions used by VA headquarters to identify veterans with special

disabilities or the methodology used to develop workload data for their

facilities. For example, one facility offered substance abuse treatment in a

day treatment program instead of a traditional substance abuse clinic. The

capacity report indicated that this facility experienced a 17-percent decline

in the treatment of seriously mentally ill patients with substance abuse

disorders and a 9-percent decline in expenditures for this population.

Facility officials believed that the capacity report understated workload for

seriously mentally ill patients with substance-related disorders because

VA’s methodology did not include the day treatment clinic as a program

serving this special disability population.

6

Other groups, such as the minority staff of the Senate Committee on Veterans’ Affairs, were

also unable to verify workload and resource data because of problems in obtaining reliable

or comparable data across facilities. See Minority Staff Review of VA Programs for Veterans

With Special Needs, prepared for Senator John D. Rockefeller IV, July 27, 1999.
B-283462

Page 13 GAO/HEHS-00-57 Disabled Veterans’ Care

We also identified deficiencies in the accuracy of VA’s resource measures

(that is, expenditures for all programs and staffing levels and beds for

spinal cord care and blind rehabilitation). Moreover, veterans’ service

organizations have reported discrepancies between the cost distribution

report (CDR), which is the data system used by VA to allocate costs, and

information reported by special disability program officials. The

information in the CDR is suspect because it relies on subjective judgments

to allocate the distribution of staff time and dollars spent in each inpatient

unit and outpatient care area. VA’s Inspector General found that servicelevel managers have broad discretion in selecting and applying cost

allocation techniques, leading to inconsistency, infrequent updates, and

disparate treatment of similar cost accounting issues.7 Clinical staff told us

that vacancies were at times deliberately hidden through the reallocation of

staff to special disability programs in the CDR or filled with individuals

possessing less skill and ability. Thus, these staff believe that the CDR can

be easily manipulated to create the appearance that staffing and

expenditure levels in the special disability program have been maintained.

VA acknowledged that its data systems need improvement, and in

December 1998 the Veterans Health Administration (VHA) held a Data

Quality Summit to identify issues related to the collection and use of data in

VHA. Multiple data quality issues were identified, including the

completeness, reliability, validity, and timeliness of ambulatory care data.

Recognizing problems with the data used to prepare the annual capacity

report, VA used a verification and data correction process to improve the

accuracy and reliability of data for the fiscal year 1998 report to the

Congress. VA headquarters shared preliminary data with medical centers,

VISNs, and program offices to identify problems. In addition, the most

recent capacity report reflects a closer working relationship among VA, its

advisory committees, and interested veterans’ service organizations.

According to VA, this collaboration has resulted in data improvements.

While the steps taken by VA to improve its data quality are commendable,

we believe that these efforts will not bear fruit in the short term because of

the myriad people and processes at the facility, VISN, and national levels

that make data collection at VA so cumbersome.

7

VA Office of Inspector General, Evaluation of Medical Center Investment in Ambulatory

Care Infrastructure, Report Number 9AY-A19-078 (Washington, D.C.: VA, Mar. 31, 1999).
B-283462

Page 14 GAO/HEHS-00-57 Disabled Veterans’ Care

Limited Outcome Measures

Further Impede Capacity

Measurement

The primary basis for VA’s conclusion that it has maintained capacity is the

increased number of veterans served by the special disability programs.

However, of the five special disability programs that reported serving more

veterans, two experienced a decrease in expenditures and two a reduction

in dedicated beds. Without outcome measures, the effect of these changes

on the appropriateness and effectiveness of treatment is unclear. For

example, although the number of veterans treated for serious mental

illness increased by 8 percent from 1996 to 1998, expenditures decreased

by 9 percent. Similarly, the number of veterans treated for PTSD increased

by 9 percent, while expenditures declined by 17 percent. VA generally

attributed expenditure reductions to increases in efficiency as outpatient

or domiciliary care replaced more costly hospital inpatient treatment.

Other stakeholders review the same data and conclude that reduced

expenditures have eroded comprehensiveness and quality of care.
B-283462

Page 15 GAO/HEHS-00-57 Disabled Veterans’ Care

Facility managers we contacted were primarily concerned with

maintaining operations given the constraints of constant budgets, staff

reductions, and increasing workloads. These managers implemented

various strategies to improve efficiency while maintaining services to all

veterans, including those served in the special disability programs. These

strategies included the use of service lines,8 “hoptel” beds,9 making

referrals to community-based service providers, and shifting care to

outpatient settings. Facility officials generally believed that newly

developed alternative care settings were appropriate for special disability

populations, although no clear evidence exists to support this position.

While facility officials believed that they were meeting the demand for

special disability services, they expressed concern that additional cost

reductions might adversely affect quality of care. One facility was able to

reduce the number of inpatient psychiatric beds from over 400 to fewer

than 100. Officials at this location were confident that the community

infrastructure was adequate for most veterans. VA case managers were

assigned to patients, and staff members were working with the community

to develop additional capacity as needed. In contrast, officials from another

facility stated that their community had few suitable alternatives, a

situation that led the facility’s chief of staff to question the strategy of

deinstitutionalizing patients with mental illness.

Assessment of patient care outcomes for VA’s special disability populations

would be a major asset in interpreting VA data and trends that showed

more veterans served with fewer resources. Although outcome measures

are difficult to develop and are not generally available in private sector

health care systems either, VA made a commitment in 1997 to develop

within 2 years comprehensive and reliable measures of treatment outcome

for the six disability groups.

The fiscal year 1998 capacity report contains performance “monitors” that

are a mixture of outcome and process measures related to the care

8

A service-line model is a health care organizational model based upon providing a

comprehensive set of clinical and administrative services to meet the needs of a particular

segment of the market (for example, veterans with mental illness or spinal cord

dysfunction). Budgetary, personnel, and reporting authorities vary in the different serviceline models.

9

Hoptels are temporary lodging, usually within facilities, that provide a cost-effective

alternative to inpatient admissions.
B-283462

Page 16 GAO/HEHS-00-57 Disabled Veterans’ Care

provided to the six disability groups. The 18 performance monitors VA

identified are designed to assess quality, functional status, and patient

satisfaction. According to VA, these measures will be revised as more

appropriate ones are identified.

Data were unavailable for 7 of the 18 performance monitors identified in

the fiscal year 1998 capacity report, and information was unavailable for

the 3 years from 1996 through 1998 for 15 monitors (see app. II). Some

performance monitors, such as continuity of care for previously

hospitalized patients and changes in functional status, appear to be useful

indicators of quality. However, others are more process-oriented and do not

support an assessment of possible improvements resulting from the care

provided. For example, the performance monitor for the care of veterans

with serious mental illness is a process measure of the percentage of

patients who are assessed on a one-time basis for their level of functioning,

not an outcome measure of their improvement. Furthermore, some

monitors are limited to a small segment of the population or address

populations broader than the special disability populations. For example,

performance monitors for the spinal cord dysfunction population include

only those patients discharged after inpatient treatment (about 55 percent

of all spinal cord dysfunction patients served), and measures for the

serious mental illness category include all psychiatric patients, and not just

those with serious mental illnesses.

Both advisory committees questioned the validity of VA’s performance

monitors and expressed concern that insufficient progress has been made

in the development of comprehensive and valid measures of treatment

outcome as VA transitions to greater reliance on outpatient delivery

systems. While VA’s development of performance monitors is a step in the

right direction, more research is needed to determine whether these

measures are adequate to assess whether the care provided to veterans in

the special disability programs is as comprehensive as, and equal in quality

to, the care provided in 1996.

VA Managers Are Not

Specifically

Accountable for

Special Disability

Programs

Special disability services are delivered at the facility level, where VISN and

facility officials face the need to become more efficient to meet the needs

of more veterans with fewer resources. The alternative to increased

efficiency is decreased services. Accountability for maintaining capacity in

the special disability programs is currently fragmented among several

organizational units in VA, and performance standards mandated by statute

have not been fully implemented for those managing resources or
B-283462

Page 17 GAO/HEHS-00-57 Disabled Veterans’ Care

allocating them to special disability programs. VA indicated in its fiscal year

1998 capacity report that it was monitoring situations in which capacity

appeared to have declined, but VA did not respond to our repeated requests

that it identify who was responsible for this monitoring.

Responsibility for

Maintaining Capacity Is

Fragmented Among

Organizational Units

Responsibility for implementing the mandate to maintain capacity in

special disability programs is divided among several headquarters units,

including the Office of Policy and Planning, the Chief Network Office, and

the Office of Patient Care Services. The Office of Policy and Planning is

responsible for developing the annual capacity report. This office

coordinates the development of capacity statistics and program definitions,

oversees the verification and validation process,10 and consults with

internal and external stakeholders in finalizing the capacity report. The

Chief Network Officer is the primary point of contact for the VISNs and

provides operational direction and supervision to the field through the 22

VISN directors. The Office of Patient Care Services houses the clinically

related headquarters programs that support the delivery of patient care

services in the field. This office develops patient care policies and

guidelines, acts as program consultant to the special disability programs,

and provides advice and consultation to VISN and facility directors.

After contacting these three headquarters units, we concluded that none of

them was responsible for monitoring field locations whose capacity to

serve special disability populations appears to have declined. Each of the

units denied responsibility for monitoring and referred us to one of the

other offices as the potentially responsible unit. Despite VA’s data

problems, enhanced monitoring and follow-up could be useful in mitigating

the limitations of VA’s current capacity measures and performance

monitors. In addition to helping identify data reliability issues, such

monitoring efforts could bring to light legitimate concerns about the

provision of services to special disability populations in alternative care

settings.

10VA refers to this as the error detection and correction process in its fiscal year 1998

capacity report.
B-283462

Page 18 GAO/HEHS-00-57 Disabled Veterans’ Care

Mandatory Job Performance

Standards Have Not Been

Fully Implemented

To improve accountability for maintaining capacity in the special disability

programs, the Congress, through the Veterans Programs Enhancement Act,

required VA to develop job performance standards for employees

responsible for allocating and managing resources for serving veterans

with special disabilities. The law also required that the standards include

measures of workload, allocation of resources, and quality of care

indicators, and that the standards be implemented by January 1, 1999.

As of January 2000, VA had implemented at least one quality of care

performance standard, or monitor, in each of the six special disability

programs. The 2000 VISN directors’ performance agreement states that

part of the performance evaluation will be based on the results of the

monitors of capacity for special populations. The agreement includes 12

monitors related to the disability populations. Achievement goals for VISN

directors have been established for the 2 spinal cord care monitors, but not

for the remaining 10 monitors. Without stated goals, it is unclear what

would be considered acceptable performance.

Also, the performance agreement is silent on the measurement of workload

and allocation of resources, which is required by the law. The mandatory

job performance standards cannot be considered fully implemented

without measures of workload and allocation of resources.

VA has demonstrated that measurable performance standards for key

management officials can promote change. For example, by including in

the VISN directors’ performance agreements a requirement to decrease the

number of days inpatients spent in acute care in fiscal year 1998, VA

reduced these acute-bed-days from 3,430 per 1,000 veterans served in fiscal

year 1994 to 1,333 per 1,000 veterans served in fiscal year 1998, a 61-percent

decrease.

Conclusions While questions remain about the accuracy of VA’s workload and resource

data, VA has committed to work with stakeholders and veterans’ service

organizations to improve the accuracy and reliability of the data. VA has

also committed itself to developing measures of quality of care in special

disability programs in order to ensure that quality is maintained or

improved. However, in view of the difficulty of developing and validating

outcome measures, it is unlikely that VA will be able to develop measures

across all special disability programs in the near future. In the meantime,

the annual capacity report can be a valuable tool for identifying specific
B-283462

Page 19 GAO/HEHS-00-57 Disabled Veterans’ Care

locations with potential problems in service delivery to special disability

populations. Enhanced monitoring of such locations could be used to

augment VA’s current limited capacity measures and performance

monitors.

Responsibility for managing VA’s response to the capacity requirement is

dispersed among several of VA’s organizational components, and none of

them has taken responsibility for investigating apparent declines in

capacity or quality of services for veterans with special disabilities.

Designating a single organization as responsible for these functions would

help focus accountability for maintaining capacity. In addition, the

accountable office could be charged with fully implementing the

congressional requirement to develop job performance standards for

employees responsible for allocating and managing resources used to serve

veterans with special disabilities.

Recommendation to

the Secretary of

Veterans Affairs

To help ensure compliance with the law, we recommend that the Secretary

of Veterans Affairs direct the Under Secretary for Health to assign lead

responsibility to a headquarters unit for

• initiating efforts to monitor and determine the causes for apparent

declines in capacity and

• developing job performance standards for employees who are

responsible for allocating and managing the resources used to serve

veterans with special disabilities.

Agency Comments and

Our Response

In commenting on the draft report, VA generally agreed with our findings

and recommendations. VA intends to take an approach to ensuring

compliance with the law that is different from assigning lead responsibility

to a headquarters unit. VA said it would renew its commitment to using

existing coordination and issue resolution mechanisms to address

compliance with the law. We continue to believe that assigning

responsibility to one office would better ensure that capacity is accurately

measured and appropriately maintained. The coordination mechanisms

have not accomplished this in the past, and focusing one office’s attention

on the issue is, in our opinion, more likely to ensure accountability in the

future. VA said that a new management structure would have to be created

if it designated a single office as responsible for ensuring compliance with

the law. We believe that a new management structure is not required and
B-283462

Page 20 GAO/HEHS-00-57 Disabled Veterans’ Care

that designating an office as accountable for compliance with the law

would be sufficient. VA has tasked a working group, the 3-year-old

Performance Management Work Group, with the development of job

performance standards. This action should emphasize and delineate

responsibility for the timely completion of the job performance standards.

VA also commented that it believes that measuring the full continuum of

care, not just the numbers of beds and FTE staff, is the most appropriate

measure of access to care. We agree with VA, but, as we have discussed in

the report, VA does not have the data or processes available to consider the

full continuum of care. In this regard, we support VA’s efforts to develop

outcome measures.

VA expressed concern that the draft report placed emphasis on the

maintenance of capacity at the VISN level, noting that the law states that

capacity should be maintained at the national level. As discussed in the

report, we included information by VISN because the nationwide data hid

the variability across the VISNs. Moreover, VA stated in its 1998 capacity

report that it was monitoring situations in which capacity appeared to have

declined in particular VISNs; however, we were unable to identify who was

responsible for this monitoring. We support VA’s initiative to monitor

declines in capacity at the VISN level, knowing that such an effort exceeds

the statutory requirement to maintain capacity nationally.

As agreed with your office, unless you publicly announce its contents

earlier, we plan no further distribution of this report until 30 days from the

date of the report. At that time, we will send copies of this report to the

Honorable Togo D. West, Jr., Secretary of Veterans Affairs; appropriate

congressional committees; and other interested parties. We will also make

copies available to others upon request.
Page 21 GAO/HEHS-00-57 Disabled Veterans’ Care

Please contact me on (202) 512-7101 if you or your staff have questions

about the report or need additional assistance. George Poindexter, Linda

Diggs, Marcia Mann, and William Stanco made key contributions to this

assignment.

Sincerely yours,

Stephen P. Backhus

Director, Veterans’ Affairs and

Military Health Care Issues
Page 22 GAO/HEHS-00-57 Disabled Veterans’ Care

Appendix I

VA Definitions of Six Special Disabilities ApApenednxidexiIs

a

DSM-IV is the standard handbook for psychodiagnosis employed by clinicians and researchers in the

United States.

Disability VA capacity report definition

Spinal cord injury

and disorders

(SCI&D)

Veterans with neurological deficit lesions involving the spinal cord, including but not limited to traumatic spinal cord

injuries; intraspinal neoplasms resulting in neurological deficit; vascular insults to the spinal cord; cauda equina

syndrome; inflammatory disease of the spinal cord; and diseases such as multiple sclerosis, unstable traumatic

lesions of the spinal cord, and degenerative spine diseases are considered to have SCI&D.

Blindness Veterans are considered disabled by blindness when the best corrected central visual acuity with ordinary

eyeglasses or contact lenses is 20/200 or less in the better eye, or when the best corrected visual acuity in the better

eye is better than 20/200 but visual field defects exist that produce a useful visual field dimension of 20 degrees or

less.

Traumatic brain

injury (TBI)

Veterans who have sustained brain trauma, including from iatrogenic causes, and who have motor or cognitive

impairments as a result of the brain injury for at least 3 months, are considered to have a TBI. These individuals

either require rehabilitative services, such as acute intervention of speech pathology and cognitive rehabilitation, in

the first several months or are so severely impaired that their rehabilitative potential is objectively so low that

rehabilitative services are not appropriate. These patients may require chronic life support measures, such as

mechanical ventilation.

Amputation Amputation applies to veterans who have a full or partial amputation of a limb, including neurologic loss of a limb

(except from stroke) and loss of the use of a limb from injuries to the brain, spinal cord, or peripheral nerves.

Serious mental

illness (SMI)

Veterans who have within the past year had a diagnosed mental, behavioral, or emotional disorder of sufficient

duration to result in a disability that meets the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition

(DSM-IV),a criteria are considered to have an SMI. Disability is defined as a functional impairment that substantially

interferes with or limits one or more major life activities, such as bathing, dressing, managing money, or taking

prescribed medication. Two subcategories include veterans who have a diagnosed DSM-IV substance-related

disorder and homeless veterans who have a disability as a result of mental illness.

Post-traumatic

stress disorder

(PTSD)

Veterans who meet the diagnostic criteria for PTSD include those who

• have been exposed to a catastrophic or traumatic event involving actual or threatened death or injury, or a threat to

the physical integrity of self or others, and who have a subjective response of intense fear, helplessness, or horror;

• have intrusive recollections or reexperiences of the event;

• persistently avoid stimuli associated with the trauma and have generally numbed responsiveness;

• have persistent symptoms of increased arousal;

• have symptoms that last 1 month or longer; and

• experience clinically significant distress or impairment in social, occupational, or other important areas of

functioning.
Page 23 GAO/HEHS-00-57 Disabled Veterans’ Care

Appendix II

Performance and Access Monitors ApendxiI

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

SCI&D Access Admission to a

spinal cord

injury (SCI)

center within 1

day of request

for patients

needing acute,

inpatient

specialty care

(average)

Process Only SCI&D

patients needing

urgent admission

to an SCI center

are included in

the

measurement,

which covers all

newly injured

veterans and a

portion of the

SCI&D population

with older injuries.

VA treats

approximately

400 newly injured

patients per year.

The newly injured

make up 8% of

the SCI&D

population

treated as

inpatients.

Survey of SCI

centers by the

SCI&D Strategic

Health Group

FY 96—41%

FY 97—91%

FY 98—100%

Acute condition

means care is

required by newly

injured veterans or

by veterans with SCI

who need urgent

care because of

medical

complications or

surgical needs.

Results are reported

as a percentage of

SCI centers that met

the requirements

established by the

indicator.

SCI&D Access SCI clinic

appointment

within 7 days of

referral

(average)

Process Covers those

patients seen in

SCI clinics at

facilities with an

SCI center

Survey by the

SCI&D Strategic

Health Group

FY 96—87%

FY 97—100%

FY 98—100%

Results are reported

as a percentage of

SCI centers that met

the requirements

established by the

indicator.

SCI&D Performance SCI inpatients

rating VA care

as “very good”

to “excellent”

Outcome Only veterans

discharged from

an SCI center can

potentially be

included in the

denominator. Of

the 5,117 patients

discharged in FY

1998, only 550, or

10%, of the

patients were

included in the

survey results.

Patient

satisfaction

survey results

FY 97—55%

FY 98—55%

Of the 23 SCI

centers, 16 had

fewer than 30

respondents to the

survey; according to

VA, such small

samples are

regarded as

unreliable and of

questionable validity.

Continued
Appendix II

Performance and Access Monitors

Page 24 GAO/HEHS-00-57 Disabled Veterans’ Care

SCI&D Performance Discharge from

SCI bed unit to

noninstitutional

community

living

Outcome Includes veterans

admitted for

rehabilitation and

other purposes,

such as annual

exams

National Patient

Care Database

on patients

discharged from

SCI bed unit

FY 97—95%

FY 98—95%

VA states that

discharge to

noninstitutional

community living

“could be” viewed as

positive. Outcome

does not necessarily

relate to quality of

care received during

inpatient treatment

and may be the

result of

socioeconomic

factors.

Blindness Access Average

waiting time for

admission to

inpatient blind

rehabilitation

program

Process Measures time

only for patients

actually admitted

to program, not

those on waiting

list who decline

admission

Self-reporting by

blind

rehabilitation

centers

FY 96—31.8

weeks

FY 97—32.2

weeks

FY 98—33.4

weeks

Measures

admissions during a

6-month period

Blindness Access Number and

percentage

change in the

number of

veterans

served by blind

rehabilitation

outpatient

specialists

(BROS)

Process There was a 36%

increase in the

number of

veterans seen by

a BROS,

representing an

increase from

7.4% in 1997 to

10% in 1998.

BROS

semiannual

reports

FY 97—873

FY 98—1,191

% change: +36

The number of

veterans served is

not the best measure

of access. A

numerator and a

denominator are

needed to determine

accessibility of the

program for

veterans.

Blindness Performance Inpatients of

blind

rehabilitation

program

reporting being

“satisfied” or

“completely

satisfied” with

VA care

Outcome Patients admitted

to a VA blind

rehabilitation

program, which

accounts for 17%

of the blinded

veteran

population

seeking care

Blind

rehabilitation

customer

satisfaction

survey

FY 97—98%

FY 98—98%

The survey response

rate in 1998 was

38%, which was an

improvement over

1997. Two of the

nine blind

rehabilitation centers

had less than 30

responses to the

survey.

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 25 GAO/HEHS-00-57 Disabled Veterans’ Care

TBI Access Average

waiting time for

admission to a

designated TBI

bed

Process Impossible to

determine without

numerators and

denominators,

which were not

provided in the

report

Survey of TBI

centers

FY 96—3 days

FY 97—2 days

FY 98—2 days

There is no

indication of the size

of the survey

sample.

TBI Access Average

waiting time for

first-time TBI

outpatient

appointment

Process Impossible to

determine without

numerators and

denominators,

which were not

provided in the

report

Survey of TBI

centers

FY 96—6 days

FY 97—5 days

FY 98—7 days

There is no

indication of the size

of the survey

sample.

TBI Performance Discharge of

“firstadmission” TBI

patients from

TBI network

medical

rehabilitation

beds to the

community

Outcome Measures only

first-admission

TBI patients

VA Functional

Status Outcome

Database

FY 98-—68% Only percentages

were provided, with

a footnote stating

that some were

based on a very

small number of

discharges.

Amputation Access Delayed

prosthetic

orders

(cumulative)

Process Measures much

more than patient

population with

amputations and

was deleted from

the list of access

measures by VA

for FY 1998

Nationaldelayed

prosthetic order

report

FY 96—1.3%

FY 97—0.7%

Amputation Performance Discharge of

lower extremity

amputees from

inpatient

rehabilitation

units to

community

setting

Outcome Unable to

determine what

percentage of

amputees the

data provided by

VA represent

VA Functional

Status and

Outcome

Database

FY 96—82%

FY 97—77%

FY 98—77%

The number of

inpatient

rehabilitation units

has decreased and,

as a result, the way

care is delivered to

veterans undergoing

amputations has

also changed,

making comparisons

by fiscal year

difficult.

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 26 GAO/HEHS-00-57 Disabled Veterans’ Care

Amputation Performance Patients at risk

for foot

amputation

who were

referred to a

foot care

specialist

Process Sample of

diabetic patients

External peer

review

program—data

collected by

outside

contractor.

Data not

available.

Indicator included in

Veterans Integrated

Service Network

(VISN) directors’

performance

measures under

clinical practice

guidelines

Amputation Performance Patient

satisfaction

with VA-issued

lower extremity

prosthetic limb

Outcome Survey to be

administered to

lower extremity

amputees

National survey

of patients

Data not

available.

SMI Access Veterans seen

in any

psychiatric

outpatient

clinic within 30

days after

discharge

Process Indicator includes

only those

veterans

discharged from a

general

psychiatry unit

(approximately

33% of the

veterans treated

for SMI); unable

to determine if

this includes all

discharges from

psychiatry or just

patients

determined to

have an SMI.

Patient

Treatment File

and outpatient

files

FY 96—52%

FY 97—53%

FY 98—58%

The indicator for

veterans with a

psychiatric diagnosis

who were seen

within 30 days of

discharge is part of

the FY 2000 VISN

directors’

performance

measures.

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 27 GAO/HEHS-00-57 Disabled Veterans’ Care

SMI Access Average time

to first

outpatient visit

following

discharge

Process Indicator includes

only those

veterans

discharged from a

general

psychiatry unit

(approximately

33% of the

veterans treated

for SMI); unable

to determine if

this includes all

discharges from

psychiatry or just

patients

determined to

have an SMI.

Patient

Treatment File

and outpatient

files

FY 96—34

days

FY 97—32

days

FY 98—31

days

SMI Performance Patients with at

least one

Global

Assessment of

Functioning

score if seen in

any Veterans

Health

Administration

mental health

inpatient or

outpatient

setting in FY

1998

Process Includes all

patients seen in a

psychiatric

inpatient or

outpatient setting,

not just those with

SMI

Mental health

package

Data not

available.

Would need

comparison of

functional scores to

measure outcome of

treatment, which VA

intends to

accomplish in FY

1999.

SMI—

substance

abuse

Access Veterans seen

in any

substance

abuse

outpatient

clinic within 30

days after

discharge

Process The majority of

substance abuse

care is now

delivered in an

outpatient setting.

This measure

includes only the

28% of veterans

treated for

substance abuse

in an inpatient

setting and

includes more

than SMI

patients.

Patient

Treatment File

and outpatient

files

FY 96—38%

FY 97—41%

FY 98—41%

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 28 GAO/HEHS-00-57 Disabled Veterans’ Care

SMI—

substance

abuse

Access Average time

to first

outpatient visit

for veterans

discharged

fromsubstance

abuse

programs

Process The majority of

substance abuse

care is now

delivered in an

outpatient setting.

This measure

includes only the

28% of veterans

treated for

substance abuse

in an inpatient

setting and

includes more

than SMI

patients.

Patient

Treatment File

and outpatient

files

FY 96—27

days

FY 97—28

days

FY 98—27

days

SMI—

substance

abuse

Performance Patients

(percentage)

that

demonstrate

improvement in

the drug and

alcohol use

scores in the

Addiction

Severity Index

Outcome Covers veterans

receiving

inpatient or

outpatient care

and would

potentially include

more than SMI

patients in the

sample

Mental health

package

Data not

available.

SMI—

homeless

Access VA had not

developed an

access

indicator for the

homeless

subcategory of

SMI at the time

the FY 1998

report was

published.

Process Not determined at

the time the FY

1998 capacity

report was

published

Not determined

at the time the

FY 1998

capacity report

was published

Data not

available.

Because of the

closure of substance

abuse and general

psychiatry beds, VA

did not include

access data for

these patients and

may not include

them in the future

because of small

sample sizes.

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 29 GAO/HEHS-00-57 Disabled Veterans’ Care

SMI—

homeless

Performance Veterans who

acquired living

arrangements

at discharge

from a

Domiciliary

Care for

Homeless

Veterans

program or

Health Care for

Homeless

Veteran

communitybased contract

residential care

program

Outcome May include

veterans without

SMI when looking

at homeless

population

discharged from a

Domiciliary Care

for Homeless

Veterans program

or Health Care for

Homeless

Veteran

community-based

contract

residential care

program

Northeast

Program

Evaluation

Center files

FY 96—51%

FY 97—52%

FY 98—52%

Sample size was not

provided in report.

SMI—

homeless

Performance Veterans who

obtained

employment at

discharge from

a Domiciliary

Care for

Homeless

Veterans

program or

communitybased contract

residential care

program

Outcome May include

veterans without

SMI if looking at

homeless

population

discharged from a

Domiciliary Care

for Homeless

Veterans program

or communitybased contract

residential care

program

Northeast

Program

Evaluation

Center files

FY 96—49%

FY 97—52%

FY 98—54%

Sample size was not

provided in report.

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 30 GAO/HEHS-00-57 Disabled Veterans’ Care

SMI—

homeless

Performance Veterans with

mental illness

(including

substance

abuse) who

had a follow-up

mental health

outpatient visit

within 30 days

of discharge

from a contract

Domiciliary

Care for

Homeless

Veterans

program or

communitybased

residential care

program

Process May include

veterans without

SMI if looking at

homeless

population

discharged from a

contract

Domiciliary Care

for Homeless

Veterans program

or communitybased residential

care program

1999 data to be

collected and

summarized by

the Northeast

Program

Evaluation

Center.

Data not

available.

PTSD Access Veterans with a

primary

diagnosis of

PTSD seen in

any psychiatric

outpatient

clinic within 30

days after

discharge

Process Covers PTSD

population who

received inpatient

care, which

accounts for

approximately

11% of the

population

Patient

Treatment File

and outpatient

files

FY 96—64%

FY 97—65%

FY 98—68%

PTSD Access Average time

to first

outpatient visit

for veterans

discharged

with a primary

PTSD

diagnosis

Process Covers PTSD

population who

received inpatient

care, which

accounts for

approximately

11% of the

population

Patient

Treatment File

and outpatient

files

FY 96—30

days

FY 97—28

days

FY 98—26

days

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 31 GAO/HEHS-00-57 Disabled Veterans’ Care

PTSD Performance Veterans

(percentage)

treated for

PTSD in

specialized

PTSD

programs with

at least one

Global

Assessment of

Functioning

score

Process Covers PTSD

population

treated in both

inpatient and

outpatient

programs

Mental health

package

Data not

available.

VA is using

functional scores to

determine the

number of veterans

disabled by PTSD for

FY 1998.

PTSD Performance Change in

PTSD

symptoms on

the short form

of the

Mississippi

Scale from

admission to

follow-up 4

months after

discharge

(national

average for

adjusted mean

scores)

Outcome Covers PTSD

population

admitted to a

specialized

intensive PTSD

program

The Northeast

Program

Evaluation

Center is using

a self-reporting

survey.

FY 96—39.05

FY 97—37.19

% change:

-1.86

PTSD Performance Change in

alcohol abuse

symptoms as

measured by

the Alcohol

Abuse

Composite of

the Addiction

Severity Index

from admission

to follow-up 4

months after

discharge

(national

average for

adjusted mean

scores)

Outcome Covers PTSD

population

admitted to a

specialized

intensive PTSD

program

The Northeast

Program

Evaluation

Center is using

a self-reporting

survey.

FY 96—0.169

FY 97—0.137

% change:

-0.032

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 32 GAO/HEHS-00-57 Disabled Veterans’ Care

a

A measure that focuses on a process that leads to a certain outcome and that, when executed well,

will increase the probability of achieving a desired outcome.

PTSD Performance Change in drug

abuse

symptoms as

measured by

the Drug

Abuse

Composite of

the Addiction

Severity Index

from admission

to follow-up 4

months after

discharge

(national

average for

adjusted mean

scores)

Outcome Covers PTSD

population

admitted to a

specialized

intensive PTSD

program

The Northeast

Program

Evaluation

Center is using

a self-reporting

survey.

FY 96—0.071

FY 97—0.059

% change:

-0.012

PTSD Performance Change in

occupational

functioning as

measured by

the number of

days the

veteran has

been employed

during the past

30 days at

admission and

follow-up 4

months after

discharge

(national

average for

adjusted mean

scores)

Outcome Covers PTSD

population

admitted to a

specialized

intensive PTSD

program

The Northeast

Program

Evaluation

Center is using

a self-reporting

survey.

Data not

available.

PTSD Performance Veterans

successfully

contacted for

outcome

assessment

after discharge

from an

intensive

PTSD program

Process

(step to

collect data

for an

outcome

measure)

Covers PTSD

population

admitted to a

specialized

intensive PTSD

program

The Northeast

Program

Evaluation

Center’s

response rate

for survey

FY 96—62.7%

FY 97—66.6%

% change:

+3.9

Disability

Access or

performance

monitor Indicator

Indicator

type

(processa

or

outcomeb

)

Portion of

population

covered

Method of data

collection

Results

reported Comments

Continued from Previous Page
Appendix II

Performance and Access Monitors

Page 33 GAO/HEHS-00-57 Disabled Veterans’ Care

b

A measure that indicates the result of the performance or nonperformance of a function(s) or

process(es).
Page 34 GAO/HEHS-00-57 Disabled Veterans’ Care

Appendix III

Comments From the Department of Veterans

Affairs ApendxiI
Appendix III

Comments From the Department of Veterans

Affairs

(406174) Letr Page 35 GAO/HEHS-00-57 Disabled Veterans’ Care
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00