Medical ADP Systems: Defense Achieves Worldwide Deployment of Composite Health Care System
Released: 1996-04-05T00:00:00

Letter

Background

CHCS is a comprehensive medical information system that Defense has developed to provide automated support to its military medical treatment facilities. As shown in figure 1, the system is multi-faceted and complex, composed of nine integrated modules and shared capabilities, such as order-entry, results retrieval, and electronic mail. The modules are used to create and update the integrated patient database, which can be accessed by all authorized users. We describe the CHCS shared capabilities and modules in more detail in appendix II. CHCS supports high-volume workloads generated by numerous physicians and other health care professionals using the system simultaneously and enhances communications within and among medical treatment facilities.

In acquiring CHCS, Defense awarded a contract to Science Applications International Corporation (SAIC), in March 1988, to design, develop, deploy, and maintain CHCS. This contract recently completed its eighth and last year and ended on February 29, 1996.

CHCS has become an important part of Defense’s inpatient and outpatient medical operations. From the time a patient is admitted into a medical facility to the time of discharge, CHCS records information on the patient’s condition and treatment and makes it available to physicians, nurses, and technicians. For example, CHCS establishes a medical record as a new patient registers at the facility. As the results of tests that physicians order (as well as other patient information) are entered into CHCS, they become immediately available for medical care decisions. Further, if medication is prescribed, CHCS, in processing the prescription, checks it against the patient’s medical record for potentially dangerous medical interactions.

CHCS is also integral to Defense’s implementation of Tricare, its nationwide managed health care program. Defense’s goals for the Tricare program are to improve access to high-quality care while containing the growth of health care costs. Tricare, which is being implemented over a 3-year period, calls for coordinating and managing care on a regional basis using all available military hospitals and clinics supplemented by contracted civilian services. The Managed Care Program submodule of CHCS is the application through which active duty members and beneficiaries choosing the health maintenance organization option will be enrolled in Tricare. Tricare managers will use CHCS to assign enrolled beneficiaries to primary care providers from either the military medical treatment facility or the civilian provider network. CHCS will also assist Tricare managers in maintaining the provider network and scheduling appointments with military and/or civilian network primary care providers and specialists. Finally, CHCS is critical to measuring Tricare’s success because it enables managers to track enrollment and disenrollment in Tricare.

Scope and Methodology

To assess Defense’s actions relating to CHCS deployment and operations, we met with program officials at the Office of the Assistant Secretary of Defense for Health Affairs and CHCS program officials at Defense, as well as contractor officials at the following eight medical treatment facilities: Walter Reed Army Medical Center, Washington, D.C.; National Naval Medical Center Bethesda, Maryland; 89th Medical Group, Andrews Air Force Base (AFB), Maryland; 20th Medical Group, Shaw AFB, South Carolina; Moncrief Army Community Hospital, Ft. Jackson, South Carolina; Naval Medical Center Portsmouth, Virginia; 1st Medical Group, Langley AFB, Virginia; and McDonald Army Community Hospital, Ft. Eustis, Virginia. We also contacted CHCS program officials by telephone and mail at the following nine CHCS medical treatment facilities: Naval Hospital Great Lakes, Illinois; Darnall Army Community Hospital, Ft. Hood, Texas; Tripler Army Medical Center, Honolulu, Hawaii; Eisenhower Army Medical Center, Ft. Gordon, Georgia; Blanchfield Army Community Hospital, Ft. Campbell, Kentucky; 59th Medical Wing, Lackland AFB, Texas; 96th Medical Group, Eglin AFB, Florida; 81st Medical Group, Keesler AFB, Mississippi; and 82nd Medical Group, Sheppard AFB, Texas.

To assess Defense’s continuing efforts to address past problems, we examined (1) Defense’s August 1994 Performance Management Plan Version 5.0, (2) Defense’s August 1993 deployment plan, Implementation and Use of the CHCS, (3) Defense deployment schedules through October 4, 1995, (4) monthly progress reports provided to Defense by the CHCS contractor through December 1995, (5) Defense’s May 1995 report on VAX/PC system sizing algorithms, (6) Defense’s June 1995 report on high-end system sizing algorithms, and (7) Defense’s July 1995 report on the high-end computing platform for CHCS.

We met with CHCS users to ascertain their use of and satisfaction with CHCS, and to observe CHCS in operation. In addition, we reviewed Defense documentation relating to the results of CHCS operational tests.

We also received formal briefings from Defense on projects and programs related to CHCS, such as Defense’s Clinical Integrated Workstation project, Defense’s managed health care program, CHCS’ Benefits Realization Improvement Program, and Defense’s Pacific Medical Network project. We worked closely with and briefed senior CHCS program officials at Defense to discuss our concerns as they arose and to confirm our understanding of potential problems and their implications for the achievement of CHCS objectives.

We requested written comments from the Secretary of Defense. They were provided by the Assistant Secretary of Defense for Health Affairs and are incorporated as appendix I.

CHCS Deployment Is Complete

At the end of 1995, Defense completed deployment of CHCS to 526 of its 815 medical treatment facilities worldwide. CHCS deployment involved the installation of computer equipment and software to carry out CHCS outpatient and inpatient functions. Given the complexity of the design and development of CHCS and the number of facilities involved, this was not an easy task. Key to the successful development and deployment of CHCS has been the leadership provided by the Deputy Assistant Secretary of Defense for Health Services Operations and Readiness and the CHCS program manager and their application of a set of fundamental information management practices that we refer to as best practices. With worldwide deployment, Defense can realize the full benefits of CHCS, such as the time savings associated with physicians having immediate and facility-wide access to patient information.

Application of Best Practices Contributes to Successful Deployment

Instrumental to the successful development and deployment of CHCS worldwide has been Defense’s application of some of the best practices of leading private and public organizations for strategic information management. For example, it has been shown that the involvement and commitment of line management are crucial to making information management decisions and implementing projects. Over the past 5 years, the Deputy Assistant Secretary of Defense for Health Services Operations and Readiness, as the chief executive for the CHCS project, obtained such line management involvement and commitment by (1) promoting tri-service (Army, Navy, and Air Force) representation within Defense’s CHCS Program Office, and (2) engaging the support of the military department surgeon general organizations, which oversee Defense’s medical treatment facilities.

The Deputy Assistant Secretary also appointed an experienced and knowledgeable CHCS program manager, who was instrumental in (1) sustaining program momentum, (2) ensuring that CHCS was developed and tested in increments, thereby mitigating the impact of large-scale software development problems, and (3) instituting a set of performance measures relating to hospital operations and medical outcomes to help guide overall program direction.

Successful organizations also manage information systems as investments rather than expenses. Two key attributes are: (1) linking information system decisions tightly to program budget decisions and focusing them on mission improvement, and (2) using a disciplined process of postimplementation reviews—based on explicit decision criteria and quantifiable measures assessing mission benefits, risk, and cost—to select, control, and evaluate information systems projects.

Defense has issued policies implementing the above two attributes. Also, the CHCS program has consistently followed these policies, which require the continuous involvement of senior Defense program, financial, and information resources management officials. For example, in order to proceed into the various system development phases (analysis, design, programming, testing, validation, and implementation), the CHCS program manager had to submit justification to and obtain approval from Defense’s Major Automated Information Systems Review Council. This justification, which included documentation, such as a functional economic analysis,served as (1) a record of system approval by senior Defense officials and (2) input to Defense’s planning, programming, and budgeting process.

Finally, successful organizations have competent line and information management professionals, and ensure that their skills and knowledge are kept current. For example, both the CHCS program and deputy program managers were required to complete the comprehensive, advanced program management training offered by the Defense Systems Management College (DSMC). They also must remain current in their clinical areas by satisfying necessary continuing professional education requirements.

Benefits of CHCS Expected to Exceed Costs

Defense currently projects total benefits of $4.1 billion to be derived from using CHCS. This amount exceeds Defense’s $2.8 billion estimated system life-cycle cost by $1.3 billion. Of the total benefits amount, 83 percent represents savings attributed to increased productivity and direct cost offsets. Productivity increases would come from improved scheduling and improved access to patient information. For example, under Defense’s prior paper-based systems, physicians would order tests on paper and the results would be maintained in a patient’s paper medical file. Physicians and other health-care providers would then have to search for either the medical file or some item that was expected to be in the file. With CHCS, this information is now entered directly into the computer and is available to every authorized system user. Health-care providers can review the test results as soon as they are entered into the computer, without having to search through paper documents, thus saving staff time. Similarly, the patient saves time, as fewer visits are unproductive due to missing information.

Direct offsets include dollar savings derived from not operating the paper-based systems used prior to CHCS and from expected decreases in malpractice claims. For example, CHCS users and officials told us that because the automated CHCS records contain complete information on the patient’s allergies and medications, fewer incidents of adverse patient reactions to drugs are expected.

Past Problems Are Being Addressed

In the past 4 years, we have issued several reports identifying problems associated with CHCS design and implementation, such as Defense’s lack of an acceptable method for physicians to enter inpatient orders into CHCSand weaknesses or deficiencies in Defense’s tools and methodology for managing CHCS performance. Defense is addressing these concerns.

User-Friendly CHCS Inpatient Order-Entry Capability Under Development

Defense originally envisioned that under the CHCS inpatient order-entry process, physicians would directly key in instructions to nurses and technicians for the treatment of hospitalized patients. Defense’s intent was to eliminate the (1) costs associated with other staff entering physicians’ orders into CHCS and (2) errors in the data other staff entered because of misinterpretations of physicians’ handwriting. In September 1991, we reported that the inpatient order-entry capability in CHCS was not considered user-friendly by many physicians because entering conditional and complex orders into CHCS took much more time than writing out the orders by hand. As a result, many physicians resisted using the inpatient order-entry features of CHCS, electing to write out their orders by hand and to have other staff enter them into the system. Further, Defense deactivated the inpatient order-entry capability at all but two of its medical treatment facilities pending further development and testing.

Defense has performed extensive analysis in the past 4 years to address the inpatient order-entry problem. It issued a request for proposals to solicit commercial inpatient order-entry-system solutions in February 1992. By mid-1992, it had developed basic requirements for an inpatient order-entry capability. Defense’s analysis of those requirements led it to conclude that in order to provide physicians with this capability, it needed to develop a clinically-oriented graphical user interface (GUI).

Defense is currently building a prototype GUI. This prototype, once successfully completed, should enable physicians to access computer screens or windows containing icons that represent activities such as ordering or modifying patients’ prescriptions, and ordering inpatient laboratory tests. It is intended that physicians will be able to look up inpatient data, review inpatient laboratory test results, and perform many other tasks by clicking on a few icons and selecting items from a few menus. The GUI is being developed to enable physicians to use CHCS more efficiently, thereby reducing the possibility of errors in the system due to data-entry mistakes and reducing costs associated with having other staff enter physicians’ orders. Defense expects to complete an operational version of this GUI during 1996, as part of the Clinical Integrated Workstation project.

Tools and Methodology for Managing CHCS Performance Strengthened

In July 1994 we reported that the tools Defense was using at its CHCS sites to measure performance did not collect all the data it needed to detect response-time problems, diagnose their causes, and determine their significance. Defense also lacked modern performance analysis tools that would help it determine the causes of response-time problems and project the impact on response time of changes in workload and/or system configuration.

In addition, we reported that Defense’s methodology for managing CHCS performance was weak. The methodology did not require routine analysis and elimination of extremely long response times that occur sporadically, but relied instead on user complaints to initiate review and resolution of such problems. At that time, we also found that Defense’s method of determining reserve CHCS capacity was unreliable and might have resulted in either excessive capacity, thereby incurring unnecessary cost, or insufficient capacity, thereby leading to unsatisfactory system performance.

Since our July 1994 report, Defense has modified several existing CHCS performance measurement and analysis tools and has purchased additional ones. These tools enable Defense to measure system response times and determine which CHCS system resources (for example, memory and disk drives) are causing the response-time problems. Appendix III describes in more detail Defense’s on-going efforts to address deficiencies in its performance management tools.

In addition, Defense has taken steps to strengthen its methodology for managing CHCS performance. Specifically, Defense has (1) updated its performance management plan to include procedures for investigating and correcting extremely long response times and (2) improved its measures of system reserve capacity by developing performance simulation models for each CHCS computer platform that forecast computer resource capacity requirements.

Enhancing CHCS’ Backup and Recovery Plan Would Minimize Disruption

Defense’s current backup and recovery plan at CHCS facilities contains provisions for (1) backup copies of CHCS software and databases to be stored in other buildings, (2) critical CHCS functions to be performed manually in emergency situations, and (3) access to emergency backup generators and related equipment if power is lost. However, the plan lacks policies and procedures for the rapid repair or replacement of CHCS equipment damaged in a disaster, such as an earthquake, fire, accident, or sabotage. If the computer room housing a hospital’s CHCS hardware were heavily damaged by a disaster, users would likely suffer serious, potentially prolonged disruptions in computer service.

Sound information system controls require agencies to ensure that they are adequately prepared to cope with disaster. A current, tested, and reliable backup and recovery plan is essential to ensuring that Defense can restore CHCS operations and data should disaster strike.

According to Defense officials, their initial strategy with respect to recovery of CHCS equipment was reactive: to wait until a disaster struck before determining how best to repair or replace damaged equipment. They cited as justification for this stance: (1) the low probability of a serious disaster affecting CHCS that would not also affect the host hospital’s entire operations, (2) the costs associated with adopting a more proactive method, and (3) the sufficiency of reverting to manual methods during periods of CHCS downtime.

We disagree with this justification. Regarding Defense’s first point, CHCS now operates, for the most part, in a regional environment, where a single CHCS host facility supports one or more geographically remote satellite CHCS facilities. In this regional configuration, each host maintains an automated central patient record that is accessed by satellite facilities on demand. A disruption in CHCS operations at a host facility due to a fire, for instance, which destroys the computer room (whether or not it also destroys the rest of the hospital) will disrupt operations in every satellite facility connected to that host.

Concerning Defense’s second point, CHCS program office officials have recently stated that improvements in technology—better, faster, and cheaper computer equipment—may now make it possible for them to adopt a more active plan for repairing or replacing damaged CHCS hardware at a reasonable cost.

Finally, with respect to Defense’s third point, health-care providers at CHCS facilities told us that they have become so dependent on the patient information in CHCS that they would experience great difficulty reverting to manual methods during an extended CHCS downtime. For example, CHCS currently provides medical treatment facilities with the capability to perform drug interaction screening, which cannot be done as effectively by a human relying on memory or reviewing paper documents as it can by the computer.

We discussed our concerns with CHCS program office officials on several occasions. In recent meetings, they said they are reviewing Defense’s CHCS backup and recovery plan to address rapid repair or replacement of damaged CHCS equipment.

Conclusions

As the backbone of Defense’s medical operations, CHCS will provide personnel with almost instant access to patient information, from medical history to current treatment and vital statistics. With CHCS, Defense can make significant improvements in the way its medical treatment facilities operate: It can lower the cost and improve the quality of its health care delivery, and better address the needs of its patients, physicians, nurses, and other system users. Patients’ access to health care has increased with better appointment availability through improved scheduling. Physicians and nurses have experienced time savings in the delivery of medical care with improved access to patient information.

If Defense is to realize all of CHCS’ potential, however, it is critical that CHCS be available to physicians and other health care providers when needed. While Defense’s backup and recovery plan provides for recovery from disruptions in computer service due to power outages, the plan does not effectively address recovery from major disruptions requiring the repair or replacement of CHCS equipment damaged as a result of disaster. Health care providers have become dependent on the patient information in the system, so any major disruption in the availability of that information could result in injury or even loss of life. This risk would be greatly minimized if Defense had a more effective backup and recovery plan for CHCS equipment.

Recommendation

We recommend that the Secretary of Defense direct the Assistant Secretary of Defense for Health Affairs to develop, test, and implement Defense-wide policies and procedures for the rapid repair or replacement of CHCS equipment damaged in disasters.

Agency Comments

In commenting on a draft of this report, the Department of Defense stated that it fully agreed with the report. Defense concurred with our recommendation to implement policies and procedures for the rapid repair or replacement of CHCS equipment damaged in disasters. Specifically, the CHCS Program Office, in coordination with the Office of the Assistant Secretary of Defense for Health Affairs, tasked a commercial vendor during January 1996, to prepare a requirements analysis and recommendations. These would enable Defense to implement policies and procedures for continuity of operations and recovery from disasters for the Military Health Services System-wide infrastructure.

We are sending copies of this report to the Chairmen and Ranking Minority Members of the House and Senate Committees on Appropriations, the Secretary of Defense, and the Director of the Office of Management and Budget. Copies will also be made available to other interested parties upon request. Please contact me at (202) 512-6252 or William Franklin, Director, at (202) 512-6234 if you have any questions concerning this report.

GAO has been monitoring and reporting on CHCS since August 1985. We conducted this latest evaluation from June through December 1995, in accordance with generally accepted government auditing standards. Major contributors to this report are listed in appendix IV.

Comments From the Department of Defense

CHCS Shared Capabilities and Modules

CHCS is composed of several shared capabilities—such as order-entry, results retrieval, and electronic mail—and nine modules. The modules provide access to an integrated electronic patient database, which facilitates collection and input of data at the point of care. This supports integration of the patient care process and immediate availability of patient information to any authorized system user. The following sections describe the CHCS shared capabilities and each CHCS module.

Capabilities shared by most CHCS modules include order-entry, which allows the entry of patient orders by health-care providers and ancillary support personnel; results retrieval, which allows direct access to test results performed under any module; and electronic mail, which allows users to communicate with each other.

The Dietetics module manages the order and delivery of patient dietary instructions.

The Clinical module manages orders for patient care and the retrieval of test results. It contains checks against the patient’s medical record for risks and contraindications, and issues a warning if necessary.

The Laboratory module manages data associated with clinical and anatomical pathology, and blood/chemical tests. This includes ordering tests, processing specimens, documenting test results, and supporting quality controls.

The Patient Administration module manages the registration of patients and their medical records.

The Patient Appointment and Scheduling module manages appointment schedules for clinics and health care providers. Its Managed Care Program submodule supports enrollment, provider network management, and health care finder activities.

The Pharmacy module manages the ordering and filling of prescriptions. It checks for drug interactions and allergies, while providing an automated inventory control capability.

The Radiology module manages the ordering and scheduling of diagnostic, radiologic, nuclear medicine, and radiation therapy testing as well as the reporting of test results.

The Medical Records and Image Files Tracking module manages and tracks patient medical records and images.

The Quality Assurance module supports the identification and documentation of recurring problems related to patient care, and tracks their solutions and resolutions. It also provides management of provider case lists and training to support the credentialing process.

Update: Deficiencies in Defense’s CHCS Performance Management Tools

In our previously cited July 1994 report, we identified deficiencies in Defense’s CHCS performance management tools. These deficiencies are summarized below, along with Defense’s ongoing efforts to resolve them.

First, Defense’s Performance Monitoring Tool did not use a representative sampling of CHCS functions in measuring system response time experienced by system users. Defense now recognizes that additional user functions need to be included in its sampling. It is currently conducting engineering analyses to determine how many additional user functions should be measured.

Second, Defense’s Option Audit tool only measured system component use by option (i.e., a menu item that a user selects, such as “Enter/Maintain Lab Orders” or “Lab Order Entry/Login”), rather than at the user-function level. Defense is now modifying this tool to enable it to measure system component use at the CHCS user-function level, collect data on the frequency with which system users employ various CHCS functions, and measure system-component use for CHCS interfaces. Defense expects these modifications to be completed during 1997.

Third, Defense did not have adequate tools for the PC-CHCS UNIX platform. It has since modified the Performance Monitoring Tool and Option Audit to support performance monitoring and analysis of PC-CHCS systems. In addition, CHCS performance engineering staff evaluated five commercial-off-the-shelf UNIX performance measurement tools, and recommended obtaining two of them: Olympus TuneUp for site-level performance monitoring and analysis and Stallion Technology Monitor for evaluation and analysis of the performance impact of changes to CHCS software.

Last, we reported that Defense did not have adequate modeling tools for its CHCS systems. It has since acquired the SES Workbench simulation modeling tool, and developed performance simulation models for all CHCS configurations, including the VAX, Alpha, and PC systems. These simulation models allow Defense to project the impact of workload growth and system configuration changes on response times. Defense recently used one of the models to project the impact of the CHCS software version 4.4 upgrade on system response time at CHCS facilities. According to Defense, the changes to response time predicted by the model were close to the actual changes resulting from the upgrade.

Major Contributors to This Report

Accounting and Information Management Division, Washington, D.C.

Related GAO Products

Defense Health Care: Issues and Challenges Confronting Military Medicine (GAO/HEHS-95-104, March 22, 1995).

Defense’s Composite Health Care System: Background Briefing for the Staff of the Senate Committee on Appropriations, Subcommittee on Defense (March 17, 1995).

Defense’s Composite Health Care System: Background Briefing for the Staff of the Senate Committee on Armed Services, Subcommittee on Force Requirements and Personnel (February 14, 1995).

Defense’s Composite Health Care System: Background Briefing for the Staff of the House Committee on National Security, Subcommittee on Military Personnel (February 14, 1995).

Medical ADP Systems: Defense’s Tools and Methodology for Managing CHCS Performance Need Strengthening (GAO/AIMD-94-61, July 15, 1994).

Composite Health Care System: Outpatient Capability Is Nearly Ready for Worldwide Deployment (GAO/IMTEC-93-11, December 15, 1992).

Medical ADP Systems: Composite Health Care System Is Not Ready To Be Deployed (GAO/IMTEC-92-54, May 20, 1992).

Medical ADP Systems: Changes in Composite Health Care System’s Deployment Strategy Are Unwise (GAO/IMTEC-91-47, September 30, 1991).

Medical ADP Systems: Composite Health Care System: Defense Faces a Difficult Task (GAO/IMTEC-90-42, March 15, 1990).

Defense’s Acquisition of the Composite Health Care System (GAO/T-IMTEC-90-04, March 15, 1990).

Medical ADP Systems: Composite Health Care System Operational Tests Extended (GAO/IMTEC-89-30, April 10, 1989).

Medical ADP Systems: Analysis of Technical Aspects of DOD’s Composite Health Care System (GAO/IMTEC-88-27, July 11, 1988).

Medical ADP Systems: Composite Health Care System Acquisition—Fair, Reasonable, and Supported (GAO/IMTEC-88-26, March 4, 1988).

Medical ADP Systems: Composite Health Care System Operational Test and Evaluation Costs (GAO/IMTEC-88-18BR, January 28, 1988).

ADP Systems: Concerns About DOD’s Composite Health Care System Development Contracts (GAO/IMTEC-87-25, June 8, 1987).

ADP Systems: Concerns About the Acquisition Plan for DOD’s Composite Health Care System (GAO/IMTEC-86-12, March 31, 1986).

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.