Organ Procurement Organizations: Alternatives Being Developed to More Accurately Assess Performance
Released: 1997-11-26T00:00:00

Letter

Background

Although the number of donors is not growing as quickly as the demand for organs, the number of donors has steadily increased since 1988. The major reason for this increase is because many more older people are becoming organ donors than in the past. Nearly two-thirds of cadaveric donors were between the ages of 18 and 49 in 1988, but, by 1996, only about one-half of donors were in this age group. The proportion of donors aged 50 and older doubled from about 12 percent in 1988 to about 26 percent in 1996. Another reason for the increase in donors is because more minorities are consenting to donate organs. Between 1988 and 1996, the percentage of organ donors who belonged to racial and ethnic minority groups increased from about 16 to 23 percent.

The organ donation process usually begins at a hospital when a patient is identified as a potential organ donor. Only those patients pronounced brain dead are considered for organ donation. Most organ donors either die from nonaccidental injuries, such as a brain hemorrhage, or accidental injuries, such as a motor vehicle accident. Other causes of death appropriate for organ donation are drowning, gunshot or stab wound, or asphyxiation.

Once a potential organ donor has been identified, a staff member of either the hospital or the OPO typically contacts the deceased’s family, which then has the opportunity to donate the organs. If the family consents to donation, OPO staff coordinate the rest of the organ procurement activities, including recovering and preserving the organs and arranging for their transport to the hospital where the transplant will be performed.

One donor may provide organs to several different patients. Each cadaveric donor provides an average of three organs. In 1996, OPOs procured kidneys from 93 percent of organ donors and livers from 82 percent of them; other organs were procured at lower rates (see fig. 1).

Role of OPOs

The national system of 63 OPOs currently in operation coordinates the retrieval, preservation, transportation, and placement of organs. For Medicare and Medicaid payment purposes, HCFA certifies that an OPO meets certain criteria and designates it as the only OPO for a particular geographic area. OPOs must meet service area and other requirements. As of January 1, 1996, each OPO must meet at least one of the following service area requirements: 1. Include an entire state or official U.S. territory. 2. Either procure organs from an average of at least 24 donors per calendar year in the 2 years before the year of redesignation or request and receive an exception to this requirement. 3. If it operates exclusively in a noncontiguous U.S. state, territory, or commonwealth, procure organs at the rate of 50 percent of the national average of all OPOs for both kidneys procured and transplanted per million population. 4. If it is a new entity, demonstrate that it can procure organs from at least 50 potential donors per calendar year.

In addition, each OPO must have a board of directors or an advisory board with the authority to recommend policies on donating, procuring, and distributing organs. The board must have a transplant surgeon from each transplant center in the OPO’s service area and representation from hospital administrations, tissue banks, voluntary health associations, and either intensive care or emergency room personnel, the public, and physicians or people skilled in human histocompatibility and neurology.

OPOs must also meet other requirements. Among these, an OPO must be a nonprofit entity and have accounting and other procedures to ensure its fiscal stability. It must also have the appropriate staff and equipment to obtain organs from donors in its service area and have working relationships with at least 75 percent of the hospitals in its service area that participate in Medicare and Medicaid. OPOs must also conduct systematic efforts to acquire all usable organs from potential donors. Furthermore, OPOs must have arrangements to cooperate with tissue banks to ensure that they obtain all usable tissues from donors.

To ensure the fair distribution and safety of organs, OPOs must have a system to equitably allocate organs to transplant patients. OPOs must also arrange for appropriate tissue typing of organs and ensure that donor screening and testing for infectious diseases, including human immunodeficiency virus (HIV), are performed.

OPOs use a variety of methods for increasing donation such as raising public awareness of organ donation and developing relationships with hospitals. The goal of public education is to promote the consent process, giving people the information they need to make decisions about organ and tissue donation and encouraging them to share their decisions with their families. Such public education campaigns include mass media advertising; presentations to schools, churches, civic organizations, and businesses; and informational displays in motor vehicle offices, city and town halls, public libraries, pharmacies, and physician and attorney offices.

The racial and ethnic makeup of an OPO’s service area can affect its ability to procure organs because minority families often do not consent to organ donation. One study found that African American families’ refusal rate for organ donation was 60 percent compared with 29 percent for white families. Organ donation among minority populations, however, has increased over time. For example, African Americans accounted for 8.9 percent of organ donors in 1988 and 12 percent in 1996. The OPOs realize the need to emphasize organ donation by minorities and are focusing on increasing donation by minority populations. To help increase minority donation, OPOs have staff sensitive to the needs of and accepted by the minority population to conduct outreach and request donations and have established ethnic task forces. These efforts have increased the number of minority organ donors. (See fig. 2.)

In addition, education efforts help hospital staff clarify organ and tissue recovery policies to ensure that potential donors are consistently recognized and referred. Such activities as educating staff both in seminars and informally and featuring hospital newsletter articles about organ donation help OPOs educate hospital staff.

OPOs also conduct hospital development activities to build strong relationships with service area hospitals to promote organ donation. OPOs try to have representatives at their larger hospitals so that they can facilitate donation when a potential donor becomes available. In addition, OPOs encourage hospital staff to get involved in the organ donation process through such activities as post-donor recovery conferences to brief staff on the results of transplantations, inform them of recipients’ status, and discuss the strengths and weaknesses of the organ recovery process. Most OPOs, as part of their hospital development activities, conduct medical records reviews to determine their procurement process’ strengths and weaknesses and to share data on missed potential donors and donation consent rates with donor hospitals.

Many Identified Potential Donors Do Not Become Organ Donors

Many potential donors referred to OPOs do not meet OPO acceptance criteria; for others, the donors’ families do not consent to donation. In addition, sometimes after donation consent is obtained, doctors find that potential donors have diseases or physical conditions that make their organs unusable. As figure 3 shows, a high proportion of potential donors do not become organ donors. These data, from an AOPO annual survey of member OPOs, indicate that in 1995 about two-thirds of patients identified as potential donors were eliminated either because the family refused consent or because the donor was ultimately judged to be unsuitable for such reasons as HIV or hepatitis infection or poor condition of the organs upon inspection.

HCFA’s OPO Oversight Role

HCFA’s regional offices oversee the entire OPO certification process.Regional offices handle the OPO application process, conduct on-site reviews of OPOs, redesignate or terminate OPOs, and settle OPO territory disputes. HCFA headquarters provides the regional offices with advice and technical assistance, reviews corrective action plans submitted by OPOs that did not fully qualify for recertification, and calculates the results of the performance assessments.

During our review, four HCFA headquarters staff oversaw OPO performance as part of their duties in the End Stage Renal Disease Program. HCFA has recently reorganized the headquarters staff, however, and assigned overseeing OPO performance to the Division of Integrated Delivery Systems within the Center for Health Plans and Providers and the Clinical Standards Group in the Office of Clinical Standards and Quality. Regional office staff are involved only during the few months every 2 years when OPO recertification takes place.

HCFA Has Established a Population-Based Performance Standard for OPOs

HCFA has developed a standard for assessing OPO performance. Starting on January 1, 1996, OPOs were required to achieve at least 75 percent of the national mean in four of the five performance categories averaged over the 2 calendar years before the year of redesignation. During the 1996 transition period, OPOs meeting numerical goals in three of the five categories were recertified if they submitted an acceptable corrective action plan to increase organ donation. Recertification was granted to five OPOs that met numerical goals in three categories and submitted corrective action plans. In addition, five OPOs met goals in fewer than three categories, failing the performance standard. Of these OPOs, adjacent OPOs took over the service areas of two and a portion of the third’s. The fourth OPO merged operations with another OPO, and the fifth, determined to be a new entity, was exempt from meeting the performance standard. Recent legislation allows HCFA to change the cycle time from 2 to 4 years for OPOs meeting the standard during the previous cycle. The five performance categories for which OPOs must achieve numerical goals based on 1 million population in the OPO service area are number of actual organ donors; kidneys recovered; kidneys transplanted; extrarenal organs (heart, liver, lung, and pancreas) recovered; and extrarenal organs transplanted.

HCFA may grant exceptions from its performance standard for OPOs operating exclusively outside the contiguous United States such as in a U.S. territory or commonwealth. Because distance from the U.S. mainland can make transporting organs difficult, the procurement rate for such areas tends to be lower. OPOs typically do not recover organs unless they can identify suitable recipients. OPOs in these areas must, however, meet a standard of 50 percent of the national average of all OPOs for kidneys recovered and transplanted per 1 million population.

We Have Reported on OPO Performance Measures in the Past

In the Transplant Amendments Act of 1990, the Congress mandated that we study and report on the effectiveness of the national organ procurement and allocation system. As part of that study, we reported on the effectiveness of OPOs in procuring organs and the extent of HHS’ monitoring of OPOs’ procurement efforts. We reported that donor procurement rates—consisting of the number of donors procured per 1 million population in a geographic area—varied by OPO. We questioned the usefulness of this procurement effectiveness measure, however, because it overlooked the number of potential organ donors. HCFA nevertheless chose population as its basis for assessing OPO organ procurement performance.

HCFA’s Current Standard Is Not the Best Measure of OPO Performance

HCFA’s current standard does not accurately measure OPOs’ performance in procuring organs usable for transplantation for several reasons. Although HCFA identified several advantages of using population data, measuring performance according to population has many inherent weaknesses. For example, in the last assessment cycle, HCFA used population data that were not current. Furthermore, the standard does not account for variations in demographics and other factors that can affect the organ donation rates of OPOs, including causes of death and nonresident donors. In addition, for the initial round of recertification, HCFA did not account for the total U.S. population.

HCFA Noted Several Advantages of Using a Population-Based Measure

HCFA chose a population-based measure because the data are readily available. Collected by the Bureau of the Census, population data for an OPO service area can be developed on the basis of the county-level data the census provides. Furthermore, the population data can be adjusted to account for hospitals that deal with OPOs outside the designated OPO service area. Another reason HCFA chose population data is that OPOs pay little if anything for these data and they are relatively easy to obtain. HCFA officials also said that the organ procurement industry, mainly AOPO, agreed with using a population-based standard.

In addition, HCFA officials said that population, unlike other measures, such as number of deaths, would not pose a disadvantage for OPOs serving urban areas. Although urban areas may be more likely to have more violent deaths than other areas, the higher incidence of HIV and other diseases would limit the number of donors. In comparing the use of population data with other alternatives, HCFA officials believe that OPOs failing to meet a population-based standard would likely fail to meet other standards such as ones based on the number of service area deaths.

Population Data Were Not Timely

In the last performance cycle, HCFA allowed OPOs to use either 1990 or 1992 census data to count their service area populations. All OPOs chose to use 1990 data. In the next performance cycle covering 1996-97, HCFA plans to require that OPOs use more current population data—for 1995.

Assessing OPO 1994-95 procurement and transplantation performance using 1990 population data fails to account for population changes from 1990 to 1994. The nation’s population grew during the period from about 249 million to about 260 million, a 4.7-percent increase, with regional increases varying. The northeast and midwest states’ population increased by 1.2 and 2.9 percent, while the southern and western states’ population increased by about 6.1 and 7.7 percent, respectively. Any OPO whose service area population had increased would have had an advantage by using the 1990 data.

Population Demographics Vary by Region

A problem with using population as the basis for the standard is that it does not account for variation in population demographics that affect organ donation potential. Age and disease, for example, influence the acceptability of individuals as organ donors. These characteristics vary by region and by OPO and can pose advantages or disadvantages to an OPO’s ability to procure donors.

About 72 percent of cadaveric organ donors in 1994 and 1995 were between the ages of 18 and 64. Although this age group constitutes 61 percent of the nation’s overall population, among the states and the District of Columbia, this group constitutes from about 56 to 66 percent. Not considering other demographic factors, OPO service areas with proportionately fewer individuals between the ages of 18 and 64 may have a disadvantage in procuring organs. Conversely, OPOs with a greater proportion of individuals in this age group may have an advantage in procuring organs over other OPOs because a greater proportion of their population would be eligible to become organ donors.

The rate of diseases, such as HIV, also varies by region, and HIV-infected individuals are not acceptable as organ donors. Annual rates of AIDS ranged from a low of 2.8 cases per 100,000 population in South Dakota to a high of 246.9 cases per 100,000 population in the District of Columbia in 1994. During 1995, the prevalence of AIDS ranged from 2.6 cases to 185.7 cases per 100,000 population in those same jurisdictions. Such factors could clearly limit the eligible donor pool in some OPO service areas.

Causes of Death That Yield Organ Donors Also Vary Geographically

In addition, a standard relying on population fails to account for regional variations in causes of death. Organ donors typically die from head trauma and accidental injuries, the rates for which vary geographically. For example, motor vehicle accidents caused the death of about 25 percent of organ donors in 1994 and 1995. The rates of these accidents in an OPO’s service area can pose an advantage or a disadvantage to an OPO’s ability to procure donors. Data from the Centers for Disease Control and Prevention show that in 1991 the number of drivers fatally injured ranged from 4.44 per 100,000 population in the District of Columbia to 17.87 per 100,000 population in Mississippi.

Some Donors Do Not Live in the Procuring OPO’s Service Area

Some OPOs may draw donors from a much larger area than their service areas. They may serve high tourist areas or have trauma centers to which patients from outside their area are transferred. This affects the validity of the population-based standard because nonresident donors are not counted in the procuring OPO’s service area population. For seven OPOs whose service areas constituted an entire state, a limited analysis of United Network for Organ Sharing (UNOS) data shows that, from about 2 to 17 percent of the donors reported by these OPOs for 1994-95 lived outside these OPOs’ service areas. Table 1 shows the number and percentage of donors who lived outside the procuring OPOs’ service areas.

These data, however, may underestimate the number of donors living outside the procuring OPOs’ service areas. OPO coordinators collect the data used to perform this analysis when a potential donor becomes available. They collect the data on a UNOS form and submit it to UNOS. UNOS does not verify the data’s accuracy, and one OPO representative said that the UNOS forms often do not capture donors’ actual residences. The coordinator may record the donor’s residence as the donor hospital’s city, state, and ZIP code when residency information is lacking.

HCFA Did Not Account for Total U.S. Population but Plans to in the Next Cycle

The population data used in assessing performance for some OPOs in 1996 varied from the population data of the states and counties that comprise OPOs’ service areas as defined by HCFA. The differing population data occur because OPOs may adjust their service area populations to account for donor hospitals that affiliate with OPOs outside their service areas.

The law requires hospitals to have an agreement for notification of potential organ donation only with the OPO designated for the area in which a hospital is located. The law does provide, however, for the Secretary of HHS to waive this requirement so that hospitals can refer potential donors to OPOs outside the service area. The Secretary must approve such a request if she determines that it (1) is expected to increase organ donation and (2) will ensure equitable treatment of patients waiting for transplant within the affected OPOs’ service areas. As of October 6, 1997, HCFA had approved 173 waivers; 11 others were pending final action. When hospitals affiliate with other OPOs, these OPOs adjust their populations to accurately reflect their service area populations. The affected OPOs agree upon and make the population adjustments, according to HCFA officials. HCFA has not prescribed a method for adjusting population data. The 1990 populations for the HCFA-defined service areas and the populations adjusted by the OPOs appear in appendix II.

In addition, for the 1996 recertification cycle, we found, using HCFA’s OPO service area definitions, that 39 counties with a total population of about 1.4 million people had not been assigned to an OPO. The accuracy of the OPO definitions HCFA used for that cycle raises concerns because three of the unassigned counties had sizable populations of about 100,000 people or more. One of these counties headquartered an OPO. (App. III lists the unassigned counties.) According to a HCFA official, for the next recertification cycle, all counties will be assigned to OPOs. Table 2 shows the 1990 U.S. population, including Puerto Rico, the total population assigned to the OPOs, and the population of the unassigned counties.

Because of adjustments to OPO populations and the unassigned counties, about 3.5 million people were not assigned to any OPO. Although the affected OPOs are to consider their total populations and agree on population adjustments, OPOs did not account for about 2.1 million people in their population data. The unassigned counties, according to a HCFA official, generally did not have a hospital in their service areas. For the next recertification cycle, HCFA regional offices are to reconcile the OPO populations to account for the total U.S. population.

An Alternative Standard Based on the Number of OPO Service Area Deaths Is Slightly More Accurate but Still a Gross Measure of Performance

HCFA considered but rejected using the number of deaths as a basis for its standard. Assessing OPO procurement and transplantation performance according to the number of deaths is slightly more suitable than using population as a standard because it limits comparisons to the portion of the population eligible for organ donation. It has several disadvantages, however, including lack of timely data and of adjustments for factors surrounding an organ donor’s death, such as whether it was an in-hospital death, cause of death, declaration of brain death, and age, among others, that do not allow for accurately assessing the number of potential donors.

HCFA Considered but Rejected a Standard Using Number of Deaths

HCFA reasoned that because states collect vital statistics data, such as mortality data, such data may be inconsistent among the states. HCFA also had concerns about OPOs’ cost in obtaining death data and its timeliness. When HCFA was developing its population-based performance standard and considering alternatives, the National Center for Health Statistics (NCHS) had public use tapes of mortality statistics available. HCFA did not want OPOs to incur expenses by having to purchase the tapes and certain computer resources and staff to analyze the data. In addition, NCHS’ mortality statistics have an approximately 2-year delay in availability.

Number of OPO Service Area Deaths Rather Than Population More Accurately Reflects Number of Potential Donors

Although some organs, typically kidneys, are obtained from living donors, OPOs recover organs from cadaveric donors. Therefore, the number of deaths in their service areas more accurately reflects the number of OPOs’ potential donors. In 1994, the United States had about 2.3 million deaths out of a population of about 260 million. Although using total deaths fails to consider other factors about and characteristics of potential donors, it would eliminate considering a portion of the population that an OPO clearly could not consider for organ donation.

National Mortality Data Are Not Complete or Timely Enough for OPO Assessment Purposes

Because collecting vital statistics is typically a state function, NCHS obtains mortality statistics from the states, the District of Columbia, and territories. Some territories, such as Puerto Rico, do not submit data to NCHS. In addition, the availability of data lags by 2 years. For example, mortality data for 1995 were not available until mid-1997. Because of this, using NCHS mortality data to assess OPO performance would result in a problem similar to that of using 1990 population data for the 1994-95 assessment cycle: namely, comparing the number of deaths for incomparable time periods. The degree to which death rates vary over time is not clear; comparing data from different time periods, however, may skew the results of this type of analysis. This situation may become less problematic when HCFA moves to a 4-year recertification cycle because data would then be available for at least part of the period under review.

Small Portion of Those Who Die May Become Organ Donors

Because only a fraction of those who die make acceptable organ donors, using number of deaths as a standard provides only a gross measure of the number of potential donors. The United States had about 2.3 million deaths in 1994; however, national estimates of potential donors vary widely—totaling 5,000 to 29,000. Organ donors’ characteristics account for the small number of acceptable organ donors compared with the number of deaths.

Many older people are not considered potential donors upon their death because they are less likely than younger people to yield organs suitable for transplantation. People 65 years of age or older accounted for 73 percent of U.S. deaths in 1994. This age group accounted for less than 5 percent of the organ donors in 1994 and 1995. In addition, organ donors are admitted to a hospital before death, most to an intensive care unit. Furthermore, certain causes of death are more likely to result in the declaration of brain death than others. The vast majority of organ donors in 1994 and 1995 died from head trauma, such as that occurring from motor vehicle accidents or violent injuries; intracranial hemorrhage or stroke; or anoxia (insufficient amount of oxygen reaching the tissues of the body) caused, for example, by drowning or asphyxiation.

Some other causes of death make organ donation unacceptable because of disease that compromises the viability of organs for transplant. These diseases include HIV infection, hepatitis B, certain cancers, and tuberculosis, among others. Cancer, the second leading cause of death in 1996, accounted for 24 percent of the deaths that year.

Because of these factors, a standard based on the number of donors and organs procured and organs transplanted per 100,000 deaths may be little better than one based on population in assessing OPOs’ performance in procuring organs from potential donors.

An Alternative Standard Using Number of Deaths Adjusted for Cause of Death and Age Would More Accurately Measure the Number of Potential Donors but Still Be Approximate

HCFA also considered and rejected using adjusted death data to assess OPO performance. Adjusting for cause of death and age would more accurately estimate the number of potential organ donors than do either population or total death statistics. Considering only those causes of death that most often result in organ donation is an indicator of the number of potential donors. In addition, because older people generally do not become organ donors, limiting consideration to certain age groups would better reflect the number of likely donors.

Unfortunately, incomplete and untimely data would make adjusting for cause of death and age problematic as it does using total number of deaths. The coding of causes of death may not sufficiently identify suitable donors, and methods for adjusting for causes of death are not standard and require special staff and equipment capabilities. These drawbacks hinder the usefulness of an adjusted cause of death and age standard for assessing OPO performance.

HCFA Considered Using Number of In-Hospital Deaths as a Standard

When HCFA was developing its performance standard, the agency suggested that the number of in-hospital deaths provided a more targeted measure of the number of an OPO’s potential donors. However, the agency had concerns that such data would be unavailable or incomplete.

Adjusting for Cause of Death and Age More Accurately Estimates Number of Potential Donors Than Number of Deaths Alone

Measuring OPO performance according to the number of service area deaths adjusted for cause of death and age more accurately reflects the number of potential donors than measuring performance according to the number of all service area deaths. The number of service area deaths adjusted for cause of death and age better estimates the number of potential donors because it accounts for the small subset of the deceased that may be suitable organ donation candidates. Adjusting for cause of death and limiting consideration to deaths of those under age 75, we found that in 1994 about 147,000, or 6 percent, of the 2.3 million U.S. deaths involved these causes of death or were from this age group. This estimate, however, is much larger than the estimates some have made of a national donor pool of from 5,000 to 29,000 people per year.

Adjusted Cause of Death Data Are Incomplete or Not Timely Enough for OPO Assessment Purposes

Adjusted cause of death data are a subset of the NCHS mortality statistics. As noted, these data have completeness and timeliness limitations. Depending on the variables used for adjusting, it may not be possible to make these adjustments to analyze these data for all OPOs. For example, Oklahoma does not distinguish whether the death occurred in or out of hospital. Just as for total number of deaths, adjusted death data have a 2-year lag in availability. Again, this may be less problematic when HCFA moves to a 4-year recertification cycle.

Coding Cause of Death Data May Not Sufficiently Identify Suitable Donors

State offices of vital statistics report mortality statistics using the International Classification of Diseases 9th Revision Clinical Modification (ICD-9-CM) codes to classify deaths by cause and circumstances. Medical staff apply these codes at the time of death. An NCHS official stated that state offices of vital statistics accurately apply these codes, and studies have shown that only 3 percent of cases have coding discrepancies. Whether physicians are appropriately diagnosing cause of death and recording it accurately on the death certificate is unknown.

ICD-9-CM codes have limitations for estimating the number of potential donors in the absence of more detailed information. For example, the codes may not allow for determining the site of a cerebrovascular accident (CVA). CVA was the cause of death in about 40 percent of donors in 1994 and 1995. The lesion’s site in a CVA determines whether brain death will occur, so knowing the site is important for determining donation potential for assessing organ procurement performance.

Methods for Adjusting for Cause of Death Not Standard and Require Certain Resources

We did not identify an agreed-upon set of variables for indicating the subset of deaths that would yield suitable organ donors. We consulted experts to identify ICD-9-CM codes most frequently associated with organ donors; however, the measure we used does not fully account for the characteristics of potential donors. For example, our definition of adjusted deaths does not include in-hospital deaths, a requisite for organ donation. In addition, data, such as from NCHS, do not reveal enough information to accurately identify deaths with organ donation potential because data on a patient’s social history and medical conditions ruling out organ donation are missing.

Another drawback of using adjusted death data is the resources needed to perform the analyses. As HCFA noted in rejecting this alternative, OPOs would need to have certain computer and staff resources to compute the number of adjusted deaths in their service areas.

More OPOs Would Have Been Subject to Termination Under Alternative Measures

As stated, using the number of deaths and adjusted deaths may be an incremental improvement over using population data because OPOs are assessed according to subsets of the population that can become organ donors. However, like population data, these measures do not accurately reflect organ procurement performance. Our analysis determined whether OPOs identified as poor performers under the current standard would fare differently under alternative measures.

To assess the OPOs using these alternative measures, we used the 1994-95 OPO data on the categories HCFA used to assess performance. (See app. I for more information on our methodology.) As shown in table 3, some but not all OPOs would have fared differently depending on the standard used to assess performance. Five OPOs would have been subject to termination for failing to meet at least 75 percent of the national average for at least three of the five performance categories using HCFA’s population-based standard; three of these five OPOs would also have failed using a standard based on the number of deaths or adjusted deaths. The two other OPOs subject to termination under the current population standard would also have failed to meet the adjusted deaths standard. An additional 10 OPOs would have been subject to termination under one or the other (or in one case, both) of the two alternative standards.

More OPOs would have been subject to termination under a standard based on the number of deaths and adjusted deaths, 7 and 12 respectively, than under a standard based on population. Thus, although population does not accurately assess OPO performance, it may mean fewer OPOs are being assessed as poor performers. Although additional OPOs are identified as poor performers under the alternative standards, this does not necessarily indicate that action against them would have been warranted but may indicate flaws in these alternate measures.

An Alternative Standard Using Medical Records Reviews Would More Accurately Determine the Number of OPOs’ Potential Donors but May Be Costly

A standard using the number of donors and the number of organs recovered and transplanted compared with the number of potential organ donors would more accurately assess OPO performance. A retrospective review of death records from hospitals in an OPO’s service area could be used to estimate the number of potential donors. In developing its standards, HCFA did not consider using medical records reviews to estimate the number of potential donors for assessing OPOs’ performance. Most OPOs are conducting some form of medical records review to gain information on the strengths and weaknesses of their organizations’ organ procurement policies and practices. AOPO has started a medical records review project to determine the feasibility of using medical records reviews to estimate the number of potential donors. Using medical records reviews for assessing performance depends on several considerations: consistency of OPOs’ reviews, their independent and valid results, the cost of the reviews, and the cooperation of donor hospitals in giving access to medical records. HCFA is considering rules that would require hospital cooperation in medical records reviews.

Medical Records Reviews Can Accurately Estimate the Number of an OPO’s Potential Donors

Systematically reviewing donor hospital medical records can help to accurately estimate the number of an OPO’s potential donors. A medical records review involves reviewing all deaths at a hospital, with an in-depth examination of those meeting certain criteria. Reviewing the charts for these patients reveals the patients’ suitability for organ donation based on several factors, including cause of death, evidence of brain death, and contraindications for donation such as age and disease. Such reviews can identify that subset of deaths in which patients could have become organ donors—the true number of potential donors for an OPO service area.

Most OPOs Conduct Some Form of Medical Records Review

A survey of 68 OPOs that we conducted in 1992 showed that 60 conducted some form of medical records review. The reviews varied from a yearly review of all major hospitals to a review of a sample of cases at some major hospitals. A more recent survey, AOPO’s 1995 annual survey of its member OPOs, showed that 43 of the 49 OPOs participating in the survey conducted records reviews, mainly in donor-producing hospitals. The surveyed OPOs, however, are increasingly reviewing records in hospitals that have not provided organ donors to determine if these hospitals have the potential for donors.

In addition, OPOs use these reviews as a management tool. They track indicators such as the rate at which hospitals identify and refer potential donors to the OPO, rates of requesting donation, and rates of consent to donation. Staff at the OPOs we visited stated that this information allows them to determine where they need to focus their efforts to increase organ donation.

Of the OPOs we visited, all were conducting some form of medical records review. This included one OPO that had instituted a voluntary system in which hospitals in its service area agreed to notify the OPO of all in-hospital deaths. About 75 percent of the hospitals in the service area participate. Even with this system, the OPO still found it valuable to conduct medical records reviews to determine the completeness and accuracy of the information reported by the participating hospitals. The OPO conducts more complete records reviews at hospitals not participating in the system.

AOPO Is Conducting a Medical Records Review Project

AOPO is conducting a medical records review project partially funded by HRSA involving 33 participating OPOs. The project’s goal is to develop a method for consistently collecting information to determine the potential donor population. The OPOs are conducting the reviews for 18 months. To ensure consistent reviews, AOPO has developed a manual for and trained staff of the participating OPOs.

AOPO estimates that the project’s conclusions will be available by mid-1998. Preliminary results of the project, however, were presented at the AOPO annual meeting in June 1997. Data were presented on, among other things, the number of potential donors identified, the number referred to the OPOs, consent rates, and the number of organ donors. Preliminary results of the project raised some concerns, including the varying levels of cooperation by donor hospitals, consistency in record reviewers’ interpretation of data, and the cost and time needed to validate self-reported data. As part of this project, AOPO plans to develop hospital demographics data collection forms to produce a model for estimating donor potential. This will reduce the effort needed to conduct medical records reviews.

Donor Acceptability Criteria Vary by OPO

Differences in donor acceptance criteria by OPO may make it difficult to consistently identify potential donors. Some OPOs are accepting organs from older donors and those with diseases such as hepatitis C, hypertension, and certain cancers. Organs from these donors can be more costly to procure, and recipient survival rates can be lower. Using such donors can increase the donor pool, however, and benefit patients who otherwise would not receive a transplant.

If HCFA were to assess OPOs’ performance according to their number of potential donors, OPOs that use liberal donor acceptance criteria for estimating purposes would not fare as well as those with more conservative donor acceptance criteria. Potential donors who are older or have compromising health conditions are less likely to become donors and may yield fewer organs than younger and healthier donors.

To illustrate, one OPO we visited is participating in the AOPO medical records review project and provides data to AOPO using the AOPO potential donor criteria. The OPO, however, for its own purposes, uses more liberal criteria than AOPO’s to identify potential donors. As a result, 28 percent of the potential donors the OPO identified using its own acceptance criteria did not meet the AOPO criteria. OPO officials conceded that most of the 28 percent of potential donors would not have been acceptable, but to maximize its number of organ donors, counted these patients as potential donors. OPOs told us that an important factor in allowing them to use liberal donor criteria is the willingness of the transplant centers in their service area to use organs from these donors. Because most organs go to the transplant centers in an OPO’s service area, the OPO’s criteria will reflect the practice styles of those transplant centers. Where a transplant center is willing to transplant organs from older or less healthy donors, the OPO will expand its criteria to recover organs from older donors and those with certain diseases and medical conditions; where transplant centers are not likely to use these organs, an OPO will not recover such organs if it does not believe it can place them.

For medical records reviews to be used for identifying the number of an OPO’s potential donors and assessing OPO performance according to its donor potential, consideration must be given to OPOs’ varying donor acceptance criteria. For OPOs that have liberal donor acceptance criteria, adjustments must be made for the lower organ yield per donor these OPOs may have.

Cost of Medical Records Reviews Must Be Considered

Although medical records reviews are a valuable tool for determining the number of potential donors, they can be expensive for an OPO. Many OPOs, however, that are conducting comprehensive records reviews are already bearing the cost of the reviews. In addition, the degree to which added expense will be incurred to conduct the reviews and analyze the results is not clear. The OPOs we visited use different approaches to conduct these reviews. Some OPOs have separate staff to conduct hospital development tasks, which include records reviews, while other OPOs rely on their procurement coordinators to conduct the reviews at their assigned hospitals. We asked the OPOs to provide information on the resources needed and the costs associated with conducting medical records reviews. One OPO reported the cost as a few thousand dollars; another OPO reported the cost as $250,000. We did not determine what these costs comprised.

The OPO with the highest records review costs increased its staff from 35 full-time equivalent positions in 1993 to 63.2 in 1997, an 81-percent increase. The additional staff were hired to perform organ procurement and hospital development as well as support services. During this same period, the OPO increased its number of organ donors by 51 percent. The increase in organ procurement and hospital development staff was critical to increasing the number of organ donors, according to OPO officials. OPO officials also noted that the growth in organ donation in the 5-year period allowed them to hold organ acquisition fees relatively constant even with the increased investment in personnel.

Medical Records Reviews Are Not Consistent by OPO

For medical records reviews to be used to accurately estimate the number of potential donors as part of HCFA’s recertification standards, they must yield consistent and valid results. The OPOs that conduct medical records reviews, however, do so to determine their operations’ weaknesses and what practices they should emphasize to increase organ donation. These OPOs design their reviews to meet their needs and available resources.

In addition, OPOs conducting records reviews generally use different methodologies for their reviews. As the AOPO project revealed, consistent records reviews would require standard collection forms, manuals, and reviewer training. As AOPO found, validating results can be costly and time consuming. To use records reviews for assessing OPO performance, HCFA would have to validate the results somehow. One way to validate results would be to include a sample validation component when inspecting OPOs. Furthermore, a minority of OPOs do not conduct medical records reviews.

These OPOs lack the experience of some other OPOs because they have not been working with hospitals to allow them access to records.

Some donor hospitals’ lack of cooperation is a major concern to OPOs. The OPOs we visited cited hospitals in their service areas that refused to cooperate with records reviews. One reason for this is the hospital’s concern for patient confidentiality. Currently, OPOs have no leverage to make hospitals cooperate in the reviews. We also learned that the degree of cooperation varies among participating hospitals: Some hospitals will provide lists of hospital deaths and facilitate access to records; at other hospitals, the reviewers have to take additional steps to locate appropriate records for review.

HCFA is considering changing requirements for hospitals participating in Medicare regarding organ donation. The agency may propose changes requiring hospitals to cooperate with OPOs in reviewing death records. Other possible changes would provide OPOs with more control over identifying potential donors, requesting donations, educating hospital staff, and managing donors while testing and placement take place.

An Alternative Standard Based on Modeling Might Be Used to Estimate the Number of Potential Donors

A team of researchers from the Partnership for Organ Donation, the Harvard Medical School, and the Harvard School of Public Health has developed a modeling method using information about hospitals to predict the number of potential donors. The goal of this effort is to design an estimating procedure that will be relatively simple to execute, inexpensive, and valid. The scope of their study includes three OPOs and a random stratified sample of 88 hospitals in the OPOs’ service areas.

The team identified variables that are statistically significant predictors of the number of potential donors. It collected medical records review data for calendar years 1993 and 1992 in the smallest hospitals in the sample. Using the number of potential donors from the medical records review as the dependent variable, the team tested the variables in a series of regressions to identify those that best predicted the number of potential donors. Variables included total number of deaths, total staffed beds, Medicare case mix, medical school affiliation, and trauma center certification.

Death data were not readily available at all sample hospitals. For example, data on the numbers of deaths were not available at 6 hospitals, and only partial death data were available at 12 hospitals. Because of this, the team identified proxy variables for death. These variables included total staffed beds. In addition, the team found case mix to be a strong predictor of the number of potential donors. Case mix is the type of patients, based on diagnosis, that are in the hospital.

Research results have shown that the estimated numbers of potential donors are reasonably close to the numbers estimated from the medical records reviews. This modeling method shows promise for accurately estimating the number of potential donors and involves fewer resources than medical records reviews. If this research effort realizes its goal, this method could be a reasonable alternative to medical records reviews for assessing OPO performance.

Conclusions

Because of the gap between the supply of organs and the demand for organ transplants, OPOs are legislatively required to conduct and participate in systematic efforts to acquire all usable organs from potential donors. HCFA’s current population-based performance standard cannot accurately assess OPOs’ ability to meet the goal of acquiring all usable organs because it does not identify the number of potential donors within the OPOs’ service areas.

We identified performance measures as alternatives to the current population-based standard. Two of these alternatives—organ procurement and transplantation compared with the number of deaths or deaths adjusted for cause of death and age—would more accurately estimate the number of potential organ donors but have drawbacks. These drawbacks include lack of timely data and inability to identify the subset of causes of death suitable for organ donation. HCFA considered and rejected each of these alternatives when it established the current standard.

Two other alternative measures that HCFA did not consider—medical records reviews and modeling—show more promise for accurately identifying the number of potential donors. Reviewing hospital medical records is the most accurate method of estimating the number of potential donors in an OPO’s service area. Most OPOs do conduct medical records reviews but at varying levels of sophistication. For such a measure to be usable, the reviews would have to be conducted consistently among OPOs and the results would need to be available for validation. The AOPO records review project has raised questions about consistency in conducting the reviews and the independent verification of their results. Although most OPOs are conducting some form of medical records reviews and therefore incurring the costs of these reviews, HCFA must consider its own and the OPOs’ additional expense involved in standardizing such reviews. Other considerations include the extent to which the reviews would add to the cost of organs and whether these costs would outweigh the benefit of more accurately measuring the number of potential donors.

Another alternative, modeling, shows promise and would be less expensive than medical records reviews. At least one group is developing a modeling method using substitute measures to provide a valid measure for estimating the number of potential donors. Using existing data would make this alternative less costly than medical records reviews; however, the accuracy of such a model has yet to be established. If the number of potential donors for an OPO can be reasonably predicted using a set of variables, this could eliminate concerns about the cost of implementing medical records reviews.

HCFA believes its current standard identifies OPOs that are “poor performers.” In its final rule, however, the agency stated that it was interested in any empirical research that would merit consideration for further refining its standard. The approaches we have identified merit HCFA’s consideration.

Recommendations

To better ensure that HCFA accurately assesses OPOs’ organ procurement performance and that OPOs are maximizing the number of organs procured and transplanted, we recommend that the Secretary of Health and Human Services direct HCFA to evaluate the ongoing development of methods for determining the number of potential donors for an OPO. These methods include medical records reviews and a model to estimate the number of potential donors. If HCFA determines that one or both of these methods can accurately estimate the number of potential donors at a reasonable cost, it should choose one and begin assessing OPO performance accordingly.

Agency Comments

HCFA was given a draft of this report but could not provide written comments in time for their inclusion in this report. We met with HCFA headquarters officials responsible for the OPO certification process, and they concurred with our recommendation.

We are sending copies of this report to the Secretary of Health and Human Services, the Administrator of the Health Care Financing Administration, and the Administrator of the Health Resources and Services Administration, and other interested parties. We will also make copies available to others upon request.

Please contact me at (202) 512-7119 if you or your staff have any questions. Major contributors to this report include Marcia Crosse, Roy Hogberg, Andrea Rozner, Joan Vogel, and Craig Winslow.

Scope and Methodology

To learn about organ procurement issues and organ procurement organization (OPO) operations and develop information on alternative performance measures, we conducted a literature review and interviewed a number of federal officials and representatives of organizations and OPOs. We interviewed officials and obtained documentation from the Health Care Financing Administration (HCFA) and the Health Resources and Services Administration’s Division of Transplantation. We met with and obtained documentation from representatives of the Association of Organ Procurement Organizations, American Congress for Organ Donation, Partnership for Organ Donation, United Network on Organ Sharing, and selected OPOs.

We also met with and received data from representatives of seven OPOs, including the

Regional Organ Procurement Agency of Southern California, Los Angeles,

Southern California Organ Procurement Center, Los Angeles, California;

Regional Organ Bank of Illinois, Chicago, Illinois;

New England Organ Bank, Newton, Massachusetts;

LifeGift Organ Donation Center, Houston, Texas;

Southwest Transplant Alliance, Dallas, Texas; and

Washington Regional Transplant Consortium, Falls Church, Virginia.

We selected these OPOs because they were reviewing medical records and because they represented different geographic locations and a range of performance rankings under the current performance standards.

To rank OPOs’ performance using standards other than HCFA’s population- based standard, we obtained 1994 county-level mortality data from the Centers for Disease Control and Prevention’s National Center for Health Statistics (NCHS). Using these data, we determined the total number of deaths and the number of deaths adjusted for the cause of death and age for 65 OPOs during 1994. (NCHS could not provide mortality data for Puerto Rico.) Although we wanted to use mortality statistics for 1994 and 1995, the most recent year for which we could obtain data was 1994.

After developing the number of deaths and adjusted deaths for each OPO, we modified them to account for adjustments OPOs made in their population data, which HCFA used to assess their performance. If an OPO adjusted its population data upwards, we increased the numbers of deaths and adjusted deaths proportionately. Likewise, if an OPO adjusted its population data downwards for assessment purposes, we decreased the number of deaths proportionately.

To determine the number of deaths adjusted for cause of death, we developed a list of causes of death that could reasonably result in brain death and from which organ donation might therefore be possible. The list was limited to deaths occurring under the age of 75 because almost no organ donors exceed this age. To develop data on deaths associated with brain death, we (1) used the Partnership for Organ Donation’s medical records review form, which identifies causes of death most likely to produce potential organ donors; (2) reviewed the ICD-9-CM, Fourth Edition to identify the codes for these causes of death; and (3) sent a list of the codes we selected to NCHS’ Mortality Branch for review and revision to ensure that we had chosen the most appropriate codes. Table I.1 lists and describes the codes we used in our search.

Difference in the 1990 OPO Service Area Population and the Population HCFA Used for Assessment Purposes

Arkansas Regional Organ Recovery Agency Organ and Tissue Acquisition Center of Southern California Regional Organ Procurement Agency of Southern California Southern California Organ Procurement Center Colorado Organ Recovery Systems, Inc.

Northeast OPO and Tissue Bank The OPO at University of Florida Organ Donor Center of Hawaii Regional Organ Bank of Illinois Indiana Organ Procurement Organization, Inc.

Iowa Statewide Organ Procurement Organization (continued)

Transplant Resource Center of Maryland Organ Procurement Agency of Michigan Upper Midwest Organ Procurement Organization, Inc.

Mississippi Organ Recovery Agency, Inc.

Nebraska Organ Retrieval System, Inc.

New Jersey Organ and Tissue Sharing Network New York Regional Transplant Program OPO of Albany Medical College University of Rochester Organ Procurement Program Upstate New York Transplant Services, Inc.

Life Share of the Carolinas (continued)

Oklahoma Organ Sharing Network, Inc.

Counties Not Assigned to Any OPO for the 1996 Recertification Cycle

(continued)

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.