VA Drug Formulary: Better Oversight Is Required, but Veterans Are Getting Needed Drugs
Released: 2001-01-29T00:00:00

Letter

Background

In fiscal year 2000, VA’s Veterans Health Administration (VHA) provided primary and specialty medical care to approximately 3.2 million veterans at a cost of about $18 billion. VA’s pharmacy benefit cost approximately $2 billion—about 12 percent of the total VHA budget—and provided approximately 86 million prescriptions. In contrast, 10 years ago VA’s pharmacy benefit represented about 6 percent of VA’s total health care budget.

Health care organizations’ efforts to control pharmacy costs and improve quality of care include (1) implementing formularies that limit the number of drug choices available; (2) establishing financial incentives, such as variable copayments, to encourage the use of formulary drugs; (3) using compliance programs, such as prior authorization, that encourage or require physicians to prescribe formulary drugs; and (4) developing clinical guidelines for prescribing drugs.

VA does not have authority to use financial incentives to encourage compliance with its formulary. VA provides outpatient pharmacy services free to veterans receiving medications for treatment of service-connected conditions and to low-income veterans whose incomes do not exceed a threshold amount. Other veterans who have prescriptions filled by VA may be charged $2 for each 30-day supply of medication.

In 1995, VA began transforming its delivery and management of health care to expand access to care and increase efficiency. As part of this transformation, VA decentralized decision-making and budgeting authority to 22 VISNs, which became responsible for managing all VA health care. VISNs were given substantial operational autonomy. Although VISN and medical center directors are held accountable in annual performance agreements for meeting certain national and local goals, attaining formulary goals has not been part of their performance standards.

VA medical centers began using formularies as early as 1955 to manage their pharmacy inventories. Because of the geographic mobility of VA patients, VA officials believed that a national formulary would improve veterans’ continuity of care. In September 1995, VA established a centralized group to manage its pharmacy benefit on a nationwide basis. In November 1995, VISNs were established, and the Under Secretary for Health directed each VISN to develop and implement a VISN-wide formulary. To develop their formularies, the VISNs generally combined existing medical center formularies and eliminated rarely prescribed drugs. VISN formularies became effective on April 30, 1996. Also in 1996, the Congress required VA to improve veterans’ access to care regardless of the region of the United States in which they live. As part of its response, VA implemented a national drug formulary on June 1, 1997, by combining the core set of drugs common to the newly developed VISN formularies. In addition to the national and VISN formularies, a few medical centers retained their own formularies.

VA’s Pharmacy Benefits Management Strategic Healthcare Group (PBM) is responsible for managing the national formulary list, maintaining databases that reflect drug use, and monitoring the use of certain drugs. VISN directors are responsible for implementing and monitoring compliance with the national formulary, ensuring that VISN restrictions placed on national formulary products are appropriate, and ensuring that a nonformulary drug approval process is functioning in all of their medical centers.

As all formularies do, VA’s national formulary limits the number of drug choices available to health care providers. VA’s formulary lists more than 1,100 unique drugs that are assigned to 1 of 254 drug classes—groups of drugs similar in chemistry, method of action, or purpose of use. After performing reviews of drug classes representing the highest costs and volume of prescriptions, VA decided that some drugs in 4 of its 254 drug classes were therapeutically interchangeable—that is, essentially equivalent in terms of efficacy, safety, and outcomes—and therefore had the same therapeutic effect. This determination allowed VA to select one or more of these drugs for its formulary to seek better prices through competitively bid committed-use contracts. Other therapeutically equivalent drugs in these classes were then excluded from the formulary. These four classes are known as “closed” classes. VA has not made clinical decisions regarding therapeutic interchange in the remaining 250 drug classes, and it does not limit the number of drugs that can be added to these classes. These are known as “open” classes.

In some cases, drugs listed on the national formulary may be restricted. Restrictions are generally placed on the use of drugs if they have the potential to be used inappropriately. For example, restrictions are placed on drugs with potentially serious side effects, such as interferon, which is used to treat such conditions as hepatitis C. VA has also adopted guidelines to assist practitioners in making decisions about the diagnosis, treatment, and management of specific clinical conditions, such as congestive heart failure. In addition, VA has adopted criteria to help standardize treatment, improve the quality of patient care, and promote cost-effective drug prescriptions. Finally, VA limits prescribing privileges for some drugs to specially trained physicians and requires consultation with a specialist before certain drugs can be prescribed.

National Formulary Standardization Not Yet Achieved

VA has made significant progress in establishing a national formulary, with most drugs being prescribed from the formulary list. Nevertheless, VA’s oversight has not been sufficient to ensure that it is fully achieving its national formulary goal of standardizing its drug benefit nationwide. We found that some facilities have omitted required national formulary drugs. In addition, the extent to which VISNs add drugs to supplement the national formulary has the potential for conflicting with VA’s ability to achieve standardization if not closely managed. Also, we found that some facilities, contrary to policy, have modified the list of drugs available in closed classes.

National Formulary Drugs Were Omitted From Formularies in Medical Centers We Visited

Almost 3 years after VA facilities were directed to make available locally all national formulary drugs, two of the three medical centers we visited did not list all national formulary drugs in the formularies used by their prescribers. VHA’s national formulary policy directive states that items listed on the national formulary shall be made available throughout the VA health care system and must be available in all VA facilities. While a physical supply of all national formulary drugs is not required to be maintained at all facilities, if a clinical need for a particular formulary drug arises in the course of treating a patient, it must be made available to the patient.

Many drugs listed on the national formulary were not available as formulary choices in two of the three medical centers we visited. In the first, about 25 percent (286 drugs) of the national formulary drugs were not available as formulary choices. These included drugs used to treat high blood pressure and mental disorders, as well as drugs used to treat the unique medical needs of women. At the second medical center, about 13 percent (147 drugs) of the national formulary drugs were omitted, including drugs used to treat certain types of cancer and others used to treat stomach conditions.

Health care providers at these two medical centers were required to seek nonformulary drug approvals for over 22,000 prescriptions of national formulary drugs from October 1999 through March 2000. If the national formulary had been properly implemented at these medical centers, prescribers would not have had to use extra time to request and obtain nonformulary drug approvals for these drugs, and patients could have started treatment earlier. Our analysis showed that over 14,000 prescriptions were filled as nonformulary drugs for 91 of the 286 drugs at the first center. No prescriptions were filled for the remaining 195 drugs. At the other medical center, over 8,000 prescriptions for 23 of the 147 drugs were filled as nonformulary drugs. No prescriptions were filled for the remaining 124 drugs.

Drugs Added to Supplement the National Formulary Decrease Standardization

VA’s policy allowing VISNs to supplement the national formulary locally has the potential for conflicting with VA’s ability to achieve standardization if not closely managed. From June 1997 through March 2000, VISNs added 244 unique drugs to supplement the list of drugs on the national formulary. The number of drugs added by each VISN varies widely, ranging from as many as 63 by VISN 20 (Portland) to as few as 5 by VISN 8 (Bay Pines). (Fig. 1 shows the number of drugs added by each VISN.) Adding drugs to supplement the national formulary is intended to allow VISNs to be responsive to the unique needs of their patients and to allow quicker formulary designation of new FDA-approved drugs. However, the wide variation in the number of drugs added by the VISNs to supplement the national formulary raises concern that this practice, if not appropriately monitored, could result in unacceptable decreases in formulary standardization. VA officials have acknowledged that this variation affects standardization and told us they plan to address it. For example, the PBM plans to review new drugs when approved by the FDA to determine if they will be added to the national formulary or if VISNs may continue to add them to their formularies to supplement the national formulary.

Medical Centers Inappropriately Modified the List of Drugs in Closed Classes

The medical centers we visited also inappropriately modified the national formulary list of drugs in the closed classes. Contrary to VA formulary policy, two of three medical centers added two different drugs to two of the four closed classes, and one facility did not make a drug available (see fig. 2).

While our analysis was performed at the medical center level, the IOM found similar nonconformity at the VISN level. Specifically, IOM reported that 16 of the 22 VISNs modified the list of national formulary drugs for the closed classes.

Most Prescribed Drugs Are on the National Formulary

From October 1999 through March 2000, 90 percent of VA outpatient prescriptions were written for national formulary drugs. The percentage of national formulary drug prescriptions filled by individual VISNs varied slightly, from 89 percent to 92 percent. We found wider variation among medical centers within VISNs—84 percent to 96 percent (see table 1).

The remaining 10 percent of prescriptions filled systemwide were for drugs VISNs and medical centers added to supplement the national formulary or for nonformulary drugs. VA’s PBM and IOM estimate that drugs added to supplement the national formulary probably account for about 7 percent of all prescriptions filled and nonformulary drugs account for approximately 3 percent of all prescriptions filled. However, at the time of our review, VA’s nationwide data could identify a filled prescription only as either a national formulary drug or not. Without specific information, VA does not know if the additions are resulting in an appropriate balance between local needs and national formulary standardization. VA officials told us that they are modifying the database to enable it to identify which drugs are added to supplement the national formulary and which are nonformulary.

Approval Processes for Nonformulary Drugs Have Weaknesses

Medical center approval processes for nonformulary drugs are not always timely, and the amount of time needed to obtain such approvals varied widely across medical centers. In addition, some VISNs have not established processes to collect and analyze data on nonformulary requests. As a result, VA does not know if approved requests met its established criteria or if denied requests were appropriate.

Time to Process Nonformulary Drug Approvals Varies Among Facilities

Although the national formulary directive requires certain criteria for approval of nonformulary drugs, it does not dictate a specific nonformulary approval process. As a result, the processes health care providers must follow to obtain nonformulary drugs differ among VA facilities regarding how requests are made, who receives them, who approves them, and how long it takes. In addition, IOM documented wide variations in the nonformulary drug approval process. Figure 3 shows the steps prescribers must generally follow to obtain nonformulary and formulary drugs.

The person who first receives a nonformulary drug approval request may not be the person who approves it. For example, 61 percent of prescribers reported that nonformulary drug requests must first be submitted to a facility pharmacist, 14 percent said they must first be submitted to facility pharmacy and therapeutics (P&T) committees, and 8 percent said they must first be sent to service chiefs. In contrast, 31 percent of prescribers reported that it is a facility pharmacist who approves nonformulary drug requests, 26 percent said that the facility P&T committee approves them, and 15 percent told us that the facility chief of staff approves them. The remaining 28 percent reported that various other facility officials or members of the medical staff approve nonformulary drug requests.

The time required to obtain approval for use of a nonformulary drug varied greatly depending on the local approval processes. The majority of prescribers (60 percent) we surveyed reported that it took an average of 9 days to obtain approval for use of nonformulary drugs. But many prescribers also reported that it took only a few hours (18 percent) or minutes (22 percent) to obtain such approvals.

During our medical center visits, we observed that some medical center approval processes are less convenient than others. For example, to obtain approval to use a nonformulary drug in one facility we visited, prescribers were required to submit a request in writing to the P&T committee for its review and approval. Because the P&T committee met only once a month, the final approval to use the requested drug was sometimes delayed as long as 30 days. The requesting prescriber, however, could write a prescription for an immediate 30-day supply if the medication need was urgent.

In contrast, in another medical center we visited, a clinical pharmacist was assigned to work directly with health care providers to help with drug selection, establish dose levels, and facilitate the approval of nonformulary drugs. In that facility, clinical pharmacists were allowed to approve the use of nonformulary drugs. If a health care provider believed that a patient should be prescribed a nonformulary drug, the physician and pharmacist could consult at the point of care and make a final decision with virtually no delay.

Prescribers in our survey were almost equally divided on the ease or difficulty of getting nonformulary drug requests approved (see table 2).

Regardless of whether the nonformulary drug approval process was perceived as easy or difficult, the vast majority of prescribers told us such requests were generally approved. According to our survey results, 65 percent of prescribers sought approval for a nonformulary drug in 1999. These prescribers reported that they made, on average, 25 such requests (the median was 10 requests). We estimated that 84 percent of all prescribers’ nonformulary requests were approved.

When a nonformulary drug request was disapproved, 60 percent of prescribers reported that they switched to a formulary drug. However, more than one-quarter of the prescribers who had nonformulary drug requests disapproved resubmitted their requests with additional information.

The majority of prescribers we surveyed told us they were more likely to convert VA patients who were on nonformulary drugs obtained at another VA facility to formulary drugs than to request a nonformulary drug (see table 3).

Consequently, patients who move from one area of the country to another or temporarily seek care in a different VA facility are likely to be switched from a nonformulary drug to a formulary drug.

Some VISNs Are Not Collecting and Analyzing Nonformulary Drug Data as Required

VA’s national formulary policy requires that a request to use a nonformulary drug be based on at least one of six criteria: (1) the formulary agent is contraindicated, (2) the patient has had an adverse reaction to the formulary agent, (3) all formulary alternatives have failed therapeutically, (4) no formulary alternative exists, (5) the patient has previously responded to the nonformulary agent and risk is associated with changing to the formulary agent, and (6) other circumstances involving compelling evidence-based reasons exist. Each VISN is responsible for establishing a process to collect and analyze data concerning nonformulary drug requests.

Contrary to the national formulary policy, not all VISNs have established a process to collect and analyze nonformulary request data at the VISN and local levels. Twelve of VA’s 22 VISNs reported that they do not collect information on approved and denied nonformulary drug requests. Three VISNs reported that they collect information only on approved nonformulary drug requests, and seven reported that they collect information for both approved and denied requests. Consequently, data that could help VISNs, medical centers, and the PBM offices are not always collected and analyzed for trends in a systematic manner. Such information could help VA at all levels to determine the extent to which nonformulary drugs are being requested and whether medical center processes for approving these requests meet established criteria. In its report, IOM noted that inadequate documentation could diminish confidence in the nonformulary process.

Prescribers Report Having Access to the Drugs Veterans Need

Seventy percent of VA prescribers in our survey reported that the formulary they use contains the drugs their patients need either to a “great extent” or to a “very great extent.” Twenty-seven percent reported that the formulary meets their patients’ needs to a “moderate extent,” with 4 percent reporting that it meets their patients’ needs to “some extent.” No VA prescribers reported that the formulary meets their patients’ needs to a “very little or no extent.” This is consistent with IOM’s conclusion that the VA formulary “is not overly restrictive.”

Overall, two and one-half times as many prescribers indicated that the formulary they currently use “helps” or “greatly helps” their ability to prescribe drugs as those who said it “hinders” or “greatly hinders” them (see table 4).

Some prescribers reported that the formulary they use helps them keep current with new drugs and helps remove some of the pressures created by direct-to-consumer advertising. Other prescribers reported that newly approved drugs are not made available on the national formulary as soon as they would like, and some reported their frustration with delays experienced when certain formulary drugs must be approved by specially trained physicians before they can be prescribed.

Prescribers we surveyed reported they were generally satisfied with the national formulary. We asked prescribers who said that they had worked for VA before the national formulary was established whether the current formulary does a better job of keeping the list of drugs in the drug classes from which they most frequently prescribe up to date, as compared with the formulary they used to use. Three-quarters told us that they had worked for VA before the national formulary was implemented in June 1997. Thirty- eight percent of these prescribers reported that the national formulary was “better” or “considerably better” than previous formularies. About half (48 percent) indicated that the current formulary was “about the same” as the one it replaced. Seven percent reported that it was “worse” or “considerably worse” than previous formularies.

Few veterans have complained about not being able to obtain the drugs they believe they need. At the VA medical centers we visited, patient advocates told us that veterans made very few complaints concerning their prescriptions. In its analysis of the patient advocates’ complaint databases, IOM found that less than one-half of one percent of veterans’ complaints were related to drug access. IOM further reported that complaints involving specific identifiable drugs often involved drugs that are marketed directly to consumers, such as sildenafil (Viagra), which is used to treat erectile dysfunction. Fifty-one percent of the prescribers in our survey reported that over the past 3 years, an increasing number of their patients have requested a drug they have seen or heard advertised in the media. Our review also indicated that the few prescription complaints made were often related to veterans’ trying to obtain “lifestyle” drugs or refusals by VA physicians and pharmacists to fill prescriptions written by non-VA health care providers. VA officials told us that VA does not fill prescriptions written by non-VA-authorized prescribers, in part to ensure that one practitioner manages a patient’s care.

Conclusions

Over the past 3½ years, VA has made significant progress in establishing its national formulary, which has generally met with prescriber acceptance. Prescribers reported that veterans are generally receiving the drugs they need and that veterans rarely register complaints concerning prescription drugs.

VA has not provided sufficient oversight, however, to ensure that VISNs and medical centers comply with formulary policies and that the flexibility given to them does not unduly compromise VA’s goal of formulary standardization. Contrary to VA formulary policy, some facilities omitted national formulary drugs or modified the closed drug classes. While adding a limited number of drugs to supplement the national formulary is permitted, as more drugs are added by VISNs, formulary differences among facilities are likely to become more pronounced, decreasing formulary standardization. While VA recognizes the trade-off between local flexibility and standardization, it lacks criteria for determining the appropriateness of adding drugs to supplement the national formulary. Consequently, VA cannot determine whether the resulting decrease in standardization is acceptable. Not all VISN directors have met their responsibilities for implementing national formulary policy.

Inefficiencies that exist in the nonformulary drug approval processes across the system can cause delays in making final treatment decisions. In addition, the processes require health care provider time and energy that might be better used for direct patient care. We believe a more efficient nonformulary drug approval process could enable facilities to benefit from lessons learned in other locations. Finally, VISNs lack the data needed to analyze nonformulary drug requests to determine whether all approved requests met approval criteria and all denied requests were appropriate.

Recommendations for Executive Action

In order to ensure more effective management of the national formulary, we recommend that the Secretary of Veterans Affairs direct the Under Secretary for Health to take the following actions: Establish a mechanism to ensure that VISN directors comply with national formulary policy.

Establish criteria that VISNs should use to determine the appropriateness of adding drugs to supplement the national formulary and monitor the VISNs’ application of these criteria.

Establish a nonformulary drug approval process for medical centers that ensures appropriate and timely decisions and provides that veterans for whom a nonformulary drug has been approved will have continued access to that drug, when appropriate, across VA’s health care system.

Enforce existing requirements that VISNs collect and analyze the data needed to determine that nonformulary drug approval processes are implemented appropriately and effectively in their medical centers, including tracking both approved and denied requests.

Agency Comments and Our Evaluation

In commenting on a draft of this report, VA agreed with our findings and concurred with our recommendations. VA highlighted key improvements planned or already in progress that should further enhance the process. VA’s actions to address our recommendations are summarized below.

VA plans to improve oversight at all organizational levels to help facilitate consistent compliance with national formulary policy. In its comments, VA discussed important components of improving compliance with the national formulary, including examining data to identify outliers. However, VA did not articulate a mechanism for ensuring that its oversight results in consistent compliance, which may reduce the effectiveness of its planned actions.

VA plans to establish criteria for VISNs to use to determine the appropriateness of adding drugs to supplement the national formulary. VA plans to establish steps for its nonformulary drug approval process that all medical centers and VISNs must follow. However, in its comments, VA did not specifically address how veterans would have continued access to previously approved nonformulary drugs across VA’s health care system. We believe such access is important.

VA plans to establish steps for reporting its nonformulary approval activities. In its comments, VA did not explicitly include tracking of denied requests as part of the nonformulary approval activities. We expect that its nonformulary approval activities will include tracking denied requests, as well as approved nonformulary drug requests, to determine the appropriateness of all medical center prescribing decisions.

VA plans to implement these corrective actions by June 2001. Its comments are included in appendix II.

We are sending copies of this report to the Honorable Anthony J. Principi, Secretary of Veterans Affairs; appropriate congressional committees; and other interested parties. We will also make copies available to others upon request.

Please call me at (202) 512-7101 if you or your staff have questions about this report or need additional assistance. Another contact and staff acknowledgments are listed in appendix III.

Scope and Methodology

To obtain policies and procedures from the 22 Veterans Integrated Service Networks (VISN), we mailed a questionnaire to each of the 22 VISN formulary leaders—pharmacists or physicians who serve on the Department of Veterans Affairs’ (VA) Pharmacy Benefits Management advisory board. To determine the extent to which VA health care providers write prescriptions for national formulary drugs, we analyzed data from VA’s national outpatient prescription database. To assess the implementation of the national formulary and obtain firsthand opinions about it, we interviewed medical and administrative staff at three VA medical centers located in three different VISNs. To obtain VA health care providers’ views on VA’s formulary, including whether or not it is restrictive, we mailed a questionnaire to a nationally representative sample of 2,000 VA health care prescribers. We also used information contained in the Institute of Medicine’s Description and Analysis of the VA National Formulary, issued in June 2000.

VISN Survey

To obtain policies and procedures from the 22 VISNs, we mailed a questionnaire to VISN formulary leaders. We asked if there were VISN- wide policies for several areas, including adding drugs to the VISN formulary, requesting nonformulary drugs, converting patients from one drug to another, and tracking requests for nonformulary drugs. In addition, we sought information on the number of drugs added to and dropped from the VISN formulary, the number of requests for nonformulary drugs, and the number of requests that were approved and denied. All 22 VISN formulary leaders completed and returned questionnaires.

Outpatient Prescription Database Analysis

VA’s national database on outpatient prescriptions contains information for each outpatient prescription filled at each VA medical center, including the drug prescribed, date of the prescription, patient and prescriber identifiers, medical center responsible for filling the prescription, and whether the prescribed drug is a national formulary drug. We used this database to develop a sample of VA health care providers who wrote prescriptions, determine the total number of outpatient prescriptions filled at VISNs and VA medical centers, determine the number of filled outpatient prescriptions written for national formulary drugs within a certain time frame, and determine how many VISN formulary drug prescriptions were filled in the three VISNs where we performed site visits.

We interviewed PBM headquarters officials who had either oversight or maintenance responsibility for the database to help assess the validity and reliability of the outpatient prescription data. We also performed our own analytic checks of the data. We found that data critical to our analysis—the data field indicating whether a prescription had been written for a national formulary drug—contained errors. We worked with PBM officials to correct the data, and they implemented a monthly routine to detect and correct these errors in the future. We reran our data checks, verified that the database had been corrected, and concluded that the data were acceptable for the purposes of our work.

Site Visits

To assess formulary implementation at the local level, we interviewed medical and administrative staff at three different VA medical centers—one located in Biloxi, Mississippi (VISN 16); one in Gainesville, Florida (VISN 8); and one in Omaha, Nebraska (VISN 14). We selected these VISNs and medical centers on the basis of formulary drug use from October through December 1999, the period for which the most recent and complete data were available at the time we did our work. For example, VISN 8 had the highest percentage of prescriptions for national formulary drugs (93 percent), VISN 16’s percentage of national formulary drug prescriptions was at the national average (90 percent), and VISN 14 had the lowest percentage of prescriptions filled using national formulary drugs (88 percent).

Questionnaire to Prescribers

We mailed questionnaires to a representative sample of 2,000 VA health care prescribers whose prescriptions had been dispensed from October 1 through December 31, 1999, to obtain their opinions and experiential data on various aspects of VA’s national formulary. We drew this random sample from VA’s most recent national outpatient prescription database—a data file that contains information, including a prescriber identifier, on all outpatient prescriptions filled in the VA health care system.

We mailed questionnaires to the entire sample of prescribers on April 17, 2000, with follow-up mailings on May 17 and June 21 to those who had not responded by those dates. We accepted returned questionnaires through September 1, 2000. Some prescribers’ responses indicated that they did not write prescriptions for drugs; their prescription privileges were limited to medical and surgical supplies, such as diabetic strips and food supplements. Other returned questionnaires indicated that the addressee had either left or retired from VA. These providers were thus considered ineligible for our purposes and were removed from the sample. Approximately 11 percent of the questionnaires were returned as undeliverable, and we received no response from approximately 16 percent of those to whom we mailed questionnaires. After adjusting the sample accordingly, we determined the number of useable returned questionnaires to be 1,217—a response rate of about 69 percent. (See table 5.)

Because this was a simple random sample, we believe that our results are projectable to all of VA’s health care providers who have outpatient drug prescribing privileges.

Sampling Errors

Surveys based on a sample are subject to sampling errors. Sampling error represents the extent to which a survey’s results differ from what would have been obtained had everyone in the universe of interest received and returned the same questionnaire—in this case, all VA health care providers who have outpatient drug prescribing privileges. Sampling errors have two elements: the width of the confidence interval around the estimate (sometimes referred to as the precision of the estimate) and the confidence level at which the confidence interval is computed.

The confidence interval reflects the fact that estimates actually encompass a range of possible values, not just a single value, or a “point estimate.” The interval is expressed as a point, plus or minus some value. For example, in our questionnaire, we asked prescribers, “To what extent does your VA formulary contain the drugs you believe your patients need?” The percentage of respondents who reported a “great extent” or “very great extent” was 69.1. This particular question had a confidence interval of plus or minus 2.6 percentage points. Thus, the “true” answer for this question may or may not be 69.1 percent, but it has a high probability of falling between 66.5 and 71.7 percent (69.1-percent point estimate, plus or minus 2.6 percentage points). Confidence intervals vary for individual questions (depending upon how many of the individuals who could have answered a question actually did so), but, unless otherwise noted, all percentages presented in this report are within a range of plus or minus 3.5 percentage points.

The confidence level is a measure of how certain we are that the “true” answer lies within a confidence interval. We used a 95-percent confidence level, which means that if we repeatedly took new samples of prescribers from the October through December prescription database and performed the same analysis of their responses each time, 95 percent of these samples would yield estimates that would fall within the confidence interval stated. In the previous example, this means that we are 95-percent certain that between 66.5 and 71.7 percent of prescribers believe that the VA formulary contains to a “great extent” or “very great extent” the drugs they believe their patients need.

Nonsampling Errors

Surveys can also be subject to other types of systematic error or bias that can affect results, known as nonsampling errors. One potential source of nonsampling error can be the questionnaire itself. To ensure that questions were clear and unbiased, we consulted with subject matter and questionnaire experts within GAO and obtained comments from individuals representing VA’s PBM and medical advisory panel, a working group of 11 practicing VA physicians and 1 practicing Department of Defense physician who help manage VA’s national formulary, as well as individuals representing the Institute of Medicine. Finally, the questionnaire was tested with 14 VA prescribers in VA medical centers in four locations: Phoenix, Arizona; Washington, D.C.; Hampton, Virginia; and Cincinnati, Ohio.

Prescribers’ Demographics

Prescribers were asked to provide demographic and VA employment information as well as opinions about the relevance and usefulness of VA’s formulary. On average, VA prescribers in our sample have worked for VA for 11 years, with most of those years at their current medical facility. Physicians and nurses constitute the largest groups of prescribers (65 and 15 percent, respectively), followed by physician assistants (7 percent) and other allied health professionals, such as dentists (14 percent). Most of the prescribers’ time working in VA is spent treating patients—on average, 26 hours each week. According to the national prescription file from which we drew our sample, VA prescribers who completed our questionnaire averaged 849 prescription fills from October through December 1999, the 3- month period we chose as the basis of our survey. The median number of filled prescriptions was relatively low—252—because a few prescribers had a large number of prescriptions filled during the period, while many prescribers had only a few prescriptions filled.

Comments From the Department of Veterans Affairs

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

George Poindexter, Stuart Fleishman, Mike O’Dell, and Kathie Kendrick made key contributions to this report.

Related GAO Products

State Pharmacy Programs: Assistance Designed to Target Coverage and Stretch Budgets (GAO/HEHS-00-162, Sept. 6, 2000).

Prescription Drug Benefits: Applying Private Sector Management Methods to Medicare (GAO/T-HEHS-00-84, Mar. 22, 2000).

VA Health Care: VA’s Management of Drugs on Its National Formulary (GAO/HEHS-00-34, Dec. 14, 1999).

Prescription Drug Benefits: Implications for Beneficiaries of Medicare HMO Use of Formularies (GAO/HEHS-99-166, July 20, 1999).

Defense Health Care: Fully Integrated Pharmacy System Would Improve Service and Cost-Effectiveness (GAO/HEHS-98-176, June 12, 1998).

Ordering Information

The first copy of each GAO report is free. Additional copies of reports are $2 each. A check or money order should be made out to the Superintendent of Documents. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders by phone: (202) 512-6000 fax: (202) 512-6061 TDD (202) 512-2537 Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

To Report Fraud, Waste, or Abuse in Federal Programs

Web site: http://www.gao.gov/fraudnet/fraudnet.htm e-mail: fraudnet@gao.gov 1-800-424-5454 (automated answering system)