VA Drug Formulary: Better Oversight Is Required, but Veterans Are Getting Needed Drugs
Published: 2001-01-29T00:00:00

Fast Facts:

Highlights:
section_title: What GAO Found
paragraphs: During the last three years, the Department of Veterans Affairs (VA) has made significant progress in establishing its national drug formulary, which has generally met with the prescriber acceptance. Most veterans are receiving the drugs the need and rarely register complaints about prescription drugs. However, VA has not been sufficient to ensure that the Veterans Integrated Service Networks (VISN) and medical centers comply with formulary policies and that the flexibility given to them does not compromise VA's goal of formulary standardization. Contrary to VA formulary policy, some facilities omitted national formulary drugs or modified the closest drug classes. Although a limited number of drugs to supplement the national formulary is permitted, formulary differences among facilities are likely to become more pronounced, as more drugs are added by VISNs, decreasing formulary standardization. VA recognizes the trade-off between local flexibility and standardization, but it lacks criteria for determining the appropriateness of adding drugs to supplement the national formulary and therefore may not be able to determine whether the decrease in standardization is acceptable.

section_title: Letter
paragraphs: Food and Drug Administration Institute of Medicine Pharmacy Benefits Management Strategic Healthcare Group pharmacy and therapeutics Department of Veterans Affairs Veterans Health Administration Veterans Integrated Service Network The large number of drugs manufactured to treat various medical  conditions has led health care organizations to develop lists of medications  that they encourage their health care providers to use when they write  prescriptions for patients. These lists of medications, grouped by  therapeutic class, are known as drug formularies. Organizations prefer, and  sometimes require, that these drugs be used as the provider’s first choice  because organizations believe these drugs represent the best value among  therapeutic choices and because by concentrating their purchases  organizations can secure better prices.
As early as 1955, the Department of Veterans Affairs’ (VA) medical centers  began using formularies to manage their pharmacy inventories, and until  recently each of VA’s 173 medical centers maintained its own formulary. In  June 1997, VA established its national formulary by consolidating locally  developed and managed formularies into a core list of drugs. The national  formulary was intended to give health care providers who have outpatient  prescription privileges the same list of formulary drugs no matter where in  the VA system they practice medicine, and to assure veterans that they will  have access to the same group of formulary drugs for their treatment at all  VA medical centers no matter where they seek care. VA medical centers  were directed to make all national formulary drugs available to prescribers.  To provide flexibility in meeting local patient needs, VA allows its 22  regional Veterans Integrated Service Networks (VISN) to add drugs locally  to supplement the national formulary and requires them to establish  approval processes for prescribers to obtain drugs not listed on their  formularies.
Over the past few years, you and others in the Congress have raised  questions about VA’s national formulary. In December 1999, we issued a  report addressing your interest in how VA manages its national formulary.  In June 2000, the Institute of Medicine (IOM), as mandated by the Congress  in House Report 105-610, issued its report on the effect VA’s national  formulary has had on the cost and quality of VA health care, the  restrictiveness of VA’s national formulary, and how the national formulary  compares with private and other government formularies.
To supplement the findings of our December 1999 report, you asked us to  review (1) the impact of national formulary implementation by VISNs and  medical centers on VA’s goal of standardizing its formulary and (2) VISN  and medical center processes for obtaining nonformulary outpatient  prescriptions. You also asked that we examine whether VA’s national  formulary allows health care providers to prescribe the medications they  believe their patients need.
To address these VA formulary issues, we reviewed the formulary policies  and activities of VA’s central office and its VISNs and compiled and  analyzed nationwide VA prescription data. To assess implementation at the  local level, we also discussed formulary issues with VA officials at three  medical centers we visited in three different VISNs. We selected these  medical centers on the basis of their percentage of prescriptions filled for  formulary drugs. At your request, we also conducted a survey of 2,000  health care providers who have VA prescription writing privileges  (prescribers) to determine whether they believe implementation of the  national formulary has adversely affected their ability to prescribe needed  medications for the veterans they treat. In addition, we reviewed the  congressionally mandated IOM assessment of VA’s national formulary  issued in June 2000, and we incorporated its pertinent results in our report.
Appendix I contains further details of our scope and methodology. We  performed our work between January 2000 and January 2001 in accordance  with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: In fiscal year 2000, VA’s Veterans Health Administration (VHA) provided  primary and specialty medical care to approximately 3.2 million veterans at  a cost of about $18 billion. VA’s pharmacy benefit cost approximately $2  billion—about 12 percent of the total VHA budget—and provided  approximately 86 million prescriptions. In contrast, 10 years ago VA’s  pharmacy benefit represented about 6 percent of VA’s total health care  budget.
Health care organizations’ efforts to control pharmacy costs and improve  quality of care include (1) implementing formularies that limit the number  of drug choices available; (2) establishing financial incentives, such as  variable copayments, to encourage the use of formulary drugs; (3) using  compliance programs, such as prior authorization, that encourage or  require physicians to prescribe formulary drugs; and (4) developing clinical  guidelines for prescribing drugs.
VA does not have authority to use financial incentives to encourage  compliance with its formulary. VA provides outpatient pharmacy services  free to veterans receiving medications for treatment of service-connected  conditions and to low-income veterans whose incomes do not exceed a  threshold amount. Other veterans who have prescriptions filled by VA may  be charged $2 for each 30-day supply of medication.
In 1995, VA began transforming its delivery and management of health care  to expand access to care and increase efficiency. As part of this  transformation, VA decentralized decision-making and budgeting authority  to 22 VISNs, which became responsible for managing all VA health care.  VISNs were given substantial operational autonomy. Although VISN and  medical center directors are held accountable in annual performance  agreements for meeting certain national and local goals, attaining  formulary goals has not been part of their performance standards.
VA medical centers began using formularies as early as 1955 to manage  their pharmacy inventories. Because of the geographic mobility of VA  patients, VA officials believed that a national formulary would improve  veterans’ continuity of care. In September 1995, VA established a  centralized group to manage its pharmacy benefit on a nationwide basis. In  November 1995, VISNs were established, and the Under Secretary for  Health directed each VISN to develop and implement a VISN-wide  formulary. To develop their formularies, the VISNs generally combined  existing medical center formularies and eliminated rarely prescribed drugs.  VISN formularies became effective on April 30, 1996. Also in 1996, the  Congress required VA to improve veterans’ access to care regardless of the  region of the United States in which they live. As part of its response, VA  implemented a national drug formulary on June 1, 1997, by combining the  core set of drugs common to the newly developed VISN formularies. In  addition to the national and VISN formularies, a few medical centers  retained their own formularies.
VA’s Pharmacy Benefits Management Strategic Healthcare Group (PBM) is  responsible for managing the national formulary list, maintaining databases  that reflect drug use, and monitoring the use of certain drugs. VISN  directors are responsible for implementing and monitoring compliance  with the national formulary, ensuring that VISN restrictions placed on  national formulary products are appropriate, and ensuring that a  nonformulary drug approval process is functioning in all of their medical  centers.
As all formularies do, VA’s national formulary limits the number of drug  choices available to health care providers. VA’s formulary lists more than  1,100 unique drugs that are assigned to 1 of 254 drug classes—groups of  drugs similar in chemistry, method of action, or purpose of use. After  performing reviews of drug classes representing the highest costs and  volume of prescriptions, VA decided that some drugs in 4 of its 254 drug  classes were therapeutically interchangeable—that is, essentially  equivalent in terms of efficacy, safety, and outcomes—and therefore had  the same therapeutic effect. This determination allowed VA to select one or  more of these drugs for its formulary to seek better prices through  competitively bid committed-use contracts. Other therapeutically  equivalent drugs in these classes were then excluded from the formulary.  These four classes are known as “closed” classes. VA has not made clinical  decisions regarding therapeutic interchange in the remaining 250 drug  classes, and it does not limit the number of drugs that can be added to  these classes. These are known as “open” classes.
In some cases, drugs listed on the national formulary may be restricted.  Restrictions are generally placed on the use of drugs if they have the  potential to be used inappropriately. For example, restrictions are placed  on drugs with potentially serious side effects, such as interferon, which is  used to treat such conditions as hepatitis C. VA has also adopted guidelines  to assist practitioners in making decisions about the diagnosis, treatment,  and management of specific clinical conditions, such as congestive heart  failure. In addition, VA has adopted criteria to help standardize treatment,  improve the quality of patient care, and promote cost-effective drug  prescriptions. Finally, VA limits prescribing privileges for some drugs to  specially trained physicians and requires consultation with a specialist  before certain drugs can be prescribed.
section_title: National Formulary Standardization Not Yet Achieved
paragraphs: VA has made significant progress in establishing a national formulary, with  most drugs being prescribed from the formulary list. Nevertheless, VA’s  oversight has not been sufficient to ensure that it is fully achieving its  national formulary goal of standardizing its drug benefit nationwide. We  found that some facilities have omitted required national formulary drugs.  In addition, the extent to which VISNs add drugs to supplement the  national formulary has the potential for conflicting with VA’s ability to  achieve standardization if not closely managed. Also, we found that some  facilities, contrary to policy, have modified the list of drugs available in  closed classes.
subsections: section_title: National Formulary Drugs Were Omitted From Formularies in Medical Centers We Visited
paragraphs: Almost 3 years after VA facilities were directed to make available locally all  national formulary drugs, two of the three medical centers we visited did  not list all national formulary drugs in the formularies used by their  prescribers. VHA’s national formulary policy directive states that items  listed on the national formulary shall be made available throughout the VA  health care system and must be available in all VA facilities. While a  physical supply of all national formulary drugs is not required to be  maintained at all facilities, if a clinical need for a particular formulary drug  arises in the course of treating a patient, it must be made available to the  patient.
Many drugs listed on the national formulary were not available as  formulary choices in two of the three medical centers we visited. In the  first, about 25 percent (286 drugs) of the national formulary drugs were not  available as formulary choices. These included drugs used to treat high  blood pressure and mental disorders, as well as drugs used to treat the  unique medical needs of women. At the second medical center, about 13  percent (147 drugs) of the national formulary drugs were omitted,  including drugs used to treat certain types of cancer and others used to  treat stomach conditions.
Health care providers at these two medical centers were required to seek  nonformulary drug approvals for over 22,000 prescriptions of national  formulary drugs from October 1999 through March 2000. If the national  formulary had been properly implemented at these medical centers,  prescribers would not have had to use extra time to request and obtain  nonformulary drug approvals for these drugs, and patients could have  started treatment earlier. Our analysis showed that over 14,000  prescriptions were filled as nonformulary drugs for 91 of the 286 drugs at  the first center. No prescriptions were filled for the remaining 195 drugs.  At the other medical center, over 8,000 prescriptions for 23 of the 147 drugs  were filled as nonformulary drugs. No prescriptions were filled for the  remaining 124 drugs.
section_title: Drugs Added to Supplement the National Formulary Decrease Standardization
paragraphs: VA’s policy allowing VISNs to supplement the national formulary locally has  the potential for conflicting with VA’s ability to achieve standardization if  not closely managed. From June 1997 through March 2000, VISNs added  244 unique drugs to supplement the list of drugs on the national formulary.  The number of drugs added by each VISN varies widely, ranging from as  many as 63 by VISN 20 (Portland) to as few as 5 by VISN 8 (Bay Pines).  (Fig. 1 shows the number of drugs added by each VISN.) Adding drugs to  supplement the national formulary is intended to allow VISNs to be  responsive to the unique needs of their patients and to allow quicker formulary designation of new FDA-approved drugs. However, the wide  variation in the number of drugs added by the VISNs to supplement the  national formulary raises concern that this practice, if not appropriately  monitored, could result in unacceptable decreases in formulary  standardization. VA officials have acknowledged that this variation affects  standardization and told us they plan to address it. For example, the PBM  plans to review new drugs when approved by the FDA to determine if they  will be added to the national formulary or if VISNs may continue to add  them to their formularies to supplement the national formulary.
section_title: Medical Centers Inappropriately Modified the List of Drugs in Closed Classes
paragraphs: The medical centers we visited also inappropriately modified the national  formulary list of drugs in the closed classes. Contrary to VA formulary  policy, two of three medical centers added two different drugs to two of the  four closed classes, and one facility did not make a drug available (see fig.  2).
While our analysis was performed at the medical center level, the IOM  found similar nonconformity at the VISN level. Specifically, IOM reported  that 16 of the 22 VISNs modified the list of national formulary drugs for the  closed classes.
section_title: Most Prescribed Drugs Are on the National Formulary
paragraphs: From October 1999 through March 2000, 90 percent of VA outpatient  prescriptions were written for national formulary drugs. The percentage of  national formulary drug prescriptions filled by individual VISNs varied  slightly, from 89 percent to 92 percent. We found wider variation among  medical centers within VISNs—84 percent to 96 percent (see table 1).
The remaining 10 percent of prescriptions filled systemwide were for drugs  VISNs and medical centers added to supplement the national formulary or  for nonformulary drugs. VA’s PBM and IOM estimate that drugs added to  supplement the national formulary probably account for about 7 percent of  all prescriptions filled and nonformulary drugs account for approximately 3  percent of all prescriptions filled. However, at the time of our review, VA’s  nationwide data could identify a filled prescription only as either a national  formulary drug or not. Without specific information, VA does not know if  the additions are resulting in an appropriate balance between local needs  and national formulary standardization. VA officials told us that they are  modifying the database to enable it to identify which drugs are added to  supplement the national formulary and which are nonformulary.
section_title: Approval Processes for Nonformulary Drugs Have Weaknesses
paragraphs: Medical center approval processes for nonformulary drugs are not always  timely, and the amount of time needed to obtain such approvals varied  widely across medical centers. In addition, some VISNs have not  established processes to collect and analyze data on nonformulary  requests. As a result, VA does not know if approved requests met its  established criteria or if denied requests were appropriate.
subsections: section_title: Time to Process Nonformulary Drug Approvals Varies Among Facilities
paragraphs: Although the national formulary directive requires certain criteria for  approval of nonformulary drugs, it does not dictate a specific nonformulary  approval process. As a result, the processes health care providers must  follow to obtain nonformulary drugs differ among VA facilities regarding  how requests are made, who receives them, who approves them, and how  long it takes. In addition, IOM documented wide variations in the  nonformulary drug approval process. Figure 3 shows the steps prescribers  must generally follow to obtain nonformulary and formulary drugs.
The person who first receives a nonformulary drug approval request may  not be the person who approves it. For example, 61 percent of prescribers  reported that nonformulary drug requests must first be submitted to a  facility pharmacist, 14 percent said they must first be submitted to facility  pharmacy and therapeutics (P&T) committees, and 8 percent said they  must first be sent to service chiefs. In contrast, 31 percent of prescribers  reported that it is a facility pharmacist who approves nonformulary drug  requests, 26 percent said that the facility P&T committee approves them,  and 15 percent told us that the facility chief of staff approves them. The  remaining 28 percent reported that various other facility officials or  members of the medical staff approve nonformulary drug requests.
The time required to obtain approval for use of a nonformulary drug varied  greatly depending on the local approval processes. The majority of  prescribers (60 percent) we surveyed reported that it took an average of 9  days to obtain approval for use of nonformulary drugs. But many  prescribers also reported that it took only a few hours (18 percent) or  minutes (22 percent) to obtain such approvals.
During our medical center visits, we observed that some medical center  approval processes are less convenient than others. For example, to obtain  approval to use a nonformulary drug in one facility we visited, prescribers  were required to submit a request in writing to the P&T committee for its  review and approval. Because the P&T committee met only once a month,  the final approval to use the requested drug was sometimes delayed as long  as 30 days. The requesting prescriber, however, could write a prescription  for an immediate 30-day supply if the medication need was urgent.
In contrast, in another medical center we visited, a clinical pharmacist was  assigned to work directly with health care providers to help with drug  selection, establish dose levels, and facilitate the approval of nonformulary  drugs. In that facility, clinical pharmacists were allowed to approve the use  of nonformulary drugs. If a health care provider believed that a patient  should be prescribed a nonformulary drug, the physician and pharmacist  could consult at the point of care and make a final decision with virtually  no delay.
Prescribers in our survey were almost equally divided on the ease or  difficulty of getting nonformulary drug requests approved (see table 2).
Regardless of whether the nonformulary drug approval process was  perceived as easy or difficult, the vast majority of prescribers told us such  requests were generally approved. According to our survey results, 65  percent of prescribers sought approval for a nonformulary drug in 1999.  These prescribers reported that they made, on average, 25 such requests  (the median was 10 requests). We estimated that 84 percent of all  prescribers’ nonformulary requests were approved.
When a nonformulary drug request was disapproved, 60 percent of  prescribers reported that they switched to a formulary drug. However,  more than one-quarter of the prescribers who had nonformulary drug  requests disapproved resubmitted their requests with additional  information.
The majority of prescribers we surveyed told us they were more likely to  convert VA patients who were on nonformulary drugs obtained at another  VA facility to formulary drugs than to request a nonformulary drug (see  table 3).
Consequently, patients who move from one area of the country to another  or temporarily seek care in a different VA facility are likely to be switched  from a nonformulary drug to a formulary drug.
section_title: Some VISNs Are Not Collecting and Analyzing Nonformulary Drug Data as Required
paragraphs: VA’s national formulary policy requires that a request to use a nonformulary  drug be based on at least one of six criteria: (1) the formulary agent is  contraindicated, (2) the patient has had an adverse reaction to the  formulary agent, (3) all formulary alternatives have failed therapeutically,  (4) no formulary alternative exists, (5) the patient has previously  responded to the nonformulary agent and risk is associated with changing  to the formulary agent, and (6) other circumstances involving compelling  evidence-based reasons exist. Each VISN is responsible for establishing a  process to collect and analyze data concerning nonformulary drug  requests.
Contrary to the national formulary policy, not all VISNs have established a  process to collect and analyze nonformulary request data at the VISN and  local levels. Twelve of VA’s 22 VISNs reported that they do not collect  information on approved and denied nonformulary drug requests. Three  VISNs reported that they collect information only on approved  nonformulary drug requests, and seven reported that they collect  information for both approved and denied requests. Consequently, data  that could help VISNs, medical centers, and the PBM offices are not always  collected and analyzed for trends in a systematic manner. Such information  could help VA at all levels to determine the extent to which nonformulary  drugs are being requested and whether medical center processes for  approving these requests meet established criteria. In its report, IOM noted  that inadequate documentation could diminish confidence in the  nonformulary process.
section_title: Prescribers Report Having Access to the Drugs Veterans Need
paragraphs: Seventy percent of VA prescribers in our survey reported that the formulary  they use contains the drugs their patients need either to a “great extent” or  to a “very great extent.” Twenty-seven percent reported that the formulary  meets their patients’ needs to a “moderate extent,” with 4 percent reporting  that it meets their patients’ needs to “some extent.” No VA prescribers  reported that the formulary meets their patients’ needs to a “very little or  no extent.” This is consistent with IOM’s conclusion that the VA formulary  “is not overly restrictive.”
Overall, two and one-half times as many prescribers indicated that the  formulary they currently use “helps” or “greatly helps” their ability to  prescribe drugs as those who said it “hinders” or “greatly hinders” them  (see table 4).
Some prescribers reported that the formulary they use helps them keep  current with new drugs and helps remove some of the pressures created by  direct-to-consumer advertising. Other prescribers reported that newly  approved drugs are not made available on the national formulary as soon as  they would like, and some reported their frustration with delays  experienced when certain formulary drugs must be approved by specially  trained physicians before they can be prescribed.
Prescribers we surveyed reported they were generally satisfied with the  national formulary. We asked prescribers who said that they had worked  for VA before the national formulary was established whether the current  formulary does a better job of keeping the list of drugs in the drug classes  from which they most frequently prescribe up to date, as compared with  the formulary they used to use. Three-quarters told us that they had worked  for VA before the national formulary was implemented in June 1997. Thirty- eight percent of these prescribers reported that the national formulary was  “better” or “considerably better” than previous formularies. About half (48  percent) indicated that the current formulary was “about the same” as the  one it replaced. Seven percent reported that it was “worse” or  “considerably worse” than previous formularies.
Few veterans have complained about not being able to obtain the drugs  they believe they need. At the VA medical centers we visited, patient  advocates told us that veterans made very few complaints concerning  their prescriptions. In its analysis of the patient advocates’ complaint  databases, IOM found that less than one-half of one percent of veterans’  complaints were related to drug access. IOM further reported that  complaints involving specific identifiable drugs often involved drugs that  are marketed directly to consumers, such as sildenafil (Viagra), which is  used to treat erectile dysfunction. Fifty-one percent of the prescribers in  our survey reported that over the past 3 years, an increasing number of  their patients have requested a drug they have seen or heard advertised in  the media. Our review also indicated that the few prescription complaints  made were often related to veterans’ trying to obtain “lifestyle” drugs or  refusals by VA physicians and pharmacists to fill prescriptions written by  non-VA health care providers. VA officials told us that VA does not fill  prescriptions written by non-VA-authorized prescribers, in part to ensure  that one practitioner manages a patient’s care.
section_title: Conclusions
paragraphs: Over the past 3½ years, VA has made significant progress in establishing its  national formulary, which has generally met with prescriber acceptance.  Prescribers reported that veterans are generally receiving the drugs they  need and that veterans rarely register complaints concerning prescription  drugs.
VA has not provided sufficient oversight, however, to ensure that VISNs and  medical centers comply with formulary policies and that the flexibility  given to them does not unduly compromise VA’s goal of formulary  standardization. Contrary to VA formulary policy, some facilities omitted  national formulary drugs or modified the closed drug classes. While adding  a limited number of drugs to supplement the national formulary is  permitted, as more drugs are added by VISNs, formulary differences among  facilities are likely to become more pronounced, decreasing formulary  standardization. While VA recognizes the trade-off between local flexibility  and standardization, it lacks criteria for determining the appropriateness of  adding drugs to supplement the national formulary. Consequently, VA  cannot determine whether the resulting decrease in standardization is  acceptable. Not all VISN directors have met their responsibilities for  implementing national formulary policy.
Inefficiencies that exist in the nonformulary drug approval processes  across the system can cause delays in making final treatment decisions. In  addition, the processes require health care provider time and energy that  might be better used for direct patient care. We believe a more efficient  nonformulary drug approval process could enable facilities to benefit from  lessons learned in other locations. Finally, VISNs lack the data needed to  analyze nonformulary drug requests to determine whether all approved  requests met approval criteria and all denied requests were appropriate.
section_title: Recommendations for Executive Action
paragraphs: In order to ensure more effective management of the national formulary,  we recommend that the Secretary of Veterans Affairs direct the Under  Secretary for Health to take the following actions:  Establish a mechanism to ensure that VISN directors comply with  national formulary policy.
Establish criteria that VISNs should use to determine the  appropriateness of adding drugs to supplement the national formulary  and monitor the VISNs’ application of these criteria.
Establish a nonformulary drug approval process for medical centers  that ensures appropriate and timely decisions and provides that  veterans for whom a nonformulary drug has been approved will have  continued access to that drug, when appropriate, across VA’s health care  system.
Enforce existing requirements that VISNs collect and analyze the data  needed to determine that nonformulary drug approval processes are  implemented appropriately and effectively in their medical centers,  including tracking both approved and denied requests.
section_title: Agency Comments and Our Evaluation
paragraphs: In commenting on a draft of this report, VA agreed with our findings and  concurred with our recommendations. VA highlighted key improvements  planned or already in progress that should further enhance the process.  VA’s actions to address our recommendations are summarized below.
VA plans to improve oversight at all organizational levels to help  facilitate consistent compliance with national formulary policy. In its  comments, VA discussed important components of improving  compliance with the national formulary, including examining data to  identify outliers. However, VA did not articulate a mechanism for  ensuring that its oversight results in consistent compliance, which may  reduce the effectiveness of its planned actions.
VA plans to establish criteria for VISNs to use to determine the  appropriateness of adding drugs to supplement the national formulary.  VA plans to establish steps for its nonformulary drug approval process  that all medical centers and VISNs must follow. However, in its  comments, VA did not specifically address how veterans would have  continued access to previously approved nonformulary drugs across  VA’s health care system. We believe such access is important.
VA plans to establish steps for reporting its nonformulary approval  activities. In its comments, VA did not explicitly include tracking of  denied requests as part of the nonformulary approval activities. We  expect that its nonformulary approval activities will include tracking  denied requests, as well as approved nonformulary drug requests, to  determine the appropriateness of all medical center prescribing  decisions.
VA plans to implement these corrective actions by June 2001. Its comments  are included in appendix II.
We are sending copies of this report to the Honorable Anthony J. Principi,  Secretary of Veterans Affairs; appropriate congressional committees; and  other interested parties. We will also make copies available to others upon  request.
Please call me at (202) 512-7101 if you or your staff have questions about  this report or need additional assistance. Another contact and staff  acknowledgments are listed in appendix III.
section_title: Scope and Methodology
paragraphs: To obtain policies and procedures from the 22 Veterans Integrated Service  Networks (VISN), we mailed a questionnaire to each of the 22 VISN  formulary leaders—pharmacists or physicians who serve on the  Department of Veterans Affairs’ (VA) Pharmacy Benefits Management  advisory board. To determine the extent to which VA health care providers  write prescriptions for national formulary drugs, we analyzed data from  VA’s national outpatient prescription database. To assess the  implementation of the national formulary and obtain firsthand opinions  about it, we interviewed medical and administrative staff at three VA  medical centers located in three different VISNs. To obtain VA health care  providers’ views on VA’s formulary, including whether or not it is  restrictive, we mailed a questionnaire to a nationally representative sample  of 2,000 VA health care prescribers. We also used information contained in  the Institute of Medicine’s Description and Analysis of the VA National  Formulary, issued in June 2000.
subsections: section_title: VISN Survey
paragraphs: To obtain policies and procedures from the 22 VISNs, we mailed a  questionnaire to VISN formulary leaders. We asked if there were VISN- wide policies for several areas, including adding drugs to the VISN  formulary, requesting nonformulary drugs, converting patients from one  drug to another, and tracking requests for nonformulary drugs. In addition,  we sought information on the number of drugs added to and dropped from  the VISN formulary, the number of requests for nonformulary drugs, and  the number of requests that were approved and denied. All 22 VISN  formulary leaders completed and returned questionnaires.
section_title: Outpatient Prescription Database Analysis
paragraphs: VA’s national database on outpatient prescriptions contains information for  each outpatient prescription filled at each VA medical center, including the  drug prescribed, date of the prescription, patient and prescriber identifiers,  medical center responsible for filling the prescription, and whether the  prescribed drug is a national formulary drug. We used this database to  develop a sample of VA health care providers who wrote prescriptions,   determine the total number of outpatient prescriptions filled at VISNs  and VA medical centers,   determine the number of filled outpatient prescriptions written for  national formulary drugs within a certain time frame, and  determine how many VISN formulary drug prescriptions were filled in  the three VISNs where we performed site visits.
We interviewed PBM headquarters officials who had either oversight or  maintenance responsibility for the database to help assess the validity and  reliability of the outpatient prescription data. We also performed our own  analytic checks of the data. We found that data critical to our analysis—the  data field indicating whether a prescription had been written for a national  formulary drug—contained errors. We worked with PBM officials to  correct the data, and they implemented a monthly routine to detect and  correct these errors in the future. We reran our data checks, verified that  the database had been corrected, and concluded that the data were  acceptable for the purposes of our work.
section_title: Site Visits
paragraphs: To assess formulary implementation at the local level, we interviewed  medical and administrative staff at three different VA medical centers—one  located in Biloxi, Mississippi (VISN 16); one in Gainesville, Florida (VISN  8); and one in Omaha, Nebraska (VISN 14). We selected these VISNs and  medical centers on the basis of formulary drug use from October through  December 1999, the period for which the most recent and complete data  were available at the time we did our work. For example, VISN 8 had the  highest percentage of prescriptions for national formulary drugs (93  percent), VISN 16’s percentage of national formulary drug prescriptions  was at the national average (90 percent), and VISN 14 had the lowest  percentage of prescriptions filled using national formulary drugs (88  percent).
section_title: Questionnaire to Prescribers
paragraphs: We mailed questionnaires to a representative sample of 2,000 VA health  care prescribers whose prescriptions had been dispensed from October 1  through December 31, 1999, to obtain their opinions and experiential data  on various aspects of VA’s national formulary. We drew this random sample  from VA’s most recent national outpatient prescription database—a data  file that contains information, including a prescriber identifier, on all  outpatient prescriptions filled in the VA health care system.
We mailed questionnaires to the entire sample of prescribers on April 17,  2000, with follow-up mailings on May 17 and June 21 to those who had not  responded by those dates. We accepted returned questionnaires through  September 1, 2000. Some prescribers’ responses indicated that they did not  write prescriptions for drugs; their prescription privileges were limited to  medical and surgical supplies, such as diabetic strips and food  supplements. Other returned questionnaires indicated that the addressee  had either left or retired from VA. These providers were thus considered  ineligible for our purposes and were removed from the sample.  Approximately 11 percent of the questionnaires were returned as  undeliverable, and we received no response from approximately 16 percent  of those to whom we mailed questionnaires. After adjusting the sample  accordingly, we determined the number of useable returned questionnaires  to be 1,217—a response rate of about 69 percent. (See table 5.)
Because this was a simple random sample, we believe that our results are  projectable to all of VA’s health care providers who have outpatient drug  prescribing privileges.
subsections: section_title: Sampling Errors
paragraphs: Surveys based on a sample are subject to sampling errors. Sampling error  represents the extent to which a survey’s results differ from what would  have been obtained had everyone in the universe of interest received and  returned the same questionnaire—in this case, all VA health care providers  who have outpatient drug prescribing privileges. Sampling errors have two  elements: the width of the confidence interval around the estimate  (sometimes referred to as the precision of the estimate) and the confidence  level at which the confidence interval is computed.
The confidence interval reflects the fact that estimates actually encompass  a range of possible values, not just a single value, or a “point estimate.” The  interval is expressed as a point, plus or minus some value. For example, in  our questionnaire, we asked prescribers, “To what extent does your VA  formulary contain the drugs you believe your patients need?” The  percentage of respondents who reported a “great extent” or “very great  extent” was 69.1. This particular question had a confidence interval of plus  or minus 2.6 percentage points. Thus, the “true” answer for this question  may or may not be 69.1 percent, but it has a high probability of falling  between 66.5 and 71.7 percent (69.1-percent point estimate, plus or minus  2.6 percentage points). Confidence intervals vary for individual questions  (depending upon how many of the individuals who could have answered a  question actually did so), but, unless otherwise noted, all percentages  presented in this report are within a range of plus or minus 3.5 percentage  points.
The confidence level is a measure of how certain we are that the “true”  answer lies within a confidence interval. We used a 95-percent confidence  level, which means that if we repeatedly took new samples of prescribers  from the October through December prescription database and performed  the same analysis of their responses each time, 95 percent of these samples  would yield estimates that would fall within the confidence interval stated.  In the previous example, this means that we are 95-percent certain that  between 66.5 and 71.7 percent of prescribers believe that the VA formulary  contains to a “great extent” or “very great extent” the drugs they believe  their patients need.
section_title: Nonsampling Errors
paragraphs: Surveys can also be subject to other types of systematic error or bias that  can affect results, known as nonsampling errors. One potential source of  nonsampling error can be the questionnaire itself. To ensure that questions  were clear and unbiased, we consulted with subject matter and  questionnaire experts within GAO and obtained comments from  individuals representing VA’s PBM and medical advisory panel, a working  group of 11 practicing VA physicians and 1 practicing Department of  Defense physician who help manage VA’s national formulary, as well as  individuals representing the Institute of Medicine. Finally, the  questionnaire was tested with 14 VA prescribers in VA medical centers in  four locations: Phoenix, Arizona; Washington, D.C.; Hampton, Virginia; and  Cincinnati, Ohio.
section_title: Prescribers’ Demographics
paragraphs: Prescribers were asked to provide demographic and VA employment  information as well as opinions about the relevance and usefulness of VA’s  formulary. On average, VA prescribers in our sample have worked for VA  for 11 years, with most of those years at their current medical facility.  Physicians and nurses constitute the largest groups of prescribers (65 and  15 percent, respectively), followed by physician assistants (7 percent) and  other allied health professionals, such as dentists (14 percent). Most of the  prescribers’ time working in VA is spent treating patients—on average, 26  hours each week. According to the national prescription file from which we  drew our sample, VA prescribers who completed our questionnaire  averaged 849 prescription fills from October through December 1999, the 3- month period we chose as the basis of our survey. The median number of  filled prescriptions was relatively low—252—because a few prescribers  had a large number of prescriptions filled during the period, while many  prescribers had only a few prescriptions filled.
section_title: Comments From the Department of Veterans Affairs
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: George Poindexter, Stuart Fleishman, Mike O’Dell, and Kathie Kendrick  made key contributions to this report.
section_title: Related GAO Products
paragraphs: State Pharmacy Programs: Assistance Designed to Target Coverage and  Stretch Budgets (GAO/HEHS-00-162, Sept. 6, 2000).
Prescription Drug Benefits: Applying Private Sector Management Methods  to Medicare (GAO/T-HEHS-00-84, Mar. 22, 2000).
VA Health Care: VA’s Management of Drugs on Its National Formulary  (GAO/HEHS-00-34, Dec. 14, 1999).
Prescription Drug Benefits: Implications for Beneficiaries of Medicare  HMO Use of Formularies (GAO/HEHS-99-166, July 20, 1999).
Defense Health Care: Fully Integrated Pharmacy System Would Improve  Service and Cost-Effectiveness (GAO/HEHS-98-176, June 12, 1998).
subsections: section_title: Ordering Information
paragraphs: The first copy of each GAO report is free. Additional copies of  reports are $2 each. A check or money order should be made out to  the Superintendent of Documents. VISA and MasterCard credit  cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address are  discounted 25 percent.
Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders by phone: (202) 512-6000 fax: (202) 512-6061 TDD (202) 512-2537 Each day, GAO issues a list of newly available reports and  testimony. To receive facsimile copies of the daily list or any list  from the past 30 days, please call (202) 512-6000 using a touchtone  phone. A recorded menu will provide information on how to obtain  these lists.
section_title: To Report Fraud, Waste, or Abuse in Federal Programs
paragraphs: Web site: http://www.gao.gov/fraudnet/fraudnet.htm  e-mail: fraudnet@gao.gov  1-800-424-5454 (automated answering system)