Department of Health and Human Services 
OFFICE OF 
INSPECTOR GENERAL 
RYAN WHITE EVALUATION SYSTEMS 
AIDS Drug Assistance Program 
JUNE GIBBS BROWN 
Inspector General 
AUGUST 1999 
OEI-05-98-00390  OFFICE OF INSPECTOR GENERAL 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, is to 
protect the integrity of the Department of Health and Human Services programs as well as the 
health and welfare of beneficiaries served by them.  This statutory mission is carried out through a 
nationwide program of audits, investigations, inspections, sanctions, and fraud alerts.  The 
Inspector General informs the Secretary of program and management problems and recommends 
legislative, regulatory, and operational approaches to correct them. 
Office of Evaluation and Inspections 
The Office of Evaluation and Inspections (OEI) is one of several components of the Office of 
Inspector General.  It conducts short-term management and program evaluations (called 
inspections) that focus on issues of concern to the Department, the Congress, and the public.  The 
inspection reports provide findings and recommendations on the efficiency, vulnerability, and 
effectiveness of departmental programs. 
OEI's Chicago Regional Office prepared this report under the direction of William Moran, 
Regional Inspector General.  Principal OEI staff included: 
REGION  HEADQUARTERS 
William Moran, Project Leader
 Elise Stein, Program Specialist 
Ann Maxwell, Lead Analyst
 Rada Spencer, Program Specialist 
Victoria Jacobs, Analyst
 Alan Levine, Program Specialist 
Nora Leibowitz, Analyst

Vince Greiber, New York Regional Office

To obtain copies of this report, please call the Chicago Regional Office at (312) 353-9867. 
Reports are also available on the World Wide Web at our home page address: 
http://www.dhhs.gov/progorg/oei  E X E C U T I V E   S U M M A R Y  
PURPOSE 
To examine the evaluation systems established by Ryan White AIDS Drug Assistance 
Programs (ADAPs) to monitor program compliance, measure client outcomes and 
measure overall effectiveness. 
BACKGROUND 
In 1990, Congress passed the Ryan White Comprehensive AIDS Resource Emergency 
(CARE) Act as a comprehensive response to the HIV epidemic and its impact on 
individuals, families, communities, cities and States.  The AIDS Drug Assistance Program 
provides medications to low-income people living with HIV/AIDS that are uninsured or 
under-insured and lack coverage for medications.  In fiscal year (FY) 1999, Congress 
appropriated $461 million of Title II funds to finance drug assistance programs.  This is an 
increase of 61 percent over FY 1998 when States received $285.5 million in Title II 
ADAP funds. 
The Ryan White CARE Act will be considered for re-authorization in the year 2000.  As 
Congress debates this re-authorization, questions are certain to be asked about the 
effectiveness and impact of the programs funded.  This inspection examines the capacity of 
AIDS Drug Assistance Programs to answer those questions. 
We conducted an examination of the evaluation mechanisms used by eight Title II ADAPs 
through an extensive documentation review and on-site interviews.  The States visited 
represent 68 percent of the living AIDS cases reported in 1997 and 61 percent of the Title 
II funding for FY 1998. 
FINDINGS 
The AIDS Drug Assistance Programs in the Eight States We Visited Have 
Sufficient Programmatic and Fiscal Monitoring Mechanisms in Place 
The ADAP administrators closely monitor program and fiscal activity through the review 
of routine reports.  Fiscal control over expenditures is further maintained by employing a 
variety of cost containment strategies to purchase the drugs at the lowest possible price. 
Program enrollment is monitored through the use of standardized enrollment procedures 
1 
Evaluation Systems: ADAP  OEI-05-98-00390  and centralized processing.  All ADAPs rely on regular, automated matches with Medicaid 
databases as the primary tool to assure that ADAP funds are being used as the payor of 
last resort. 
While Most ADAPs Rely on Process Measures to Gauge Program Progress, a Few 
are Developing Ways to Measure Client Outcomes 
Most ADAP administrators reported that they rely on program utilization data to assess 
the success of their program.  However, an increasing number of administrators are taking 
on the responsibility of monitoring program impact to improve program efficiency and 
effectiveness and to respond to demands for greater accountability.  On the other hand, 
some ADAP administrators are of the opinion that measuring outcomes is not an 
appropriate way to expend limited resources.  They argue that it is best to rely on medical 
research professionals to measure clinical outcomes and that requiring ADAPs to measure 
outcomes would duplicate work already being done in the medical field. 
ADAPs are Conducting Minimal Assessments of Whether They are Reaching 
Populations in Need 
All ADAPs indicated that they match the demographics of enrolled clients to the 
demographic profile of the epidemic as a measure of how they are meeting population 
needs.  Despite the understood disadvantages of relying on this method to assess program 
effectiveness, ADAPs have not found other, more sophisticated means to assess whether 
they are reaching populations in need. 
RECOMMENDATIONS 
To Promote Outcome Measurement, HRSA Should Encourage ADAPs to Compare 
Utilization Data to Recommended Clinical Guidelines 
The Health Resources and Services Administration (HRSA) should require drug assistance 
programs to review whether drugs are dispensed in accordance with the Guidelines for the 
Use of Antiretroviral Agents in HIV Infected Adults and Adolescents.  Requiring ADAPs 
to compare their utilization data to these recommended clinical guidelines provides a 
means to extrapolate medically indicated client health outcomes from utilization data. 
Given that ADAP clients may also be receiving HIV-related drugs from other sources, 
HRSA should encourage ADAPs to expand their relationships with Medicaid to enable an 
annual exchange of information on prescription drugs dispensed to ADAP clients.. 
While not a rigorous scientific measure of program impact, with or without secondary 
data sources, this comparison would provide a general sense of ADAPs contribution to 
2 
Evaluation Systems: ADAP  OEI-05-98-00390  client health outcomes.  It would also allow for the analysis of patterns and trends that 
would suggest more in-depth explorations of client outcomes and program impact. 
Finally, it is a logical next step in understanding the impact of the ADAP program.  The 
current Ryan White application guidance requires ADAPs to report the number of clients 
receiving protease inhibitors.  The HRSA could simply expand this requirement to include 
a broader range of drugs along with a narrative comparison of how utilization patterns 
match the recommended clinical guidelines. 
The HRSA Should Encourage Collaborative Efforts among Ryan White Titles to 
Monitor Client Adherence to Drug Regimens 
Since positive health outcomes cannot be expected unless clients adhere to prescribed 
drug regimens, HRSA should work with adherence initiatives where they exist and 
encourage other Title I and Title II grantees, in consultation with ADAP, to promote and 
evaluate adherence.  Given the difficulty of this complex behavioral and socio-economic 
issue, initiatives will likely require a combination of local resources. 
The HRSA Should Instruct ADAPs to Collaborate with Title I and Title II Needs 
Assessment Efforts to Assure That They Explicitly Evaluate Medication Therapy 
Needs 
To better assess unmet need, ADAP liaisons should work with and support current Title I 
or Title II needs assessment efforts.  The ADAP liaisons should also ensure that needs 
assessment efforts actively solicit information from current ADAP clients as well as 
HIV-infected persons currently not enrolled in ADAP. 
AGENCY COMMENTS 
The HRSA provided comments on the draft report in which they did not concur with the 
first and third recommendations.  The complete text of HRSAs comments can be found in 
Appendix A. 
Regarding the first recommendation, HRSA stated that they did not have the statutory 
authority to require ADAPs to match client utilization data to NIH/PHS guidelines.  The 
HRSA also stated that the data available to ADAPs would be insufficient to make this 
analysis meaningful. 
In response to HRSAs concern regarding appropriate authority, the recommendation was 
altered to remove the implication of a statutory requirement.  However, developing 
outcome measures for ADAP remains an important step in the evolution of the program. 
Matching utilization patterns to the National Institutes for Health/Public Health Service 
(NIH/PHS) Guidelines for the Use of Antiretroviral Agents in HIV Infected Adults and 
Adolescents would provide a general sense of ADAPs contribution to client health 
3 
Evaluation Systems: ADAP  OEI-05-98-00390  outcomes.  It is also a logical first step in outcome development.  The HRSA already 
requires ADAPs to present a break-out of the utilization of protease inhibitors in their 
annual application for funds.  Our recommendation would simply broaden this requirement 
to include an analysis of utilization by nationally recommended drug therapies. 
We appreciate HRSAs concerns regarding the impact of incomplete utilization data.  It is 
for this reason that we suggested ADAPs expand their current relationships with Medicaid 
to include a data match on drug utilization.  This data sharing agreement could be based 
on the model data sharing agreement disseminated as part of  the 1998 HRSA, Health 
Care Financing Administration (HCFA), and Centers for Disease Control (CDC) inter-
agency agreement to promote the development of integrated public health information 
systems. 
Even with data limitations, the comparison of utilization data to drug therapy guidelines 
provides a sense of program contribution.  Our recommendation to use this limited 
measure of program outcome builds on the United Way model which allows programs to 
take credit for their contribution to client outcomes while recognizing the practical 
limitations of assessing attribution in the absence of complete data. 
The HRSA also did not concur with the third recommendation that ADAPs should 
collaborate with Title I and Title II needs assessment efforts.  They stated that ADAP was 
not the appropriate entity to take the lead in collaborative efforts with the other Ryan 
White programs. 
It was not our intention to imply that ADAP should take the lead in collaborating between 
the Ryan White programs.  Rather, our recommendation directed ADAPs to more closely 
participate in the required Title I and Title II needs assessment efforts in order to have 
their evaluation needs better met.  The HRSAs stated intention to include instructions in 
the Title I application guidance regarding collaboration with ADAP reflects our intention 
to promote the level of meaningful collaboration between ADAP and the other CARE Act 
programs. 
4 
Evaluation Systems: ADAP  OEI-05-98-00390  T A B L E   O F   C O N T E N T S  
PAGE

EXECUTIVE SUMMARY  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

INTRODUCTION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  6

FINDINGS

Sufficient Monitoring Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Most use Utilization Data, Few Developing Outcome Measures . . . . . . . . . . . . . . . . . .  13

Minimal Assessments of Unmet Need  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  16

RECOMMENDATIONS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  17

AGENCY COMMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  22

APPENDIX

A: Agency Comments  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  24

B: Related Office of Inspector General Reports  . . . . . . . . . . . . . . . . . . . . . . . . . . .  32

5 
Evaluation Systems: ADAP  OEI-05-98-00392  I N T R O D U C T I O N  
PURPOSE 
To examine the evaluation systems established by Ryan White AIDS Drug Assistance 
Programs (ADAPs) to monitor program compliance, measure client outcomes and 
measure overall effectiveness. 
BACKGROUND 
This study complements our reports entitled Ryan White Evaluation Systems, Title I: 
Grants to Metropolitan Areas (OEI-05-98-00392) and Ryan White Evaluation Systems, 
Title II: Grants to States (OEI-05-98-00393). 
Ryan White CARE Act 
In 1990, Congress passed the Ryan White Comprehensive AIDS Resource Emergency 
(CARE) Act as a comprehensive response to the HIV epidemic and its impact on 
individuals, families, communities, cities and States.  The Ryan White programs provide 
health care and support services to persons with HIV/AIDS who would otherwise not 
have access to care.  The Health Resources and Services Administration (HRSA) directs 
Ryan White resources to various local entities through the Act's four titles and Part F. 
The Ryan White programs constitute slightly over $1 billion of the $6.8 billion in 
HIV/AIDS expenditures included in the U.S. Department of Health and Human 
Services (DHHS) fiscal year (FY) 1998 budget.  The bulk of Ryan White funds are 
allocated through Titles I and II of the Act.  In FY 1998, nearly 90 percent of the Acts 
funds were awarded to Title I and Title II grantees. 
The Ryan White CARE Act was re-authorized in 1996 through the year 2000.  Changes to 
the Act in the re-authorization included a directive to Title I and Title II grantees to 
prioritize a portion of their funds for emerging populations. 
AIDS Drug Assistance Program 
Title II of the Ryan White CARE Act provides grants to States to improve the quality, 
availability and organization of health care and support services for individuals and 
families with HIV/AIDS.  Section 2616(a) of the CARE Act requires States to use a 
portion of their Title II funds to establish an AIDS Drug Assistance Program.  States are 
allowed to determine unique financial and medical eligibility for the ADAP in their State. 
6 
Evaluation Systems: ADAP  OEI-05-98-00390  States are also allowed to determine what drugs will be included in their formularies, how 
the drugs will be purchased and how the drugs will be distributed to clients. 
In FY 1999, Congress appropriated $461 million in Title II funds to be used for ADAP. 
This is an increase of 61 percent over FY 1998 when States received $285.5 million in 
Title II ADAP funds.  These ADAP funds are typically augmented by contributions from 
State general revenue funds and, to a lesser extent, Title I grants.  The mission of ADAP is 
to serve low-income people living with HIV/AIDS that are uninsured or under-insured and 
lack coverage for medications. 
The need for financial assistance to support expensive HIV/AIDS drug regimens was first 
identified in 1987 when Congress provided an emergency appropriation of $30 million to 
States.  Beginning in late 1995, combination antiretroviral therapies, which include 
protease inhibitors, emerged as a new standard of care and ADAPs experienced explosive 
growth in enrollment and utilization.  In November 1997, DHHS published Guidelines for 
the Use of Antiretroviral Agents in HIV Infected Adults and Adolescents developed by the 
Panel on Clinical Practices for Treatment of HIV Infection convened by the Department 
and the Henry J. Kaiser Family Foundation.  These guidelines encouraged aggressive 
antiretroviral therapy for a broad range of HIV infected individuals, creating even greater 
demand for the program.  In 1998, ADAPs served approximately 108,000 people with 
HIV/AIDS. 
Ryan White CARE Act Evaluation Requirements 
Section 2616(c)(5) of the Ryan White CARE Act requires that States document the 
progress made in making therapeutics available to individuals eligible for assistance. 
Section 2616(d) requires that the Secretary review the current status of State drug 
reimbursement programs, assess the barriers to the expanded availability of treatments, 
and examine the extent to which States coordinate with other Ryan White grantees to 
reduce barriers to the expanded availability of the treatments. 
The Ryan White CARE Act includes caps on the amount of funds Title II grantees, 
inclusive of ADAP, can spend on administrative and planning/evaluation functions. 
Grantees cannot exceed a 10 percent administrative cap and a 10 percent 
planning/evaluation cap, the combined total of which cannot exceed 15 percent.  Grantees 
can pay for needs assessments, program evaluation and service delivery assessments out of 
their planning/evaluation cap. 
HRSAs Current Evaluation Requirements 
In the FY 1998 Application Guidance, States were asked, for the first time, to complete an 
ADAP State Profile.  The FY 1998 guidance also required a narrative description of 
7 
Evaluation Systems: ADAP  OEI-05-98-00390  agency administration, funding sources, formulary, eligibility criteria, client utilization, 
cost-savings strategies and coordination with other Ryan White funded services. 
The FY 1998 guidance also asked ADAPs to: 1) describe their system for tracking 
utilization and expenditure information on a monthly basis, 2) describe any significant 
changes in client and drug utilization, 3) compare the demographic characteristics of 
ADAP clients to the demographic characteristics of the HIV epidemic of their State and 
discuss any discrepancies, and 4) describe the barriers to access and plans to enroll under-
served populations.  The FY 1999 application asks that ADAPs provide a brief overview 
of client enrollment and utilization trends.  It does not provide direction on how this 
should be achieved. 
The ADAPs are also required, as a condition of their award, to file monthly and annual 
reports with HRSA.  The monthly reports track enrollment and average monthly 
utilization.  They specify a break-out for the utilization of protease inhibitors.  The annual 
administrative report requires ADAPs to document the services they provide and the 
characteristics of clients receiving services. 
As a program administered by the State under Title II, ADAPs are required to outline 
goals and objectives for the upcoming year in an Implementation Plan.  These must be 
accompanied by the service or outcome measure used to evaluate goal attainment.  The 
application guidance includes people served and units of service as examples of 
outcome/service measures.  The achievement of these goals must be reported in the next 
years application. 
Re-authorization 
The Ryan White CARE Act of 1990 will be considered for its second re-authorization in 
2000.  As Congress debates this re-authorization, questions are certain to be asked about 
the effectiveness and impact of the Ryan White programs.  Grantees and administrators 
will also be asked to account for whether and how the programs have evolved to meet the 
shifting needs of the communities served. 
SCOPE AND METHODOLOGY 
This inspection is an examination of the evaluation systems established by State AIDS 
Drug Assistance Programs.  Specifically, this report documents the mechanisms ADAPs 
use to monitor their program compliance, measure client outcomes, and assess their 
effectiveness in meeting the evolving needs of the population affected by the epidemic. 
This evaluation does not judge the compliance, measure the outcomes or assess the 
effectiveness of the ADAPs, but rather judges whether the drug assistance programs have 
8 
Evaluation Systems: ADAP  OEI-05-98-00390  and appropriately utilize the tools to perform these evaluation functions.  This study also 
does not examine HRSAs monitoring and evaluation of the ADAPs. 
For the purposes of this analysis, we define compliance as serving the population, 
providing the services and spending the resources as agreed to in return for the award of 
program funds.  Outcomes are defined as measures of the benefits clients derived as a 
result of the program.  Finally, we define overall effectiveness as how well the mix of 
services meets the evolving needs of the population affected by the epidemic. 
A purposive sample of eight Title II drug assistance programs were selected for this 
evaluation.  The sampled ADAPs consist of New York, New Jersey, Maryland, Florida, 
Illinois, Texas, California and the District of Columbia.  The sites were selected based on 
the number of reported AIDS cases, FY 1998 Ryan White funding, high incidence of 
intravenous drug use transmission, the demographics of the AIDS/HIV population, and 
the regional diversity of the sites.  These seven States and DC represent 68 percent of the 
living AIDS cases reported in 1997 and 61 percent of the Title II funding for FY 1998. 
For each of the eight drug assistance programs selected, we conducted an extensive 
document review.  We reviewed their  FY 1998 grant application, templates of their 
programmatic and fiscal reports, enrollment guidelines, client satisfaction surveys and any 
other documents pertaining to their use of outcomes or evaluations of the overall 
effectiveness of the program. 
We also conducted on-site interviews with the sampled ADAPs between October and 
December 1998.  We interviewed the ADAP administrator and any staff responsible for 
monitoring, outcomes or evaluation. 
In addition to our survey work, we interviewed knowledgeable organizations such as the 
Center for Disease Control (CDC), the HIV/AIDS Bureau of the Health Resources and 
Services Administration and the Office of HIV/AIDS Policy.  These meetings were 
complimented by background materials on the Ryan White program provided by HRSA. 
We conducted our review in accordance with the Quality Standards for Inspections issued 
by the Presidents Council on Integrity and Efficiency. 
9 
Evaluation Systems: ADAP  OEI-05-98-00390  F I N D I N G S  
The AIDS Drug Assistance Programs in the eight States we 
visited have sufficient programmatic and fiscal monitoring 
mechanisms in place 
Routine program and fiscal reports 
The ADAP administrators monitor program activity through the review of routine 
programmatic reports.  These activity reports are typically reviewed on a monthly and, 
sometimes weekly, basis.  They contain aggregate data on program processes such as the 
number of clients enrolled, newly approved applications, the number of clients served, the 
number of prescriptions filled by drug and the number of clients receiving each covered 
drug.  One ADAP receives a standard set of 13 tables every month that includes an 
unduplicated client count, new enrollees by ethnic group and county, CD4 count by age 
group, client income by county, insurance status, etc. Typically, the point-in-time data 
provided on a monthly basis is analyzed longitudinally to detect enrollment and utilization 
trends and project utilization needs for the future. 
The ADAPs ensure the fiscal integrity of their programs by monitoring regular reports of 
their program expenditures.  Weekly batches of pharmacy claims are bundled into monthly 
expenditure reports.  These financial reports usually provide an accounting of expenditures 
during the reporting month and year-to-date figures.  Some ADAPs  produce more 
complex financial reports that include items such as average cost per prescription, average 
cost per client and average fills per client.  Expenditure reports are typically accompanied 
by supporting documentation such as the payroll register, pharmacy claims for drugs 
dispensed and orders for drugs. 
Centralized databases facilitate routine reporting, but remain under-utilized 
All of the programmatic and fiscal reports used to monitor program performance are 
created off of centralized computer data management systems.  These administrative data 
systems track confidential client demographic and service information as well as claims 
information.  One State maintains 1,000 fields on each client including basic 
demographics, income information, insurance, CD4 count, viral load, disease staging 
information, treatment history, risk factor, and so forth. 
Despite the rich resource of data contained in these centralized data systems, some 
administrators do not fully utilize the capabilities at their disposal.  Most of the ADAP 
10 
Evaluation Systems: ADAP  OEI-05-98-00390  administrators indicated that their centralized data systems allow instant and easy access to 
a variety of up-to-the-minute, client level program data and analyses.  Further, they stated 
that the databases provide the capacity to respond to emerging program and policy issues 
with quantitative analysis.  However, only half of the ADAPs mentioned actually using 
their databases in this manner.  For example, one of these ADAPs recently used their 
database to respond to concerns regarding the number of pediatric cases, the number of 
women served, and high utilization rates in particular counties and pharmacies.  Other 
administrators feel that while they are collecting all of the necessary programmatic data, 
they have yet to realize the full potential of the data to assist in program management and 
evaluation. 
Standard eligibility criteria are uniformly enforced 
Standard enrollment forms, and training on how to appropriately complete them, are used 
to ensure that all required eligibility information is obtained in a consistent manner. 
Standard forms enhance the efficiency of enrollment procedures and the ability to evaluate 
the accuracy of the data. These forms are extensive and typically request demographic, 
financial and medical information in accordance with State eligibility rules.  Appropriate 
support documentation is also required such as personal identification cards, income 
verification statements and a physician signed medical certification of HIV diagnosis.  One 
State AIDS Drug Assistance Program trimmed 2,000 duplicates from its rolls last year 
when it standardized its eligibility screening and enrollment practices. 
All ADAPs centrally review enrollment forms and the supporting documentation for 
completeness and accuracy.  The central screening and processing of enrollment 
applications helps ensure a uniform enforcement of the program eligibility rules.  It also 
helps prevent duplication of services and other fraud.  Some ADAPs have further 
centralized the enrollment process by requiring all application forms to be sent to the 
central administration office for initial eligibility determination. 
Seven of the ADAPs visited redetermine eligibility on an annual or semi-annual basis.  One 
AIDS Drug Assistance Program only initiates redetermination procedures if alerted to a 
potential change in Medicaid status or other eligibility criteria. 
Payor of last resort status ensured mainly through automated Medicaid matches 
All ADAPs rely on regular, automated matches with Medicaid databases as the primary 
tool to assure that ADAP funds are being used as the payor of last resort.  The ADAPs 
focus their enforcement energies on Medicaid since they feel it is the most likely source of 
overlapping funding and they have the greatest ability to enforce compliance.  Other 
techniques used to enforce payor of the last resort status include explicit instructions 
11 
Evaluation Systems: ADAP  OEI-05-98-00390  written into pharmacy contracts and highlighted on enrollment forms and soliciting 
information regarding other third-party insurance on client enrollment forms. 
Medicaid matches are a part of the standard ADAP enrollment process.  In the majority of 
ADAPs we visited, Medicaid receipt disqualifies applicants from ADAP eligibility.  In 
order to enforce this, ADAPs automatically cross check all applicants against the Medicaid 
rolls.  The ADAPs we visited all have on-line capacity to perform this check.  The ease 
and speed with which this automated match occurs allows ADAPs to perform this check 
with every application at the time of application and without compromising a clients 
access to necessary drugs. 
Periodic matches of enrollment files to Medicaid membership files are also conducted to 
monitor client Medicaid eligibility status which can fluctuate over time.  It is typical 
procedure to approve applicants for ADAP who appear to be Medicaid eligible while also 
helping them apply for Medicaid benefits. Medicaid eligible HIV-infected persons can also 
receive ADAP to aid in spend down or cost sharing requirements of Medicaid or if they 
have limited Medicaid coverage.  The periodic checks against the Medicaid rolls allows 
the ADAPs to monitor on-going client transitions from Medicaid Pending status to full 
Medicaid benefits so the appropriate program is billed.  The ADAPs vary tremendously in 
how often they perform this on-going, automated check.  Some ADAPs check on a 
monthly basis, some weekly, and others check with every prescription request. 
One weakness in the current system is the difficulty of recouping payments from Medicaid 
for those ADAP clients who cycle onto Medicaid while still receiving benefits through 
ADAP.  Since the matches do not occur on a daily basis, and some ADAPs go as long as a 
month before checking the Medicaid rolls, there is the potential for ADAPs to incur costs 
for Medicaid-covered services.  Of the eight ADAPs interviewed, only one mentioned 
having a way to bill Medicaid retroactively.  Upon discovering an overpayment, this drug 
assistance program withholds future payments to the respective pharmacy for the amount 
billable to Medicaid.  One other ADAP indicated in their application for funds that they 
have negotiated a way to retroactively bill Medicaid. 
Cost containment strategies are used to limit drug expenses 
Fiscal control over expenditures is further maintained by employing a variety of cost 
containment strategies to purchase the drugs at the lowest possible price.  Six of the eight 
ADAPs purchase drugs through Section 602 of the Veterans Health Care Act.  This 
allows them to purchase drugs at a discounted Public Health Service (PHS) price or 
access a rebate option.  The Office of Drug Pricing that administers the Federal Drug 
Pricing Program for purchasing drugs at the PHS price requires ADAPs to follow 
established regulations in order to qualify.  These regulations were designed to avoid the 
diversion of drugs purchased under this program to non-eligible patients. Thus, for these 
drug assistance programs, there is an additional level of Federal control. 
12 
Evaluation Systems: ADAP  OEI-05-98-00390  For those States where the Federal drug discount pricing program is not an available 
option due to program structure, multiple other strategies are pursued.  These include 
negotiating savings off of the Average Wholesale Price (AWP) and voluntary rebate 
programs.  The States we visited have negotiated from 10 to 15 percent off of the AWP. 
These States also typically search for further savings through avoiding or reducing 
infrastructure costs potentially incurred through storing and distributing the drugs. 
Despite the efforts individual State ADAPs are making to ensure they are purchasing 
drugs at the lowest possible price, variations in the purchase prices States are paying 
remain.  Each AIDS Drug Assistance Program is allowed to craft their own cost 
containment strategy.  This allows for maximum State flexibility, but it also means States 
have only their own resources and ingenuity to rely upon to overcome the barriers to 
accessing the lowest drug prices.  Some ADAPs suggested that HRSA and the Health 
Care Financing Administration (HCFA) assume a leadership role at the national level in 
order to obtain the best possible drug prices for the ADAP program. 
While most ADAPs rely on process measures to gauge 
program progress, a few are developing ways to measure 
client outcomes 
The AIDS drug assistance program was conceived as a means to provide prescription 
drugs to HIV infected individuals who were uninsured or underinsured.  In response to 
this and tremendous funding pressures, some ADAPs have limited their activities to the 
process of supplying prescription drugs to HIV-infected persons in need.  They view the 
program primarily as a financing and distribution mechanism.  However, an increasing 
number of administrators are taking on the responsibility of monitoring program impact to 
improve program efficiency and effectiveness and to respond to demands for greater 
accountability. 
Most ADAPs track client utilization of drugs to monitor program success 
Most ADAP administrators reported that they rely on the utilization data reported in their 
regular program reports to assess the success of their program.  The utilization reports 
typically provide, in aggregate, the number of clients actively enrolled and the number of 
drugs prescribed by drug class or specific drug.  Often they will break out utilization by 
gender, race/ethnicity and age.  A few ADAPs also regularly track enrollment and 
utilization trends by sub-population or targeted neighborhoods.  Utilization goals are 
outlined in the Implementation Plan in each years application for funding. 
A few of the AIDS Drug Assistance Programs currently, or plan to, take this review of 
utilization data a step further by using the data as a basis to extrapolate health outcomes. 
13 
Evaluation Systems: ADAP  OEI-05-98-00390  This analysis involves reviewing drug utilization data to determine how closely the drugs 
dispensed match the aggressive drug regimens recommended by the National Institutes of 
Health/Public Health Service (NIH/PHS) Guidelines for the Use of Antiretroviral Agents 
in HIV Infected Adults and Adolescents1.  If the analysis reveals that ADAP prescriptions 
accurately mirror NIH/PHS guidelines, these ADAP administrators assume their clients 
are benefitting from the positive health outcomes attributed to these drug therapies in the 
medical research.  One ADAP conducts a quarterly analysis comparing drug usage to 
NIH/PHS guidelines.  Another ADAP is planning on examining prescribing patterns as 
well as utilization patterns by reviewing a random selection of ADAP clients medical 
charts. 
A few ADAPs are planning on tracking individual client health outcomes to 
determine program success 
While the majority of AIDS Drug Assistance Programs are currently using process 
measures to track program functioning and outcomes, a few ADAPs are planning on 
incorporating client health outcomes into their tracking systems in order to document 
program impact.  Outcome measures under consideration include CD4 counts, viral loads, 
opportunistic infections, and hospital admissions.  One ADAP is struggling to move 
beyond strict measures of health outcomes by developing broader measures of quality of 
life.  They are interested in tracking how long their clients are ambulatory, to what extent 
their clients can take care of themselves, the number and length of hospitalizations, and the 
development and duration of opportunistic infections. 
Most ADAPs developing outcome measures plan to track the data for these measures by 
augmenting their current client-level, administrative data systems.  They plan to request 
pertinent clinical information on the original enrollment form and subsequent forms 
submitted for re-eligibility determination.  A few ADAPs are already capturing CD4 
counts in this manner, but none are currently tracking the variations over time as a 
measure of client health.  The clinical information obtained on the enrollment form will be 
input into the current database where it can be linked to existing client utilization data for 
analysis.  One ADAP indicated they would be tracking outcome measures in another year. 
Other ADAPs felt the development of outcome measures would take a few years. 
One of the ADAPs planning on developing outcome measures also plans to assess client 
outcomes by means of an independent evaluation.  Over the next year and a half, they plan 
1	 These guidelines contain recommendations for the clinical use of antiretroviral agents in the 
treatment of HIV-infected adults and adolescents. A separate document contains 
pediatric-specific issues related to antiretroviral therapy. These guidelines were developed to 
serve as a companion document to the therapeutic principles formulated by the National Institute 
of Health Panel to Define Principles of Therapy of HIV infection. 
14 
Evaluation Systems: ADAP  OEI-05-98-00390  to review a random selection of clients.  This special project will focus on clients clinical 
outcomes, length of participation in the ADAP program and the extent to which non-
adherence to the prescribed drug regimen negatively impacts health outcomes. 
Some ADAP administrators feel that assessing client health outcomes is beyond 
the scope of ADAP 
Some ADAP administrators feel that measuring outcomes is not an appropriate way to 
expend limited resources. They believe the process measures tracked in their utilization 
reports are sufficient to judge their programs performance against the goal of efficiently 
distributing prescription drugs on the formulary to uninsured or underinsured HIV-
infected individuals.  As one ADAP administrator stated, We are just responsible for 
getting drugs to folks who need them and this is where our limited money should go. 
Some ADAP administrators also argue that it is best to rely on medical research 
professionals to measure clinical outcomes.  According to these respondents, in order to 
get reliable results on clinical outcomes, outcome evaluations should be controlled clinical 
studies.  These types of studies are best directed by skilled researchers who have the 
expertise to properly conduct such evaluations and access to the necessary resources. 
Finally, several ADAP administrators feel that requiring ADAPs to measure client 
outcomes would duplicate work completed by the medical evaluation field.  Medical 
studies already exist on the effects of the drugs since all approved drugs are put through 
extensive clinical trials prior to Food and Drug Administration approval.  These studies 
demonstrate what the expected outcomes for a given drug regimen should be.  Only drugs 
that are proven to reduce HIV/AIDS symptoms and prolong life are placed on the 
formulary to be distributed by ADAP.  Thus, the ADAP administrators reason, as long as 
the program ensures that it is fulfilling prescrip`tions according to the prescribed 
guidelines, they should be able to assume that their medical outcomes reflect the outcomes 
demonstrated in medical research.  As one administrator put it, ADAP outcomes can only 
be the outcomes predicted by medical studies. 
15 
Evaluation Systems: ADAP  OEI-05-98-00390  ADAPs are conducting minimal assessments of whether they 
are reaching populations in need 
While all of the sample ADAPs conduct periodic matches of client and 
epidemiological data, they conduct little additional research to determine whether 
they are reaching the populations in need 
All ADAPs indicated that they match client demographics to the demographic profile of 
the epidemic to measure how they are meeting population needs.  The majority of ADAPs 
perform this analysis annually.  Some ADAPs perform this analysis more frequently, on an 
as needed basis.  Typically, the analysis consists of comparing the demographics of 
enrolled clients to AIDS surveillance data broken out by gender, race/ethnicity and age.  A 
few of the ADAPs we visited make use of HIV prevalence estimates.  The two data sets 
are compared in order to locate any discrepancies.  These discrepancies are explored as 
possible areas of unmet need to address in the future. 
However, many ADAP administrators expressed great reservations about judging the 
effectiveness of their programs by this simple methodology.  Their major concern is that 
Ryan White AIDS Drug Assistance Programs are not the only source of funding for 
HIV/AIDS medication.  A certain proportion of HIV/AIDS cases access drugs through 
State Medicaid programs, private insurance or clinical trials.  One ADAP study found that 
90 percent of the HIV-infected residents in an area with disproportionately low ADAP 
enrollment rates were enrolled in Medicaid. 
Despite the disadvantages of relying on an epidemiological match to assess program 
effectiveness and growing pressure to better assess the needs of traditionally underserved 
populations, ADAPs have not found other, more sophisticated means to assess whether 
they are reaching populations in need.  The sources ADAPs mentioned they rely on for 
information regarding unmet need and access to services include activist groups, the 
ADAP advisory boards, and the Statewide Coordinated Statement of Needs assessment 
of overall State needs.  Only three of the ADAPs we visited indicated receiving 
information from planning council or consortia needs assessment efforts. 
16 
Evaluation Systems: ADAP  OEI-05-98-00390  R E C O M M E N D A T I O N S  
To promote outcome measurement, HRSA should encourage 
ADAPs to compare utilization data to recommended clinical 
guidelines 
As a first step in gathering a better sense of program impact, HRSA should encourage 
drug assistance programs to review whether drugs are dispensed in accordance with the 
NIH/PHS Guidelines for the Use of Antiretroviral Agents in HIV Infected Adults and 
Adolescents.  To the extent that the comparison reveals that drug distribution accurately 
mirrors the guidelines, ADAP administrators and HRSA can assume that clients are 
benefitting from the positive health outcomes attributed to specific drug therapies 
evaluated in medical research studies. 
To create a more meaningful picture of program impact, HRSA should also encourage 
ADAPs to expand their relationships with Medicaid to enable an annual exchange of 
information on HIV-related drugs dispensed to ADAP clients.  The resulting data match 
would render a more complete picture of clients access to prescription drugs.  It would 
also be helpful, where possible, to establish data exchanges with the other drug 
reimbursement sources such as private insurance companies and other pharmacy assistance 
programs. 
Most ADAPs already have data relationships with State Medicaid offices in order to verify 
payor of last resort status.  The HRSA could encourage ADAPs to expand these data 
sharing arrangements to include an interchange of utilization data.  The model data sharing 
agreement disseminated as part of the 1998 inter-agency agreement between HRSA, 
HCFA, and the CDC could be used.  This inter-agency effort was undertaken to facilitate 
data sharing between State Medicaid and other health agencies to support program 
monitoring, promote the development of common performance measures across multiple 
programs and improve the quality of care for vulnerable populations. 
While matching ADAP distribution patterns with the NIH/PHS guidelines would 
inevitably lack some information regarding drug utilization, it would, nonetheless, provide 
a general sense of ADAPs contribution to client health outcomes.  This approach to 
outcome measurement corresponds with a highly regarded method of outcome 
development promoted by the United Way.  This approach recognizes the practical 
limitations faced by program administrators searching for a way to gage program impact. 
These limitations typically mean that rigorous program impact evaluations are beyond the 
means and scope of these agencies.  Thus the United Way endorses using outcome 
measures of contribution as the first step in building meaningful outcome measures. 
17 
Evaluation Systems: ADAP  OEI-05-98-00390  Beyond providing a broad measure of contribution, this comparison would also allow for 
the analysis of patterns and trends that would suggest more in-depth explorations of client 
outcomes and program impact.  For example, if disparities in the quality of care are 
detected in the client population through this broad analysis, ADAPs could engage in a 
more in-depth examination of utilization in specific sub-populations.  This data and 
analysis could be provided  to clinical providers as a form of feedback.  The ADAPs might 
also want to take the next step of surveying physician prescribing practices. 
This approach to outcome measurement also integrates the United Way philosophy of 
using data sources readily available to agencies.  Most AIDS drug assistance programs are 
already relying on client utilization data as a means to gage program success.  Asking 
them to analyze these utilization patterns against a national standard is a logical next step 
to a more sophisticated understanding of program impact.  In fact, some ADAPs already 
routinely perform this analysis.  All of the ADAPs that we interviewed have utilization 
data at their disposal and the capacity to perform this analysis.  Further, the current Ryan 
White application guidance already requires ADAPs to report the number of clients 
receiving protease inhibitors.  The HRSA could simply expand this requirement to include 
a broader range of drugs along with a narrative comparison of how utilization patterns 
match the recommended clinical guidelines. 
HRSA should encourage collaborative efforts among Ryan 
White Titles to monitor client adherence to drug regimens 
Since positive health outcomes cannot be expected unless clients adhere to prescribed 
drug regimens, HRSA should work with adherence initiatives where they exist and 
encourage other Title I and Title II grantees, in consultation with ADAPs, to promote and 
evaluate adherence.  Given the difficulty of this complex behavioral and socio-economic 
issue, initiatives will likely require a combination of resources and a collaboration of 
organizations.  Medical providers funded by Title I and Title II are responsible for 
prescribing the most appropriate drug regimen for the patient.  Medical and other support 
service providers that have on-going personal interaction with the clients have the capacity 
to educate and monitor patients on adherence issues.  The ADAP program could 
contribute utilization trend data and information on which drugs clients are obtaining. 
The HRSA could encourage collaborative adherence initiatives by disseminating the latest 
medical research on the impact of non-adherence and offering technical assistance on how 
to promote and evaluate adherence.  Recommendations, best practices and implementation 
models could stem from adherence studies currently underway. 
Recognizing that adherence ensures their clients will obtain the maximum benefit from 
advanced drug therapy, several locations are already grappling with this difficult issue. 
One planning council outlined their plans to make adherence a service category in order to 
18 
Evaluation Systems: ADAP  OEI-05-98-00390  allocate funds to an adherence initiative.  One of the grantees we visited has been awarded 
funds through the Special Projects of National Significance program to study adherence. 
Two of the ADAPs we interviewed are involved in adherence initiatives.  One is 
monitoring patient adherence by generating a list of clients who do not pick up their refills 
from the pharmacy and are past due to receive their medications.  Pharmacists perform a 
weekly manual review of refill logs to see which patients have not picked up their 
prescriptions.  The file is flagged for further follow-up initiatives including contacting the 
patient directly or contacting the patients case manager. 
Techniques for adherence promotion include the creation and dissemination of patient 
education materials and behavior modification programs.  Monitoring methods include 
supervised drug intake, on-going contact through beepers and regularly convened peer-
support groups.  In cases where limited resources do not allow for such intense 
monitoring, adherence information could be collected through limited adherence studies 
using random sampling techniques. 
The HRSA should facilitate the spread of these efforts to help ensure that clients are 
reaping the benefits of advanced drug therapy.  Gathering adherence information is the 
only way that ADAP programs can assure their clients are benefitting from the drugs the 
program is providing and that the drugs are having the intended impact. 
The HRSA should instruct ADAPs to collaborate with Title I 
and Title II needs assessment efforts to assure that they 
explicitly evaluate medication therapy needs 
The ADAPs need a more sophisticated way to assess whether or not they are meeting the 
medication needs of the HIV/AIDS population.  The ADAPs are being called to greater 
accountability for their efforts to reach and serve traditionally underserved populations. 
The current method of matching the demographics of ADAP clients with the 
demographics of local HIV cases provides useful information on the needs of infected 
persons.  However, as ADAP administrators indicated, it is a simple method that paints an 
incomplete picture.  Several ADAP administrators pointed out that relying on a match of 
ADAP cases to existing public health data overestimates unmet need since it does not take 
into account other programs providing drug therapy.  Other shortcomings include lack of 
HIV prevalence data in some States and the reliance on enrollment data rather than active 
cases to estimate program utilization. 
Not only is the currently utilized methodology unreliable, but it only provides a general 
picture of unmet need.  It does not provide the ADAPs with a reliable way to pinpoint 
specific subpopulations in various regions of the State.  It also does not present a 
19 
Evaluation Systems: ADAP  OEI-05-98-00390  qualitative picture of unmet need.  Barriers to care, historic mistrust of the medical 
profession and the stigma of HIV add a complexity to the needs of various sub-
populations of infected persons which are often not reflected through a quantitative 
analysis of unmet need. 
Finally, properly judging unmet need is essential not only to ensure that medications are 
equitably distributed to populations in need, but also to better target current ADAP 
outreach efforts.  The U. S. Surgeon General David Satcher has called for ADAP 
programs to do a better job of targeting minority populations.  The ADAP programs must 
have a reliable way to determine the subpopulations in their State that require further 
attention in order to better target their outreach initiatives. 
While ADAPs may lack the resources and funding to conduct intensive outreach efforts 
aimed at specific subpopulations, they do have the responsibility to make sure that all 
those in need of HIV medication are aware of the program in order to ensure that existing 
access to health care disparities are not simply perpetuated by ADAP.  One way to 
accomplish this would be to target outreach efforts to specific organizations that focus on 
traditionally underserved populations. 
To better assess unmet need, HRSA should instruct ADAPs to appoint liaisons or create a 
mechanism to work with and support Title I and Title II needs assessment efforts to assure 
a comprehensive assessment of prescription drug needs.  Title I and Title II grantees are 
required to conduct routine needs assessments as part of their priority setting and 
allocation processes.  The surveys and focus groups used to collect information for these 
needs assessments typically capture information regarding unmet need for prescription 
drugs.  However, since the goal of the assessment is to gather information on primary 
medical care and support services, the information on medication therapy needs tends to 
be very basic.  Furthermore, it does not appear that this information is routinely relayed to 
ADAP administrators. 
The ADAPs should also collaborate with Title I and Title II needs assessment efforts to 
make sure that the Title I and Title II grantees actively solicit respondents from the 
currently enrolled ADAP population as well as those HIV-infected persons who are 
currently not enrolled in the ADAP.  This would allow ADAPs to access information from 
a broader spectrum of HIV-infected individuals and provide a richer understanding of 
unmet need and barriers to care. 
The ADAP needs assessment liaison could be appointed from the administrative staff or 
the ADAP advisory boards.  In some States, these advisory boards already take on 
responsibilities beyond determining the ADAP formulary.  In particular, two of the 
ADAPs visited have advisory bodies that provide overall program and policy guidance and 
recommendations.  Like Title I planning councils and Title II consortia, they advise on all 
programmatic issues and strive to be representative of the epidemic.  The liaison could 
20 
Evaluation Systems: ADAP  OEI-05-98-00390  work with the Title I and Title II planning bodies to assure that they collect needed 
information to assist the ADAPs in fulfilling their mission of supplying drug therapy to 
HIV-infected individuals in need. 
21 
Evaluation Systems: ADAP  OEI-05-98-00390  A G E N C Y   C O M M E N T S  
The HRSA provided comments on the draft report in which they did not concur with the 
first and third recommendations.  Where appropriate, we changed the report to reflect 
their comments, concerns and technical corrections.  The complete text of HRSAs 
comments can be found in Appendix A. 
Regarding the first recommendation to encourage ADAPs to match client utilization data 
to NIH/PHS guidelines as a broad measure of program impact, HRSA stated that they did 
not have the statutory authority to require ADAPs to perform this analysis.  The HRSA 
also stated that the data available to ADAPs would be insufficient to make meaningful 
comparisons and extrapolate client health outcomes. 
In response to HRSAs concern regarding their ability to require ADAPs to perform 
evaluative functions, the recommendation was altered to remove the implication of a 
statutory requirement.  However, developing outcome measures for ADAP remains an 
important step in the evolution of the program.  In our interviews, we found that many 
ADAPs have moved beyond a limited sense of mission and accountability.  They do not 
view the drug assistance program as merely a financing mechanism and recognize the 
importance of documenting program impact.  They recognize that measuring outcomes 
provides a means to promote the achievements of the program in terms of its most 
valuable contributions - the quality and prolonged life of its clients. 
Further, the match is a logical first step in outcome development.  The HRSA already 
requires ADAPs to present a break-out of the utilization of protease inhibitors in their 
annual application for funds.  Our recommendation would simply broaden this requirement 
to include an analysis of utilization of nationally recommended drug therapies. 
We appreciate HRSAs concerns regarding the impact of incomplete utilization data. 
Since ADAP is only one of many reimbursement mechanisms for HIV drug therapies, it is 
true that any picture of client drug utilization constructed by ADAP would not accurately 
represent total client access to HIV medication.  It is for this reason that we suggested 
that ADAP expand their current relationships with Medicaid to include a data match on 
drug utilization.  This data sharing agreement could build on the relationships already 
established with Medicaid in order to verify payor of last resort status and the 1998 
HRSA, HCFA and CDC inter-agency agreement to promote the development of 
integrated public health information systems. 
While matching ADAP distribution patterns with the NIH/PHS guidelines would 
inevitably lack some information regarding drug utilization, it would, nonetheless, provide 
a general sense of ADAPs contribution to client health outcomes.  In fact, the measure 
22 
Evaluation Systems: ADAP  OEI-05-98-00390  was never intended to serve as a rigorous measure of program impact.  We recognize that 
this type of evaluation is beyond the scope of ADAP agencies and that leading medical 
institutions around the country are involved in efforts to understand the impact of various 
drug combinations on client health outcomes.  Our recommendation is based on the 
United Way model which allows programs to take credit for their contribution to client 
outcomes while recognizing the practical limitations on assessing attribution in the absence 
of complete data.  A few ADAPs are already engaged in this type of analysis and have 
found it a useful tool for understanding program impact. 
The HRSA also did not concur with the recommendation that ADAPs should collaborate 
with Title I and Title II needs assessment efforts.  They stated that the drug assistance 
programs were not the appropriate entities to take the lead in collaborative efforts with the 
other Ryan White programs.  They were also concerned by the suggestion that the ADAP 
advisory boards could take on the responsibility of collaboration since ADAP advisory 
boards are typically clinical bodies supplying ADAP formulary advice. 
It was not our intention to imply that ADAP should take the lead in collaborating between 
the Ryan White programs.  Rather, our recommendation directed ADAPs to more closely 
participate in the required Title I and Title II needs assessment efforts in order to have 
their evaluation needs better met.  We offered the ADAP advisory boards as a possible 
liaison source only where the board was already functioning in a larger capacity than 
formulary advisement.  The specific designation of liaison responsibilities would be the 
responsibility of each ADAP administration. Of ultimate importance is that meaningful 
collaboration happens on a continuing basis.  The HRSAs stated intention to include 
instructions in the Title I application guidance regarding collaboration with ADAP reflects 
our intention to promote the level of meaningful collaboration between ADAP and the 
other Ryan White programs. 
23 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

24 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

25 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

26 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

27 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

28 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

29 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

30 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX A 
Agency Comments

31 
Evaluation Systems: ADAP  OEI-05-98-00390  APPENDIX B 
Related Office of Inspector General Reports

The Ryan White CARE Act: Local Implementation Issues (OEI-05-93-00336) 
The Ryan White CARE Act: Examples of Local Coordination (OEI-05-93-00335) 
The Ryan White CARE Act: Special Projects of National Significance (OEI-05-93-00332) 
The Ryan White CARE Act: Technical Report of 1992 Expenditures (OEI-05-93-00334) 
The Ryan White CARE Act: Funding Formulas (OEI-05-93-00330) 
Medicaid Managed Care and HIV/AIDS (OEI-05-97-00210) 
Audit of State AIDS Drug Assistance Programs Use of Drug Price Discounts (A-01-97-01501) 
Review of Maximizing Drug Discounts Under the Missouri Ryan White Program 
(A-01-97-00926) 
Audit of the Ryan White Comprehensive AIDS Resource Emergency Act of 1990, Title II, 
Administered by the Health Resources and Services Administration (A-01-97-01500) 
Audit of Eligibility Under Title I of the Ryan White Comprehensive AIDS Resource Emergency 
Act of 1990, New York Metropolitan Area (A-02-95-02517) 
Audit of the Ryan White Comprehensive AIDS Resource Emergency Act of 1990, Title II, 
Administered by the State of Connecticut (A-01-96-01501) 
Audit of the Ryan White Comprehensive AIDS Resource Emergency Act of 1990 in the Boston 
Metropolitan Area for Fiscal Year 1994 (A-01-95-01504) 
32 
Evaluation Systems: ADAP  OEI-05-98-00390