# Presentation: 711178

## Measuring What Matters: Allocation, Planning and Quality Assessment for the Ryan White CARE Act

- Presentation to
- Presidential Advisory Council on HIV and AIDS
- March 29, 2004

## Charge to the Committee(Ryan White CARE Act Amendments of 2000, P.L. 106-345)

- To assess whether reported HIV cases are **adequate, reliable, and sufficiently accurate for inclusion in formula grants** for RWCA Titles I and II, and to make recommendations for improving HIV reporting systems;
- To **identify data and tools for assessing a community’s severity of need** and how that information can be used in allocation decisions; and
- To identify available health outcome and other data that can be used to **measure the quality** of and access to RWCA-funded services.

**Notes:**
RWCA is safety net program that provides funding to cities, states, other public/private entities for HIV/AIDS care for people with low incomes and little or no insurance.

This study was mandated by Congress as part of the 2000 reauthorization legislation of RWCA.  
Congress asked the committee to examine three issues:  
Are reported HIV cases “adequate, reliable and sufficiently” for inclusion in the Title I and II formulas?
What data are available for assessing communities’ needs for resources?
What data could be used to measure the quality of and access to RWCA services?

## Motivation for Study

- 	Dramatic changes in epidemiology, need, and treatment options have occurred since the beginning of the epidemic.
- Due to advances in treatment, HIV (non-AIDS) cases account for a higher percent of all infected
- Data on AIDS cases traditionally used to allocate treatment resources, but HIV data thought to be a better reflection of resource needs
- Disparities still exist in access to and quality of care (e.g., by insurance status, gender, race/ethnicity)

**Notes:**
The questions posed by Congress were motivated by dramatic changes in the epidemic, needs, and quality of care. Examples:  
Due to treatment advances, mortality and morbidity have declined substantially. HIV (non-AIDS) cases are now a higher percent of all infected. 
AIDS data have traditionally been used to allocate treatment resources, but given shift in epidemic, expectation developed that data on total HIV cases would be a better reflection of the epidemic and related resource needs.
Concerns exist about disparities in access to and quality of HIV/AIDS care. Several large studies (e.g., HCSUS) showed that disparities persist by insurance status, sex, and race/ethnicity when controlling for other variables. 

## Charge 1: HIV Data and Formulas

- Formulas based on estimated living AIDS cases (ELCs)
- Concerns that using ELCs may disadvantage areas with “emerging” epidemics and areas that provide early access to treatment; total HIV cases a better reflection of resource needs
- Congress specified that HIV cases should be incorporated into Title I and II formulas between FY 2005-2007—provided that the Secretary determines that such data are available from all areas, and that they are “**sufficiently accurate and reliable**.”
- IOM asked to assist the Secretary in assessing the readiness of states to produce adequate and reliable HIV case-reporting data

**Notes:**
Estimated living AIDS cases have been used as primary factor in formulas for allocating RWCA funds.  
Reasons why: 
AIDS data have been collected in all areas of the U.S. since the early years of the epidemic, 
uniform standards exist for collection and reporting of AIDS data, and 
AIDS data are considered to be very high-quality. 

Given shifts in the epidemic though, there was an expectation that reported HIV data would better reflect resource needs.

Given this expectation, Congress specified that HIV case reports be incorporated into formulas by fiscal years 2005-2007 – provided that Secretary determined the HIV data were “sufficiently accurate” for this purpose. The Committee was asked to help in making that determination.  

Important reminder: Our charge was very focused –only looked at the role of HIV reporting vis-à-vis formula allocations. Did not evaluate broader functions of HIV reporting. 

## Charge 1: Evaluation Criteria

- All states should be capable of providing data
- Data should be of comparable quality across jurisdictions
- HIV data should produce more accurate assessments of the relative disease burden and resource needs
- Including HIV data should result in significant variation in the relative size of awards to states and EMAs

**Notes:**
Main question:  Does including HIV data in the formula increase the fairness of the allocations? 

Committee identified 4 criteria to answer this question. 
1) Every eligible area would need to be capable of providing the HIV data required for the formula.
2) The quality of the HIV data across jurisdictions would need to be comparable.
3) HIV data should provide more accurate assessment of the epidemic and related resource needs across jurisdictions than AIDS data alone.
4) Including HIV data should produce significant variation in relative size of allocations (if including the data doesn’t produce any change, doesn’t make any difference and AIDS data should continue to be used).

## Charge 1: Key Findings 

- Areas are not capable of providing comparable data
- At time report was released, Georgia and Philadelphia did not have HIV reporting systems
- Some systems require more time to mature
- CDC does not utilize data from code-based reporting states due to a lack of deduplication methods
- Not clear that inclusion of HIV case reports in formulas will produce fairer allocations. Could be less accurate than AIDS case reports.
- Formula features dampen any change that would result from using HIV data—double counting of EMA cases, hold harmless, minimum awards, etc.

**Notes:**
States are not capable of providing adequate, comparable data on relative disease burden. Three reasons: 
At time of the report, Georgia and Philadelphia did not have HIV reporting systems – they have since put systems in place
Maturity of systems differs – some areas (e.g., Georgia, California) only recently implemented HIV reporting; takes awhile for backlog of cases to be reported, providers need to be educated, etc. 
CDC does not utilize data from code-based reporting states owing to lack of methods for deduplicating these cases. 

Not clear that HIV case reporting will result in fairer allocations. Our analyses (based on available data) showed that including would not likely result in significantly different allocations. Even if it did, because the HIV data are accurate/uniform, including the data could decrease the equity of the overall allocations. 

Formula features dampen impact of including HIV in the formula. E.g., double counting of EMA cases, minimum awards, etc.

Bottom line/other points to make: 
There is clear variation in relative HIV burden across jurisdictions. And, there are disparities in the $/ELC received by jurisdictions (Give an example from Appendix C). 
But, based on available data (combination of estimated and reported HIV cases), our analyses suggest that including HIV data in formulas would not have as big an impact on allocations as we expected. We were surprised by this result.
Any effect/benefit of including HIV data is small when compared to the effect of the structural features of the formula on allocations.
Even if including HIV data did make a difference in the allocations, it may not make the allocations more fair.

## Charge 1: Key Recommendations

- Continue using ELCs in the formulas for now, but:
- Improve HIV reporting:
- Develop de-duplication methods that permit full use of all reported HIV cases, including reports from states with code-based systems.
- Enhance laboratory reporting (HIV viral load, CD4)
- Consider developing alternative estimates of HIV cases (survey or model-based estimation)
- Congress should reevaluate the formula:
- Impact of formula features on allocations
- Match between intent and results

**Notes:**
Continue to use estimated living AIDS cases as the basis for the formula for the next several years, but should also take several steps: 
Improve HIV reporting
Develop deduplication methods so that data from code-based states can be used.  (Right now, CDC does not accept data from the 15 code-based reporting states into HARS because of lack of deduplication methods.)
Enhance lab (electronic in particular) reporting (VL, CD4 – reporting requirements vary by areas; some don’t require it. Improving laboratory reporting could enhance the consistency, completeness, and timeliness (if electronic based) of HIV case reports. 
Should also consider developing alternative estimates of HIV prevalence using survey and/or model based approaches. These approaches have the potential to produce more timely, representative data. They also allow estimates of the total HIV infected population including the undiagnosed (this is particularly important for quality of care). A combination survey/model-based approach was suggested in previous IOM reports (No Time to Lose) and committee believes this approach should be reconsidered.
Congress should also reconsider two aspects of the formulas: 
Reexamine structural features of formula (minimums, thresholds, hold harmless, etc) – are these consistent with goal of improving the fairness of the allocations? (Note that the committee was not asked to and did not make a judgment on that issue – we merely elucidated the effect of these features on allocations.) 
Reexamine whether the intent of the legislation is consistent with structure of the formulas (RWCA is payer of last resort and is intended to benefit low-income, uninsured individuals. Problem is that allocations are made on total counts – insurance status is not incorporated into the decision.)

Emphasize that findings don’t mean that we don’t support use HIV reporting for other purposes. We weren’t asked to evaluate HIV reporting for other functions. Only considered use of HIV data for the formula.

## Charge 2: Resource Needs

- Several discretionary programs attempt to target funds to areas with most “severe need”
- Committee’s analysis focused on Title I supplemental award
- Title I supp. awards are determined by a competitive grant application process
- Severity of need accounts for 33% of application score
- Grantees provide data on HIV/AIDS epidemiology, poverty/insurance status, comorbidities, and special populations

**Notes:**
This section of the report focused on discretionary grants rather than the formula grants.
RWCA has several discretionary grants that attempt to target funds to areas with most “severe need”; these try to take into account other factors than just case counts.  
The Committee focused on Title I supplemental award (which constitutes 1/2 of total Title I award) because it is the largest of the RWCA discretionary programs. The recommendations could apply to other discretionary grant programs (Title II ADAP supplemental award; Titles III/IV)
Title I supplemental awards are determined based on a competitive application process involving all EMAs. 
Applications are scored on a number of factors, but 33% of the application score is determined by “severity of need”. 
In the severity of need section of the application, grantees are asked to provide information on the epidemic in their area, poverty/insurance status, comorbidities, and special populations. 

## Charge 2: Conceptual Framework

- Resource needs determined by:
- Disease burden (# of cases, complexity of cases)
- Cost of Care (wages, supply costs, etc.)
- Available resources (# of physicians, Medicaid coverage, etc.)
- Resource needs = (disease burden * cost of care) – available resources

**Notes:**
Resource needs determined by broad array of factors. We grouped these factors into three categories:
Disease burden (Ex: # of cases and the complexity of those cases)
Cost of Care (Ex: even if have same number of cases in two areas, it may cost more to deliver care in one area because wages, transportation, or supply costs are higher)
Available resources (Ex: # of doctors, Medicaid eligibility and services covered; e.g., may require more resources to deliver care in rural area where there are few physicians)

Key point: needs are not just counts!  

## Charge 2: Key Findings

- Title I supplemental award process relies on nonstandard measures of local need.
- Application process is burdensome for grantees. High correlation between supplemental and base awards, effort seems unjustified.

**Notes:**

Committee reviewed the severity of need component of the 51 Title I applications (FY2002) to see the types of indicators that grantees were using.  
Main problem is lack of standardization in the measures used and the definitions of those measures across jurisdictions.  For example, grantees may use different measures to describe comorbidities (prevalence, incidence, period prevalence, etc.). Even when grantees used the same measure, the definitions (numerator and denominator) often varied.  Lack of standardization makes it impossible to make objective, comparative assessments. 
Furthermore, Title I supplemental grants entail a very comprehensive application and elaborate review process that is time-consuming and burdensome for both grantees and for HRSA. Despite all the effort, the correlation between the formula and supplemental awards is very high (0.87), meaning that this very time-consuming process makes very little difference in the overall awards to areas.

## Charge 2: Key Recommendations

- Title I supplemental award should be based on two components:
- quantitatively defined need based on small number of standardized measures;
- locally defined need described in _short _narrative by grantees.
- Predominance of weight for determining Title I supplemental awards should be given to quantitative measures.

**Notes:**
We recommend that the awards be determined based on two factors : 
Quantitatively defined need based on small number of standard measures that can be calculated by HRSA (goal is to remove burden from grantees)
Short narrative describing local needs that maybe are not captured by standardized measures
Quantitative measures should be predominate factor in determining Title I supplemental awards (this is consistent with Congressional intent)

Problem is that this is hard to do. The measures don’t exist or HRSA would have done it already. 

## Charge 2: Key Recommendations (cont.)

- Two approaches for estimating resource needs:
- Direct measures – survey people and ask them their needs (e.g., HCSUS). Problem: expensive and complex
- Indirect measures– identify predictors of need based on combination of targeted surveys and publicly available data (report contains “proof of concept” using data from the HCSUS and Area Resource File)
- Secretary should evaluate the cost and utility of redesigning and coordinating studies conducted by HRSA and CDC to assess the needs of people living with HIV (efforts underway).

**Notes:**
The Committee suggests two possible approaches for estimating resource needs: 
Direct measures – One approach is to develop direct measures of resource needs by surveying people with HIV/AIDS and asking them what their needs are. 
The HCSUS study, which surveyed a random sample of individuals with HIV/AIDS in care, is an example of this type of direct approach. 
The problem with this type of approach for assessing needs for RWCA is that one would have to conduct repeated surveys in all eligible areas.  This can be expensive and complex to do. 
Indirect measures– An alternative strategy is to identify predictors, or indirect measures, of need using a combination of targeted surveys in a few areas and then combining that with publicly available data to develop estimates for all eligible areas. 
Data weren’t available for us to develop the predictors, but the committee did a “proof of concept” analysis of this approach using data from the HCSUS and Area Resource File (publicly available data set containing thousands of variables on factors such as providers in an area, social and demographic factors). 
For example, several variables (e.g., percent of the population in poverty) were predictive of needs and/or costs of care. This type of model could be used to develop predictors of needs in all eligible areas.  

HRSA will need to evaluate the feasibility and usefulness of these alternative approaches.

On related note, the Committee also recommended that the Secretary of HHS evaluate the cost and utility of redesigning and coordinating special studies conducted by HRSA and CDC to assess the needs of people living with HIV (efforts are currently underway).

## Charge 3: Quality of Care

- Considerable effort by HRSA and grantees and others to assess quality of HIV/AIDS care
- RWCA Amendments of 2000 required all title programs to establish quality management programs. While many Title III and IV grantees  had QM in place, this was a new requirement for many Title I and II grantees.
- Committee developed a conceptual framework, reviewed quality measures that are already in use, and recommended an initial set of measures.

**Notes:**
A lot of work being done by HRSA, grantees, and nationally in the field on how to measure the quality of HIV/AIDS care. 
We developed a framework for thinking about HIV/AIDS quality assessment, identified an initial set of measures, and suggested a process for refining those measures. 

## Charge 3: Conceptual Framework

*Population of interest* (not diagnosed and not in care, diagnosed and not in care, diagnosed and in care)
*Level of assessment* (individual, provider or clinic, population or area)
*Type of measure* (structure, process, outcome)
*Spectrum of services* (e.g., counseling and testing, referral to treatment, social support services, prevention of transmission, prevention of AIDS, prophylaxis, treatment and service integration and coordination)

**Notes:**
Four dimensional conceptual framework for quality of care:  
Population- Non-DX; DX but not in TX; in TX – most of the focus has been on those who are diagnosed and in treatment. Need to expand the focus to include those who are Undo or Do and not in treatment.
Level Assessment—indiv, clinic, pop – most of the work is done at the clinic level. Need to expand measures to the broader population (e.g., proportion of people with HIV who died within 12 months of diagnosis). Important indicator of overall health of the population.
Type—structure, process, outcome –Little work has been done on measures of structure (e.g., no measures of experience of providers/clinic in providing HIV/AIDS care, despite research demonstrating importance of this in outcomes).
Spectrum of services — measures should  be expanded to include the entire spectrum of services. Ex: Focus has been on medical care services, while little has been done in measuring the quality of social support services. 

## Charge 3: Review of HIV/AIDS Measures

- Committee reviewed nine key sources for  HIV/AIDS quality measures
- There are many measures and approaches already out there that can be used to measure quality of HIV/AIDS care
- _Problem:_ measures are not standardized

**Notes:**
The committee reviewed quality measures used by nine key sources (HIVQUAL, FACCT, HCSUS, etc.)
While many measures exists, measures aren’t standardized

## Charge 3: Key Recommendations

- Specified initial set of standardized quality measures
- Specified a process for defining and operationalizing suggested measures
-  Recommend additional resources to develop infrastructure for monitoring quality

**Notes:**
We identified an initial set of standardized quality indicators for RWCA, but these need further refinement  before they can be operationalized (definitions, criteria need to be decided on)
Committee suggested a process by which one could select and refine measures.  HRSA should develop a working group with a variety of stakeholders. The NCQA and the national Quality Forum have models for doing this (Ex: Committee on Performance Measurement) 
Committee also recommends additional resources for infrastructure for quality monitoring – can’t do this for free

## For more information...

- The full report is available to view and for purchase on-line at _[www.nap.edu](http://www.nap.edu/)__.  _
- More information about the committee members and the project is available at _[www.iom.edu/ryanwhite.com](http://www.iom.edu/ryanwhite.com)__._
- 	The Institute of Medicine is a private, nonprofit institution that provides health policy advice under a congressional charter granted to the National Academy of Sciences.