Health Resources and Services Administration: Many Underserved Areas Lack a Health Center Site, and the Health Center Program Needs More Oversight
Published: 2008-08-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Health centers funded through grants under the Health Center Program--managed by the Health Resources and Services Administration (HRSA), an agency in the U.S. Department of Health and Human Services (HHS)--provide comprehensive primary care services for the medically underserved. HRSA provides funding for training and technical assistance (TA) cooperative agreement recipients to assist grant applicants. GAO was asked to examine (1) to what extent medically underserved areas (MUA) lacked health center sites in 2006 and 2007 and (2) HRSA's oversight of training and TA cooperative agreement recipients' assistance to grant applicants and its provision of written feedback provided to unsuccessful applicants. To do this, GAO obtained and analyzed HRSA data, grant applications, and the written feedback provided to unsuccessful grant applicants and interviewed HRSA officials.
section_title: What GAO Found
paragraphs: Grant awards for new health center sites in 2007 reduced the overall percentage of MUAs lacking a health center site from 47 percent in 2006 to 43 percent in 2007. In addition, GAO found wide geographic variation in the percentage of MUAs that lacked a health center site in both years. Most of the 2007 nationwide decline in the number of MUAs that lacked a site occurred in the South census region, in large part, because half of all awards made in 2007 for new health center sites were granted to the South census region. GAO also found that HRSA lacks readily available data on the services provided at individual health center sites. HRSA oversees training and TA cooperative agreement recipients, but its oversight is limited in key respects and it does not always provide clear feedback to unsuccessful grant applicants. HRSA oversees recipients using a number of methods, including regular communications, review of cooperative agreement applications, and comprehensive on-site reviews. However, the agency's oversight is limited because it lacks standardized performance measures to assess the performance of the cooperative agreement recipients and it is unlikely to meet its policy goal of conducting comprehensive on-site reviews of these recipients every 3 to 5 years. The lack of standardized performance measures limits HRSA's ability to effectively evaluate cooperative agreement recipients' activities that support the Health Center Program's goals with comparable measures. In addition, without timely comprehensive on-site reviews, HRSA does not have up-to-date comprehensive information on the performance of these recipients in supporting the Health Center Program. HRSA officials stated that they are in the process of developing standardized performance measures. Moreover, more than a third of the written feedback HRSA sent to unsuccessful Health Center Program grant applicants in fiscal years 2005 and 2007 contained unclear statements. The lack of clarity in this written feedback may undermine its usefulness rather than enhance the ability of applicants to successfully compete for grants in the future.

section_title: Letter
paragraphs: Health centers in the federal Health Center Program provide  comprehensive primary health care services—preventive, diagnostic,  treatment, and emergency services as well as referrals to specialty care— to federally designated medically underserved populations (MUP) or those  individuals residing in federally designated medically underserved areas  (MUA). To fulfill the Health Center Program’s mission of increasing  access to primary health care services for the medically underserved, the  Health Resources and Services Administration (HRSA)—the agency within  the U.S. Department of Health and Human Services (HHS) that administers  the Health Center Program—provides grants to health centers. These  grants, along with other federal benefits available to health center grantees  through the Health Center Program, are an important part of successful  health center operations and viability. In 2006, Health Center Program  grants made up about 20 percent of all health center grantees’ revenues. A  health center grantee may provide services at one or more delivery sites— known as health center sites. Not all health center sites are required to  provide the full range of comprehensive primary care services; some  health center sites may provide only limited services, such as dental and  mental health services. In 2006, approximately 1,000 health center grantees  operated more than 6,000 health center sites while serving more than   15 million people.
Beginning in fiscal year 2002, HRSA significantly expanded the Health  Center Program under a 5-year effort—the President’s Health Centers  Initiative—to increase access to comprehensive primary care services for  underserved populations, including those in MUAs. Under the initiative,  HRSA set a goal of awarding 630 grants to open new health center sites— such grants are known as new access point grants—and 570 grants to  expand services at existing health center sites by the end of fiscal year  2006. New access point grants fund one or more new health center sites  operated by either new or existing health center grantees. In July 2005, we  reported challenges HRSA encountered during this expansion of the  Health Center Program. In particular, we found that HRSA’s process for  awarding new access point grants might not sufficiently target  communities with the greatest need for services, though we concluded  that changes HRSA had made to its grant award process could help the  agency appropriately consider community need when distributing federal  resources. We also reported that HRSA lacked reliable information on the  number and location of the sites where health centers provide care, and  we recommended that HRSA collect this information. In response to our  recommendation, HRSA took steps to improve its data collection efforts in  2006 to more reliably account for the number and location of health center  sites funded under the Health Center Program.
By the end of fiscal year 2007, HRSA had achieved its grant goals under the  original President’s Health Centers Initiative and launched a second  nationwide effort, the High Poverty County Presidential Initiative. In fiscal  year 2007, HRSA held two new access point competitions, one focused on  opening new health center sites in up to 200 HRSA-selected counties that  lacked a health center site—part of the High Poverty County Presidential  Initiative—and one that was an open competition.
To assist potential health center grantees in applying for new access point  grants, HRSA provides funds to national, regional, and state organizations  to promote Health Center Program grant opportunities and help applicants  secure funding. This funding mechanism is known as a training and  technical assistance (TA) cooperative agreement. For fiscal year 2007,  HRSA awarded nearly $53 million in cooperative agreements to national  organizations—specifically, those that assist broadly with health center  operations as well as expand access to health care for underserved  populations—and regional and state primary care associations (PCA),  organizations that also support health centers and other safety net  providers in increasing access to primary care services. HRSA also assists  potential grantees by providing written feedback to applicants that apply  for, but are not awarded, HRSA grants through the Health Center Program.  This written feedback—known as summary statements—characterizes the  strengths and weaknesses of the applications. The summary statements  are intended to help unsuccessful applicants improve the quality—and  therefore success—of future grant applications. The summary statements  are prepared by objective review committees selected by HRSA to  evaluate health center grant applications. Before HRSA releases the  statements to unsuccessful applicants, the agency removes any internal  recommendations made by the committee and reviews them for accuracy.
Given the expansion of the Health Center Program under the President’s  Health Centers Initiative and the High Poverty County Initiative as well as  HRSA’s past challenges in targeting its new access point grant awards to  serve needy areas, you asked us to examine the extent to which MUAs  contain health center sites as well as HRSA’s management of the Health  Center Program, specifically, efforts to assist applicants for new access  point grants. In this report, we examine (1) for 2006, the extent to which  MUAs lacked health center sites and the services provided by each site in  an MUA; (2) how new access point grants awarded in 2007 changed the  extent to which MUAs lacked health center sites; and (3) HRSA’s oversight  of cooperative agreement recipients’ assistance to new access point  applicants and feedback the agency provides to unsuccessful applicants.
To examine the extent to which MUAs lacked health center sites  nationwide and the services provided by each site in 2006, we interviewed  HRSA officials and obtained health center site data from HRSA’s uniform  data system (UDS). The UDS provided the zip code location of health  center sites as of December 31, 2006. We also obtained from HRSA data  on the geographic location of MUAs designated for 2006. We linked the  location of the MUAs to their associated zip codes using a geographic  crosswalk file based on U.S. Census Bureau data. We then compared the  location of health center sites with the location of MUAs by census region  and state. We limited our analysis to health center sites operated by  grantees that received community health center funding—the type of  funding that requires sites to provide services to all residents of the service  area regardless of their ability to pay. In addition, because HRSA takes  into account the location of federally qualified health center look-alike  sites—facilities that operate like health center sites but do not receive  HRSA funding—when deciding where to award new access point grants,  we obtained from HRSA the location of health center look-alike sites in  2006 and compared them with the location of MUAs.
To examine how new access point grants awarded in 2007 changed the  extent to which MUAs lacked health center sites nationwide, we obtained  from HRSA the applications submitted for the new access point  competitions held in fiscal year 2007 and the list of funded applicants for  these competitions. We reviewed the applications to determine the zip  code location of proposed new health center sites, that is, sites for which  the applicants requested funding, and the list of funded applicants to  determine the location of the new health center sites for which grants  were awarded in 2007. We also obtained from HRSA data on the location  of MUAs in 2007. We then compared the location of proposed and funded  new health center sites in 2007 with the location of MUAs in 2007. As  with the 2006 analysis, we limited our review to health center sites  operated by grantees that requested community health center funding— the type of funding that requires sites to provide services to all residents of  the service area regardless of their ability to pay. As we did for the 2006  analysis, we obtained from HRSA the location of health center look-alike  sites in 2007 and compared them to the location of MUAs in 2007.
To examine HRSA’s oversight of cooperative agreement recipients’  assistance to new access point applicants, we first interviewed HRSA  officials and representatives from organizations that had training and TA  cooperative agreements with HRSA for fiscal year 2007 to provide  assistance to applicants for health center grants. Specifically, we  interviewed representatives of the eight national organizations that target  assistance to new access point applicants and a judgmental sample of 10  geographically diverse state PCAs. We reviewed copies of the  organizations’ notices of grant awards, work plans (documents detailing  health center training and technical assistance activities), and semiannual  and annual progress reports submitted to HRSA. We examined  documents obtained from HRSA relating to its review of these cooperative  agreement recipients’ fiscal year 2007 annual noncompeting continuation  applications and periodic comprehensive on-site reviews conducted by  HRSA. To evaluate HRSA’s feedback to unsuccessful applicants, we  obtained from HRSA the summary statements that were issued to  unsuccessful applicants in connection with each of the three new access  point grant competitions held in fiscal years 2005 and 2007. We selected a  random sample of 30 percent of the summary statements based on  application score. This resulted in a sample of 69 summary statements out  of the universe of 230 sent to unsuccessful applicants. The results of our  analysis are generalizable to this universe. For each summary statement,  we reviewed the information provided on the application’s strengths and  weaknesses for each of the eight criteria used to evaluate new access  point grant applications.
We discussed our data sources with knowledgeable agency officials and  performed data reliability checks, such as examining the data for missing  values and obvious errors, to test the internal consistency and reliability of  the data. After taking these steps, we determined that the data were  sufficiently reliable for our purposes. We conducted our work from April  2007 through July 2008 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The Health Center Program is governed by section 330 of the Public Health  Service Act. By law, grantees with community health center funding must  operate health center sites that    serve, in whole or in part, an MUA or MUP;    provide comprehensive primary care services as well as enabling services,  such as translation and transportation, that facilitate access to health care;   are available to all residents of the health center service area, with fees on  a sliding scale based on patients’ ability to pay;   are governed by a community board of which at least 51 percent of the  members are patients of the health center; and   meet performance and accountability requirements regarding  administrative, clinical, and financial operations.
subsections: section_title: HRSA’s MUA Designation Criteria
paragraphs: HRSA may designate a geographic area—such as a group of contiguous  counties, a single county, or a portion of a county—as an MUA based on  the agency’s index of medical underservice, composed of a weighted sum  of the area’s infant mortality rate, percentage of population below the  federal poverty level, ratio of population to the number of primary care  physicians, and percentage of population aged 65 and over.
In previous reports, we identified problems with HRSA’s methodology for  designating MUAs, including the agency’s lack of timeliness in updating its  designation criteria. HRSA published a notice of proposed rule making in  1998 to revise the MUA designation system, but it was withdrawn because  of a number of issues raised in over 800 public comments. In February  2008, HRSA published a revised proposal and the period for pubic  comment closed in June 2008.
section_title: HRSA’s New Access Point Grant Process
paragraphs: HRSA uses a competitive process to award Health Center Program grants.  There are four types of health center grants available through the Health  Center Program, but only new access point grants are used to establish  new health center sites. Since 2005, HRSA has evaluated applications for  new access point grants using eight criteria for which an application can  receive a maximum of 100 points (see table 1).
Grant applications are evaluated by an objective review committee—a  panel of independent experts, selected by HRSA, who have health center- related experience. The objective review committee scores the  applications by awarding up to the maximum number of points allowed for  each criterion and prepares summary statements that detail an  application’s strengths and weaknesses in each evaluative criterion. The  summary statements also contain the committee’s recommended funding  amounts and advisory comments for HRSA’s internal use; for example, the  committee may recommend that HRSA consider whether the applicant’s  budgeted amount for physician salaries is appropriate. The committee  develops a rank order list—a list of all evaluated applications in  descending order by score. HRSA uses the internal comments— recommended funding amounts and advisory comments—from the  summary statements and the rank order list when making final funding  decisions. In addition, HRSA is required to take into account the  urban/rural distribution of grants, the distribution of funds to different  types of health centers, and whether a health center site is located in a  sparsely populated rural area. HRSA also considers the geographic  distribution of health center sites—to determine if overlap exists in the  areas served by the sites—as well as the financial viability of grantees.  After the funding decisions are made, HRSA officials review the summary  statements for accuracy, remove the recommended funding amounts and  any advisory comments, and send the summary statements to  unsuccessful applicants as feedback.
section_title: HRSA’s Training and TA Cooperative Agreements
paragraphs: For fiscal year 2007, HRSA funded 60 training and TA cooperative  agreements with various national, regional, and state organizations to  support the Health Center Program, in part, by providing training and  technical assistance to health center grant applicants. Cooperative  agreements are a type of federal assistance that entails substantial  involvement between the government agency—in this case, HRSA—and  the funding recipient—that is, the national, regional, and state  organizations. HRSA relies on these training and TA cooperative  agreement recipients to identify underserved areas and populations across  the country in order to assist the agency in increasing access to primary  care services for underserved people. In addition, these cooperative  agreement recipients serve as HRSA’s primary form of outreach to  potential applicants for health center grants.
For each cooperative agreement recipient, HRSA assigns a project officer  who serves as a recipient’s main point of contact with the agency. The  duration of a cooperative agreement, known as the project period, is  generally 2 or 3 years, with each year known as a budget period. As a  condition of the cooperative agreements, HRSA project officers and the  organizations jointly develop work plans detailing the specific training and  technical assistance activities to be conducted during each budget period.  Activities targeted to new access point applicants can include assistance  with assessing community needs, disseminating information in  underserved communities regarding health center program requirements,  and developing and writing grant applications. After cooperative  agreement recipients secure funding through a competitive process, they  reapply for annual funding through what is known as a noncompeting  continuation application each budget period until the end of their project  period. These continuation applications typically include a work plan and  budget for the upcoming budget period and progress report on the  organization’s current activities.
HRSA policy states that cooperative agreement recipients will undergo a  comprehensive on-site review by agency officials once every 3 to 5 years.  During these comprehensive on-site reviews, HRSA evaluates the  cooperative agreement recipients using selected performance measures— developed in collaboration with the organizations—and requires recipients  to develop action plans to improve operations if necessary. The purpose of  these reviews is for the agency to evaluate the overall operations of all its  funding recipients and improve the performance of its programs.
section_title: Almost Half of MUAs Lacked a Health Center Site in 2006, and the Types of Services Provided by Each Site Could Not Be Determined
paragraphs: Almost half of MUAs nationwide lacked a health center site in 2006. The  percentage of MUAs that lacked a health center site varied widely across  census regions and states. We could not determine the types of primary  care services provided by health center sites in MUAs because HRSA does  not maintain data on the types of services offered at each site. Because of  this, the extent to which individuals in MUAs have access to the full range  of comprehensive primary care services provided by health center sites is  unknown.
subsections: section_title: Almost Half of MUAs Nationwide Lacked Health Center Sites in 2006, and the Percentage of MUAs Lacking Sites Varied Widely by Census Region and State
paragraphs: Based on our analysis of HRSA data, we found that 47 percent of MUAs  nationwide—1,600 of 3,421—lacked a health center site in 2006. We  found wide variation among census regions—Northeast, Midwest, South,  and West—and across states in the percentage of MUAs that lacked health  center sites. (See fig. 1.) The Midwest census region had the most MUAs  that lacked a health center site (62 percent) while the West census region  had the fewest MUAs that lacked a health center site (32 percent).
More than three-quarters of the MUAs in 4 states—Nebraska (91 percent),  Iowa (82 percent), Minnesota (77 percent), and Montana (77 percent)— lacked a health center site; in contrast, fewer than one-quarter of the  MUAs in 13 states—including Colorado (21 percent), California   (20 percent), Mississippi (20 percent), and West Virginia (19 percent)— lacked a health center site. (See app. I for more detail on the percentage of  MUAs in each state and the U.S. territories that lacked a health center site  in 2006.)
In 2006, among all MUAs, 32 percent contained more than one health  center site; among MUAs with at least one health center site, 60 percent  contained multiple health center sites. Almost half of all MUAs in the West  census region contained more than one health center site while less than  one-quarter of MUAs in the Midwest contained multiple health center  sites. The states with three-quarters or more of their MUAs containing  more than one health center site were Alaska, Connecticut, the District of  Columbia, Hawaii, New Hampshire, and Rhode Island. In contrast,  Nebraska, Iowa, and North Dakota were the states where less than   10 percent of MUAs contained multiple sites.
section_title: The Types of Services Provided at Individual Sites Could Not Be Determined Because Data Were Not Readily Available
paragraphs: We could not determine the types of primary care services provided at  each health center site because HRSA does not collect and maintain  readily available data on the types of services provided at individual health  center sites. While HRSA requests information from applicants in their  grant applications on the services each site provides, in order for HRSA to  access and analyze individual health center site information on the  services provided, HRSA would have to retrieve this information from the  grant applications manually. HRSA separately collects data through the  UDS from each grantee on the types of services it provides across all of its  health center sites, but it does not collect data on services provided at  each site. Although each grantee with community health center funding is  required to provide the full range of comprehensive primary care services,  it is not required to provide all services at each health center site it  operates. HRSA officials told us that some sites provide limited services— such as dental or mental health services. Because HRSA lacks readily  available data on the types of services provided at individual sites, it  cannot determine the extent to which individuals in MUAs have access to  the full range of comprehensive primary care services provided by health  center sites. This lack of basic information can limit HRSA’s ability to  assess the full range of primary care services available in needy areas  when considering the placement of new access points and limit the  agency’s ability to evaluate service area overlap in MUAs.
section_title: 2007 Awards Reduced the Number of MUAs That Lacked a Health Center Site, but Wide Geographic Variation Remained
paragraphs: Our analysis of new access point grants awarded in 2007 found that these  awards reduced the number of MUAs that lacked a health center site by  about 7 percent. Specifically, 113 fewer MUAs in 2007—or 1,487 MUAs in  all—lacked a health center site when compared with the 1,600 MUAs that  lacked a health center site in 2006. As a result, 43 percent of MUAs  nationwide lacked a health center site in 2007.
Despite the overall reduction in the percentage of MUAs nationwide that  lacked health center sites in 2007, regional variation remained. The West  and Midwest census regions continued to show the lowest and highest  percentages of MUAs that lacked health center sites, respectively. (See   fig. 2.) Three of the census regions showed a 1 or 2 percentage point  change since 2006, while the South census region showed a 5 percentage  point change.
The minimal impact of the 2007 awards on regional variation is due, in  large part, to the fact that more than two-thirds of the nationwide decline  in the number of MUAs that lacked a health center site—77 out of the 113  MUAs—occurred in the South census region. (See table 2.) In contrast,  only 24 of the 113 MUAs were located in the Midwest census region, even  though the Midwest had nearly as many MUAs that lacked a health center  site in 2006 as the South census region. Overall, while the South census  region experienced a decline of 12 percent in the number of MUAs that  lacked a health center site, the other census regions experienced declines  of approximately 4 percent.
The South census region experienced the greatest decline in the number of  MUAs lacking a health center site in 2007 compared to other census  regions, in large part, because it was awarded more new access point  grants that year than any other region. (See table 3.) Specifically, half of all  new access point awards made in 2007—from two separate new access  point competitions—went to applicants from the South census region.
When we examined the High Poverty County new access point  competition, in which 200 counties were targeted by HRSA for new health  center sites, we found that 69 percent of those awards were granted to  applicants from the South census region. (See fig. 3.) The greater number  of awards made to the South census region for this competition may be  explained by the fact that nearly two-thirds of the 200 counties targeted  were located in the South census region. (For detail on the High Poverty  County new access point competition by census region and state, see   app. II.)
When we examined the open new access point competition, which did not  target specific areas, we found that the South census region also received  a greater number of awards than any other region under that competition.  Specifically, the South census region was granted nearly 40 percent of  awards; in contrast, the Midwest received only 17 percent of awards. (See  table 4.)
section_title: HRSA Oversees Cooperative Agreement Recipients but Oversight Is Limited in Key Respects, and Its Feedback to Unsuccessful Applicants Is Not Always Clear
paragraphs: HRSA oversees cooperative agreement recipients, but the agency’s  oversight is limited because it does not have standardized performance  measures to assess the performance of the cooperative agreement  recipients in assisting new access point applicants and the agency is  unlikely to meet its policy timeline for conducting comprehensive on-site  reviews. Although HRSA officials told us that they were developing  standardized performance measures, they provided no details on the  specific measures that may be implemented. Moreover, more than a third  of the summary statements sent to unsuccessful applicants for new access  point competitions held in fiscal years 2005 and 2007 contained unclear  feedback.
subsections: section_title: HRSA Oversees Cooperative Agreement Recipients but Lacks Standardized Performance Measures and Likely Will Not Complete All Comprehensive On-site Reviews in a Timely Manner
paragraphs: HRSA oversees the activities of its cooperative agreement recipients using  a number of methods. HRSA officials told us that over the course of a  budget period, project officers use regular telephone and electronic  communications to discuss cooperative agreement recipients’ activities as  specified in work plans, review the status of these activities, and help set  priorities. According to HRSA officials, there is no standard protocol for  these communications, and their frequency, duration, and content vary  over the course of a budget period and by recipient. HRSA staff also  reviews annual noncompeting continuation applications to determine  whether the cooperative agreement recipients provided an update on their  progress, described their activities and challenges, and developed a  suitable work plan and budget for the upcoming budget period. The  progress reports submitted by cooperative agreement recipients in these  annual applications serve as HRSA’s primary form of documentation on  the status of cooperative agreement recipients’ activities.
HRSA’s oversight of training and TA cooperative agreement recipients is  based on performance measures tailored to the individual organization  rather than performance measures that are standardized across all  recipients. Specifically, HRSA uses individualized performance measures  in cooperative agreement recipients’ work plans and comprehensive on- site reviews to assess recipients’ performance. For cooperative agreement  recipients’ work plans, recipients propose training and technical  assistance activities in response to HRSA’s cooperative agreement  application guidance, in which the agency provides general guidelines and  goals for the provision of training and technical assistance to health center  grant applicants. The guidance requires recipients to develop performance  measures for each activity in their work plans. When we analyzed the  work plans of the 8 national organizations and 10 PCAs with training and  TA cooperative agreements, we found that these measures varied by  cooperative agreement recipient. For example, we found that for national  organizations, performance measures varied from (1) documenting that  the organization’s marketing materials were sent to PCAs to (2) recording  the number of specific technical assistance requests the organization  received to (3) producing monthly reports for HRSA detailing information  about potential applicants. For state PCAs, measures varied from (1) the  PCA providing application review as requested to (2) holding specific  training opportunities—such as community development or board  development—to (3) identifying a specific number of applicants the PCA  would assist during the budget period. Because these performance  measures vary for cooperative agreement recipients’ activities, HRSA does  not have comparable measures to evaluate the performance of these  activities across recipients.
HRSA’s oversight of cooperative agreement recipients is limited in some  key respects. One limitation is that the agency does not have standardized  measures for its assessment of recipients’ performance of training and  technical assistance activities. Without standardized performance  measures, HRSA cannot effectively assess the performance of its  cooperative agreement recipients with respect to the training and  technical assistance they provide to support Health Center Program goals.  For example, HRSA does not require that all training and TA cooperative  agreement recipients be held to a performance measure that would report  the number of successful applicants each cooperative agreement recipient  helped develop in underserved communities, including MUAs.  Standardized performance measures could help HRSA identify how to  better focus its resources to help strengthen the performance of  cooperative agreement recipients.
HRSA officials told us that they are developing performance measures for  the agency’s cooperative agreement recipients, which they plan to  implement beginning with the next competitive funding announcement,  scheduled for fiscal year 2009. However, HRSA officials did not provide  details on the particular measures that it will implement, so it is unclear to  what extent the proposed measures will allow HRSA to assess the  performance of cooperative agreement recipients in supporting Health  Center Program goals through such efforts as developing successful new  access point grant applicants.
HRSA’s oversight is also limited because the agency’s comprehensive on- site reviews of cooperative agreement recipients do not occur as  frequently as HRSA policy states. According to HRSA’s stated policy, the  agency will conduct these reviews for each cooperative agreement  recipient every 3 to 5 years. The reviews are intended to assess—and  thereby potentially improve—the performance of the cooperative  agreement recipients in supporting the overall goals of the Health Center  Program. This support can include helping potential applicants apply for  health center grants, identifying underserved areas and populations across  the country, and helping HRSA increase access to primary care services  for underserved populations.
As part of the comprehensive on-site reviews, HRSA officials consult with  the relevant project officer, examine the scope of the activities  cooperative agreement recipients have described in their work plans and  reported in their progress reports, and develop performance measures in  collaboration with the recipient. Similar to the performance measures in  cooperative agreement recipients’ work plans, the performance measures  used during comprehensive on-site reviews are also individually tailored  and vary by recipient. For example, during these reviews, some recipients  are assessed using performance measures that include the number of  training and technical assistance hours the recipients provided; other  recipients are assessed using measures that include the number of  applicants that were funded after receiving technical assistance from the  recipient or the percentage of the state’s uninsured population that is  served by health center sites in the Health Center Program.
After an assessment, HRSA asks the recipient to develop an action plan. In  these action plans, the reviewing HRSA officials may recommend  additional activities to improve the performance of the specific measures  they had identified during the review. For example, if the agency  concludes that a cooperative agreement recipient needs to increase the  percentage of the state’s uninsured population served by health center  sites in the Health Center Program, it may recommend that the recipient  pursue strategies to develop a statewide health professional recruitment  program and identify other funding sources to improve its ability to  increase access to primary care for underserved people.
Although HRSA’s stated policy is to conduct on-site comprehensive  reviews of cooperative agreement recipients every 3 to 5 years, HRSA is  unlikely to meet this goal for its training and TA cooperative recipients  that target assistance to new access point applicants. In the 4 years since  HRSA implemented its policy for these reviews in 2004, the agency has  evaluated only about 20 percent of cooperative agreement recipients that  provide training and technical assistance to grant applicants. HRSA  officials told us that they have limited resources each year with which to  fund the reviews. However, without these reviews, HRSA does not have a  means of obtaining comprehensive information on the performance of  cooperative agreement recipients in supporting the Health Center  Program, including information on ways the recipients could improve the  assistance they provide to new access point applicants.
section_title: HRSA Provided Unclear Written Feedback to More Than a Third of Unsuccessful Applicants
paragraphs: More than a third of summary statements sent to unsuccessful applicants  from new access point grant competitions held in fiscal years 2005 and  2007 contained unclear feedback. Based on our analysis of 69 summary  statements, we found that 38 percent contained unclear feedback  associated with at least one of the eight evaluative criteria, while   13 percent contained unclear feedback in more than one criterion. We  defined feedback as unclear when, in regard to a particular criterion, a  characteristic of the application was noted as both a strength and a  weakness without a detailed explanation supporting each conclusion. We  found that 26 summary statements contained unclear feedback. We found  41 distinct examples of unclear feedback in the summary statements. (See  table 5.) HRSA’s stated purpose in providing summary statements to  unsuccessful applicants is to improve the quality of future grant  applications. However, if the feedback HRSA provides in these statements  is unclear, it may undermine the usefulness of the feedback for applicants  and their ability to successfully compete for new access point grants.
Based on our analysis, the largest number of examples of unclear  feedback was found in the need criterion, in which applications are  evaluated on the description of the service area, communities, target  population—including the number served, encounter information, and  barriers-—and the health care environment. For example, one summary  statement indicated that the application clearly demonstrated and  provided a compelling case for the significant health access problems for  the underserved target population. However, the summary statement also  noted that the application was insufficiently detailed and brief in its  description of the target population.
Seven of the examples of unclear feedback were found in the response  criterion, in which applications are evaluated on the applicant’s proposal  to respond the target population’s need. One summary statement indicated  that the application detailed a comprehensive plan for health care services  to be provided directly by the applicant or through its established linkages  with other providers, including a description of procedures for follow-up  on referrals or services with external providers. The summary statement  also indicated that the application did not provide a clear plan of health  service delivery, including accountability among and between all  subcontractors.
section_title: Conclusions
paragraphs: Awarding new access point grants is central to HRSA’s ongoing efforts to  increase access to primary health care services in MUAs. From 2006 to  2007, HRSA’s recent new access point awards achieved modest success in  reducing the percentage of MUAs nationwide that lacked a health center  site. However, in 2007, 43 percent of MUAs continue to lack a health  center site, and the new access point awards made in 2007 had little  impact on the wide variation among census regions and states in the  percentage of MUAs lacking a health center site. The relatively small effect  of the 2007 awards on geographic variation may be explained, in part,  because the South census region received a greater number of awards  than other regions, even though the South was not the region with the  highest percentage of MUAs lacking a health center site in 2006.
HRSA awards new access point grants to open new health center sites,  thus increasing access to primary health care services for underserved  populations in needy areas, including MUAs. However, HRSA’s ability to  target these awards and place new health center sites in locations where  they are most needed is limited because HRSA does not collect and  maintain readily available information on the services provided at  individual health center sites. Having readily available information on the  services provided at each site is important for HRSA’s effective  consideration of need when distributing federal resources for new health  center sites because each health center site may not provide the full range  of comprehensive primary care services. This information can also help  HRSA assess any potential overlap of services provided by health center  sites in MUAs.
HRSA could improve the number and quality of grant applications it  receives—and thereby broaden its potential pool of applicants—by better  monitoring the performance of cooperative agreement recipients that  assist applicants and by ensuring that the feedback unsuccessful  applicants receive is clear. However, limitations in HRSA’s oversight of the  training and TA cooperative agreement recipients hamper the agency’s  ability to identify recipients most in need of assistance. Because HRSA  does not have standardized performance measures for these recipients— either for their work plan activities or for the comprehensive on-site  reviews—the agency cannot assess recipients’ performance using  comparable measures and determine the extent to which they support the  overall goals of the Health Center Program. One standardized performance  measure that could help HRSA evaluate the success of cooperative  agreement recipients that assist new access point applicants is the number  of successful grant applicants each cooperative agreement recipient  develops; this standardized performance measure could assist HRSA in  determining where to focus its resources to strengthen the performance of  cooperative agreement recipients.
HRSA’s allocation of available resources has made it unlikely that it will  meet its goal of conducting comprehensive on-site reviews of each  cooperative agreement recipient every 3 to 5 years. Without these reviews,  HRSA does not have comprehensive information on the effectiveness of  training and TA cooperative agreement recipients in supporting the Health  Center Program, including ways in which they could improve their efforts  to help grant applicants. Given the agency’s concern regarding available  resources for its comprehensive on-site reviews, developing and  implementing standardized performance measures for training and TA  cooperative agreement recipients could assist HRSA in determining the  cost-effectiveness of its current comprehensive on-site review policy and  where to focus its limited resources.
HRSA could potentially improve its pool of future applicants by increasing  the extent to which it provides clear feedback to unsuccessful applicants  on the strengths and weaknesses of their applications. HRSA intends for  these summary statements to be used by applicants to improve the quality  of future grant applications. However, the unclear feedback HRSA has  provided to some unsuccessful applicants in fiscal years 2005 and 2007  does not provide those applicants with clear information that could help  them improve their future applications. This could limit HRSA’s ability to  award new access point grants to locations where such grants are needed  most.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Administrator of HRSA take the following four  actions to improve the Health Center Program:    Collect and maintain readily available data on the types of services  provided at each health center site to improve the agency’s ability to  measure access to comprehensive primary care services in MUAs.
Develop and implement standardized performance measures for training  and TA cooperative recipients that assist applicants to improve HRSA’s  ability to evaluate the performance of its training and TA cooperative  agreements. These standardized performance measures should include a  measure of the number of successful applicants a recipient assisted.
Reevaluate its policy of requiring comprehensive on-site reviews of Health  Center Program training and TA cooperative agreement recipients every   3 to 5 years and consider targeting its available resources at  comprehensive on-site reviews for cooperative agreement recipients that  would benefit most from such oversight.
Identify and take appropriate action to ensure that the discussion of an  applicant’s strengths and weaknesses in all summary statements is clear.
section_title: Agency Comments and Our Evaluation
paragraphs: In commenting on a draft of this report, HHS raised concerns regarding  the scope of the report and one of our recommendations and concurred  with the other three recommendations. (HHS’s comments are reprinted in  app. III.) HHS also provided technical comments, which we incorporated  as appropriate.
HHS said its most significant concern was with our focus on MUAs and the  exclusion of MUPs from the scope of our report. In our analysis, we  included the health center sites of 90 percent of all Health Center Program  grantees. We excluded from our review sites that were associated with the  remaining 10 percent of grantees that received HRSA funding to serve  specific MUPs only because they are not required to serve all residents of  the service area. Given our research objective to determine the location  of health center sites that provide services to residents of an MUA, we  excluded these specific MUPs and informed HRSA of our focus on health  center sites and MUAs. We agree with HHS’s comment that it could be  beneficial to have information on the number of grants awarded to  programs serving both MUAs and MUPs generally to fully assess the  coverage of health center sites.
HHS also commented that our methodology did not account for the  proximity of potential health center sites located outside the boundary of  an MUA. While we did not explicitly account for the proximity of potential  health center sites located outside an MUA, we did include the entire area  of all zip codes associated with an MUA. As a result, the geographic  boundary of an MUA in our analysis may be larger than that defined by  HRSA, so our methodology erred on the side of overestimating the number  of MUAs that contained a health center site.
With regard to our reporting on the percentage of MUAs that lacked a  health center site, HHS stated that this indicator may be of limited utility,  because not all programs serving MUAs and MUPs are comparable to each  other due to differences in size, geographic location, and specific  demographic characteristics. Specifically, HHS commented that our  analysis presumed that the presence of one health center site was  sufficient to serve an MUA. In our work, we did not examine whether  MUAs were sufficiently served because this was beyond the scope of our  work. Moreover, since HRSA does not maintain site-specific information  on services provided and each site does not provide the same services, we  could not assess whether an MUA was sufficiently served. HHS also noted  that a health center site may not be the appropriate solution for some  small population MUAs; however, we believe it is reasonable to expect  that residents of an MUA—regardless of its size, geographic location, and  specific demographic characteristics—have access to the full range of  primary care services.
With regard to our first recommendation that HRSA collect and maintain  site-specific data on the services provided at each health center site, HHS  acknowledged that site-specific information would be helpful for many  purposes, but it said collecting this information would place a significant  burden on grantees and raise the program’s administrative expenses. We  believe that having site-specific information on services provided would  help HRSA better measure access to comprehensive primary health care  services in MUAs when considering the placement of new health center  sites and facilitate the agency’s ability to evaluate service area overlap in  MUAs.
HHS concurred with our three other recommendations. With regard to our  second recommendation, HHS stated that HRSA will include standardized  performance measures with its fiscal year 2009 competitive application  cycle for state PCAs and that HRSA plans to develop such measures for  the national training and TA cooperative agreement recipients in future  funding opportunities. With regard to our third recommendation, HHS  commented that HRSA has developed a 5-year schedule for reviewing all  state PCA grantees. HHS also stated that HRSA is examining ways to  better target onsite reviews for national training and TA cooperative  agreement recipients that would most benefit from such a review. Finally,  HHS agreed with our fourth recommendation and stated that HRSA is  continuously identifying ways to improve the review of applications.
As arranged with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution of it until 30 days after  its issue date. At that time, we will send copies of this report to the  Secretary of HHS, the Administrator of HRSA, appropriate congressional  committees, and other interested parties. We will also make copies of this  report available to others upon request. In addition, the report will be  available at no charge on the GAO Web site at http:///www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-7114 or bascettac@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Staff members who made major contributions to this report  are listed in appendix IV.
section_title: Appendix I: Number and Percentage of Medically Underserved Areas (MUA) Lacking a Health Center Site, 2006 and 2007
section_title: Appendix II: Data on the 2007 High Poverty County New Access Point Competition, by Census Region and State
section_title: Appendix III: Comments from the U.S. Department of Health and Human Services
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Nancy Edwards, Assistant  Director; Stella Chiang; Krister Friday; Karen Howard; Daniel Ries; Jessica  Cobert Smith; Laurie F. Thurber; Jennifer Whitworth; Rachael Wojnowicz;  and Suzanne Worth made key contributions to this report.