Health Resources and Services Administration: Action Taken to Train and Oversee Grantee Monitoring Staff, but Certain Guidance Could Be Improved
Published: 2014-09-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In fiscal year 2013, HRSA awarded over $6 billion in grants, which accounted for over 80 percent of the agency's appropriations. HRSA's workforce—its staff and contractors— is responsible for monitoring the agency's grantees to ensure they are financially sound, using federal funds appropriately, and in compliance with program requirements. In recent years, GAO has reported on weaknesses in HRSA's monitoring of certain grant programs and made recommendations for the agency to improve grantee monitoring.
GAO was asked to review HRSA's oversight mechanisms for its grantee monitoring workforce. This report examines (1) the extent to which HRSA has developed guidance for staff who monitor grantees; (2) the extent to which it implemented training for this staff; (3) HRSA's oversight of its staff; and (4) its practices to ensure contractors who conduct grantee monitoring activities are qualified and carry out their work as appropriate. GAO reviewed HRSA training documentation and guidance related to grantee monitoring and interviewed HRSA officials. GAO also reviewed the contracts that HRSA identified as including grantee monitoring activities in fiscal year 2013.
section_title: What GAO Found
paragraphs: In 2012, the Department of Health and Human Services' (HHS) Health Resources and Services Administration (HRSA) began systematically developing guidance for the key staff involved in grantee monitoring—project officers in its programmatic bureaus and grants management specialists and financial integrity staff in its Office of Federal Assistance Management (OFAM). Specifically, HRSA issued the first agency-wide guidance written primarily for project officers in November 2012 and is in the process of developing new guidance and revising existing guidance for staff in OFAM. Additionally, HRSA required each of the bureaus to develop more detailed guidance for project officers by the end of 2012. To assist the bureaus in systematically developing this guidance, HRSA provided a template outlining the minimum components to be included in the documents. While all of HRSA's bureaus developed guidance for their project officers, GAO found that the bureaus' guidance did not address all of the key components of grantee monitoring identified by HRSA, such as components related to conducting site visits. Although HRSA conducted reviews of the bureaus' guidance, these reviews were not sufficient to ensure that the guidance addressed all required grantee monitoring components. Consequently, there is an increased possibility that project officers will not conduct, or document, all the monitoring activities required by HRSA, which puts the agency at risk for incomplete or insufficient grantee monitoring.
Since 2013, HRSA has implemented agency-wide training programs for key staff responsible for conducting monitoring of grantees—a project officer certification program, which the majority of project officers have completed, and a career development program for grants management specialists. Both programs include courses that cover material related to grantee monitoring. In addition to these training programs, the bureaus and OFAM offer formal and informal training related to grantee monitoring, such as training on specific grant program requirements.
HRSA has four main methods for overseeing its staff's monitoring of grantees. According to HRSA officials, supervisors
review information in, and reports from, HRSA's online system for documenting grantee monitoring activities, such as staff's assessments of grantee compliance;
regularly communicate with staff in one-on-one and other meetings; and
conduct annual performance appraisals to hold staff accountable for their monitoring responsibilities.
Additionally, HRSA has mechanisms in place to ensure that the contractors that conduct grantee monitoring activities are qualified, including reviewing staffing proposals prior to awarding the contract, requiring contractor staff to have specific qualifications, and requiring training for contractor staff. To oversee contractors' work, HRSA staff regularly communicate with the contractors, review their deliverables, and obtain feedback from project officers and grantees on contractor performance.
section_title: What GAO Recommends
paragraphs: GAO recommends that HRSA develop a process to ensure that the guidance developed by its bureaus addresses all of the key components of grantee monitoring as established by the agency. HHS agreed with GAO's recommendation and said HRSA has begun to take action to implement it.

section_title: Letter
paragraphs: The Health Resources and Services Administration (HRSA), an agency  within the Department of Health and Human Services (HHS), is charged  with improving access to health care services for people who are  uninsured, isolated, or medically vulnerable and enhancing the capacity  of the health care workforce. To carry out its mission, HRSA provides  funding and support for a wide variety of programs, most commonly  through grants. In fiscal year 2013, HRSA awarded over $6 billion in  grants, which accounted for over 80 percent of the agency’s  appropriations. Through these grants, HRSA provided financial support to  more than 3,000 recipients, including organizations in every state. These  grantees serve millions of people each year through programs that  provide health care to uninsured individuals; people with human  immunodeficiency virus and acquired immunodeficiency syndrome  (HIV/AIDS); pregnant women, mothers, and children; people living in rural  communities; and others.
To manage its grant programs, HRSA employs project officers in its  programmatic bureaus and staff in its Office of Federal Assistance  Management (OFAM). These staff are supplemented by contractors that  perform a variety of tasks to support HRSA’s programs and operations.  HRSA’s workforce—its staff and contractors—is responsible for  monitoring grantees. Specifically, they are responsible for collecting and  assessing information from reports, audits, site visits, and other sources  to ensure that grantees are in compliance with the terms of their grants.  The effective monitoring of grantees is critical to ensure that they are  financially sound, in compliance with programmatic and administrative  requirements, and appropriately utilizing federal funds. In addition,  monitoring is essential to ensure that grantees receive the support and  assistance they need so that funding is used as effectively as possible for  the populations HRSA serves. However, in 2012 we reported on  weaknesses in HRSA’s monitoring of certain grant programs. Specifically,  we reported that HRSA did not consistently follow HHS regulations and  guidance in its monitoring of grantees under the Ryan White  Comprehensive AIDS Resources Emergency Act of 1990, as amended.  We also reported that HRSA’s monitoring of its Health Center Program  grantees was insufficient to ensure that the agency consistently identified  all instances of grantee noncompliance with program requirements. We  recommended actions to improve HRSA’s monitoring of these grantees.  HHS concurred with our recommendations, some of which have already  been implemented by HRSA.
You asked us to review HRSA’s management and oversight mechanisms,  including the systems HRSA has in place to train and oversee its  workforce responsible for monitoring grantees. In this report, we examine  (1) the extent to which HRSA has developed guidance for its staff  responsible for monitoring grantees; (2) the extent to which HRSA has  implemented training for its staff responsible for monitoring grantees;   (3) HRSA’s oversight of its staff responsible for monitoring grantees; and  (4) what practices, if any, HRSA has in place to ensure that the  contractors conducting grantee monitoring activities are qualified and  carry out their work as appropriate.
To examine the extent to which HRSA has developed guidance for its  staff responsible for monitoring grantees, we reviewed guidance that  HRSA developed related to monitoring grantees, including standard  operating procedures (SOP) developed by HRSA’s programmatic  bureaus and OFAM. We assessed HRSA’s guidance on grantee  monitoring against federal internal control standards related to  monitoring, as well as standards that HRSA established for the guidance  developed by its bureaus. In addition, we conducted interviews with  HRSA officials, including representatives from HRSA’s SOP workgroup— assembled by HRSA to facilitate the bureaus’ development of guidance  on monitoring grantees—and leadership of OFAM and each of the  bureaus about guidance for staff on grantee monitoring at both the  agency and bureau level. We also conducted interviews with groups of  four to six project officers and their supervisors from each of four selected  bureaus—the Bureau of Health Professions, the Bureau of Primary  Health Care, the HIV/AIDS Bureau, and the Maternal and Child Health  Bureau—in order to gain their perspectives and experiences with HRSA’s  guidance on grantee monitoring. The four selected bureaus had the  largest number of project officers and active grants and the greatest  amount of grant awards in fiscal year 2013.that varied in their level of HRSA experience (including new project  officers hired after June 1, 2013) and the number of grants that they  monitored.
To examine the extent to which HRSA has implemented training for its  staff responsible for monitoring grantees, we reviewed documentation of  HRSA’s training, including training materials and announcements, and  HRSA’s mechanisms for tracking and evaluating training. We assessed  HRSA’s training practices against leading federal practices for workforce  training. In addition, we interviewed HRSA officials, including  representatives from the HRSA Learning Institute—the training center  HRSA established in 2010—and leadership from OFAM and each of the  bureaus. We also used our small group interviews with project officers  and their supervisors to gain their perspectives and experience with the  training HRSA has implemented related to grantee monitoring.
To examine HRSA’s oversight of its staff responsible for monitoring  grantees, we reviewed HRSA’s written policies and documentation  related to oversight of staff, including sample performance appraisals and  performance elements for staff with grantee monitoring responsibilities.  We discussed methods used to oversee staff responsible for grantee  monitoring during our interviews with HRSA, OFAM, and bureau  leadership, as well as during our project officer and supervisor small  group interviews.
To examine what practices, if any, HRSA has in place to ensure that the  contractors conducting grantee monitoring activities are qualified and  carry out their work as appropriate, we reviewed and analyzed the 13  contracts HRSA identified as including grantee monitoring activities, such  as grantee site visits and financial reviews, in fiscal year 2013.appendix I for more information on each of the contracts we reviewed. We  also discussed oversight of contractors during our interviews with HRSA  and OFAM leadership, leaders from the Bureau of Primary Health Care  and the HIV/AIDS Bureau—the two bureaus that used contractors to  conduct grantee monitoring activities in fiscal year 2013—as well as our  small group interviews with project officers and supervisors from these  two bureaus. In addition, we interviewed representatives from the  contractors with the two largest contracts (based on dollar amount of   See  obligations), as well as the HRSA officials responsible for oversight of  these contractors.
We conducted this performance audit from July 2013 to September 2014  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: HRSA provides grants to a variety of entities using the HHS grant  management process, which consists of seven phases. The first five  phases of the grant management process cover the planning of the grant  funding opportunity through the awarding of the grant funds. The sixth  phase of the process, which occurs postaward, is the monitoring phase  during which HRSA is to review grantees to identify potential problems  and areas of noncompliance with programmatic and administrative  requirements that must be addressed by the grantee. This monitoring  continues for as long as HRSA retains a financial interest in the grantee’s  activities and is followed by the seventh and final phase—closeout.
Within HRSA, the grants management process, including grantee  monitoring, is a coordinated effort of project officers located in the  programmatic bureaus, and grants management specialists and financial  integrity staff in OFAM.
Project officers. The nearly 400 project officers within the bureaus  have primary responsibility for the day-to-day monitoring of an  assigned portfolio of grantees. Project officer monitoring activities  include reviewing grantee reports on various topics to ensure  performance requirements are met and to identify performance or  financial problems; communicating with grantees regularly and  documenting that communication; consulting with grantees on  programmatic issues; and conducting and arranging for site visits. For  example, project officers conduct the initial review of requests from  grantees for prior approval of certain actions, such as carrying over  unobligated funds to subsequent funding periods; review grantee  progress and financial reports; and are generally expected to have at  least quarterly contact with their grantees. They are expected to keep  their supervisors and relevant OFAM staff informed regarding grantee  performance and the need for any intervention.
Grants management specialists. Located in OFAM’s Division of  Grants Management Operations, grants management specialists are  responsible for managing business and financial issues for a portfolio  of grantees. For example, they review the Federal Financial Report, a  required report to document the financial progress of a grant award,  which includes information on the funds a grantee has received and  how they were spent. Grants management specialists and project  officers are expected to coordinate in monitoring grantees; for  example, grants management specialists are responsible for  approving or disapproving prior approval requests after the project  officer’s initial review. As of June 2014, there were nearly 70 grants  management specialists within OFAM.
Financial integrity staff. Located in OFAM’s Division of Financial  Integrity, financial integrity staff are responsible for the financial  oversight and monitoring of grantees, which includes conducting or  coordinating financial audits and reviews of grantees. For example,  some financial integrity staff conduct program integrity analyses to  ensure that financially at-risk grantees are identified for targeted  intervention; others work with grantees to ensure the resolution of  financial audit findings. As of June 2014, there were 34 financial  integrity staff with grantee monitoring responsibilities.
HRSA supplements its staff with contractors that perform grantee  monitoring activities such as conducting site visits to monitor compliance  with grant requirements, or financial audits and reviews. Each contractor  is overseen by a HRSA contracting officer’s representative (COR) who is  generally located within the bureau or office that requested the grantee  monitoring support, but works on behalf of the contracting officer. The  COR is responsible for overseeing certain aspects of the contractor’s  performance, including reviewing required reports and assessing  timeliness of deliverables.
section_title: HRSA Has Developed Guidance for Staff on Grantee Monitoring, but Bureau-Level Guidance Did Not Address All of the Key Components of Monitoring
paragraphs: HRSA began systematically developing guidance for the key staff  responsible for grantee monitoring in 2012. However, we found that  guidance developed by the bureaus for project officers did not address all  of the key components of grantee monitoring established by HRSA.
subsections: section_title: HRSA Began Systematically Developing Guidance in 2012 for the Key Staff Responsible for Grantee Monitoring
paragraphs: In 2012, HRSA began systematically developing guidance for project  officers and staff in OFAM responsible for monitoring grantees.  Specifically, HRSA issued the first agency-wide guidance written primarily  for project officers in November 2012, and it required each of the bureaus  to develop more detailed guidance for project officers by the end of 2012.  In addition, HRSA is in the process of developing new guidance and  revising existing guidance for grants management specialists and  financial integrity staff in OFAM. Collectively, this guidance addresses the  2007 HHS Awarding Agency Grants Administration Manual requirement  for each agency to develop monitoring plans.
Agency-wide guidance for project officers. HRSA issued its Grantee  Oversight Manual in November 2012—the first agency-wide guidance  developed primarily for project officers. This manual, updated in 2014, is  intended to outline a common set of minimum requirements for monitoring  grantees. It generally describes the roles and responsibilities of project  officers and other staff responsible for conducting monitoring of grantees  and builds on guidance that HRSA and HHS had previously developed.  Prior to the issuance of the HRSA Grantee Oversight Manual, HRSA’s  main source of guidance on monitoring of grantees was OFAM’s 2009  Division of Grants Management Operations Operational and Procedures  Manual; however, this manual was primarily written for grants  management specialists in OFAM. Additionally, HRSA has periodically  issued Grants Policy Operations Memoranda to provide guidance to staff  on various topics, some of which are relevant to project officers’  monitoring of grantees, such as carrying forward unobligated funds to  subsequent funding periods. Project officers also relied on the  overarching HHS policies for guidance on monitoring of grantees,  including the Awarding Agency Grants Administration Manual.
Bureau-specific guidance for project officers. In addition to developing  the agency-wide guidance, HRSA required each of its bureaus to develop  monitoring plans and document them in the form of SOPs. HRSA  specified that these SOPs should supplement the Grantee Oversight  Manual by including more detailed, program-specific monitoring  requirements and be developed by the end of 2012. To assist the  bureaus in systematically developing their SOPs, HRSA provided the  bureaus with a template outlining the minimum components to be  included in their documents and established a workgroup to facilitate  implementation of the SOPs across the agency. At the time of our review,  all of the bureaus had developed SOPs. Officials from four of the bureaus  told us that, prior to the agency’s 2012 requirement to develop SOPs,  they did not have any in place.
In addition to SOPs, HRSA officials described other sources of guidance  developed by the bureaus, such as site visit guides, policy notices,  frequently asked questions, checklists, and templates.and supervisors we interviewed generally found the guidance on grantee   Project officers  monitoring to be helpful, and one project officer who had been with HRSA  for over 10 years stated that the level of guidance provided to project  officers has significantly improved in recent years. However, a few project  officers and supervisors we interviewed identified areas where they could  use additional guidance, such as guidance on specific requirements for  certain programs and additional guidance on the information that should  be entered into HRSA’s Electronic Handbooks (EHB)—the agency’s  online system for documenting grantee monitoring activities.
Guidance for OFAM staff. OFAM has also developed new guidance for  grants management specialists and financial integrity staff. The agency is  in the process of replacing the 2009 Division of Grants Management  Operations Operational and Procedures Manual for grants management  specialists with a series of SOPs. In addition to making necessary  updates, HRSA officials said that this approach would allow them greater  flexibility in revising and updating the written guidance in the future. As of  July 2014, OFAM had developed 4 SOPs for grants management  specialists related to grantee monitoring—such as “Reviewing Financial  Reports”—and indicated plans to develop approximately 10 more. HRSA  officials said that staff can still rely on the guidance provided by the  Operational and Procedures Manual during the transition to SOPs. In  addition to the guidance for grants management specialists, OFAM’s  Division of Financial Integrity has 3 SOPs related to grantee monitoring  for financial integrity staff—”Audit Resolution,” “Special Reviews,” and  “EHB Financial Assessments”—all of which were developed or revised  since 2013.
section_title: Bureau-Level Guidance Did Not Address All of the Key Components of Grantee Monitoring
paragraphs: While HRSA’s bureaus have developed bureau-level guidance for project  officers on monitoring of grantees, we found that the bureaus’ SOPs did  not address all of the key components of grantee monitoring identified by  HRSA. Federal internal control standards state that management is  responsible for developing detailed policies to fit their agency’s  operations. HRSA specified the detail that bureaus should include in  their SOPs by providing a template outlining the key components of  monitoring that should, at a minimum, be included. The key components  of monitoring identified in the template fall into three main categories— communication, reviewing grantee reports, and conducting site visits.
Across the 13 SOPs that were developed by the bureaus as of April 2014,  we found that the SOPs most thoroughly addressed the components  related to communication and least thoroughly addressed those related to  conducting site visits. Specifically, 7 of the 13 SOPs fully addressed the  six key components related to communication; each of the remaining 6  SOPs did not fully address one communication component. The  communication components that were not fully addressed by the SOPs  were specifying the frequency and purpose of communication with  grantees (3 SOPs), the process for communicating concerns or  noncompliance (1 SOP), and coordinating with OFAM (2 SOPs). For  reviewing grantee reports, 5 of the 13 SOPs fully addressed the five key  components. Of the 8 remaining SOPs, the majority did not fully  address at least two components and only one of these SOPs addressed  the component on documentation of grantee compliance. Of the 10  SOPs for which the site visit components were applicable, none fully  addressed all nine key site visit components. More than half of the  SOPs did not fully address at least four of the nine site visit components,  and 1 SOP did not address any of them. Additionally, one component in  particular—requirements for tracking site visits—was not addressed by  any of the SOPs. (See fig. 1 for information about the extent to which  the SOPs addressed each of the key components of grantee monitoring  by category.)
Members of HRSA’s SOP workgroup told us that they conducted an  initial, high-level review of the bureaus’ SOPs to assess their consistency  with the template, but the review was not formal or in-depth. Additionally,  leaders from each of the bureaus reported having processes for internally  developing and reviewing the SOPs, such as establishing internal  workgroups, soliciting participation and feedback from project officers,  reviewing the HRSA template, and conducting reviews at the bureau- leadership level. However, neither the SOP workgroup’s review nor those  done by the bureaus ensured that the SOPs addressed all required  grantee monitoring components. Because the SOPs do not include the  level of detail specified by HRSA in the template, there is an increased  possibility that project officers will not conduct, or appropriately document,  all of the required monitoring activities, which puts HRSA at greater risk  for having incomplete or ineffective grantee monitoring.
section_title: HRSA Implemented Agency-Wide Training Programs for Grantee Monitoring Staff Beginning in 2013; Bureau-Level Training Varied
paragraphs: Since 2013, HRSA has implemented agency-wide training programs for  key staff responsible for monitoring grantees, including a project officer  certification program and a training program for grants management  specialists. HRSA’s project officer certification program, launched in  May 2013, includes a standard set of seven courses that were to be  completed by all experienced project officers by April 30, 2014. Under  the program, new project officers hired after June 1, 2013, are to  complete 40 hours of self-study and in-person training within 6 months of  being hired. HRSA reported that 86 percent of experienced project  officers completed the certification program by the April 2014 deadline  and that 90 percent of new project officers completed the program by  June 2014. After the initial certification, all project officers are required  to meet annual continuing education requirements. Both the new and  experienced project officer training includes courses that cover material  related to grantee monitoring. For example, these courses cover the  goals of grantee monitoring, project officer responsibilities, sources of  information, and helpful tips for effective monitoring of grantees. Project  officers, supervisors, and bureau-leadership officials we interviewed said  they were generally satisfied with the certification program, and that the  level of training it provides is sufficient and helpful.
HRSA also initiated a new training program for grants management  specialists in 2014. HRSA officials described this as a career  development program with tracks at the entry, mid-career, or advanced- career level; staff work with their supervisors to determine which track is  most appropriate. Officials said that each level offers grants management  courses with a focus on monitoring. For example, there are courses that  cover grants management specialist responsibilities, coordinating with  project officers, and various types of reports that grants management  specialists are responsible for reviewing.
Competencies describe measurable patterns of knowledge, skills, abilities, behaviors,  and other characteristics that an individual needs to perform work roles or occupational  functions successfully. Understood another way, competencies specify what the person  needs to do the job successfully. See GAO-13-591.
September 2014 for project officers to provide feedback and suggestions  for improving the program. Officials noted that moving forward, the focus  for project officer training will be on continuing education as all existing  project officers should have a foundation of knowledge from completing  the initial certification.
In addition to agency-wide training programs, the bureaus provided their  own training—such as training on specific grant programs—to varying  degrees. Three bureaus—the Bureau of Health Professions, the Bureau  of Primary Health Care, and the HIV/AIDs Bureau—provided some formal  training on grantee monitoring in 2013. These bureaus, which  represented three of the four largest bureaus at HRSA in 2013, offered  trainings ranging from a single course on grantee performance to a series  of courses on specific grant program requirements and monitoring  activities, such as financial monitoring and site visits. Officials from the  HIV/AIDs Bureau said they had mechanisms in place for formally tracking  and evaluating the training they offered. Leadership from three of the four  bureaus that did not provide formal training related to grantee monitoring  explained that their bureaus are relatively small. Thus, the officials  indicated that their project officers benefit from frequent communication  with their supervisors in lieu of formal training. Leadership from the  remaining bureau that did not provide formal training said that because  the bureau has many different programs with different requirements, it  would not make sense to offer formal bureau-wide training.
Officials from all of the bureaus described various informal training  mechanisms in place that help provide project officers with the knowledge  and skills to conduct their work, including one-on-one meetings between  project officers and their supervisors, mentoring programs, and having  new project officers shadow more experienced project officers on site  visits prior to conducting them. For example, the Bureau of Health  Professions developed a tool for new project officers that specifies  timelines for completing certain tasks that will prepare them for their role  as project officers within the bureau, such as reviewing key bureau and  HRSA guidance, shadowing a senior project officer and identifying a  mentor, and reviewing bureau resources. In addition, supervisors we  spoke with generally agreed that it is their responsibility to provide on-the- job training to project officers, particularly related to program-specific  grant requirements. While project officers and supervisors we interviewed  generally felt that the training provided by HRSA and the bureaus related  to grantee monitoring was sufficient, a few identified areas where project  officers could use more training; for example, training on financial  monitoring, effectively conducting monitoring calls with a large number of  grantees, and documenting monitoring activities in EHB. A few more  experienced project officers told us that training on grantee monitoring  has improved significantly in recent years.
Similar to the bureaus, OFAM officials described additional training for  their staff related to monitoring grantees. For example, OFAM officials  told us they have offered training related to the new SOPs for grants  management specialists, such as sessions on federal financial reviews,  and they plan to provide training on each of the additional SOPs as they  are developed and rolled out to staff. Officials said that the expectation is  that all staff participate in these trainings and that they are tracking staff  completion. OFAM officials said they also provide training to financial  integrity staff on changing requirements and relevant topics, such as  consistent treatment of audit findings, on an as-needed basis.
section_title: HRSA Has Four Main Methods for Overseeing Its Staff’s Monitoring of Grantees
paragraphs: HRSA has four main methods for overseeing project officers, grants  management specialists, and financial integrity staff that are responsible  for monitoring its grantees—supervisors reviewing information in, and  reports from, EHB; participating in staff interactions with grantees;  communicating regularly with staff; and conducting performance  appraisals.
Reviewing information in, and reports from, EHB. HRSA officials and  supervisors we interviewed identified reviewing information in, and reports  from, EHB as one of the main methods for overseeing staff responsible  for monitoring grantees; EHB is the agency’s online system for  documenting grantee monitoring activities, as well as corresponding and  exchanging documents with grantees. There are three main ways that  supervisors use EHB for oversight purposes.
Supervisors review certain documents or decisions made by staff  about grantees for quality, accuracy, and completeness; the EHB  system requires supervisory review and approval to occur before  certain grant-related actions can be taken. Examples of items  requiring supervisory review and approval in EHB include project  officers’ assessments of grantee compliance, noncompliant grantees’  plans and time frames to address their noncompliance, and grantee  prior approval requests, as well as certain financial assessments that  are completed by financial integrity staff. Additionally, officials told us  that there are multiple levels of required supervisory review and  approval in EHB for certain monetary actions made by grants  management specialists. In reviewing staff’s work, supervisors told us  that they consider the documentation used to support staff’s  recommendations. For example, when reviewing a project officer’s  assessments of grantee compliance, a supervisor may review the  reports submitted by the grantee, which also are to be documented in  EHB. In doing so, the supervisor is able to determine if the EHB  record includes the necessary documentation and if that  documentation supports the project officer’s assessment, and then  provide feedback to the project officer.
Supervisors also spot-check staff’s day-to-day activities in EHB to  oversee their work. For example, at any time a project officer’s  supervisor can access a grantee’s EHB record to assess whether a  project officer has communicated with the grantee at the required  intervals and documented that communication. By reviewing the  documentation of the communications, the supervisor can determine  the information the project officer and grantee discussed. Supervisors  can also see whether site visit reports have been entered or uploaded  into EHB within the established time frames, and can review the audit  reports completed by financial integrity staff, including reviewing the  supporting materials from EHB.
Supervisors also use EHB-generated reports to oversee staff.
According to the HRSA officials we spoke with, supervisors generate  EHB reports to review information, such as the number and types of  tasks in a staff member’s queue, the timeliness of task completion,  and whether anything is overdue.
Participating in interactions with grantees. Supervisors provide  oversight of staff by participating in some of their interactions with  grantees. According to HRSA officials, supervisors may participate in  monitoring calls between project officers and grantees. Supervisors may  also review project officers’ email or other written correspondence in  advance of it being sent to grantees, or may be copied on email  exchanges between project officers and grantees, so that they are  cognizant of grantee interactions. In addition, supervisors may participate  in grantee site visits with project officers. According to officials,  participating in these interactions allows supervisors to observe how the  staff interact with the grantees, how they handle any problems that come  up, and the customer service staff provide. If they identify any problems  or concerns with these interactions, supervisors told us they discuss it  with their staff.
Communicating regularly with staff. Supervisors use regular  communications with staff as another oversight method. For example,  according to HRSA officials, supervisors hold individual, one-on-one,  meetings with members of their staff on a regular basis, ranging from  weekly to quarterly. Two officials said that the frequency of one-on-one  meetings varies based on the staff’s experience. Topics discussed during  these meetings include updates on the grantees that the staff is  responsible for monitoring, any grantee compliance issues, upcoming or  recent site visits, and submissions from grantees. In preparation for these  meetings, officials we interviewed reported that supervisors often review  the staff’s EHB documentation, and then discuss it during their meetings.  Officials from one bureau stated that supervisors use EHB information to  guide conversations with their staff during their one-on-one meetings to  discuss grantee progress, concerns, and best practices. In addition to  one-on-one meetings, some supervisors hold regular group meetings with  all of their staff responsible for grantee monitoring, where, among other  things, staff can report on any issues they are encountering with their  grantees or ask questions of their peers or supervisors.
Conducting performance appraisals. HRSA also uses its annual  performance appraisal process to hold its staff accountable for their  grantee monitoring responsibilities. The performance appraisals for all  staff with responsibility for monitoring grantees include performance  elements related to monitoring. The specific elements upon which staff  are assessed are based on the roles and responsibilities for their job  function, and can vary by individual. Table 1 provides examples of  performance elements related to monitoring for the three key positions  with responsibility for grantee monitoring. In addition, supervisors and  bureau and office leadership also are held accountable through their  performance appraisals for their oversight of the staff who perform  grantee monitoring activities. For example, HRSA officials told us bureau  and office leadership are held accountable for overseeing the monitoring  of grantees through the business acumen element of their appraisal,  under which they are expected to ensure program integrity and  responsible stewardship of resources.
section_title: HRSA Has Several Mechanisms in Place Designed to Ensure Contractors That Monitor Grantees Are Qualified and to Oversee Their Work
paragraphs: HRSA has several mechanisms in place designed to ensure that HRSA  contractors that conduct grantee monitoring activities are qualified and to  oversee the work performed by these contractors. HRSA officials told us  that the agency’s process for awarding contracts is designed to ensure  the contractors that conduct grantee monitoring activities are qualified. As  part of the process for awarding such contracts, HRSA considers staffing  proposals and the expectations outlined in the statement of work—a  section of the contract solicitation that describes the specific tasks  required of the contractor. For all 13 contracts that HRSA identified as  including grantee monitoring activities in fiscal year 2013, the personnel  proposed by contractors were a factor in HRSA’s evaluation and selection  of the contractors. For each of these contracts, HRSA required  prospective contractors to submit staff résumés and, for some contracts,  also required letters of commitment or other information regarding the  expertise and qualifications for certain staff designated as “key  personnel”—contractor staff who are proposed to lead the work, such as  the project director. Officials told us they considered the experience of  these individuals when evaluating the potential contractors.
For some contracts, HRSA identified specific qualifications required for  contractor staff. Nine of the 13 contracts—4 used by OFAM for financial  reviews and 5 used by programmatic bureaus for grantee monitoring— included specific requirements for the qualifications or characteristics of  contractor staff. For example, the OFAM contracts included information  on educational requirements, years of experience, and skill sets needed  for certain positions including senior financial analysts and senior  auditors. One of the bureau contracts required contractor site visit staff to  have specific clinical experience, as well as experience with  administrative and management information systems, fiscal management,  and familiarity with the support services available for the population  served by the grant.
Additionally, almost all of the contracts (12 of 13) included a clause that  specifies procedures for the contractor to follow if it makes a change to  key personnel. For example, some contracts stated that if the contractor  needed to change any key personnel, it would need to notify HRSA in  advance in writing and provide justification for the change. The contractor  would then need to receive written consent from HRSA to make the  change in personnel.
After a contractor is selected, HRSA has other mechanisms designed to  ensure that the contractor staff conducting grantee site visits are qualified  to conduct this monitoring activity. For example, some of the contracts  used by the HIV/AIDS Bureau and the Bureau of Primary Health Care  stated that the contractor is responsible for providing training to its staff.  One contract specified that contractor staff must achieve a particular  score on a posttraining assessment before they are considered eligible to  perform site visits. HRSA officials and the contractors we spoke to  explained that development of this training is a joint effort; that is, HRSA  and the contractor work together to develop the training and ensure it  covers the necessary information. Contractor staff may also attend HRSA  webinars or other trainings offered to grantees that discuss grant program  requirements.
In addition to training, HRSA has resources available for contractor staff  that conduct site visits. For example, contractor staff have access to site  visit guides, which help to ensure that the staff are knowledgeable about  the requirements of the grant programs for which they are conducting site  visits. Additionally, HRSA officials and contractors we interviewed told us  that new contractor staff may shadow more experienced contractor staff  during site visits, allowing them to observe an actual site visit prior to  being responsible for conducting one on their own. HRSA officials also  stated that when possible, HRSA project officers attend the site visits to  help new contractor staff understand what to look for when assessing a  grantee’s compliance with program requirements. HRSA officials and one  of the contractors we interviewed told us that HRSA staff also provide  input to contractors about the staff available or selected for grantee  monitoring activities, such as site visits; however, the contractor is  ultimately responsible for selecting its staff and assigning them to  particular assignments. Specifically, officials said that when there is a  specific need for a particular type of expertise among the contractor staff  conducting grantee monitoring activities, HRSA officials may recommend  to the contractor that certain individuals be added to the roster of  contractor staff available to provide services or assigned to conduct a  particular site visit.
HRSA officials reported several mechanisms they use to oversee  contractors’ work, including regular communications with contractors, the  review of contractors’ deliverables, and the collection of feedback from  project officers and grantees on contractor performance.
The contracts we reviewed all required regular communications  between HRSA and contractors for the purpose of, among other  things, discussing the status of the contractor’s work or any problems  the contractor encountered with the grantees. For example, several  contracts required contractors to provide monthly progress reports to  HRSA staff on the status of their accomplishments for the month and  any problems encountered. CORs and contractors we spoke with  stated they have standing meetings ranging in frequency from weekly  to monthly and have more frequent informal communication as  needed to carry out the day-to-day activities of the contract.  Additionally, the contractor we spoke with that conducts site visits of  Bureau of Primary Health Care grantees told us that it has quarterly  meetings with representatives from the bureau, which include  discussion of the site visit process and any grantee trends staff are  seeing during the site visits.
CORs and other HRSA officials review contractor deliverables, such  as routine progress reports and site visit reports, to assess timeliness  and quality of the products. The CORs we interviewed said they  regularly rely on project officers, who have the programmatic  expertise, to review and provide feedback on site visit reports  submitted by contractor staff. Contractor staff are then responsible for  making any necessary revisions to the reports. Officials also said that  CORs work with financial integrity staff when reviewing certain  contractor deliverables, such as financial ratio analyses to assess  grantees’ financial health and viability, as they have expertise related  to these financial reviews.
For at least some contracts, contractors collect feedback from  grantees and project officers that interact with the contractor staff. For  these contracts, grantees have the opportunity to provide feedback on  contractor staff site visit performance by completing a postvisit survey.  Contractors for the HIV/AIDS Bureau and the Bureau of Primary  Heath Care told us that they administer such surveys to grantees.  Additionally, the Bureau of Primary Heath Care contractor we spoke  with sends project officers a similar postvisit survey about their  experiences with contractor staff, while the HIV/AIDS Bureau  contractor we spoke with solicits contractor staff feedback from  debriefing calls with the project officer after the site visit is completed.
section_title: Conclusions
paragraphs: HRSA uses a large portion of its appropriations—over 80 percent in fiscal  year 2013—to award grants to external entities. As a grant-making  agency, HRSA is responsible for monitoring its grantees to ensure that  they are meeting the requirements of their grants. Effectively monitoring  grantees is essential to ensure that government funds are being used  appropriately and efficiently. In the past few years, HRSA has taken steps  to increase and improve its guidance and training related to monitoring  grantees, including implementing new training programs for key staff  responsible for grantee monitoring and developing SOPs for these staff at  both the agency and bureau level. These efforts reflect HRSA’s recent  emphasis on systematically providing guidance and training on grantee  monitoring. To assist with the development of the SOPs, HRSA provided  its bureaus with a template outlining the key components of grantee  monitoring that should be addressed in the guidance. However, we found  that the bureau-level SOPs did not address all of the key components of  grantee monitoring specified in that template; the lack of detailed policies  is inconsistent with federal standards for internal control. Because HRSA  did not ensure that the bureaus’ SOPs included the required information  and detail, and thus provided comprehensive guidance to staff  responsible for monitoring grantees, the agency has put itself at risk for  incomplete or ineffective grantee monitoring, which could jeopardize its  stewardship of federal dollars.
section_title: Recommendation for Executive Action
paragraphs: To help ensure complete and effective monitoring of grantees, we  recommend that the Administrator of HRSA develop a process, either at  the agency or bureau level, to ensure that bureaus’ SOPs address all of  the key components of grantee monitoring as established by the agency.
section_title: Agency Comments
paragraphs: We provided a draft of this report to HHS for comment. In its written  comments, reproduced in appendix II, HHS concurred with our  recommendation. In addition, HHS provided information on its plans for  implementing the recommendation, stating that HRSA officials had  already begun to take action.
We are sending copies of this report to the Secretary of Health and  Human Services and the Administrator of HRSA. In addition, the report  will be available at no charge on the GAO website at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  me at (202) 512-7114 or at draperd@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.
section_title: Appendix I: Health Resources and Services Administration Contracts with Grantee Monitoring Activities, Fiscal Year 2013
paragraphs: Appendix I: Health Resources and Services  Administration Contracts with Grantee  Monitoring Activities, Fiscal Year 2013 Contract purpose  To assist HRSA’s Division of Financial Integrity in  conducting financial ratio analyses to assess grantees’  ability to manage federal funds in accordance with federal  grants management rules and regulations.
To assist HRSA’s Division of Financial Integrity in  conducting financial assessments to assess grantees’  ability to manage federal funds in accordance with federal  grants management rules and regulations.
To assist HRSA’s Division of Financial Integrity with  conducting follow-up on grantees that are delinquent in  the completion of their financial audit.
To conduct a limited scope review of a Bureau of Primary  Health Care grantee to determine if the grantee has  financial systems adequate to support the grant awards  received, has the financial capability to perform grant  activities, and has adequate policies and procedures in  place to meet the program guidelines and terms and  conditions of the grant.
To conduct a limited scope review to determine if a  Bureau of Primary Health Care and Office of Rural Health  Policy grantee used grant funds in accordance with grant  terms and conditions and grant laws and regulations; is  properly allocating and monitoring grant funds; has  adequate internal controls in place, including, but not  limited to, for disbursements of funds, procurement  procedures, and allowable costs; and if there are conflicts  of interest involving the grantee’s Chief Executive Officer.
To conduct a limited scope review to determine if a  Bureau of Primary Health Care grantee has adequate  financial systems in place, has spent grant funds properly,  has adequate policies and procedures in place to meet  program guidelines, and is addressing previously  identified issues.
To provide logistical support and conduct diagnostic  compliance site visits to grantees.
To provide technical assistance and conduct diagnostic  compliance site visits to grantees.
To conduct a limited scope review of an HIV/AIDS Bureau  grantee in response to concerns about the grantee’s use  of grant funds.
Contract purpose  To provide assistance and support to grantees and HRSA  staff by providing on-site visits, including visits to assess  compliance with program requirements; document  reviews; and consultation to new and existing grantees.
To provide School-Based Health Center Capital Program  application review support, ensure grantees are in  compliance with federal requirements, and analyze  grantees’ progress in implementing funded projects.
Seamon Corporation  To provide logistical support and conduct site visits to  assess grantee compliance.
To provide support for application reviews, assess  compliance with federal requirements, and provide  postaward oversight.
Legend: FY = fiscal year.
We included contract obligations for both fiscal years 2012 and 2013 because work carried out in  fiscal year 2013 may have been a result of funds obligated in fiscal year 2012. The amount of  obligations represents the total amount of the contract and is not limited to obligations related to  grantee monitoring activities.
section_title: Appendix II: Comments from the Department of Health and Human Services
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Michelle B. Rosenberg,   Assistant Director; Jill K. Center; Kathleen Diamond;   Cathleen J. Hamann; Emily Loriso; Kaitlin McConnell; Julie T. Stewart;  and Jennifer M. Whitworth made key contributions to this report.