Global Health: The Global Fund to Fight AIDS, TB and Malaria Is Responding to Challenges but Needs Better Information and Documentation for Performance-Based Funding
Released: 2005-06-10T00:00:00

Letter

Background

Human immunodeficiency virus/acquired immune deficiency syndrome (HIV/AIDS), tuberculosis (TB), and malaria, are devastating millions of individuals and families, thousands of communities, and dozens of nations around the world according to the UN’s WHO.

HIV/AIDS, the retrovirus that causes AIDS, is usually transmitted (1) sexually; (2) from mothers to children before or at birth or through breastfeeding; or (3) through contact with contaminated blood, such as through the use of contaminated hypodermic needles. In 2004, it led to between 2.8 and 3.5 million deaths, most of them in sub-Saharan Africa, which is home to more than 60 percent of people living with the virus.

The number of people infected with HIV has risen in every region of the world, with the steepest increases occurring in East Asia, Eastern Europe, and Central Asia. In China, HIV/AIDS is now found in all 31 provinces, autonomous regions, and municipalities; and in India, as of 2003, 2.5 to 8.5 million people had been infected. In Eastern Europe and Central Asia, the number of HIV-positive people has risen ninefold in less than 10 years.

TB, a bacterial infection transmitted by inhalation of airborne organisms, ranks just behind HIV/AIDS as the leading infectious cause of adult mortality, each year killing up to 2 million people, mostly between the ages of 15 and 54 years. It is the most common killer of people whose immune systems are compromised by HIV.

Malaria, caused by a parasite, is transmitted in human populations through the bite of infected mosquitoes. The disease kills more than one million people per year, mostly young African children.

The Global Fund, established as a private foundation in Switzerland in 2002, was created as a partnership between governments, civil society, the private sector and affected communities to increase resources to fight the three diseases. As shown in table 1, 45 percent of the Global Fund’s 271 grants, as of April 15, 2005, were focused on HIV/AIDS; 45 percent went to recipients in sub-Saharan Africa; and 59 percent went to government recipients.

In March 2005, the Global Fund reported that across all grants, it had provided antiretroviral treatment to 130,000 people with AIDS; tested more than one million people voluntarily for HIV; supported 385,000 TB patients with directly observed short-course given more than 300,000 people new, more effective treatments for malaria; and supplied more than 1.35 million families with insecticide-treated mosquito nets.

The Global Fund’s key principles are to (1) operate as a financial instrument, not an implementing entity; (2) make available and leverage additional resources; (3) support programs that evolve from national plans and priorities; (4) operate in a balanced manner with respect to geographic regions, diseases, and health-care interventions; (5) pursue an integrated and balanced approach to prevention, treatment, care and support; (6) evaluate proposals through an independent review process; and (7) operate in a transparent and accountable manner and employ a simplified, rapid, and innovative grant-making process.

Grant Management Process Has Many Participants and Occurs in Two Phases

Numerous entities participate in the Global Fund’s processes for managing grants. The Global Fund manages its grants in two phases, generally over a 5-year period. During phase 1, the Global Fund signs a 2-year grant agreement with the principal recipient and periodically reviews recipients’ performance to determine whether to disburse additional funds. Near the end of phase 1, the board reviews the grant’s progress to determine whether to renew the grant for an additional 3 years; if the board approves continued funding, the grant enters phase 2. The Global Fund board approved the first round of grants in April 2002 and approved 33 grants to enter phase 2 as of April 25, 2005.

Numerous Entities Are Involved in Managing Grants

The following entities participate in the Global Fund’s grant management process (see fig. 1).

A country coordinating mechanism (CCM) representing country-level stakeholders submits grant proposals to the Global Fund and nominates a principal recipient to be responsible for implementing the grant. According to the Global Fund, the CCM should be made up of high-level host government representatives, representatives of nongovernmental organizations (NGO), multilateral and bilateral donors, the private sector, and individuals living with HIV/AIDS, TB, or malaria. CCMs are to develop and forward grant proposals to the Global Fund, monitor grant implementation, and advise the Global Fund on the viability of grants for continued funding after 2 years.

The principal recipient is a local entity nominated by the CCM that signs an agreement with the Global Fund to implement a grant in a recipient country. There may be multiple public and private principal recipients for a single grant. The principal recipient is responsible for overseeing the activities of any subrecipients implementing grant activities and for distributing grant money to them.

The secretariat is responsible for the Global Fund’s day-to-day operations, including managing the grant proposal process; overseeing and managing grant implementation to ensure financial and programmatic accountability; and acting as a liaison between grant recipients and bilateral, multilateral, and nongovernmental partners to ensure that activities at the country level receive necessary technical assistance and are well coordinated. As of April 15, 2005, the secretariat had 165 staff. Within the secretariat, the fund portfolio manager, or grant manager, is responsible for reviewing grant progress and deciding whether to disburse additional funds to the principal recipient.

The secretariat reports to the Global Fund’s board of directors. The 23-member board is responsible for overall governance of the Global Fund and approval of grants. The board includes 19 voting representatives of donor and recipient governments, NGOs, the private sector (including businesses and foundations), and affected communities. Key international development partners, including WHO, the Joint UN Programme on HIV/AIDS (UNAIDS), and the World Bank, participate as nonvoting members. The World Bank also serves as the Global Fund's trustee.

The local fund agent is the Global Fund’s representative in each recipient country and is responsible for financial and program oversight of grant recipients. This oversight role includes an assessment of recipients prior to their receiving money from the Global Fund. To date, the Global Fund has contracted with the following entities to serve as local fund agents: four private firms, KPMG, PricewaterhouseCoopers (PWC), Chemonics International, Inc., and Deloitte Emerging Markets; one private foundation that was formerly a public corporation, Crown Agents; the Swiss Tropical Institute; and two multilateral entities, the World Bank and the UN Office for Project Services (UNOPS). PWC and KPMG serve as the local fund agents in 91 of the 110 countries for which the Global Fund has contracted local fund agents.

Global Fund Implements Grants in Two Phases

After the board approves a proposal submitted by a CCM and vetted by an independent, multinational technical review panel, typically for a 5-year grant, the secretariat signs a 2-year grant agreement with the principal recipient. This initial 2-year period represents phase 1 of the grant; if the board approves continued funding, the grant enters phase 2.

Phase 1

For grants approved in 2002 and 2003, the local fund agent conducted, or contracted with other entities to conduct, assessments of the recipient’s capacity to (1) manage, evaluate, and report on program activities; (2) manage and account for funds, including disbursing to subrecipients; and (3) procure goods and services and maintain a reliable supply of drugs and other commodities. The local fund agent initially conducted these assessments after the signing of the grant agreement but now conducts them before the Global Fund signs an agreement with a principal recipient. After the local fund agent determines that the results of its assessments are satisfactory, the Global Fund instructs the World Bank to disburse the first tranche of funds to the principal recipient.

According to its policy, the Global Fund disburses subsequent tranches based on performance to ensure that investments are made where impact in alleviating the burden of HIV/AIDS, TB, and malaria can be achieved. During the grant period, the portfolio managers are to link disbursements to periodic demonstrations of program progress and financial accountability. The grant agreements initially specified that principal recipients would report their progress and request additional disbursements on a quarterly basis. In July 2004, the secretariat changed the default reporting/disbursement request cycle to every 6 months. As of April 15, 2005, about 20 percent of the Global Fund’s grants were on a 6-month schedule. According to the secretariat, some grant recipients may choose to remain on a quarterly schedule or the secretariat may decide, based on a grant’s risk profile, to disburse only one quarter at a time.

According to secretariat officials, grant managers use four sources of information to determine whether to disburse additional funds to grant recipients.

Recipient progress reports. Principal recipients submit progress reports on meeting designated targets along with requests for further funding at the end of each disbursement period. If program results or expenses differ significantly from plans attached to the grant agreement, the principal recipient is to explain the reasons for these deviations and may also provide an overview of other program results achieved, potential issues and lessons learned, as well as any planned changes in the program and budget. The recipient forwards its progress report and disbursement request to the Global Fund secretariat through the local fund agent.

Recipient expenditure data. The progress reports contain cash-flow information. Principal recipients are to outline expenditures for the previous disbursement period, comparing amounts budgeted for grant activities with amounts spent. Recipients are then to reconcile expenditures and provide a current cash balance. Budgets may vary from initial projections, owing to cost savings, additional expenditures, or currency fluctuations.

Local fund agent assessments. The local fund agent reviews and validates the information in the progress update, performs ad-hoc verifications of program performance and financial accountability, and advises the Global Fund on the next disbursement. Local fund agents are to highlight achievements and potential problems to support their advice and may identify performance gaps to be addressed. Representatives of one local fund agent, which covers grants in 29 countries, said that they base their disbursement recommendations on two considerations: (1) the Global Fund’s level of risk in making additional disbursements to a recipient that uses funds ineffectively and (2) the immediate effect of withholding disbursement on program implementation, including the delivery of disease-mitigating services. These representatives said that, overall, they strive to tie the progress update to projected results in the grant agreement.

Contextual information. The secretariat also uses additional information relevant to interpreting grant progress, such as news of civil unrest, political disturbance, allegations of corruption, conflict, major currency crisis, change of principal recipient, and natural disasters. A secretariat official said that the secretariat did not document requirements for such information for phase-1 decisions but did allow grant managers to consider any information that would adversely affect grant implementation in their decisions to disburse. This information is typically obtained through informal communications with grant recipients, bilateral and multilateral donors, or other development partners, according to secretariat officials.

The principal recipient is to provide the CCM with copies of its disbursement requests and progress reports, and CCM members may comment on the progress of implementation based on their local knowledge and experience, either through the local fund agent or directly to the secretariat. If the secretariat decides to approve the disbursement request, it may specify the level of disbursement and actions that the principal recipient must take. The secretariat then instructs the World Bank to make the disbursement. The secretariat may also decide not to approve the disbursement request.

Phase 2

When a grant reaches its sixteenth month, the Global Fund invites the CCM to submit a request for continued funding for the period following the initial 2 years. The Global Fund refers to this period as phase 2 of the grant (see fig. 2).

The CCM is to submit its request to the Global Fund by month 18, and the secretariat is to evaluate the CCM’s request using the four sources of information described earlier. Based on its assessment of this information, as informed by its professional judgment, the secretariat gives the grant one of four scores, as shown in figure 3. It then provides its assessment and recommendation—called a grant scorecard—to the board regarding approval of the request, and the board decides on the request by month 20. If the board approves the request, the principal recipient and the Global Fund negotiate and sign a grant agreement extension over the next 2 months. At month 22, the Global Fund instructs the World Bank to make the first phase-2 disbursement.

The secretariat sends its recommended scores to the board members, who vote on the recommendations via e-mail. A “go” decision means that the Global Fund approves proceeding to phase 2. “Conditional go” means that the Global Fund approves proceeding to phase 2 after the principal recipient undertakes specific actions within the time frame specified. “Revised go” means that the principal recipient must reprogram the grant and substantially revise the targets and budgets for phase 2. “No go” means that the Global Fund does not approve the grant’s proceeding to phase 2. Currently, recipients denied further funding (“no go”) cannot formally appeal the board’s decision. However, a board subcommittee may consider a formal appeal process. If the “no go” decision affects patients on lifelong treatment, the principal recipient may be eligible to receive funding to sustain treatment for 2 more years.

As figure 4 shows, the Global Fund board approved the first round of grant proposals in April 2002. The second, third, and fourth rounds were approved in January 2003, October 2003, and June 2004, respectively. The board is expected to approve a fifth round of proposals in September 2005. As of April 25, 2005, the secretariat had reviewed 36 grants that became eligible for continued funding under phase 2. The board approved 20 grants, conditionally approved 13, denied 1, and is still considering 2. The board will continue to evaluate grants for phase 2 on a rolling basis as they become eligible.

Several Factors Affected Grant Performance

According to Global Fund officials and other knowledgeable entities, recipient countries’ capacity to implement grants was an underlying factor in grant performance. In addition, principal recipients for the 38 grants as well as Global Fund and development partner officials frequently cited four factors associated with challenges or successes in grant performance: (1) guidance, (2) coordination, (3) planning, and (4) contracting and procurement. We found no significant association between the type of principal recipient, grant size, or disease targeted and the percentage of a grant’s funds disbursed to the principal recipient.

Limited Capacity in Recipient Countries Affected Performance, but Some Governments Have Strengthened Health-Sector Capacity

Global Fund and development partner officials cited limited capacity in recipient countries as an underlying factor that can negatively affect grant performance. Global Fund grant managers said that in many cases, grants experienced early delays because of weaknesses in recipients’ financial, procurement, and monitoring and evaluation systems. For example, Indonesia’s local fund agent found the principal recipient’s management and financial plans to be insufficient and asked the recipient to rework them seven times before the local fund agent recommended grant disbursements. Also in Indonesia, TB spending increased fivefold, greatly straining capacity, particularly for monitoring and evaluating activities at the district level. In Kenya, a lack of designated, adequately trained staff at the principal recipient (the ministry of finance) and immediate subrecipients (the ministry of health and the National AIDS Control Council) slowed disbursements from the principal recipient and from the immediate subrecipients to implementing organizations. Ethiopia has been slow in implementing its first three grants, particularly those for TB and malaria, owing to lack of monitoring and reporting capacity within the ministry of health, delays in recruiting staff to manage financial systems, slow decision-making processes, delays in starting the procurement process, and cumbersome procurement procedures.

Despite limited overall capacity in recipient countries, we found instances where recipient governments had worked with development partners to strengthen capacity in the health sector, thus facilitating grant performance. For example, according to the Indonesian government and WHO officials, the Global Fund grant in Indonesia is building on a strong foundation, using the country’s 5-year strategic plan for TB, a joint effort of the Indonesian government and WHO. Between 2000 and 2003, the Dutch government helped train TB “soldiers” in Indonesian provinces, which improved outreach and case-detection efforts under the Global Fund grant; Indonesia’s ministry of health had already established mechanisms to quickly disburse funds to districts. In addition, the Zambian government has worked with donors and other development partners to strengthen its health sector financing mechanisms. As a result, donors, including the Global Fund, contribute directly to an existing mechanism, the pooled health sector “basket,” and use the health sector donor group overseeing these funds to monitor and evaluate grant progress in meeting targets.

Zambia’s health sector also has mechanisms in place to quickly channel funds to the country’s more than 70 districts. In Mongolia, the local fund agent reported that the principal recipient and subrecipients had adequate financial management systems in place to account for funds and that the principal recipient could immediately start implementing the program with little, if any, technical assistance. Some countries and grant recipients are also seeking to strengthen their capacity through Global Fund grants, according to the Global Fund and principal recipients.

Lack of Clear Guidance Impeded Performance, but in Some Cases Guidance Helped Recipients Meet Targets

Grant recipients frequently reported that a lack of guidance from the recipient country’s government or the Global Fund caused them to fall short of grant targets. For example, recipients in three countries reported that they could not meet their targets because they had not received approved national treatment guidelines. Indonesia’s ministry of health did not have guidelines ready for the voluntary counseling and testing component of its HIV grant, delaying distribution of information to the provinces. Senegal’s ministry of health, another principal recipient, did not have treatment plans needed for implementing its malaria grant, preventing the principal recipient from receiving antimalarial medication. In addition, some stakeholders reported that guidance from the Global Fund was lacking or unclear or that they encountered difficulties with Global Fund grant policies. For example, in at least one instance, U.S. government officials reported that spending delays in Kenya resulted from unclear guidance from the Global Fund regarding altering programs to allow the use of newer, more effective but expensive malaria drugs. The Global Fund recognized that procedures for early grants were unclear and that this lack of clarity caused program delays. Further, WHO officials and at least one recipient voiced concerns over lack of flexibility when recipients sought to modify grant activities. For example, one subrecipient in Thailand expressed concern that it could not use Global Fund money to build or maintain a shelter for HIV-positive women because this type of activity was not written into the grant. Grant recipients also said that continued staff turnover in the Global Fund’s grant management teams made it difficult to receive clear, consistent guidance. For example, recipients in Thailand said that they had worked with four different grant managers over the life of their grants and that this turnover had complicated communication.

However, several grant recipients reported that, under certain circumstances, Global Fund guidance allowed them to quickly redirect funds to meet existing targets. For example, the principal recipient in Indonesia cited grant flexibility as a factor positively affecting performance in both its TB and HIV/AIDS round-1 grants, because this flexibility allowed it to adjust its funding priorities in line with its targets. Similarly, in Thailand, one subrecipient stated that the Global Fund allowed it to change training modules to meet educational needs, contributing to success.

Poor Coordination Slowed Grant Implementation, but Good Coordination Facilitated It

The Global Fund secretariat reported that, in some cases, poor coordination negatively affected grant implementation. For example, in Ghana, internal rivalries between ministry of health units with different responsibilities in the program are slowing implementation. In Senegal, the Global Fund reported that the principal recipient did not meet its target for coordinating and developing partnerships to promote community-based programs for combating malaria.

However, effective coordination between grant recipients and local community groups or development partners sometimes contributed to recipients’ meeting or exceeding their goals. Zambia’s HIV/AIDS grant exceeded its targets for training and provision of services because of development partner, NGO, and private sector contributions. Similarly, in Kenya, one NGO principal recipient leveraged the activities of other groups providing HIV care kits. Another recipient in Kenya exceeded its targets for condom distribution by working with local intermediaries to increase demand by approaching new types of clients, such as shoe shiners, open vehicle cleaners, security officers, staff at petrol stations, and young men at salons. In Indonesia, a grant subrecipient was able to provide treatment to a larger number of TB patients by partnering with private physicians, because a significant number of patients sought treatment at private clinics.

Planning Difficulties Made It Hard to Meet Targets, but Adept Planning Made Them Attainable

Planning difficulties affected some recipients’ ability to meet grant targets. Recipients reported that they sometimes did not achieve targets for a variety of reasons, including not budgeting sufficient time or money to complete targets, or scheduling activities for the wrong time period. One recipient in Zambia underestimated the time needed to analyze baseline data on constituent needs prior to the planned distribution of educational materials on malaria prevention to 1,000 households. The recipient eventually printed the materials but did not reach the targeted households within the planned time frame. In Sri Lanka, a malaria grant recipient underestimated the cost of establishing a community center and had to redesign its program plan to remain within the grant budget. According to the progress report, the principal recipient established new targets to use the funds originally budgeted to build the center, delaying grant implementation. Further, a recipient in Kenya did not conduct 3,000 planned community education skits aimed at preventing HIV during one disbursement period and attributed the shortfall to a conflict with annual school examinations. The Global Fund recognized that recipients’ difficulty in setting targets for the initial grants derived in part from the fact that it was developing procedures and guidelines at the same time that it was approving and signing round-1 and round-2 grants.

Conversely, in some cases, adept planning positively affected grant performance. In Indonesia, several grant recipients reported that effective planning for TB treatment allowed various districts to complete work plans early in the grant, in turn allowing the provinces that oversee those districts to meet their target of developing budgets on time. In Haiti, one principal recipient exceeded its targets by planning activities around World AIDS Day, increasing the demand for, and the principal recipient’s provision of, AIDS-related services such as condom distribution.

Delays in Contracting and Procurement Hindered Grant Performance, While Efficient Procurement Helped

Recipients frequently reported that contracting delays with subrecipients, vendors, or other service providers caused them to miss quarterly targets. For example, UNDP, a principal recipient in Haiti, was unable to hold a planned HIV conference because of delays in signing a contract with a subrecipient. Delays in selecting and reaching contracts with subrecipients caused the Argentine grant to start slowly, the Global Fund secretariat reported. In Thailand, the ministry of public health recipient could not establish TB treatment services because of a subrecipient delay in selecting a site and contract. Grant recipients and the Global Fund secretariat also cited procurement delays as reasons for missing quarterly targets. For example, recipients of malaria grants in Tanzania and Zambia reported that they did not distribute the targeted number of bed nets due to lengthy government procurement processes. In addition, during our visit to Zambia, we found that local spending restrictions also affected recipients’ ability to meet and report on targets. A district health director explained that spending restrictions delayed her purchase of a new hard drive for her office’s computer, which slowed the district’s grant activities and reports to the principal recipient. In Kenya, we found that the limited capacity of the Kenyan health ministry’s procurement agency and the ministry’s reluctance to contract with outside procurement experts led to delays and, as a result, to gaps in the supply of HIV test kits, which bilateral donors had to fill. In Ghana, according to the Global Fund secretariat, the government’s slow, bureaucratic procurement processes caused delays that contributed to the grant’s poor performance in reaching people with HIV/AIDS and opportunistic infections.

However, the Global Fund secretariat reported that some principal recipients’ efficient procurement helped them meet their targets. For example, a principal recipient in Madagascar managed procurement exceptionally well throughout the grant and, as a result, exceeded its targets for distributing bed nets. The Global Fund disbursed this grant’s phase-2 funding early, because the recipient had implemented the program rapidly and was therefore able to use the additional funds. Another recipient in Madagascar consistently met targets, and its disbursements to subrecipients accelerated. The Global Fund also reported that, after initially strengthening its capacity, the principal recipient in Moldova made substantial progress with procurement activities, thereby lowering treatment costs per patient and realizing significant savings due to lower acquisition costs.

No Significant Association Found between Grant Characteristics and Percentage of Funds Disbursed

To determine whether certain grant characteristics were factors associated with the percentage of funds disbursed, we analyzed 130 grants with first disbursements on or before December 31, 2003. We found no significant association between the type of principal recipient, grant size, or disease targeted and the percentage of a grant’s funds disbursed, after taking into account the time elapsed since the first disbursement. (See app. II for details of our analysis.) For example, the Global Fund disbursed a smaller percentage of grants to government recipients than to recipients in the private sector and faith-based organizations, but these differences do not incorporate other factors such as grant size or time elapsed since the first disbursement. We also considered whether disbursements were made in a timely manner, that is, within 135 days (a 90-day quarter plus a 45-day grace period allowed by the Global Fund for reporting). Overall, we found that 35 percent of the disbursements were made within 135 days and that later disbursements were more timely than earlier ones. The number of timely disbursements was too small at any given disbursement stage to determine whether timeliness varied according to recipient or disease type, or grant size.

Sources Informing Funding Decisions Have Limitations, and Decisions Are Not Clearly Documented

We noted problems associated with the four information sources that the secretariat draws on for periodic disbursement and renewal decisions. In addition, although the Global Fund’s stated policy is to disburse funds based on performance and to operate in a transparent and accountable manner, we found that the secretariat did not document its reasons for periodic disbursement decisions during phase 1. Similarly, some of the secretariat’s recommendations regarding grant renewals for phase 2 have not been fully documented, and stakeholders have raised additional concerns regarding the timing of the phase-2 renewal process, dated information, low grant expenditure, and potential politicization of disbursement decisions.

Information Sources for Disbursement and Renewal Decisions Have Limitations

We found the following problems associated with the sources of information that the secretariat uses in making periodic disbursement decisions during phase 1 and determining whether to renew grants during phase 2.

Recipient progress reports vary in quality. Some reports do not explain why recipients missed targets, and the limited monitoring and evaluation capabilities of many recipients raise questions about the accuracy of their reporting. Secretariat officials acknowledged that guidance for planning program activities, setting indicators, and monitoring and evaluating progress was not available when initial grants were signed. However, Global Fund secretariat and other officials have raised questions about the ability of principal recipients to discharge their responsibility for reviewing and monitoring the activities of subrecipients to which they disburse funds. According to the Global Fund official in charge of grant operations, many early grant proposals were overly ambitious and hurriedly assembled; he said more recent proposals were more realistic and better designed. UNAIDS officials also stated that when principal recipients’ progress updates show poor performance, it is not always clear whether grants are underperforming or recipients are failing to effectively report performance. For example, when a progress update shows failure to achieve targets, the principal recipient and subrecipients may have actually completed the activities but not understood how to record them.

Recipient expenditure data are incomplete. Recipients’ cash-flow reports do not include data on expenditures below the level of the principal recipient. In addition, principal recipients may not always document their disbursal of money to subrecipients. Moreover, Global Fund and other officials have questioned whether some principal recipients have the expertise needed to monitor subrecipients’ expenditures. Further, secretariat officials stated that although the achievement of program targets and cash flow are closely linked, recipients’ expenditures do not necessarily indicate that they are meeting their targets. The officials stated that utilizing this source of information is essential to guard against treatment interruptions or irreparable harm to struggling programs that are not yet viable but show strong potential.

Local fund agent assessments are inconsistent. According to Global Fund secretariat officials and others, the ability of local fund agents to effectively verify program activities varies widely. A secretariat– commissioned assessment reported that the current local fund agent system does not provide grant managers with a sufficient level of risk assurance for continued funding. The study, as well as Global Fund and development partner officials, reported that although most local fund agents are competent to assess and verify financial accountability, they often lack the knowledge and experience needed to assess and verify recipients’ performance—specifically, recipients’ ability to meet program targets, monitor and evaluate progress, and procure and manage drugs and other medical supplies. The study also stated that local fund agents’ assessments of financial and program-related capacity and verifications of activities are limited and rarely include site visits to implementing subrecipients.

Contextual information is systematically collected for phase 2 but not for phase 1. To better understand why recipients received phase-1 disbursements when they did not meet many of their performance targets, we requested full disbursement dossiers from the secretariat; however, the dossiers contained very little contextual information supporting the disbursement decisions. The contextual information provided was often in the form of hand-written notes or e-mail correspondence that had been collected ad hoc. Secretariat officials acknowledged that while they collect contextual information through detailed questions on the scorecards for phase-2 decisions, they have no systematic method for collecting such information for phase-1 decisions. Although the Global Fund considers contextual information in its funding decisions, it does not document the extent to which it uses such information.

Little Documentation Provided to Support Phase 1 Disbursement Decisions

Although the files for the 38 grants we reviewed contained information on progress toward targets and cash flow, they contained little or no documentation explaining why the Global Fund approved the disbursements. Overall, for the 38 grants we reviewed, we determined that recipients met, on average, 50 percent of their targets; partially met 21 percent; and failed to meet 24 percent. For 6 percent of the targets, the information in the progress reports was insufficient to determine whether the target had been met, partially met, or not met. In some of these cases, the Global Fund disbursed funds to recipients even though they reported that they had met few or none of their targets. For example: The principal recipient for Sri Lanka’s second malaria grant received disbursements for its third and fourth quarters, although it had submitted two progress updates showing that it met only 2 of its 14 targets for the third quarter and 4 of 13 targets for the fourth quarter. The secretariat provided no written information explaining its approval of the third-quarter disbursement and provided only a one-sentence declaration of agreement regarding the fourth-quarter disbursement. In both cases, the local fund agent had recommended that the recipient receive less than the amount requested, citing cash-flow considerations but not mentioning performance against targets. In each case, the secretariat disbursed the amount that the local fund agent recommended.

The principal recipient for Thailand’s TB grant received its second disbursement although it had met only 1 of 29 performance targets. The secretariat approved the full amount requested, stating that the recipient had not requested sufficient funds in its previous disbursement request, although the grant manager did not provide documentation to validate this assessment. The local fund agent had noted the grant’s poor performance and, acknowledging the grant’s low cash reserves, suggested a disbursement of 25 percent of the recipient’s request.

Further, the Global Fund secretariat does not systematically track denied disbursement requests or publicly document denials. Secretariat officials acknowledged that they currently have no mechanism for tracking or documenting these instances. According to these officials, the denial may eventually be documented in a memorandum on the grant’s disbursement request history once a disbursement is approved or, if the grant is ultimately canceled without further disbursement, in a grant-closing memorandum.

Some Renewal Recommendations Were Not Clearly Documented

According to grant management officials, the secretariat is to unequivocally demonstrate satisfactory performance of all grants recommended to the board for continued funding under phase 2. However, we found that the secretariat did not always clearly explain the overall score it assigned each grant when it recommended the grant for continued or conditional funding. Although a substantial part of the score is to be based on recipients’ performance against agreed-on targets (e.g., the number of people to be reached by disease mitigation services), the final score can also reflect grant managers’ professional judgment, contextual information from multilateral and bilateral donors, and past disbursement rate data. Secretariat officials said that decisions based on these information sources should be documented when an overall score does not seem to reflect recipients’ achievement of individual targets. However, we did not find such documentation in the grant scorecards for 8 of 25 early grants that the Global Fund has considered for continued funding after an initial 2-year period. The secretariat gave 3 of the grants an overall score of B2 yet recommended “conditional go,” which corresponds to a B1 score. For another grant, the secretariat gave a B1 score for three indicators, two of which concern the number of people reached by treatment, care, or other disease mitigation services, yet made an overall recommendation of “go,” which corresponds to an A score. Such discrepancies between scores and recommendations are significant, because the recommendations determine the levels of action that recipients are to undertake before receiving phase-2 funding. Seven of the scorecards also raised concerns about the quality of recipients’ data and their monitoring and evaluation capabilities. Of the 25 grants, the Global Fund decided to cancel one, and the secretariat’s scorecard clearly explained the reasons for recommending that the board cancel the grant.

Stakeholders Also Raised Concerns Regarding Phase-2 Renewal Process

According to the Global Fund, the phase-2 renewal process is a critical checkpoint to ensure that grants show results and financial accountability. However, some stakeholders raised concerns about the process that the Global Fund used to review the first set of grants eligible for renewal. For example, a representative of a local fund agent stated that this process may occur too early in the life of a grant and that progress may be better evaluated when a grant approaches the 3-year mark. Further, officials representing a Global Fund board member stated that data provided to the board during the first round of renewal decisions did not contain expenditure data. These officials stated that when they sought expenditure data (i.e., amounts spent by grant recipients on program activities) on the Global Fund’s Web site in March 2005, the most recent information for their grants of concern had been posted in June 2004. Subsequent data submitted to the board for phase-2 renewal decisions contained expenditure information. In one case, a recipient applying for phase-2 funding and recommended by the secretariat for continued funding had received more than 75 percent of its 2-year grant amount yet had transferred only 12 percent of this money to subrecipients for program activities. These officials also raised concerns over the potential for the politicization of board decisions because the board had returned three “no go” recommendations to the secretariat for further consideration after some recipients and NGOs lobbied board members.

Global Fund Is Taking Several Steps to Refine Grant Management and Performance

The Global Fund’s secretariat is launching a range of initiatives to address challenges to grant performance and improve the overall management of grants. Systemwide, the secretariat is (1) reorganizing and strengthening its units, (2) developing a risk assessment mechanism and early warning system, (3) streamlining reporting and funding procedures, (4) working with partners to strengthen recipient capacity, and (5) clarifying guidance for CCMs. However, the board has not clearly defined the CCMs’ role in overseeing grant implementation. The Global Fund has also responded to country-specific challenges in Kenya and Ukraine.

Secretariat Is Reorganizing and Increasing Its Staff to Better Manage Grants

To improve grant management and documentation of funding decisions and to better support underperforming grants, the Global Fund took the following actions in 2004: Reorganized the secretariat’s operations unit and increased the number of staff from 118 to 165. For example, it added eight grant manager positions and established regional teams, each with a team leader, so that more than one grant manager is responsible for a set of grants in the countries within a regional team. To better document periodic disbursement decisions, the secretariat added a new position, known as a program officer, to its grant management structure. The secretariat is currently recruiting program officers for each regional team, who are to be responsible for documenting disbursement and other decisions and keeping track of grant milestones. Further, secretariat officials said that the Global Fund is planning to recruit additional grant management staff to conduct increased day-to-day recipient monitoring and assistance. The program officers and the additional grant management staff accounted for most of the increase in staff at the secretariat between 2004 and 2005, according to a Global Fund administrative official.

Created the Operational, Partnerships and Country Support Unit to focus on problem grants. According to secretariat officials, this unit—which also includes new positions to liaise with development and technical assistance partners, local fund agents, and CCMs—will enable the secretariat to address grant performance issues before they become serious problems and will thereby better manage risk exposure. For example, the unit could mobilize intervention by high-level recipient government officials, solicit technical assistance from partners, or engage the United Nations Children’s Fund (UNICEF) to procure health commodities until the recipient government can set up a viable procurement system.

Strengthened its strategic information and evaluation unit to improve monitoring and evaluation, data reliability, and quality assurance.

To enhance the quality and consistency of the data that recipients report, in June 2004 the secretariat issued a monitoring and evaluation “toolkit” developed in cooperation with other donors and development assistance partners. This toolkit guides grant recipients to select consistent indicators to measure progress toward key program goals, such as the number of people with AIDS who were reached with drug treatment or the number of people given insecticide-treated bed nets to prevent malaria. The secretariat has also required attachments to each grant agreement that outline program indicators and the specific activities that enable recipients to meet these indicators and overall program goals. According to Global Fund officials, progress will more easily and consistently be measured when all grants have aligned their indicators and activities to this toolkit. Grant managers are currently working to accomplish this goal with the recipients they cover. According to the Global Fund, these developments have been important in harmonizing monitoring and evaluation approaches among partners at national and international levels and will help simplify country-level reporting to multiple donors by ensuring the use of a common set of indicators to measure interventions. Partners provided training on the toolkit in 2004. According to the secretariat, training is to continue in 2005. Recipients we met with in Thailand confirmed that they had received the toolkit. However, they said that it was not in their native language and therefore was not useful.

In March 2004, the board approved establishing a Technical Evaluation Review Group with members from UNAIDS, WHO, and other partners to develop a system for assessing and ensuring data reliability. The group first met in September 2004. According to Global Fund officials, these efforts will result in more systematic reporting and analysis by recipients, the Global Fund, and partners and, consequently, in better comparisons of grants.

To strengthen strategic information for monitoring grant performance, in fall 2004, the secretariat created a “data warehouse” that contains information from recipients’ progress reports and disbursement requests, donors, CCMs, and local fund agent assessments. Secretariat staff use the database to prepare “scorecards” that rank grants for the phase-2 renewal process.

Secretariat Is Implementing a Risk Assessment and Early Warning System to Identify Potential Nonperforming Grants

The secretariat has devised a risk-assessment model and early warning system to identify poorly performing grants and to more systematically alert grant managers when they need to intervene. Because the Global Fund disburses grants to recipients in countries with varying levels of economic development and capacity, its risk-assessment model will incorporate grant size and performance as well as country development and corruption indicators. By tracking key events in the context of grant and country risk, the grant portfolio managers can determine whether recipients have missed important milestones. The early warning system is to generate reports using indicators—for example, time elapsed between disbursements—to flag problems and trigger possible interventions. The system will also incorporate contextual information from country-based partners. When the system identifies slow-moving grants, staff from the secretariat’s Operational, Partnerships and Country Support unit will be able to assess and follow up with the appropriate level of intervention. For example, if a grant recipient in a high-risk country does not submit a progress report and disbursement request at the expected time, the system will alert staff that follow-up is needed.

Although the system has not been fully implemented, secretariat officials said that their recent intervention in Tanzania exemplifies the way the system should work. The Tanzania malaria grant was not demonstrating progress after 1 year, as measured by the amount of funds disbursed compared with the amount that the secretariat expected to disburse. After following up with the principal recipient, secretariat staff realized that political infighting—rather than technical limitations—were inhibiting progress of the malaria program: competing groups were vying for control of grant funds and uncertain of how to procure and distribute bed nets to vulnerable groups, such as pregnant women or women with young children. The government decided to give vouchers to members of vulnerable groups to enable them to purchase bed nets at a lower price; however, the ministry of health did not print or distribute the vouchers or specify where they should be distributed. The Director of the Operational, Partnerships and Country Support Unit traveled to Tanzania and met with development partners and high-ranking host government officials to encourage the government to take action. The Global Fund brought in UNICEF, a key development partner in Tanzania, to work with the government’s malaria advisor as well as experts from the Swiss Tropical Medicine Institute to resolve the problems and get the program back on track.

Secretariat Is Streamlining Procedures

In response to concerns that grantee reporting requirements are difficult and time consuming for recipients, grant managers, and local fund agents, the secretariat instituted a new policy that changes the default for reporting from quarterly to every 6 months. In addition, the secretariat is considering new, more streamlined funding mechanisms than the current round-based approach. However, the board has not endorsed these changes, and some board members, including the United States, are opposed to them at this time.

To decrease the administrative burden on grantees and to bring its practice more in line with other donor agencies, the secretariat instituted a semiannual reporting policy in July 2004. Some recipients still report quarterly, such as those implementing grants that the secretariat identified as high risk—for example, in countries with limited human resource capacity—while others have the option of using quarterly disbursements to meet their needs—for example, as a hedge against currency fluctuations. However, this policy change did not require board approval, and some board members, including the United States, do not support it.

Although the Global Fund strives to be a funding mechanism that seamlessly fits into many country programs by providing additional funding where needed, it recognizes that its current practice of financing grants through rounds can disrupt countries’ planning and time lines and strains recipient capacity. In addition, some associated with the Global Fund said that rounds might lead CCMs and recipients to concentrate their energy on developing new proposals rather than implementing existing grants and that repeated rounds add greatly to the secretariat’s workload. A document submitted to the board by the secretariat stated that although the round- based grant approval system worked well for launching the Global Fund and identifying countries that submitted strong proposals, this system forced recipients to adapt their planning cycles to those of the Global Fund (rather than building on preexisting planning cycles), encouraged the submission of smaller proposals, and left a considerable amount of time between proposal submission and approval. This document presented several options for the board, such as creating two continuous funding streams—one for governments and another for civil society recipients. For example, government applicants could submit their national strategic plans for the coming years, highlighting financing gaps and facilitating integration of Global Fund financing with existing planning and budgeting systems, such as sectorwide approaches. According to the document, this approach would create incentives for CCMs to improve and accelerate the disbursement of funds and would ease the secretariat’s workload, allowing secretariat staff to spend more time managing grants and less time negotiating grant agreements. The board has not set time frames for further discussing this issue. According to U.S. board members, the board has not yet fully discussed or approved these changes, and a majority of board members oppose them at this time.

Global Fund Is Working with Partners to Improve Technical Capacity at the Country Level

Because most grant performance problems are associated with limited capacity at the country level, where the Global Fund has no presence and plays no part in program implementation, the Global Fund relies on its technical partners to provide technical expertise to grant recipients. Although the partners we spoke with expressed their strong support for the Global Fund, they also voiced concern that they have not received additional resources to provide the technical support that grant recipients have requested.

The Global Fund and partners reported that partners provided essential support that strengthened recipients’ capacity to prepare applications for Global Fund financing and helped address the underlying problems that affected grant performance. For example: UNAIDS, a key technical partner, has added about 30 monitoring and evaluation officers in various countries who are available to support CCMs in preparing grant performance reports for phase-2 renewals. UNAIDS has also intensified its capacity-building support at the country level.

Several WHO departments have provided critical technical support. For example, WHO’s Stop TB unit supported 50 countries when they developed their applications for Global Fund financing. The Global TB Drug Facility worked with recipients in eight countries to identify and resolve procurement and supply management bottlenecks. WHO’s HIV/AIDS Department helped to develop comprehensive technical support plans for accelerating the scale-up of antiretroviral therapy and prevention services in 15 to 20 countries. In addition, according to the Global Fund, it collaborated closely with WHO’s Roll Back Malaria Department in 2004 to incorporate into existing grants new, more effective malaria treatments that use artemisinin-based combination therapy.

USAID and the U.S. Centers for Disease Control and Prevention (HHS/CDC) are assisting grantees in a number of countries. For example, USAID is supporting TB grants in numerous ways, including providing training on procuring and managing medical supplies, addressing country-level financial management constraints, and conducting human resource assessments to determine existing capacity needs. In another instance, HHS/CDC is assisting one grantee in reporting and monitoring activities and revising project funds to improve grant implementation. HHS/CDC has also coordinated the implementation and monitoring of activities under another country’s TB grant, participating in supervisory visits to districts to assess their progress and compiling and submitting quarterly reports for the TB grant to the ministry of health.

UNAIDS and WHO officials in Geneva and in the field expressed strong support for the Global Fund but consistently raised concerns about their organizations’ ability to respond to increasing numbers of requests from grant recipients for help in addressing issues underlying performance problems. For example, although UNAIDS recently added about 30 monitoring and evaluation officers in its country and regional offices, officials said that the agency’s resources are being stretched thin and that it cannot provide assistance to all Global Fund grant recipients. Likewise, WHO officials said that its regional and country staff are dedicated to providing technical assistance, but because WHO is not funded to support Global Fund grants it is often unable to respond to all recipients’ requests for help. According to officials from WHO’s HIV/AIDS, Stop TB, and Roll Back Malaria departments, the Global Fund works under the assumption that UN agencies have a mandate to provide technical assistance. However, unless it gets more money from its member countries for this purpose, WHO does not have the resources to keep up with the massive increase in need for technical assistance owing to Global Fund grants. In addition, WHO officials pointed out that the Global Fund does not encourage recipients in African countries to take advantage of WHO’s Global Drug Facility to procure quality-assured TB drugs at the cheapest prices available; instead, the Global Fund encourages competition and reliance on local industry.

Global Fund Has Clarified Guidance and Taken Steps to Strengthen CCMs, but Some Issues Remain

To strengthen accountability in recipient countries, the board has clarified some roles and responsibilities for the CCMs. The board has stated that CCMs are responsible for overseeing grant implementation and are therefore to play an important role in deciding whether grants should be renewed for phase-2 funding. To enhance and clarify CCM functioning, the secretariat in March 2005, convened regional workshops in Zambia and India on CCM best practices. In addition, to improve communication between the Global Fund and CCMs, the secretariat is compiling contact information for all CCM members. This information will enable it to communicate directly with the members instead of relying on the CCM chairperson to disseminate information. Secretariat officials acknowledged that no formal studies conclusively demonstrate a link between CCM functioning and grant performance. However, the Global Fund’s March 2005 report stated that many of the (then) 27 grants eligible for phase-2 funding benefited from several factors, including full levels of participation by CCM members in that body. Further, the report stated that low levels of participation and involvement by CCM members were a key factor in poor performance. Secretariat officials stated that they plan to initiate a study at the end of 2005 to systematically investigate links between CCM functioning and grant performance, given that a number of additional grants will then have neared the 2-year mark and gone through the phase-2 decision process.

In response to findings from several earlier studies commissioned by the Global Fund on CCM functioning, in November 2004, the board agreed on specific requirements for CCMs. However, it has not clearly defined CCMs’ role in monitoring grant implementation. In April 2005, the board directed CCMs to develop tools and procedures for overseeing grants, stating generally that these tools and procedures “should include but need not be limited to” a list of five activities such as recording key oversight actions and developing a work plan that “could include” site visits. The board noted that because CCMs vary from country to country, these guidelines can be adapted and their application paced as needed. According to secretariat staff, the board has not reached consensus regarding CCMs’ oversight role because some members want clear, specific requirements for CCMs while others prefer the more general guidelines. In addition, in 2004, the board agreed on a checklist for measuring CCM performance that focuses mostly on the makeup of the CCMs, participation and communication among members, and governance and management. However, the checklist did not include parameters for measuring the effectiveness of CCMs in overseeing grant performance. Participants at the Zambia workshop recommended that the secretariat develop more specific guidelines defining the oversight role of the CCM.

Global Fund Intervened in at Least Two Countries to Address Grant Performance

The Global Fund secretariat intervened in at least two countries in response to grant performance problems. For example, in Kenya, the secretariat intervened in 2004 at the request of donors and board members to encourage the principal recipient to hold regular meetings with subrecipients and designate staff to administer and monitor the grants. The secretariat also intervened in Kenya to improve coordination by facilitating new CCM procedures, such as designating multiple minute-takers to ensure the accuracy of the minutes and making sure that minutes are circulated promptly. According to one CCM member, two additional people now take notes at each meeting; however, the minutes are not being circulated in advance of the next meeting. In commenting on a draft of this report, U.S. government officials said that, despite these interventions, problems persist. For example, they said that CCM meetings in Kenya are too infrequent and poorly prepared; decisions are made outside of the meetings; and the minutes are often inaccurate.

In Ukraine, the Global Fund suspended three HIV/AIDS grants in January 2004 after investigating irregularities in the principal recipients’ procurements that development partners had brought to its attention a month earlier. The secretariat had also found that after nearly 12 months of a 24-month program, the recipients had spent less than 4 percent of the total 2-year amount for the three grants. The Global Fund had disbursed a total of $7.1 million to the principal recipients, from whom it obtained $6.3 million in reimbursements. In March 2004, the secretariat signed an agreement with a new principal recipient to continue the HIV/AIDS mitigation activities specified in the original grants; in addition, it transferred $300,000 to this entity to avoid interrupting ongoing programs.

Conclusions

The Global Fund’s mandate reflects inherent tensions. On the one hand, the Global Fund is to function solely as a funding entity with no implementing role and to encourage recipient country bodies such as the CCM to be responsible for implementing and overseeing grants. On the other hand, it is to disburse funds rapidly while also ensuring that recipients are able to account for expenditures and produce measurable results in addressing the three diseases.

In seeking to balance these tensions and further improve its performance, the Global Fund has revised—and continues to revise—its processes. Some systemwide changes require board approval or will take time to fully implement, whereas others can be implemented relatively quickly. Capacity in recipient countries, guidance, coordination, planning, and contracting and procurement are pivotal to grant performance and therefore merit continued attention. However, local fund agents’ frequent lack of expertise in assessing these factors, and many recipients’ limited monitoring and evaluation capabilities, raise questions about the accuracy and completeness of the information that the secretariat uses to make its periodic disbursement and funding renewal decisions. In addition, despite recent improvements, the Global Fund’s lack of consistent, clear, and convincing documentation of its funding decisions may hamper its ability to justify these decisions to donors and other stakeholders, in accordance with its principles of transparency and accountability. To ensure that all funding decisions are clearly based on grant performance and reliable data, it is critical that the Global Fund resolve these issues in a timely manner.

Recommendations for Executive Action

To improve the quality of the information on which the Global Fund bases its funding decisions and the documentation explaining these decisions, we recommend that the U.S. Global AIDS Coordinator work with the Global Fund’s Board Chair and Executive Director to take the following three actions: complete efforts to ensure that local fund agents have the necessary expertise to evaluate performance data on disease mitigation that recipients submit, continue to work with development partners to strengthen the quality and consistency of that data by enhancing recipients’ capacity for monitoring and evaluating their financial and program-related activities, and continue efforts to clearly document the Global Fund’s reasons for periodically disbursing funds and renewing grant agreements.

Agency Comments and Our Evaluation

We requested comments on a draft of this report from the Executive Director of the Global Fund, the Secretaries of State and HHS, and the Administrator of USAID, or their designees. We received formal comments from the Global Fund as well as a combined formal response from State, HHS, and USAID (see apps. III and IV). The Global Fund concurred with the report’s conclusion and recommendations and noted steps it is taking to improve documentation of grant performance such as organizing regional training of principal recipient staff to improve the quality of their reporting; defining universal and detailed performance indicators for each grant to more systematically track performance; and tailoring grant oversight and terms of reference for local fund agents based on grant risk. State, HHS, and USAID largely concurred with the report’s conclusions but did not comment on the recommendations in their formal response. Both the Global Fund and the U.S. agencies also submitted informal, technical comments, which we have incorporated into the report as appropriate.

We are sending copies of this report to the Global Fund Executive Director, the U.S. Global AIDS Coordinator, the Secretary of HHS, the Administrator of USAID, and interested congressional committees. Copies of this report will also be made available to other interested parties on request. In addition, this report will be made available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3149. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Objectives, Scope, and Methodology

In May 2003, the President signed a law directing the Comptroller General to monitor and evaluate projects supported by the Global Fund. This report reflects our review of grants that the Global Fund began disbursing before the beginning of 2004—that is, grants that have had at least 1 year to perform. In this report, we (1) describe the Global Fund’s process for managing grants and disbursing funds, (2) identify factors that have affected grant performance, (3) review the basis for, and documentation of, the Global Fund’s performance-based funding, and (4) describe the Global Fund’s recent refinements for managing grants and improving their performance.

Methodology for Describing the Global Fund’s Process for Managing Grants and Disbursing Funds

To describe the Global Fund’s process for managing grants and disbursing funds, we reviewed Global Fund documents, including The Global Fund Operational Policy Manual and related guidance documents; A Force for Change: The Global Fund at 30 Months; The Global Fund to Fight AIDS, Tuberculosis and Malaria: Annual Report 2002/2003; and Investing in the Future: The Global Fund at Three Years. We also interviewed Global Fund officials in Washington, D.C., and in Geneva, Switzerland.

Methodology for Identifying Factors That Affected Grant Performance

To identify factors affecting grant performance, we conducted three types of analysis. First, we selected 13 countries that had grants with a first disbursement on or before December 31, 2003, to allow for at least 1 year of performance, and that had grants covering more than one principal recipient. In addition, all but 4 of these countries had grants covering more than one disease. We reviewed Global Fund dossiers for 38 grants to recipients in these countries and categorized reasons given for deviation from performance targets. (Our initial scope included the 45 grants to these countries, but for 7 of the grants there were no disbursement requests available during the period of our review.) We found 75 progress reports/disbursement requests and 51 local fund agent assessments associated with 24 of the 38 grants. These 38 grants represent 29 percent of the 130 grants that had received a first disbursement by the end of 2003.

Starting with the grant’s second disbursement, we included all disbursement requests from each grant that were available on the Global Fund Web site as of November, 2005, and a few that we received subsequently from the Global Fund. We requested full disbursement dossiers from the Global Fund. These dossiers contained principal recipients’ progress reports and cash-flow/expenditure data, local fund agents’ reviews of the recipients’ information and recommendations about further disbursements, and, in most cases, additional documents such as correspondence between the Global Fund secretariat and the principal recipient. Using this information, we coded reasons given for deviation from grantees’ agreed-upon performance targets into 1 of about 30 categories. We grouped this information into 5 major categories— resources or capacity; coordination; programmatic problems, needs, or changes; procurement; and factors beyond recipients’ control. Within these categories, we developed specific subcategories such as guidance, decisions or plans not made, done, or available; signing of contracts or agreements delayed or not done; and limited trained human resources.

As in any exercise of this type, the categories developed can vary when produced by different analysts. To address this issue, two GAO analysts reviewed a sample of the progress reports and independently proposed categories, separately identifying major factors and then agreeing on a common set of subcategories. We refined these subcategories during the coding exercise that followed. We then analyzed the reasons for deviations from all of the recipients’ progress reports and placed them into one of the subcategories. When information in the progress reports was insufficient to determine how to code a reason, we consulted the local fund agents’ reports. We tallied the counts in each subcategory and identified the subcategories mentioned in the greatest number of grants.

The information in the disbursement requests varied in detail and quality. The two analysts, together with a methodologist, therefore discussed and documented categorization criteria and procedures throughout the analysis, and the methodologist reviewed the entire analysis as a final step. As a validity check on our document analysis and to identify frequently cited factors that affected grant performance, we compared the information in the subcategories mentioned in the most grants to information available from our fieldwork (see below) and determined that both sources of information reported similar findings.

In addition, we reviewed the 75 progress reports/disbursement requests associated with these 38 grants and tallied the total number of targets in each request. We ranked each target using the same numeric rating system the Global Fund uses for phase 2 (see fig. 3). Because many of the total 1,125 targets were nonnumeric (e.g., developing a monitoring and evaluation plan), we did the following: (1) if the principal recipient clearly met the target, we ranked the target as met (“meeting or exceeding expectations”) and included it with the numeric targets that fell into that category; (2) if the principal recipient clearly showed no progress toward meeting the target, we ranked it as not met (“unacceptable”) and included it with the numeric targets in that category; and (3) if the principal recipient partially met the target, we gave it a separate ranking—“partially met nonnumeric target,” and characterized it as partially met, along with the partially met numeric targets. To arrive at percentages for the targets in each category, we first calculated the percentage for each progress report/disbursement request and then averaged the percentages for the category from all the reports. Because the number of targets in each report varied greatly, we averaged the percentages rather than the numbers of targets to ensure that each report was given equal weight. We excluded from our calculations those few targets for which the information available was not adequate to determine whether or to what extent the target was met.

Second, we reviewed documents obtained from field visits to four countries—Indonesia, Kenya, Thailand, and Zambia—and interviewed a wide variety of government, civil society, and bilateral and multilateral development officials in these countries involved in grant implementation or oversight. All four of these countries received more than $36 million in committed funds for several grants that covered more than one disease, and three of them (Kenya, Thailand, and Zambia) also have grants that cover both government and civil society recipients. In addition, we interviewed officials from the Global Fund, World Health Organization (WHO), and the Joint UN Programme on HIV/AIDS (UNAIDS) in Washington, D.C., and Geneva, Switzerland.

Finally, to determine whether the percentage of funds disbursed for each grant (after the first disbursement) and the timeliness of the disbursements were associated with grant characteristics such as type of principal recipient, grant size, or disease targeted, we analyzed 130 grants with first disbursements on or before December 31, 2003. (See app. II for a more detailed discussion of this methodology.) To assess the reliability of the Global Fund’s data, we (1) posed a set of standard data reliability questions to knowledgeable agency officials, (2) performed basic electronic reasonableness tests, and (3) interviewed officials about a few small anomalies that we found during our analysis. We found only one minor limitation, namely that disbursement dates were not reported for less than 5 percent of the disbursements. Based on our assessment, we determined that the data were sufficiently reliable to generate descriptive statistics about the program, and to be used for advanced statistical modeling work.

Methodology for Reviewing the Basis for, and Documentation of, the Global Fund’s Performance- Based Funding

To review the basis for, and documentation of, the Global Fund’s performance-based funding, we examined Global Fund documents— including The Global Fund Operational Policy Manual and related guidance documents, the dossiers for the 38 grants that had a first disbursement on or before December 31, 2003, and documents supporting Global Fund decisions to continue or discontinue funding 25 of 28 grants that had reached their phase-2 renewal point and been reviewed by the secretariat as of March 31, 2005. We also analyzed local fund agents’ assessments to determine how often grant managers documented disbursement decisions. In addition, we interviewed Global Fund officials in Washington, D.C., and in Geneva, Switzerland, and officials from the Departments of State and Health and Human Services (HHS), and the U.S. Agency for International Development (USAID).

Methodology for Describing the Global Fund’s Recent Refinements for Managing Grants and Improving Their Performance

To describe the Global Fund’s recent refinements for managing grants and improving their performance, we reviewed Global Fund documents including The Global Fund Operational Policy Manual and related guidance documents and organization charts, and job descriptions for the positions of local fund agent officer, country coordinating mechanism (CCM) coordinator, program officer, and fund portfolio manager (grant manager). We also examined Global Fund papers, including the Discussion Paper on the Core Business Model for a Mature Global Fund; Update on New Measures of Performance and Early Warning System; Update on the Global Fund Information Management Platform; Revised Guidelines on the Purpose, Structure and Composition of Country Coordinating Mechanisms and Requirements for Grant Eligibility; and Performance Standards and Indicators for CCM Monitoring. In addition, we reviewed the report Investing in the Future: The Global Fund at Three Years and the Monitoring and Evaluation Toolkit for HIV/AIDS, Tuberculosis, and Malaria. We also reviewed documents from a March 2005 CCM workshop conducted in Zambia. Further, we reviewed documents obtained during fieldwork in Kenya, conducted follow-up correspondence with CCM members in Kenya, and reviewed Global Fund documents concerning grants to Ukraine. Additionally, we interviewed officials from the Global Fund, the Departments of State and HHS, USAID, UNAIDS, and WHO.

We conducted our work from June 2004 through March 2005, in accordance with generally accepted government auditing standards.

Methodologies Used to Compare Grants’ Disbursements

This appendix provides descriptive information related to the 130 grants that had received their first disbursements from the Global Fund on or before December 31, 2003, and the results of analyses we undertook to determine whether some types of grants had disbursed a larger percentage of their 2-year funds than others and to estimate the number of disbursements that were made in a timely fashion. Disbursements refer to those from the Global Fund to the principal recipient, not from the principal recipient to subrecipients. Data were current as of February 4, 2005.

Grant Characteristics

Table 2 shows selected characteristics of the 130 grants we reviewed.

No Significant Differences between Grant Type and Percentage of Grant Disbursed

Table 3 shows the performance of different grant types with respect to receiving disbursements. Some of these characteristics varied by type of grant, although many of the differences were not significant.

Ministries of finance, on average, made a smaller number of disbursements and disbursed a lower percentage of their grants, although they also had made their first disbursements later and therefore had less time to make disbursements. Similarly, larger grants disbursed lower percentages of their grant amounts than smaller grants; but again, differences in the time elapsed make it difficult to know whether these differences reflect anything more than the time they had to make disbursements. Differences in disbursements, percentages disbursed, and average days since the first disbursement were insignificant across grants dedicated to the different types of diseases.

Methodologies Used

To determine whether the differences in the percentage disbursed varied by type of grant, we used ordinary regression techniques. The Global Fund also analyzed grant disbursements, reporting in March 2005 that disbursements are indicative of performance. Our analysis differed from the Global Fund’s in that we looked at the percentage of the 2-year grant amount disbursed since the first disbursement, whereas the Global Fund looked at the percentage that was disbursed relative to the percentage that was expected to be disbursed since the first disbursement. Because the actual effect of time turns out to be nonlinear—meaning that although time elapsed since the first disbursement has a significant effect on the percentage disbursed, that effect decreases over time—we estimated the effect of time directly before estimating differences in the percentage disbursed across different types of grants.

We fit bivariate regression models (models 1-4 in table 4) to estimate and test the significance of the gross effects of time since first disbursement, principal recipient type, disease type, and grant size (or the effects of each of these factors, ignoring all others) and a multivariate regression model (model 5) to estimate the net effects of each (or the effects of each after controlling for the effects of the others). Table 4 shows these results.

Model 1 in table 4 shows the effect that time, or days between the first disbursement and February 2, 2005, has on the percentage of the grant disbursed. The significant time-squared term indicates that the effect of time is nonlinear. This nonlinearity makes the interpretation of the time coefficients somewhat less straightforward, but the positive time coefficients in table 4 indicate, not surprisingly, that grants that have had more time to make disbursements have disbursed a larger percentage of the 2-year grant amount remaining after the first disbursement. The negative coefficients associated with the squared term means that over time, time is less of a factor, or that the difference in the percentage disbursed between 100 days and 200 days is greater than the difference between 300 and 400 days. The sizable F-statistic at the base of the column for model 1 attests to the significance of the effect of time, and this nonlinear effect of time explains 17 percent of the variation in the amount disbursed of the total remaining after the first disbursement.

Models 2, 3, and 4 estimate the gross effects of principal recipient type, disease type, and grant size on the percentages disbursed, after subtracting the amount of the first disbursement. These differences are estimated using dummy variables to indicate the differences between the grant categories named in the table and the omitted referent category (civil society for principal recipient type, HIV/AIDS for disease type, and less than $2 million for grant size). The constants for each of these models reflect the percentages disbursed for grants in the referent categories, and the coefficients indicate the differences between the percentages for the categories in the table and the percentages for the referent categories. The overall percentage disbursed was 21 points lower for ministry of finance grants, and roughly 15 points lower for ministry of health grants and other government grants, than for civil society grants. These differences reflect the results of ignoring, rather than controlling for, differences in time since first disbursement, grant size, and disease targeted.

Model 5 estimates all of these effects simultaneously and, as such, provides us with net effect estimates, or estimates of each effect, controlling for the others. It shows that the time each grant has had since its first disbursement is the principal determinant of the amount disbursed. Grants that made their first disbursement earlier disbursed larger amounts of their remaining 2-year awards. After these effects were controlled for, the differences between principal recipient types in the percentages disbursed became smaller than they appeared before controls; and the only difference, which had appeared marginally significant before controls, became insignificant afterward. We found no significant differences by disease type or grant size when looking at either the gross or net effects.

Timeliness of Disbursements

We also looked at the extent to which disbursements were made in a timely fashion (i.e., in 135 days or less). As table 5 shows, 35 percent of all disbursements were timely, or within 135 days, and the extent to which disbursements were timely was greater for later disbursements than for earlier disbursements. The number of timely disbursements is too small at most stages for us to determine whether timeliness varies across grant types.

Comments from the Global Fund

Joint Comments from the Departments of State and Health and Human Services, and the U.S. Agency for International Development

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the person named above, Candace Carpenter, Martin de Alteriis, David Dornisch, Cheryl Goodman, Kay Halpern, Bruce Kutnick, Reid Lowe, Jerome Sandau, Douglas Sloane, Julie Trinder, and Tom Zingale made key contributions to this report.