Foreign Assistance: Actions Needed to Improve Transparency and Quality of Data on ForeignAssistance.gov
Released: 2016-09-07T00:00:00

Letter

Background

The OMB bulletin establishes ForeignAssistance.gov as an official site for U.S. government foreign assistance data and requires U.S. agencies to report, among other things, funding and activity-level data by implementing mechanism (e.g., contract or grant), including activity purpose, description, and location. In addition, the bulletin indicates that agencies are required to provide transaction-level data for each activity. Transactions are individual financial records of obligations and disbursements in an agency’s accounting system. Data on the website are categorized under the nine U.S. foreign assistance framework categories (economic development; education and social services; health; peace and security; democracy, human rights, and governance; environment; humanitarian assistance; program management; and multisector). In general, as of July 2016, data were available from fiscal years 2006 to 2017. Users can view the data through graphic presentations, including maps; filter data by agency, country, and sector; and download data in a spreadsheet. Figure 1 shows a graphic presentation of data (funding, agencies, and fiscal year) available on ForeignAssistance.gov.

Among other things, the United States is publishing data on ForeignAssistance.gov to meet international commitments and domestic data transparency initiatives. The website incorporates key elements necessary for the United States to meet its IATI commitment, such as frequency of reporting (quarterly), activity-level data, and publishing format. In addition, a 2015 State guidance document (toolkit) indicates that ForeignAssistance.gov is expected to meet key domestic data reporting requirements on U.S. government activities, including those in the Digital Accountability and Transparency Act of 2014 (DATA Act). The DATA Act aims to improve the transparency and quality of the federal spending data by requiring that agencies begin reporting data on all federal spending—including grants, contracts, and other types of financial assistance—using governmentwide data standards by May 2017 and publish these data in a computer-readable format by May 2018.

The specific reporting guidelines for ForeignAssistance.gov are outlined in the bulletin, which also notes two existing reports on U.S. foreign assistance—the U.S. Overseas Loans and Grants: Obligations and Loans Authorizations (Greenbook) report to Congress and the U.S. Annual Assistance Report to the Organisation for Economic Co-operation and Development’s Development Assistance Committee (OECD/DAC). The bulletin states that USAID will be the lead agency for verifying the data and assembling these reports. USAID publishes Greenbook and OECD/DAC data on Foreign Aid Explorer. Table 1 describes the key characteristics of U.S. foreign assistance reporting, including lead agency, frequency, and type of data collected.

The bulletin outlines a quarterly data collection process for agencies to submit data for ForeignAssistance.gov. The process begins with State providing agencies a data submission template to assist with the collection of agency data.

The quarterly process outlined in the bulletin includes the following key steps:

Agencies are required to submit data on their ongoing foreign assistance activities on a quarterly basis.

State is responsible for working with each agency’s designated point of contact to coordinate on the data submitted for ForeignAssistance.gov and identify areas in which agencies may need to make corrections.

State is responsible for publishing agency data on ForeignAssistance.gov. In addition, on a quarterly basis, State is responsible for developing the U.S. government IATI-formatted file and submitting it to the IATI Registry.

State Has Established a Process for Collecting and Publishing ForeignAssistance.gov Data

State Is Collecting and Publishing ForeignAssistance.gov Data for 10 Agencies

Since 2013, State has collected and published data from 10 of the 22 agencies identified in the bulletin. State focused on these 10 agencies because they are responsible for providing 98 percent of U.S. foreign assistance, according to State. In addition, State officials told us that they prioritized improving data quality before collecting data from the 12 agencies that are not yet reporting data for ForeignAssistance.gov.

Figure 2 illustrates State’s data collection and publishing process for the 10 agencies currently reporting data to ForeignAssistance.gov. The 10 agencies are the Department of Defense (DOD), the Department of Health and Human Services (HHS), the Inter-American Foundation (IAF), the Millennium Challenge Corporation (MCC), the Peace Corps, State, the Department of the Treasury (Treasury), the U.S. African Development Foundation (USADF), USAID, and the Department of Agriculture (USDA).

The process consists of five key steps, which occur on a quarterly basis: 1. State’s ForeignAssistance.gov team reaches out to each of the agency points of contact to collect that quarter’s data and provides technical guidance documents, including the bulletin. 2. Each agency point of contact e-mails data to State in a spreadsheet or the extensible markup language (XML) format, which can contain as many as 189 data fields per activity. In 2014, State expanded the number of data fields to a total of 189 to align with the IATI Standard. State officials indicated that some of the fields (e.g., currency and U.S. government) are auto-populated by the ForeignAssistance.gov team and that not all data fields are relevant to every agency. State officials told us that 55 of the 189 data fields are the most relevant to users and can be downloaded from ForeignAssistance.gov in spreadsheet format (see fig. 3). In November 2015, State prioritized 37 data fields that are critical to the U.S. government’s foreign assistance reporting, according to State. According to State officials, on average, the 10 agencies reporting data for ForeignAssistance.gov submit quarterly data for 40 to 50 data fields. 3. After converting the agency-submitted spreadsheet data to XML format for agencies that do not have conversion capability, State checks the agency data to determine whether all required fields are populated. 4. State relays any missing values or possible data reporting errors to agencies and allows them to review and make corrections before it publishes the data on the public website. 5. Using the agency-corrected data, State creates and publishes downloadable data files on ForeignAssistance.gov. During this final step, State also simultaneously links the quarterly data files to the IATI Registry.

In November 2015, State created a community of practice website to allow agency points of contact to engage online, clarify any issues, and share lessons learned. However, as of May 2016, agencies had not posted comments or questions on the website.

State Has Made Some Progress in Preparing the 12 Agencies Not Yet Reporting data for ForeignAssistance.gov

In 2015, State developed a process to prepare the 12 agencies that had not yet reported data for ForeignAssistance.gov. The 12 agencies are the Department of Commerce (Commerce), the Department of Energy (DOE), the Department of Homeland Security (DHS), the Department of the Interior (DOI), the Department of Justice (DOJ), the Department of Labor (DOL), the Department of Transportation (DOT), the Environmental Protection Agency (EPA), the Export-Import Bank of the United States (Ex-Im), the Federal Trade Commission (FTC), the Overseas Private Investment Corporation (OPIC), and the U.S. Trade and Development Agency (USTDA). In September 2015, officials from all 12 agencies told us that State had not reached out to them with specific reporting instructions. Between November 2015 and May 2016, State provided the toolkit to all 12 agencies and conducted information sessions with most of them on reporting data for ForeignAssistance.gov.

The toolkit contains descriptions of the data fields and resources to support the five-phase data publishing process. It also includes a list of 37 priority data fields, which, according to State, will make it easier for agencies to identify where they should focus their data collection efforts. The five phases of the publishing process are the following: 1. Planning phase: Agencies collaborate with the ForeignAssistance.gov team to understand the general reporting requirements. 2. Discovery phase: Agencies review their systems to identify foreign assistance data and better understand the data sources and needs. 3. Preparation phase: Agencies create and submit data samples to the ForeignAssistance.gov team, which provides feedback. 4. Processing phase: Agencies submit a final dataset to the ForeignAssistance.gov team and conducts quality assurance checks. 5. Execution phase: State uploads the final approved dataset to ForeignAssistance.gov, and agencies can issue press releases to external stakeholders.

Between December 2015 and January 2016, State conducted two information sessions on the toolkit. Six agencies (DOE, DOJ, DOL, EPA, FTC, and OPIC) attended one or both sessions. As of May 2016, State had introduced all 12 agencies to the data publishing process, but none had published data on ForeignAssistance.gov. According to State, as of May 2016, of the 12 agencies,

Six—Commerce, DHS, DOI, DOT, EPA, and USTDA—were in the planning phase. These agencies had received the toolkit, and some were beginning to review their systems to identify foreign assistance data and better understand the data required.

Two—DOJ and Ex-Im—were in the preparation phase. These agencies prepared or provided sample activity-level and organization- level datasets for State to review as part of toolkit implementation.

Four—DOE, DOL, FTC, and OPIC—were in the processing phase.

These agencies had submitted at least one quarterly dataset for the ForeignAssistance.gov team to review.

State has identified limited staff resources as a key constraint in collecting and publishing ForeignAssistance.gov data. According to State officials, the ForeignAssistance.gov team would be challenged to manage data provided by the 12 agencies not yet reporting, while continuing to publish data and work on data quality improvements with the 10 agencies that are reporting.

Agencies Identified Various Impediments to Collecting ForeignAssistance.gov Data

Our analysis shows that agencies face impediments in collecting the required information. Most of the 10 agencies reporting data for ForeignAssistance.gov identified limitations in their information technology (IT) systems as a key impediment in collecting and reporting data, while most of the 12 agencies not yet reporting data identified lack of staff time as a key impediment that they anticipate facing. In November 2015, we surveyed officials at all 22 agencies to identify and rate key factors that may impede their ability to collect and report data for ForeignAssistance.gov.

Agencies Reporting Data Identified Limited Information Technology Systems and Availability of Data as Key Impediments

Based on our analysis of survey data, the top three factors most of the 10 agencies reporting data identified as presenting a moderate or great impediment to collecting and reporting data for ForeignAssistance.gov were (1) limitations in their IT systems, (2) lack of available data, and (3) having to adjust available data to fulfill ForeignAssistance.gov requirements (see table 2).

More than half of the agencies reporting data also identified as moderate or great impediments to collecting and reporting ForeignAssistance.gov data the lack of a single agency internal IT system from which to pull all ForeignAssistance.gov data and the number of data fields required (see table 2). No agency reporting data identified that a lack of a State point of contact or a lack of a governmentwide dedicated server presented a moderate or great impediment.

Based on interviews conducted prior to the survey, most agency officials noted that their existing IT systems were limited in that they did not track data at a level of detail required by ForeignAssistance.gov. Several agencies whose main mission is not foreign assistance explained that updating their existing systems for these requirements was not a priority. Two agencies whose main mission is foreign assistance, State and USAID, undertook assessments of their current systems to understand and better align the capabilities of their systems with these reporting requirements. One agency, DOD, told us that it had plans to update its IT system to be able to report quarterly data, but that it would take some time.

Most agency officials we interviewed also told us that they lacked an integrated system to track both the financial and project data required for ForeignAssistance.gov. To fill out the ForeignAssistance.gov spreadsheet data template provided by State, they said they had to collect data from key documents and multiple internal systems for accounting and project management. However, one agency—MCC—indicated in interviews that it already had an integrated system and was therefore able to consolidate its reporting. The agency noted recent updates to its existing IT system and attributed its upgraded system to being a newer agency.

Agencies Not Yet Reporting Data Identified Lack of Staff Time as a Key Anticipated Impediment

Based on our analysis of survey data, lack of staff time was the top factor identified as an anticipated impediment by most of the 12 agencies not yet reporting data for ForeignAssistance.gov (see table 3).

More than half of the agencies reporting data also identified lack of funding, number of data fields, and limitations in their IT systems as moderate or great impediments to collecting and reporting data for ForeignAssistance.gov (see table 3).

Based on interviews conducted prior to the survey, some of the agency officials noted that they would have to add ForeignAssistance.gov reporting duties to existing staff’s primary job functions, which could be burdensome, especially for agencies with smaller foreign assistance portfolios. Other agency officials said they anticipated that staff time would be an issue, because the collection process would involve many people throughout the agency. Furthermore, some of the agency officials also noted their limited capacity to provide data quarterly for the number of data fields that ForeignAssistance.gov requires. Officials from one agency explained that completing the annual requests for data for the Greenbook and OECD/DAC was already time-consuming, as it required sending a data call to subcomponents and field staff, compiling the data into one spreadsheet, and going through multiple layers of review. See appendix II for key factors identified by agencies reporting and not yet reporting data as impediments to their data collection process.

ForeignAssistance.gov Is Not Transparent about Data Limitations, and Data Are Not Updated Annually to Ensure Quality

Data from some agencies that report on their foreign assistance to ForeignAssistance.gov are incomplete at the aggregate level. We found that in the aggregate, 14 percent of obligations and 26 percent of disbursements for fiscal year 2014 were not published on the website, when compared to USAID’s verified data. We also found that for some high-priority data fields, information was missing or inconsistent with State’s definition for each data field. In addition, although ForeignAssistance.gov discloses that published data are incomplete, we found that the website is not fully transparent about these data limitations. Moreover, the data published on ForeignAssistance.gov are not annually updated against verified—complete and accurate—foreign assistance data, as required in the bulletin.

Data Published on ForeignAssistance.gov Are Incomplete

We analyzed fiscal year 2014 data downloaded from ForeignAssistance.gov to assess the completeness of aggregate funding data as well as the completeness and consistency of information in selected data fields with State’s definitions for those data fields. We found that data on ForeignAssistance.gov were incomplete at the aggregate funding level as well as for some disaggregated data at the transaction level. In addition, we found that data for selected high-priority data fields were inconsistent with State’s definitions. State officials told us that they rely on agencies to provide complete and accurate information for ForeignAssistance.gov.

Fourteen Percent of Obligations and 26 Percent of Disbursements for Fiscal Year 2014 Were Not Published on ForeignAssistance.gov

We found that in the aggregate, 14 percent of obligations and 26 percent of disbursements for fiscal year 2014 from the 10 agencies reporting data for ForeignAssistance.gov were not published on the website as of April 2016. Our comparison of funding data on ForeignAssistance.gov to funding data published on the Foreign Aid Explorer website showed that ForeignAssistance.gov did not reflect more than $10 billion in disbursements and about $6 billion in obligations provided by the 10 agencies in fiscal year 2014. Data on these two websites are generally comparable because both essentially use the same definition of foreign assistance, based on the Foreign Assistance Act of 1961, as amended. The OMB bulletin indicates that USAID’s data are verified—checked for completeness and accuracy. The bulletin notes that USAID’s verification includes checking for common errors, comparing with third-party sources to identify gaps and more complex errors, accommodating negative entries, and taking other steps to ensure data quality. Additionally, according to USAID officials, as part of the verification process for the data published on Foreign Aid Explorer, they check to ensure that there are no anomalies, errors, duplicates, or missing values. Furthermore, USAID checks to ensure that data are consistent with those for prior years and verifies the data against official U.S. government documents. We selected fiscal year 2014 because all 10 agencies published data for that year on both websites and because it was the most recent year for which fully reported and verified USAID foreign assistance data were available at the time of our analysis.

Based on our analysis of the fiscal year 2014 funding data for the 10 agencies (see table 4), the total obligations on ForeignAssistance.gov were $36.1 billion, almost $6 billion (14 percent) less than the $42 billion identified on Foreign Aid Explorer. The total disbursements on ForeignAssistance.gov were $29.9 billion, more than $10 billion (26 percent) less than the $40.4 billion on Foreign Aid Explorer in the same year. Specifically, fiscal year 2014 obligations and disbursements were about the same or identical for four agencies (IAF, MCC, the Peace Corps, and USADF) and had a difference of less than 10 percent for two agencies (Treasury and USAID). However, the differences in obligations or disbursements exceeded 10 percent for DOD, HHS, State, and USDA, with DOD’s and USDA’s data showing the largest differences.

The four agencies whose fiscal year 2014 ForeignAssistance.gov funding data showed a difference of more than 10 percent from the Foreign Aid Explorer data for the same year provided the following explanations:

DOD. According to DOD officials, two factors explain the discrepancies. First, the two websites attribute funding for a significant portion of U.S. security assistance differently: on ForeignAssistance.gov, the assistance—which State funds and DOD implements—is attributed to State, whereas on Foreign Aid Explorer, it is attributed to DOD. The second factor, according to DOD officials, is inconsistent reporting of fiscal year 2014 funding data for ForeignAssistance.gov: DOD reported obligations, but not disbursements, for some programs, and disbursements, but not obligations, for other programs. For example, DOD did not report $4.7 billion in fiscal year 2014 obligations or disbursements for the Afghanistan Security Forces Fund on ForeignAssistance.gov.

HHS. HHS officials stated that the data the agency published on Foreign Aid Explorer more accurately reflect the agency’s foreign assistance portfolio than the data the agency published on ForeignAssistance.gov. HHS officials suggested that their agency data for fiscal years 2013, 2014, and 2015 on ForeignAssistance.gov should not be used until the quality of the data published on the website is improved. They did not explain the differences in the funding data on the two websites.

State. State officials told us that to some extent the discrepancies came about because funding for peacekeeping and U.S. contributions to international organizations was not included in the fiscal year 2014 data on ForeignAssistance.gov. State officials also noted that data published on Foreign Aid Explorer are considered to be more fully reported because they are submitted to USAID by State’s bureaus, which manually enter detailed data; by comparison, State’s data for ForeignAssistance.gov are generated from the agency’s accounting system. In addition, State’s accounting system at present includes transactions reported by State’s main office in Washington, D.C., but does not include transactions of overseas locations.

USDA. USDA officials told us that they reported incorrect fiscal year 2014 obligations for ForeignAssistance.gov because USDA misinterpreted State’s guidance. They also noted that USDA is working with State and USAID to ensure that USDA’s foreign assistance data are accurate and consistent on both websites.

DOD, HHS, State, and USDA officials said that they were aware of the differences in their funding data published on the two websites and were working on improving the quality of the data published on ForeignAssistance.gov. State officials could not indicate when the 10 reporting agencies would be able to report complete funding data for ForeignAssistance.gov and stated that it is the agencies’ responsibility to report complete and accurate data.

Information Was Missing or Inconsistent with Definitions for Some High-Priority Data Fields

In addition to discrepancies in the aggregate funding data, we found that for some of the high-priority data fields the information was either missing or inconsistent with definitions that State developed based on the IATI Standard. We analyzed fiscal year 2014 data for six of the data fields prioritized by IATI—implementing organization, award title, award description, award status, award location, and award sector— to determine if agencies populated these data fields with information and if the information was consistent with State’s definitions. According to State, data fields prioritized by IATI should be populated because they provide key information necessary to track a specific activity. We analyzed the content of the six data fields using a probability sample of 106 transactions drawn from the fiscal year 2014 data. We found that for three data fields—implementing organization, award location, and award sector—information was provided and was consistent with State’s definition for a majority or all of the transactions (see table 5).

For the other three data fields—award title, award description, and award status—the information was missing or inconsistent with the definitions for the majority of the transactions in the sample. For example, for award title, 82 percent of the transactions were either missing information or had information that was inconsistent with the definition for this data field. We also found that for award title, agencies often provided program or sector descriptors, and for award description, agencies routinely provided shorthand descriptions, acronyms, or terminology that could only be understood by officials at the agency that made the award. For example, an award description would contain “Train, Eval & Oth Related Act” or “AIDSTAR Sector II Award.” Only three transactions in our sample contained award descriptions that were somewhat consistent with the definition of a brief narrative that provided an understanding of the undertaking for which the award was funded, its objectives, and the hypothesis of the award’s development impact. The lack of clarity for data fields, such as award description, could make it difficult for domestic and international users of ForeignAssistance.gov to understand the data.

State officials told us that our findings were consistent with their observations of the data. They explained that the reporting agencies’ data systems were not currently capable of capturing and generating data that would fully meet IATI’s expectations for detailed, transaction-level information. Our analysis also shows that most of the reporting agencies identified impediments in collecting and reporting data, including limitations in their IT systems and lack of available data required by ForeignAssistance.gov.

State Relies on Agencies to Provide Complete and Accurate Data

State officials told us that they rely on agencies to provide complete and accurate data because, according to the bulletin, reporting agencies are responsible for the accuracy, completeness, and integrity of their data submissions. Additionally, State officials noted that the ForeignAssistance.gov team has neither the expertise nor the resources to check the data for completeness or accuracy on a quarterly basis. They explained that collecting and publishing data from 10 agencies on a quarterly basis is inherently challenging and can lead to trade-offs between quality and timeliness. Nevertheless, State’s ForeignAssistance.gov team conducts certain checks, such as ensuring that agency data are formatted properly and that dates are within valid ranges. (See table 6 in app. III for data field format values.)

ForeignAssistance.gov Is Not Fully Transparent about Data Limitations

Although ForeignAssistance.gov discloses—through a graphic summary and agency notes—that published data may not be complete, it is not fully transparent about data limitations. Specifically, we found that ForeignAssistance.gov does not clearly identify what data are missing and consistently identify and publish data limitations by agency.

According to OMB’s Open Government Directive, which is Memorandum M-10-06, agencies should take specific actions to implement the principles of transparency, among other things, in publishing government information online. The memorandum indicates steps to improve transparency in publishing information, which include identifying high- value information not yet available for online publication. In addition, according to prior GAO work, useful practices that help foster transparency to the public and assist decision makers in understanding the quality of agency data include, among other things, discussion of data quality and limitations.

ForeignAssistance.gov Is Not Clear about Missing Data

Although ForeignAssistance.gov discloses that some data are not yet available, it does not clearly state what data are missing. State presents a graphic summary of data from the 10 reporting agencies published on the website (fig. 4). It uses the terms “full” and “partial” to characterize agency datasets, but does not explain what these terms mean. Specifically, as of July 2016, ForeignAssistance.gov indicated that six agencies—IAF, MCC, the Peace Corps, Treasury, USADF, and USAID—reported a full set of data and that four agencies—DOD, HHS, State, and USDA—reported a partial set of data (see fig. 4). In particular, it is not clear whether full reporting means that an agency reported data (1) reflecting all of its foreign assistance funding or (2) across all of the required data fields.

While the agency notes published on ForeignAssistance.gov provide some information to help understand an agency’s foreign assistance activities, not all agencies report this information and, moreover, these notes do not clearly identify data limitations for reporting agencies.

We found that 6 of the 10 reporting agencies (DOD, the Peace Corps, State, Treasury, USAID, and USDA) provided agency notes on ForeignAssistance.gov as of July 2016. All six agencies provided general descriptions of their foreign assistance programs and accounts. However, the agency notes for three of the four agencies characterized on ForeignAssistance.gov as reporting partial data (see fig. 4)—DOD, State, and USDA—did not identify data limitations for a given year. For example, they did not comment on gaps in obligation and disbursement amounts generally or by fiscal year. The fourth agency characterized on ForeignAssistance.gov as reporting partial data—HHS—did not post agency notes. In contrast, other publishers of U.S. data provide more detailed information on data limitations or changes to the data in a given year. For example, the 2014 Greenbook identified an agency that provided data for two additional programs that year, an agency that continued to not report data, and another agency that was unable to report on certain data in detail.

Agency notes are not required by State or the bulletin, which states that agencies may provide supplemental narratives and can include data explanations and other clarifying information. Additionally, State’s most recent guidance (toolkit) on reporting to ForeignAssistance.gov does not mention agency notes or provide instructions for agencies to identify the limitations of their data.

State officials told us that they rely on agencies to report data limitations. However, if State does not provide agencies with guidance to identify data limitations that State can clearly disclose on ForeignAssistance.gov, it may undermine the website’s goal of improving public knowledge and accountability of U.S. foreign assistance.

ForeignAssistance.gov Data Are Not Updated Annually with Verified Foreign Assistance Data

As of May 2016, State had not updated ForeignAssistance.gov data with verified—complete and accurate—annual foreign assistance data to improve the quality and ensure consistency in the reporting of U.S. foreign assistance. OMB indicates in its bulletin that ForeignAssistance.gov data should be updated at the end of each calendar year using verified data reported by USAID for the Greenbook and OECD/DAC to ensure consistency in published information. Additionally, the bulletin indicated that in 2014, USAID, State, OMB, the National Security Staff, and the Office of Science and Technology were expected to undertake a review of the first 2 years’ experience to assess whether agencies whose data are published on ForeignAssistance.gov had demonstrated sufficient internal data quality control to graduate from the USAID verification process. However, as of May 2016, this interagency review had not taken place because, according to OMB officials, only 3 of the 10 reporting agencies were providing data to ForeignAssistance.gov of sufficient quality to meet the Greenbook and OECD/DAC reporting requirements. Since the majority of the agencies’ data were not yet of sufficient quality, OMB officials noted that a review to graduate agencies from USAID’s verification process was premature.

State and USAID officials told us that they are unable to update ForeignAssistance.gov with USAID’s verified data because of differences between the two datasets. State officials cited three key differences:

Number of data fields. ForeignAssistance.gov data includes up to 189 data fields; however, USAID verifies the information only for a subset of about 20 data fields.

Frequency of data reported. ForeignAssistance.gov captures quarterly data; however, USAID uses annual data.

Transaction-level data are stored differently.

ForeignAssistance.gov captures transaction-level data for each activity; however, USAID aggregates the transaction-level data to the activity level.

USAID noted that reconciling ForeignAssistance.gov data with verified Greenbook and OECD/DAC data would be problematic, especially for the seven agencies whose data do not meet the quality standards for Greenbook and OECD/DAC reporting. For these agencies, USAID (1) obtains missing information for some data fields (e.g., recipient country) directly from agency officials or units that report the information, and (2) assigns sector codes and other fields—which are not always provided in the data that agencies report for ForeignAssistance.gov—based on OECD/DAC statistical reporting directives and methodologies.

OMB officials agreed that the bulletin’s requirement for annually updating data published on ForeignAssistance.gov with USAID-verified data has not been feasible. They also acknowledge that the quality of ForeignAssistance.gov data needs to be improved.

Since State, USAID, and OMB recognize that a key step outlined in the bulletin to ensure data quality may not be feasible, and in the absence of the 2-year review on data verification or guidance on how to address the quality of the data on ForeignAssistance.gov, data will likely remain inconsistent across the range of U.S. foreign assistance reporting.

Conclusions

In response to domestic and international initiatives in the last decade, the U.S. government has increased the frequency and amount of foreign assistance data made available to the public. In 2011, the U.S. government made an international commitment to publishing more detailed and timely funding and activity data for users, including partner country governments, civil society organizations, and taxpayers. As the U.S. government’s lead agency for this reporting, State established ForeignAssistance.gov, with guidance from OMB, to provide detailed foreign assistance data on a quarterly basis. Given the magnitude and frequency of data collection, State prioritized collection and publishing of data for 10 agencies that account for the majority of U.S. foreign assistance. Facing trade-offs—which agencies recognize—between data quality and timeliness in reporting, State has experienced challenges in ensuring transparency and data quality on ForeignAssistance.gov. In particular, in the absence of guidance from State to reporting agencies to clearly identify their data limitations, State has not fully disclosed data limitations of ForeignAssistance.gov, thereby undermining the website’s goal of increasing the transparency of U.S. foreign assistance information. Moreover, because updating ForeignAssistance.gov with USAID verified data has not been feasible and the interagency assessment of the process to ensure sufficient quality control has not been done, gaps in data quality remain unaddressed, and users may risk using inaccurate or incomplete information for decision-making and accountability purposes.

Recommendations for Executive Action

To improve the transparency of ForeignAssistance.gov, we recommend that the Secretary of State provide guidance to agencies to identify data limitations that State can clearly disclose on the website.

To improve the quality of the data published on ForeignAssistance.gov and help ensure consistency in published information, we recommend that the Secretary of State, in consultation with the Director of OMB and the USAID Administrator, take the following two actions: undertake a review of the efforts to date on ensuring data quality and develop additional guidance that takes into consideration current challenges to updating ForeignAssistance.gov with verified data.

Agency Comments and Our Evaluation

We provided a draft of this report to OMB and the 22 agencies reviewed in this report (Commerce, DHS, DOD, DOE, DOI, DOJ, DOL, DOT, EPA, Ex-Im, FTC, HHS, IAF, MCC, OPIC, PC, Treasury, State, USADF, USAID, USDA, and USTDA) for review and comment. In written comments, reprinted in appendix IV and V, State and USAID agreed with our recommendations. OMB also agreed with our recommendation and provided us with the following comments in an e-mail: it will continue to work with State and USAID to help guide agencies in improving the quality and consistency of the data published on ForeignAssistance.gov.

However, State expressed concern that the report did not provide specific, actionable recommendations to the other 20 agencies responsible for reporting ForeignAssistance.gov. As noted in the report, OMB Bulletin No. 12-01 provides overall guidance on data standards and requirements for the other 20 agencies. We made the recommendations to State, in consultation with OMB and USAID, because these agencies are responsible for improving guidance and reporting requirements that can help achieve the website’s goal of improving public knowledge and accountability of U.S. foreign assistance. DOD, EPA, FTC, HHS, MCC, State, USAID, and USDA provided technical comments that we incorporated as appropriate.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 13 days from the report date. At that time, we will send copies of this report to the Directors of the Office of Management and Budget, Peace Corps, and U.S. Trade and Development Agency; the Administrators of the Environmental Protection Agency and U.S. Agency for International Development; the Secretaries of State, Agriculture, Commerce, Defense, Energy, Health and Human Services, Homeland Security, the Interior, Labor, Transportation, and the Treasury; the Attorney General of the United States; General Counsel of the Federal Trade Commission; and the Chief Executive Officers of the Export-Import Bank, Inter-American Foundation, Millennium Challenge Corporation, Overseas Private Investment Corporation, and U.S. African Development Foundation. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3149 or GootnickD@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

To review the collection and reporting of data for ForeignAssistance.gov, this report examines (1) the Department of State’s (State) data collection and publishing process for ForeignAssistance.gov; (2) key impediments, if any, that agencies face in collecting and reporting data for ForeignAssistance.gov; and (3) the data published on ForeignAssistance.gov.

To conduct this work, we analyzed key guidance documents relating to the data collection process: the 2012 Office of Management and Budget (OMB) Bulletin No. 12-01, Guidance on Collection of U.S. Foreign Assistance Data (Sept. 25, 2012) (bulletin), the 2009 OMB Memorandum M-10-06, Open Government Directive (Dec. 8, 2009), and State’s November 2015 Agency Reporting Toolkit (toolkit). We also reviewed key U.S. government plans and international agreements that outline steps for ensuring transparency in foreign assistance reporting, including the 2011 and 2013 U.S. Open Government National Action Plan, the 2005 Paris Declaration on Aid Effectiveness, and the 2011 Busan Outcome Agreement.

To examine State’s data collection process for ForeignAssistance.gov, we conducted semistructured interviews with officials of 22 U.S. agencies— the Departments of Agriculture, Commerce, Defense, Energy, Health and Human Services, Homeland Security, Justice, the Interior, Labor, State, Transportation, and the Treasury; the Environmental Protection Agency; the Export-Import Bank of the United States; the Federal Trade Commission; the Inter-American Foundation; the Millennium Challenge Corporation; the Overseas Private Investment Corporation; the Peace Corps; the U.S. African Development Foundation; the U.S. Agency for International Development; and the U.S. Trade and Development Agency—concerning four areas of the data collection process, including (1) data collection and validation processes; (2) guidance provided by State on the data collection process; (3) resource needs to collect data, such as infrastructure, staff, and training; and (4) impediments that agencies may face in collecting and reporting data for ForeignAssistance.gov. We also interviewed OMB on its monitoring role for ForeignAssistance.gov.

To identify the impediments that agencies face in collecting and reporting data for ForeignAssistance.gov, we surveyed the same 22 U.S. agencies to identify key impediments, if any, that agencies may face in collecting and reporting data for ForeignAssistance.gov, including agency reporting systems; required data fields in ForeignAssistance.gov; resources; and guidance provided that may impede an agency’s ability to collect and report data for ForeignAssistance.gov. For the 12 agencies that do not yet report data for ForeignAssistance.gov, we modified our survey instrument to probe the extent to which they anticipated impediments in collecting and reporting foreign assistance data to State. The survey was administered in November 2015, and agencies provided their responses between November 2015 and January 2016. All 22 agencies responded to the survey.

To assess the data from the 10 reporting agencies published on ForeignAssistance.gov, we compared fiscal year 2014 funding (obligation and disbursement) data published on Foreign Assistance.gov to the data collected and published by the U.S. Agency for International Development’s (USAID) data on the Foreign Aid Explorer website (http://explorer.usaid.gov/). Foreign assistance data available on these two websites are based on essentially the same definition of foreign assistance. To determine the reliability of the Foreign Aid Explorer data, we interviewed USAID officials, reviewed documentation about the data, and examined the data published on USAID’s website. We determined that USAID’s verification processes for the Foreign Aid Explorer data include checks to identify potential anomalies, duplicates, missing values, and other errors. In addition, we found that USAID compares the agencies’ data submissions to other available sources as completeness checks. We determined that data published on Foreign Aid Explorer were sufficiently reliable to serve as a reasonable comparison for the ForeignAssistance.gov data for the purposes of our reporting objectives. However, it was beyond the scope of this engagement to independently verify agency source data.

We downloaded fiscal year 2014 data from the two websites in April 2016. To examine the completeness of ForeignAssistance.gov data across data fields for the 10 reporting agencies, we analyzed the entire fiscal year 2014 dataset downloaded from ForeignAssistance.gov, which contained 176,651 transactions and 55 data fields, for missing values. We also conducted a more in-depth analysis of specific data fields using a random stratified sample of 106 transactions drawn from the fiscal year 2014 ForeignAssistance.gov data. We stratified the records by agency and allocated sample units to each agency’s stratum in proportion to its representation in the population of 176,651 transactions. Using this sample, we analyzed the information reported by the agencies in six data fields—implementing organization, award title, award description, award status, award location, and award sector.

Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95-percent confidence interval (e.g., plus-or-minus 10 percentage points). This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn.

We also interviewed State officials and contractors on the ForeignAssistance.gov team as well as key users of ForeignAssistance.gov—representatives from a consortium of nongovernmental organizations and the International Aid Transparency Initiative (IATI).

We conducted this performance audit from June 2015 to August 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Key Factors Identified by Agencies Reporting and Not Reporting ForeignAssistance.gov Data as Impediments to Their Data Collection Process

In November 2015, we surveyed officials of the 22 agencies identified in the OMB Bulletin No. 12-01 required to collect and report data for ForeignAssistance.gov. The survey asked the respondents to rate factors that impede their ability to collect and report data as follows: not at all an impediment; slight impediment; moderate impediment; great impediment; or no basis to judge. Figure 5 shows the factors that were identified by the 10 agencies reporting data for ForeignAssistance.gov that presented a great impediment.

Figure 6 shows the factors that were identified by the 12 agencies that have yet to report data for ForeignAssistance.gov that they anticipate as presenting a great impediment.

Appendix III: Analysis of Fiscal Year 2014 Transaction-Level Data on ForeignAssistance.gov

We analyzed fiscal year 2014 data that we downloaded from ForeignAssistance.gov in April 2016. This downloadable data file, in the comma-separated values format published by the Department of State (State) on ForeignAssistance.gov, contained 55 data fields and 175,651 transactions for 10 agencies reporting foreign assistance data. In the data file, each data field is represented by a column and each transaction by a row of data. According to State officials, these 55 data fields contain the most useful information about U.S. foreign assistance for website users and are a subset of the 189 data fields for which State collects foreign assistance data for ForeignAssistance.gov. A transaction is an individual financial record for each activity in an agency’s accounting system that has been processed in the given time period for program work with implementing partners and other administrative expenses.

We found that 24 of the data fields had fully reported information for all transactions and that the remaining 31 data fields were missing information, including 17 data fields for which 50 to 100 percent of the transactions had no data. ForeignAssistance.gov does not explain the reasons for missing information. However, State and other agency officials told us that a data field without any data may not necessarily mean that the agency did not provide required information because (1) the data field may not be relevant to the agencies’ reporting of foreign assistance, or (2) such data are not yet available for U.S. foreign assistance. For example, the data field for award interagency transfer status may not be relevant for an agency if there are no interagency funds to report. Additionally, other data fields may not be reported by any agency because of the nature of U.S. foreign assistance. For example, data fields for the budget period may not be populated because U.S. agencies simply do not provide such information, according to State. Table 6 below provides the data field name, data field value format, and definition for each of the 55 data fields, as well as the number and percentage of transactions that contained no data for each data field. In the table, if all transactions for a data field were populated with data, then the number of transactions (and the percentage of transactions) with no data are zero.

Appendix IV: Comments from the Department of State

Appendix V: Comments from the U.S. Agency for International Development

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Mona Sehgal (Assistant Director), Farahnaaz Khakoo-Mausel, Sada Aksartova, Melissa Wohlgemuth, Bridgette Savino, Debbie Chung, Martin De Alteriis, Carl Ramirez, Alexander Welsh, David W. Hancock, Julie Hirshen, David Dayton, Neil Doherty, and Alexandra Jeszeck made key contributions to this report. Kevin Walsh, Peter DelToro, and Shirley Hwang provided technical assistance.