Foreign Assistance: Actions Needed to Improve Transparency and Quality of Data on ForeignAssistance.gov
Published: 2016-08-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The overarching goal of ForeignAssistance.gov is to enable a variety of stakeholders—including U.S. citizens, civil society organizations, the U.S. Congress, federal agencies, partner countries, and other donors—to research and track U.S. government foreign assistance investments in an accessible and easily understood format.
GAO was asked to review the collection and reporting of ForeignAssistance.gov data. GAO examined (1) State's data collection and publishing process; (2) key impediments, if any, that agencies face in collecting and reporting data to State; and (3) the data published on ForeignAssistance.gov. GAO reviewed agency documents, assessed ForeignAssistance.gov data for completeness by comparing them to USAID's Foreign Aid Explorer data on U.S. foreign assistance for fiscal year 2014, and conducted semistructured interviews with the 22 agencies on their data collection and verification processes. GAO also interviewed OMB officials.
section_title: What GAO Found
paragraphs: Since 2013, the Department of State (State) has collected and published quarterly data on ForeignAssistance.gov from 10 agencies that provide the majority of U.S. foreign assistance and more recently has initiated a process to prepare the12 remaining agencies to collect and report data. The 2012 Office of Management and Budget (OMB) Bulletin No. 12-01 outlined requirements for collecting and publishing data from 22 agencies and designated State as the lead agency for implementing ForeignAssistance.gov.
GAO's survey showed that most of the 10 agencies reporting data for ForeignAssistance.gov identified limitations in their information technology systems and data availability as key impediments in collecting and reporting data, while most of the 12 agencies not yet reporting data identified lack of staff time as a potential key impediment.
GAO found that the data on ForeignAssistance.gov were incomplete and that State was not fully transparent about such limitations on the website. In addition, State has not updated ForeignAssistance.gov with verified annual data to ensure quality. GAO's analysis of fiscal year 2014 data showed that ForeignAssistance.gov did not report over $10 billion in disbursements and about $6 billion in obligations provided by the 10 reporting agencies, compared to U.S. Agency for International Development (USAID)-verified data (see fig.). A 2009 OMB memorandum requires agencies to improve transparency in published information, which includes identifying high-value information not yet available online. However, State, as the publisher of this information, does not provide agencies with guidance to identify data limitations that it can clearly disclose on the website and noted that it relies on agencies to report these. The absence of clear information on data limitations may undermine the goal of ForeignAssistance.gov to improve public knowledge and accountability of U.S. foreign assistance. Moreover, State, as the lead agency, has not updated ForeignAssistance.gov with verified data even though OMB Bulletin No. 12-01 indicates that these data should be updated annually using USAID-verified data. State and USAID officials told GAO that they are unable to update ForeignAssistance.gov with verified data because of differences in their datasets. OMB also noted that a review to assess whether agencies had sufficient internal data quality controls did not take place, although it was required by the bulletin. In the absence of a review or additional guidance to address the quality of the data on ForeignAssistance.gov, data will likely remain incomplete.
section_title: What GAO Recommends
paragraphs: To improve the transparency and quality of the data on ForeignAssistance.gov, State (1) should provide guidance to agencies on identifying data limitations and clearly disclose those limitations on the website; and (2) in consultation with the OMB Director and the USAID Administrator, undertake a review of efforts to ensure data quality and develop guidance on improving the quality of ForeignAssistance.gov data. State, OMB, and USAID concurred with the recommendations.

section_title: Letter
paragraphs: For more than a decade, partner countries and U.S. stakeholders have  raised concerns about the lack of timely and detailed data on foreign  assistance activities. Since 2009, the U.S. government has taken steps to  provide greater transparency on its foreign assistance activities around  the world. A series of international and domestic initiatives have outlined  key areas in which transparency could be enhanced, including timely  reporting and ensuring data accessibility to a variety of users. In 2011, the  United States became a signatory to the International Aid Transparency  Initiative (IATI), a global campaign that seeks to provide timely, forward- looking, and comprehensive data on foreign assistance funding and  activities in a standard electronic format. In 2012, the Office of  Management and Budget (OMB) issued Bulletin No. 12-01 (bulletin) to  provide guidance on its requirement for agencies to publicly report data  on their foreign assistance activities for ForeignAssistance.gov.  Congress also established standards and requirements, including a  common definition, for comprehensive reporting of all federal spending.  The overarching goal of ForeignAssistance.gov is to enable a variety of  stakeholders—including U.S. citizens, civil society organizations, the U.S.  Congress, federal agencies, partner countries, and other donors—to  research and track U.S. government foreign assistance investments in an  accessible and easily understood format. The bulletin identifies the  Department of State (State) as the lead agency responsible for collecting  and publishing data from 22 U.S. agencies for ForeignAssistance.gov.
You asked us to review the collection and reporting of data for  ForeignAssistance.gov. In this report, we examine (1) State’s data  collection and publishing process for ForeignAssistance.gov; (2) key  impediments, if any, that agencies face in collecting and reporting data for  ForeignAssistance.gov; and (3) the data published on  ForeignAssistance.gov.
To address these objectives, we analyzed key guidance documents  relating to the data collection process, including the 2012 OMB Bulletin  No. 12-01 (bulletin) and State’s November 2015 Agency Reporting Toolkit  (toolkit). To examine the data collection process, we conducted  semistructured interviews with officials of the 22 U.S. agencies on the  following topics: (1) data collection and validation processes; (2) guidance  provided by State on the data collection process; (3) resources needed to  collect data, such as infrastructure, staff, and training; and (4)  impediments that agencies may face in collecting and reporting  ForeignAssistance.gov data. We also interviewed OMB on its role in  providing guidance for ForeignAssistance.gov. Based on the agency  responses in the fourth area, we surveyed the same 22 U.S. agencies to  identify key impediments that agencies may face in collecting and  reporting data for ForeignAssistance.gov, including agency reporting  systems; required data fields in ForeignAssistance.gov; resources; and  sufficiency of the guidance provided. Finally, we assessed the data by  comparing fiscal year 2014 funding data published on  ForeignAssistance.gov to comparable funding assistance data collected  by the U.S. Agency for International Development (USAID) and published  on a website known as Foreign Aid Explorer. We determined that the  data were sufficiently reliable to serve as a reasonable comparison for the  purposes of our reporting objectives. Additionally, we analyzed a random  probability sample of 106 fiscal year 2014 transactions to assess the  completeness and consistency of the information provided by agencies  for selected data fields with State’s definition for each data field. We also  interviewed key users of ForeignAssistance.gov, including  representatives from a consortium of nongovernmental organizations and  IATI. See appendix I for more details on our objectives, scope, and  methodology.
We conducted this performance audit from June 2015 to August 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The OMB bulletin establishes ForeignAssistance.gov as an official site for  U.S. government foreign assistance data and requires U.S. agencies to  report, among other things, funding and activity-level data by  implementing mechanism (e.g., contract or grant), including activity  purpose, description, and location. In addition, the bulletin indicates that  agencies are required to provide transaction-level data for each activity.  Transactions are individual financial records of obligations and  disbursements in an agency’s accounting system. Data on the website  are categorized under the nine U.S. foreign assistance framework  categories (economic development; education and social services; health;  peace and security; democracy, human rights, and governance;  environment; humanitarian assistance; program management; and  multisector). In general, as of July 2016, data were available from fiscal  years 2006 to 2017. Users can view the data through graphic  presentations, including maps; filter data by agency, country, and sector;  and download data in a spreadsheet. Figure 1 shows a graphic  presentation of data (funding, agencies, and fiscal year) available on  ForeignAssistance.gov.
Among other things, the United States is publishing data on  ForeignAssistance.gov to meet international commitments and domestic  data transparency initiatives. The website incorporates key elements  necessary for the United States to meet its IATI commitment, such as  frequency of reporting (quarterly), activity-level data, and publishing  format. In addition, a 2015 State guidance document (toolkit) indicates  that ForeignAssistance.gov is expected to meet key domestic data  reporting requirements on U.S. government activities, including those in  the Digital Accountability and Transparency Act of 2014 (DATA Act). The  DATA Act aims to improve the transparency and quality of the federal  spending data by requiring that agencies begin reporting data on all  federal spending—including grants, contracts, and other types of financial  assistance—using governmentwide data standards by May 2017 and  publish these data in a computer-readable format by May 2018.
The specific reporting guidelines for ForeignAssistance.gov are outlined  in the bulletin, which also notes two existing reports on U.S. foreign  assistance—the U.S. Overseas Loans and Grants: Obligations and Loans  Authorizations (Greenbook) report to Congress and the U.S. Annual  Assistance Report to the Organisation for Economic Co-operation and  Development’s Development Assistance Committee (OECD/DAC). The  bulletin states that USAID will be the lead agency for verifying the data  and assembling these reports. USAID publishes Greenbook and  OECD/DAC data on Foreign Aid Explorer. Table 1 describes the key  characteristics of U.S. foreign assistance reporting, including lead  agency, frequency, and type of data collected.
The bulletin outlines a quarterly data collection process for agencies to  submit data for ForeignAssistance.gov. The process begins with State  providing agencies a data submission template to assist with the  collection of agency data.
The quarterly process outlined in the bulletin includes the following key  steps:
Agencies are required to submit data on their ongoing foreign  assistance activities on a quarterly basis.
State is responsible for working with each agency’s designated point  of contact to coordinate on the data submitted for  ForeignAssistance.gov and identify areas in which agencies may  need to make corrections.
State is responsible for publishing agency data on  ForeignAssistance.gov. In addition, on a quarterly basis, State is  responsible for developing the U.S. government IATI-formatted file  and submitting it to the IATI Registry.
section_title: State Has Established a Process for Collecting and Publishing ForeignAssistance.gov Data
subsections: section_title: State Is Collecting and Publishing ForeignAssistance.gov Data for 10 Agencies
paragraphs: Since 2013, State has collected and published data from 10 of the 22  agencies identified in the bulletin. State focused on these 10 agencies  because they are responsible for providing 98 percent of U.S. foreign  assistance, according to State. In addition, State officials told us that they  prioritized improving data quality before collecting data from the 12  agencies that are not yet reporting data for ForeignAssistance.gov.
Figure 2 illustrates State’s data collection and publishing process for the  10 agencies currently reporting data to ForeignAssistance.gov. The 10  agencies are the Department of Defense (DOD), the Department of  Health and Human Services (HHS), the Inter-American Foundation (IAF),  the Millennium Challenge Corporation (MCC), the Peace Corps, State,  the Department of the Treasury (Treasury), the U.S. African Development  Foundation (USADF), USAID, and the Department of Agriculture (USDA).
The process consists of five key steps, which occur on a quarterly basis:  1.  State’s ForeignAssistance.gov team reaches out to each of the  agency points of contact to collect that quarter’s data and provides  technical guidance documents, including the bulletin.  2.  Each agency point of contact e-mails data to State in a spreadsheet  or the extensible markup language (XML) format, which can contain  as many as 189 data fields per activity. In 2014, State expanded the  number of data fields to a total of 189 to align with the IATI Standard.  State officials indicated that some of the fields (e.g., currency and  U.S. government) are auto-populated by the ForeignAssistance.gov  team and that not all data fields are relevant to every agency. State  officials told us that 55 of the 189 data fields are the most relevant to  users and can be downloaded from ForeignAssistance.gov in  spreadsheet format (see fig. 3). In November 2015, State prioritized  37 data fields that are critical to the U.S. government’s foreign  assistance reporting, according to State. According to State officials,  on average, the 10 agencies reporting data for ForeignAssistance.gov  submit quarterly data for 40 to 50 data fields.  3.  After converting the agency-submitted spreadsheet data to XML  format for agencies that do not have conversion capability, State  checks the agency data to determine whether all required fields are  populated.  4.  State relays any missing values or possible data reporting errors to  agencies and allows them to review and make corrections before it  publishes the data on the public website.  5.  Using the agency-corrected data, State creates and publishes  downloadable data files on ForeignAssistance.gov. During this final  step, State also simultaneously links the quarterly data files to the IATI  Registry.
In November 2015, State created a community of practice website to  allow agency points of contact to engage online, clarify any issues, and  share lessons learned. However, as of May 2016, agencies had not  posted comments or questions on the website.
section_title: State Has Made Some Progress in Preparing the 12 Agencies Not Yet Reporting data for ForeignAssistance.gov
paragraphs: In 2015, State developed a process to prepare the 12 agencies that had  not yet reported data for ForeignAssistance.gov. The 12 agencies are the  Department of Commerce (Commerce), the Department of Energy  (DOE), the Department of Homeland Security (DHS), the Department of  the Interior (DOI), the Department of Justice (DOJ), the Department of  Labor (DOL), the Department of Transportation (DOT), the Environmental  Protection Agency (EPA), the Export-Import Bank of the United States  (Ex-Im), the Federal Trade Commission (FTC), the Overseas Private  Investment Corporation (OPIC), and the U.S. Trade and Development  Agency (USTDA). In September 2015, officials from all 12 agencies told  us that State had not reached out to them with specific reporting  instructions. Between November 2015 and May 2016, State provided the  toolkit to all 12 agencies and conducted information sessions with most of  them on reporting data for ForeignAssistance.gov.
The toolkit contains descriptions of the data fields and resources to  support the five-phase data publishing process. It also includes a list of  37 priority data fields, which, according to State, will make it easier for  agencies to identify where they should focus their data collection efforts.  The five phases of the publishing process are the following:  1.  Planning phase: Agencies collaborate with the  ForeignAssistance.gov team to understand the general reporting  requirements.  2.  Discovery phase: Agencies review their systems to identify foreign  assistance data and better understand the data sources and needs.  3.  Preparation phase: Agencies create and submit data samples to the  ForeignAssistance.gov team, which provides feedback.  4.  Processing phase: Agencies submit a final dataset to the  ForeignAssistance.gov team and conducts quality assurance checks.  5.  Execution phase: State uploads the final approved dataset to  ForeignAssistance.gov, and agencies can issue press releases to  external stakeholders.
Between December 2015 and January 2016, State conducted two  information sessions on the toolkit. Six agencies (DOE, DOJ, DOL, EPA,  FTC, and OPIC) attended one or both sessions. As of May 2016, State  had introduced all 12 agencies to the data publishing process, but none  had published data on ForeignAssistance.gov. According to State, as of  May 2016, of the 12 agencies,
Six—Commerce, DHS, DOI, DOT, EPA, and USTDA—were in the  planning phase. These agencies had received the toolkit, and some  were beginning to review their systems to identify foreign assistance  data and better understand the data required.
Two—DOJ and Ex-Im—were in the preparation phase. These  agencies prepared or provided sample activity-level and organization- level datasets for State to review as part of toolkit implementation.
Four—DOE, DOL, FTC, and OPIC—were in the processing phase.
These agencies had submitted at least one quarterly dataset for the  ForeignAssistance.gov team to review.
State has identified limited staff resources as a key constraint in collecting  and publishing ForeignAssistance.gov data. According to State officials,  the ForeignAssistance.gov team would be challenged to manage data  provided by the 12 agencies not yet reporting, while continuing to publish  data and work on data quality improvements with the 10 agencies that are  reporting.
section_title: Agencies Identified Various Impediments to Collecting ForeignAssistance.gov Data
paragraphs: Our analysis shows that agencies face impediments in collecting the  required information. Most of the 10 agencies reporting data for  ForeignAssistance.gov identified limitations in their information  technology (IT) systems as a key impediment in collecting and reporting  data, while most of the 12 agencies not yet reporting data identified lack  of staff time as a key impediment that they anticipate facing. In November  2015, we surveyed officials at all 22 agencies to identify and rate key  factors that may impede their ability to collect and report data for  ForeignAssistance.gov.
subsections: section_title: Agencies Reporting Data Identified Limited Information Technology Systems and Availability of Data as Key Impediments
paragraphs: Based on our analysis of survey data, the top three factors most of the 10  agencies reporting data identified as presenting a moderate or great  impediment to collecting and reporting data for ForeignAssistance.gov  were (1) limitations in their IT systems, (2) lack of available data, and (3)  having to adjust available data to fulfill ForeignAssistance.gov  requirements (see table 2).
More than half of the agencies reporting data also identified as moderate  or great impediments to collecting and reporting ForeignAssistance.gov  data the lack of a single agency internal IT system from which to pull all  ForeignAssistance.gov data and the number of data fields required (see  table 2). No agency reporting data identified that a lack of a State point of  contact or a lack of a governmentwide dedicated server presented a  moderate or great impediment.
Based on interviews conducted prior to the survey, most agency officials  noted that their existing IT systems were limited in that they did not track  data at a level of detail required by ForeignAssistance.gov. Several  agencies whose main mission is not foreign assistance explained that  updating their existing systems for these requirements was not a priority.  Two agencies whose main mission is foreign assistance, State and  USAID, undertook assessments of their current systems to understand  and better align the capabilities of their systems with these reporting  requirements. One agency, DOD, told us that it had plans to update its IT  system to be able to report quarterly data, but that it would take some  time.
Most agency officials we interviewed also told us that they lacked an  integrated system to track both the financial and project data required for  ForeignAssistance.gov. To fill out the ForeignAssistance.gov spreadsheet  data template provided by State, they said they had to collect data from  key documents and multiple internal systems for accounting and project  management. However, one agency—MCC—indicated in interviews that  it already had an integrated system and was therefore able to consolidate  its reporting. The agency noted recent updates to its existing IT system  and attributed its upgraded system to being a newer agency.
section_title: Agencies Not Yet Reporting Data Identified Lack of Staff Time as a Key Anticipated Impediment
paragraphs: Based on our analysis of survey data, lack of staff time was the top factor  identified as an anticipated impediment by most of the 12 agencies not  yet reporting data for ForeignAssistance.gov (see table 3).
More than half of the agencies reporting data also identified lack of  funding, number of data fields, and limitations in their IT systems as  moderate or great impediments to collecting and reporting data for  ForeignAssistance.gov (see table 3).
Based on interviews conducted prior to the survey, some of the agency  officials noted that they would have to add ForeignAssistance.gov  reporting duties to existing staff’s primary job functions, which could be  burdensome, especially for agencies with smaller foreign assistance  portfolios. Other agency officials said they anticipated that staff time  would be an issue, because the collection process would involve many  people throughout the agency. Furthermore, some of the agency officials  also noted their limited capacity to provide data quarterly for the number  of data fields that ForeignAssistance.gov requires. Officials from one  agency explained that completing the annual requests for data for the  Greenbook and OECD/DAC was already time-consuming, as it required  sending a data call to subcomponents and field staff, compiling the data  into one spreadsheet, and going through multiple layers of review. See  appendix II for key factors identified by agencies reporting and not yet  reporting data as impediments to their data collection process.
section_title: ForeignAssistance.gov Is Not Transparent about Data Limitations, and Data Are Not Updated Annually to Ensure Quality
paragraphs: Data from some agencies that report on their foreign assistance to  ForeignAssistance.gov are incomplete at the aggregate level. We found  that in the aggregate, 14 percent of obligations and 26 percent of  disbursements for fiscal year 2014 were not published on the website,  when compared to USAID’s verified data. We also found that for some  high-priority data fields, information was missing or inconsistent with  State’s definition for each data field. In addition, although  ForeignAssistance.gov discloses that published data are incomplete, we  found that the website is not fully transparent about these data limitations.  Moreover, the data published on ForeignAssistance.gov are not annually  updated against verified—complete and accurate—foreign assistance  data, as required in the bulletin.
subsections: section_title: Data Published on ForeignAssistance.gov Are Incomplete
paragraphs: We analyzed fiscal year 2014 data downloaded from  ForeignAssistance.gov to assess the completeness of aggregate funding  data as well as the completeness and consistency of information in  selected data fields with State’s definitions for those data fields. We found  that data on ForeignAssistance.gov were incomplete at the aggregate  funding level as well as for some disaggregated data at the transaction  level. In addition, we found that data for selected high-priority data fields  were inconsistent with State’s definitions. State officials told us that they  rely on agencies to provide complete and accurate information for  ForeignAssistance.gov.
subsections: section_title: Fourteen Percent of Obligations and 26 Percent of Disbursements for Fiscal Year 2014 Were Not Published on ForeignAssistance.gov
paragraphs: We found that in the aggregate, 14 percent of obligations and 26 percent  of disbursements for fiscal year 2014 from the 10 agencies reporting data  for ForeignAssistance.gov were not published on the website as of April  2016. Our comparison of funding data on ForeignAssistance.gov to  funding data published on the Foreign Aid Explorer website showed that  ForeignAssistance.gov did not reflect more than $10 billion in  disbursements and about $6 billion in obligations provided by the 10  agencies in fiscal year 2014. Data on these two websites are generally  comparable because both essentially use the same definition of foreign  assistance, based on the Foreign Assistance Act of 1961, as amended.  The OMB bulletin indicates that USAID’s data are verified—checked for  completeness and accuracy. The bulletin notes that USAID’s verification  includes checking for common errors, comparing with third-party sources  to identify gaps and more complex errors, accommodating negative  entries, and taking other steps to ensure data quality. Additionally,  according to USAID officials, as part of the verification process for the  data published on Foreign Aid Explorer, they check to ensure that there  are no anomalies, errors, duplicates, or missing values. Furthermore,  USAID checks to ensure that data are consistent with those for prior  years and verifies the data against official U.S. government documents.  We selected fiscal year 2014 because all 10 agencies published data for  that year on both websites and because it was the most recent year for  which fully reported and verified USAID foreign assistance data were  available at the time of our analysis.
Based on our analysis of the fiscal year 2014 funding data for the 10  agencies (see table 4), the total obligations on ForeignAssistance.gov  were $36.1 billion, almost $6 billion (14 percent) less than the $42 billion  identified on Foreign Aid Explorer. The total disbursements on  ForeignAssistance.gov were $29.9 billion, more than $10 billion (26  percent) less than the $40.4 billion on Foreign Aid Explorer in the same  year. Specifically, fiscal year 2014 obligations and disbursements were  about the same or identical for four agencies (IAF, MCC, the Peace  Corps, and USADF) and had a difference of less than 10 percent for two  agencies (Treasury and USAID). However, the differences in obligations  or disbursements exceeded 10 percent for DOD, HHS, State, and USDA,  with DOD’s and USDA’s data showing the largest differences.
The four agencies whose fiscal year 2014 ForeignAssistance.gov funding  data showed a difference of more than 10 percent from the Foreign Aid  Explorer data for the same year provided the following explanations:
DOD. According to DOD officials, two factors explain the  discrepancies. First, the two websites attribute funding for a significant  portion of U.S. security assistance differently: on  ForeignAssistance.gov, the assistance—which State funds and DOD  implements—is attributed to State, whereas on Foreign Aid Explorer,  it is attributed to DOD. The second factor, according to DOD  officials, is inconsistent reporting of fiscal year 2014 funding data for  ForeignAssistance.gov: DOD reported obligations, but not  disbursements, for some programs, and disbursements, but not  obligations, for other programs. For example, DOD did not report $4.7  billion in fiscal year 2014 obligations or disbursements for the  Afghanistan Security Forces Fund on ForeignAssistance.gov.
HHS. HHS officials stated that the data the agency published on  Foreign Aid Explorer more accurately reflect the agency’s foreign  assistance portfolio than the data the agency published on  ForeignAssistance.gov. HHS officials suggested that their agency  data for fiscal years 2013, 2014, and 2015 on ForeignAssistance.gov  should not be used until the quality of the data published on the  website is improved. They did not explain the differences in the  funding data on the two websites.
State. State officials told us that to some extent the discrepancies  came about because funding for peacekeeping and U.S. contributions  to international organizations was not included in the fiscal year 2014  data on ForeignAssistance.gov. State officials also noted that data  published on Foreign Aid Explorer are considered to be more fully  reported because they are submitted to USAID by State’s bureaus,  which manually enter detailed data; by comparison, State’s data for  ForeignAssistance.gov are generated from the agency’s accounting  system. In addition, State’s accounting system at present includes  transactions reported by State’s main office in Washington, D.C., but  does not include transactions of overseas locations.
USDA. USDA officials told us that they reported incorrect fiscal year  2014 obligations for ForeignAssistance.gov because USDA  misinterpreted State’s guidance. They also noted that USDA is  working with State and USAID to ensure that USDA’s foreign  assistance data are accurate and consistent on both websites.
DOD, HHS, State, and USDA officials said that they were aware of the  differences in their funding data published on the two websites and were  working on improving the quality of the data published on  ForeignAssistance.gov. State officials could not indicate when the 10  reporting agencies would be able to report complete funding data for  ForeignAssistance.gov and stated that it is the agencies’ responsibility to  report complete and accurate data.
section_title: Information Was Missing or Inconsistent with Definitions for Some High-Priority Data Fields
paragraphs: In addition to discrepancies in the aggregate funding data, we found that  for some of the high-priority data fields the information was either missing  or inconsistent with definitions that State developed based on the IATI  Standard. We analyzed fiscal year 2014 data for six of the data fields  prioritized by IATI—implementing organization, award title, award  description, award status, award location, and award sector— to  determine if agencies populated these data fields with information and if  the information was consistent with State’s definitions. According to State,  data fields prioritized by IATI should be populated because they provide  key information necessary to track a specific activity. We analyzed the  content of the six data fields using a probability sample of 106  transactions drawn from the fiscal year 2014 data. We found that for three  data fields—implementing organization, award location, and award  sector—information was provided and was consistent with State’s  definition for a majority or all of the transactions (see table 5).
For the other three data fields—award title, award description, and award  status—the information was missing or inconsistent with the definitions for  the majority of the transactions in the sample. For example, for award  title, 82 percent of the transactions were either missing information or had  information that was inconsistent with the definition for this data field. We  also found that for award title, agencies often provided program or sector  descriptors, and for award description, agencies routinely provided  shorthand descriptions, acronyms, or terminology that could only be  understood by officials at the agency that made the award. For example,  an award description would contain “Train, Eval & Oth Related Act” or  “AIDSTAR Sector II Award.” Only three transactions in our sample  contained award descriptions that were somewhat consistent with the  definition of a brief narrative that provided an understanding of the  undertaking for which the award was funded, its objectives, and the  hypothesis of the award’s development impact. The lack of clarity for data  fields, such as award description, could make it difficult for domestic and  international users of ForeignAssistance.gov to understand the data.
State officials told us that our findings were consistent with their  observations of the data. They explained that the reporting agencies’ data  systems were not currently capable of capturing and generating data that  would fully meet IATI’s expectations for detailed, transaction-level  information. Our analysis also shows that most of the reporting agencies  identified impediments in collecting and reporting data, including  limitations in their IT systems and lack of available data required by  ForeignAssistance.gov.
section_title: State Relies on Agencies to Provide Complete and Accurate Data
paragraphs: State officials told us that they rely on agencies to provide complete and  accurate data because, according to the bulletin, reporting agencies are  responsible for the accuracy, completeness, and integrity of their data  submissions. Additionally, State officials noted that the  ForeignAssistance.gov team has neither the expertise nor the resources  to check the data for completeness or accuracy on a quarterly basis.  They explained that collecting and publishing data from 10 agencies on a  quarterly basis is inherently challenging and can lead to trade-offs  between quality and timeliness. Nevertheless, State’s  ForeignAssistance.gov team conducts certain checks, such as ensuring  that agency data are formatted properly and that dates are within valid  ranges. (See table 6 in app. III for data field format values.)
section_title: ForeignAssistance.gov Is Not Fully Transparent about Data Limitations
paragraphs: Although ForeignAssistance.gov discloses—through a graphic summary  and agency notes—that published data may not be complete, it is not fully  transparent about data limitations. Specifically, we found that  ForeignAssistance.gov does not clearly identify what data are missing  and consistently identify and publish data limitations by agency.
According to OMB’s Open Government Directive, which is Memorandum  M-10-06, agencies should take specific actions to implement the  principles of transparency, among other things, in publishing government  information online. The memorandum indicates steps to improve  transparency in publishing information, which include identifying high- value information not yet available for online publication. In addition,  according to prior GAO work, useful practices that help foster  transparency to the public and assist decision makers in understanding  the quality of agency data include, among other things, discussion of data  quality and limitations.
subsections: section_title: ForeignAssistance.gov Is Not Clear about Missing Data
paragraphs: Although ForeignAssistance.gov discloses that some data are not yet  available, it does not clearly state what data are missing. State presents a  graphic summary of data from the 10 reporting agencies published on the  website (fig. 4). It uses the terms “full” and “partial” to characterize agency  datasets, but does not explain what these terms mean. Specifically, as of  July 2016, ForeignAssistance.gov indicated that six agencies—IAF, MCC,  the Peace Corps, Treasury, USADF, and USAID—reported a full set of  data and that four agencies—DOD, HHS, State, and USDA—reported a  partial set of data (see fig. 4). In particular, it is not clear whether full  reporting means that an agency reported data (1) reflecting all of its  foreign assistance funding or (2) across all of the required data fields.
While the agency notes published on ForeignAssistance.gov provide  some information to help understand an agency’s foreign assistance  activities, not all agencies report this information and, moreover, these  notes do not clearly identify data limitations for reporting agencies.
We found that 6 of the 10 reporting agencies (DOD, the Peace Corps,  State, Treasury, USAID, and USDA) provided agency notes on  ForeignAssistance.gov as of July 2016. All six agencies provided general  descriptions of their foreign assistance programs and accounts. However,  the agency notes for three of the four agencies characterized on  ForeignAssistance.gov as reporting partial data (see fig. 4)—DOD, State,  and USDA—did not identify data limitations for a given year. For example,  they did not comment on gaps in obligation and disbursement amounts  generally or by fiscal year. The fourth agency characterized on  ForeignAssistance.gov as reporting partial data—HHS—did not post  agency notes. In contrast, other publishers of U.S. data provide more  detailed information on data limitations or changes to the data in a given  year. For example, the 2014 Greenbook identified an agency that  provided data for two additional programs that year, an agency that  continued to not report data, and another agency that was unable to  report on certain data in detail.
Agency notes are not required by State or the bulletin, which states that  agencies may provide supplemental narratives and can include data  explanations and other clarifying information. Additionally, State’s most  recent guidance (toolkit) on reporting to ForeignAssistance.gov does not  mention agency notes or provide instructions for agencies to identify the  limitations of their data.
State officials told us that they rely on agencies to report data limitations.  However, if State does not provide agencies with guidance to identify  data limitations that State can clearly disclose on ForeignAssistance.gov,  it may undermine the website’s goal of improving public knowledge and  accountability of U.S. foreign assistance.
section_title: ForeignAssistance.gov Data Are Not Updated Annually with Verified Foreign Assistance Data
paragraphs: As of May 2016, State had not updated ForeignAssistance.gov data with  verified—complete and accurate—annual foreign assistance data to  improve the quality and ensure consistency in the reporting of U.S.  foreign assistance. OMB indicates in its bulletin that  ForeignAssistance.gov data should be updated at the end of each  calendar year using verified data reported by USAID for the Greenbook  and OECD/DAC to ensure consistency in published information.  Additionally, the bulletin indicated that in 2014, USAID, State, OMB, the  National Security Staff, and the Office of Science and Technology were  expected to undertake a review of the first 2 years’ experience to assess  whether agencies whose data are published on ForeignAssistance.gov  had demonstrated sufficient internal data quality control to graduate from  the USAID verification process. However, as of May 2016, this  interagency review had not taken place because, according to OMB  officials, only 3 of the 10 reporting agencies were providing data to  ForeignAssistance.gov of sufficient quality to meet the Greenbook and  OECD/DAC reporting requirements. Since the majority of the agencies’  data were not yet of sufficient quality, OMB officials noted that a review to  graduate agencies from USAID’s verification process was premature.
State and USAID officials told us that they are unable to update  ForeignAssistance.gov with USAID’s verified data because of differences  between the two datasets. State officials cited three key differences:
Number of data fields. ForeignAssistance.gov data includes up to  189 data fields; however, USAID verifies the information only for a  subset of about 20 data fields.
Frequency of data reported. ForeignAssistance.gov captures  quarterly data; however, USAID uses annual data.
Transaction-level data are stored differently.
ForeignAssistance.gov captures transaction-level data for each  activity; however, USAID aggregates the transaction-level data to the  activity level.
USAID noted that reconciling ForeignAssistance.gov data with verified  Greenbook and OECD/DAC data would be problematic, especially for the  seven agencies whose data do not meet the quality standards for  Greenbook and OECD/DAC reporting. For these agencies, USAID (1)  obtains missing information for some data fields (e.g., recipient country)  directly from agency officials or units that report the information, and (2)  assigns sector codes and other fields—which are not always provided in  the data that agencies report for ForeignAssistance.gov—based on  OECD/DAC statistical reporting directives and methodologies.
OMB officials agreed that the bulletin’s requirement for annually updating  data published on ForeignAssistance.gov with USAID-verified data has  not been feasible. They also acknowledge that the quality of  ForeignAssistance.gov data needs to be improved.
Since State, USAID, and OMB recognize that a key step outlined in the  bulletin to ensure data quality may not be feasible, and in the absence of  the 2-year review on data verification or guidance on how to address the  quality of the data on ForeignAssistance.gov, data will likely remain  inconsistent across the range of U.S. foreign assistance reporting.
section_title: Conclusions
paragraphs: In response to domestic and international initiatives in the last decade,  the U.S. government has increased the frequency and amount of foreign  assistance data made available to the public. In 2011, the U.S.  government made an international commitment to publishing more  detailed and timely funding and activity data for users, including partner  country governments, civil society organizations, and taxpayers. As the  U.S. government’s lead agency for this reporting, State established  ForeignAssistance.gov, with guidance from OMB, to provide detailed  foreign assistance data on a quarterly basis. Given the magnitude and  frequency of data collection, State prioritized collection and publishing of  data for 10 agencies that account for the majority of U.S. foreign  assistance. Facing trade-offs—which agencies recognize—between data  quality and timeliness in reporting, State has experienced challenges in  ensuring transparency and data quality on ForeignAssistance.gov. In  particular, in the absence of guidance from State to reporting agencies to  clearly identify their data limitations, State has not fully disclosed data  limitations of ForeignAssistance.gov, thereby undermining the website’s  goal of increasing the transparency of U.S. foreign assistance  information. Moreover, because updating ForeignAssistance.gov with  USAID verified data has not been feasible and the interagency  assessment of the process to ensure sufficient quality control has not  been done, gaps in data quality remain unaddressed, and users may risk  using inaccurate or incomplete information for decision-making and  accountability purposes.
section_title: Recommendations for Executive Action
paragraphs: To improve the transparency of ForeignAssistance.gov, we recommend  that the Secretary of State provide guidance to agencies to identify data  limitations that State can clearly disclose on the website.
To improve the quality of the data published on ForeignAssistance.gov  and help ensure consistency in published information, we recommend  that the Secretary of State, in consultation with the Director of OMB and  the USAID Administrator, take the following two actions: undertake a review of the efforts to date on ensuring data quality and develop additional guidance that takes into consideration current  challenges to updating ForeignAssistance.gov with verified data.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to OMB and the 22 agencies reviewed  in this report (Commerce, DHS, DOD, DOE, DOI, DOJ, DOL, DOT, EPA,  Ex-Im, FTC, HHS, IAF, MCC, OPIC, PC, Treasury, State, USADF,  USAID, USDA, and USTDA) for review and comment. In written  comments, reprinted in appendix IV and V, State and USAID agreed with  our recommendations. OMB also agreed with our recommendation and  provided us with the following comments in an e-mail: it will continue to  work with State and USAID to help guide agencies in improving the  quality and consistency of the data published on ForeignAssistance.gov.
However, State expressed concern that the report did not provide  specific, actionable recommendations to the other 20 agencies  responsible for reporting ForeignAssistance.gov. As noted in the report,  OMB Bulletin No. 12-01 provides overall guidance on data standards and  requirements for the other 20 agencies. We made the recommendations  to State, in consultation with OMB and USAID, because these agencies  are responsible for improving guidance and reporting requirements that  can help achieve the website’s goal of improving public knowledge and  accountability of U.S. foreign assistance. DOD, EPA, FTC, HHS, MCC,  State, USAID, and USDA provided technical comments that we  incorporated as appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 13 days from the  report date. At that time, we will send copies of this report to the Directors  of the Office of Management and Budget, Peace Corps, and U.S. Trade  and Development Agency; the Administrators of the Environmental  Protection Agency and U.S. Agency for International Development; the  Secretaries of State, Agriculture, Commerce, Defense, Energy, Health  and Human Services, Homeland Security, the Interior, Labor,  Transportation, and the Treasury; the Attorney General of the United  States; General Counsel of the Federal Trade Commission; and the Chief  Executive Officers of the Export-Import Bank, Inter-American Foundation,  Millennium Challenge Corporation, Overseas Private Investment  Corporation, and U.S. African Development Foundation. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3149 or GootnickD@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: To review the collection and reporting of data for ForeignAssistance.gov,  this report examines (1) the Department of State’s (State) data collection  and publishing process for ForeignAssistance.gov; (2) key impediments, if  any, that agencies face in collecting and reporting data for  ForeignAssistance.gov; and (3) the data published on  ForeignAssistance.gov.
To conduct this work, we analyzed key guidance documents relating to  the data collection process: the 2012 Office of Management and Budget  (OMB) Bulletin No. 12-01, Guidance on Collection of U.S. Foreign  Assistance Data (Sept. 25, 2012) (bulletin), the 2009 OMB Memorandum  M-10-06, Open Government Directive (Dec. 8, 2009), and State’s  November 2015 Agency Reporting Toolkit (toolkit). We also reviewed key  U.S. government plans and international agreements that outline steps for  ensuring transparency in foreign assistance reporting, including the 2011  and 2013 U.S. Open Government National Action Plan, the 2005 Paris  Declaration on Aid Effectiveness, and the 2011 Busan Outcome  Agreement.
To examine State’s data collection process for ForeignAssistance.gov, we  conducted semistructured interviews with officials of 22 U.S. agencies— the Departments of Agriculture, Commerce, Defense, Energy, Health and  Human Services, Homeland Security, Justice, the Interior, Labor, State,  Transportation, and the Treasury; the Environmental Protection Agency;  the Export-Import Bank of the United States; the Federal Trade  Commission; the Inter-American Foundation; the Millennium Challenge  Corporation; the Overseas Private Investment Corporation; the Peace  Corps; the U.S. African Development Foundation; the U.S. Agency for  International Development; and the U.S. Trade and Development  Agency—concerning four areas of the data collection process, including  (1) data collection and validation processes; (2) guidance provided by  State on the data collection process; (3) resource needs to collect data,  such as infrastructure, staff, and training; and (4) impediments that  agencies may face in collecting and reporting data for  ForeignAssistance.gov. We also interviewed OMB on its monitoring role  for ForeignAssistance.gov.
To identify the impediments that agencies face in collecting and reporting  data for ForeignAssistance.gov, we surveyed the same 22 U.S. agencies  to identify key impediments, if any, that agencies may face in collecting  and reporting data for ForeignAssistance.gov, including agency reporting  systems; required data fields in ForeignAssistance.gov; resources; and  guidance provided that may impede an agency’s ability to collect and  report data for ForeignAssistance.gov. For the 12 agencies that do not yet  report data for ForeignAssistance.gov, we modified our survey instrument  to probe the extent to which they anticipated impediments in collecting  and reporting foreign assistance data to State. The survey was  administered in November 2015, and agencies provided their responses  between November 2015 and January 2016. All 22 agencies responded  to the survey.
To assess the data from the 10 reporting agencies published on  ForeignAssistance.gov, we compared fiscal year 2014 funding (obligation  and disbursement) data published on Foreign Assistance.gov to the data  collected and published by the U.S. Agency for International  Development’s (USAID) data on the Foreign Aid Explorer website  (http://explorer.usaid.gov/). Foreign assistance data available on these  two websites are based on essentially the same definition of foreign  assistance. To determine the reliability of the Foreign Aid Explorer data,  we interviewed USAID officials, reviewed documentation about the data,  and examined the data published on USAID’s website. We determined  that USAID’s verification processes for the Foreign Aid Explorer data  include checks to identify potential anomalies, duplicates, missing values,  and other errors. In addition, we found that USAID compares the  agencies’ data submissions to other available sources as completeness  checks. We determined that data published on Foreign Aid Explorer were  sufficiently reliable to serve as a reasonable comparison for the  ForeignAssistance.gov data for the purposes of our reporting objectives.  However, it was beyond the scope of this engagement to independently  verify agency source data.
We downloaded fiscal year 2014 data from the two websites in April  2016. To examine the completeness of ForeignAssistance.gov data  across data fields for the 10 reporting agencies, we analyzed the entire  fiscal year 2014 dataset downloaded from ForeignAssistance.gov, which  contained 176,651 transactions and 55 data fields, for missing values. We  also conducted a more in-depth analysis of specific data fields using a  random stratified sample of 106 transactions drawn from the fiscal year  2014 ForeignAssistance.gov data. We stratified the records by agency  and allocated sample units to each agency’s stratum in proportion to its  representation in the population of 176,651 transactions. Using this  sample, we analyzed the information reported by the agencies in six data  fields—implementing organization, award title, award description, award  status, award location, and award sector.
Because we followed a probability procedure based on random  selections, our sample is only one of a large number of samples that we  might have drawn. Since each sample could have provided different  estimates, we express our confidence in the precision of our particular  sample’s results as a 95-percent confidence interval (e.g., plus-or-minus  10 percentage points). This is the interval that would contain the actual  population value for 95 percent of the samples we could have drawn.
We also interviewed State officials and contractors on the  ForeignAssistance.gov team as well as key users of  ForeignAssistance.gov—representatives from a consortium of  nongovernmental organizations and the International Aid Transparency  Initiative (IATI).
We conducted this performance audit from June 2015 to August 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Key Factors Identified by Agencies Reporting and Not Reporting ForeignAssistance.gov Data as Impediments to Their Data Collection Process
paragraphs: In November 2015, we surveyed officials of the 22 agencies identified in  the OMB Bulletin No. 12-01 required to collect and report data for  ForeignAssistance.gov. The survey asked the respondents to rate factors  that impede their ability to collect and report data as follows: not at all an  impediment; slight impediment; moderate impediment; great impediment;  or no basis to judge. Figure 5 shows the factors that were identified by the  10 agencies reporting data for ForeignAssistance.gov that presented a  great impediment.
Figure 6 shows the factors that were identified by the 12 agencies that  have yet to report data for ForeignAssistance.gov that they anticipate as  presenting a great impediment.
section_title: Appendix III: Analysis of Fiscal Year 2014 Transaction-Level Data on ForeignAssistance.gov
paragraphs: We analyzed fiscal year 2014 data that we downloaded from  ForeignAssistance.gov in April 2016. This downloadable data file, in the  comma-separated values format published by the Department of State  (State) on ForeignAssistance.gov, contained 55 data fields and 175,651  transactions for 10 agencies reporting foreign assistance data. In the data  file, each data field is represented by a column and each transaction by a  row of data. According to State officials, these 55 data fields contain the  most useful information about U.S. foreign assistance for website users  and are a subset of the 189 data fields for which State collects foreign  assistance data for ForeignAssistance.gov. A transaction is an individual  financial record for each activity in an agency’s accounting system that  has been processed in the given time period for program work with  implementing partners and other administrative expenses.
We found that 24 of the data fields had fully reported information for all  transactions and that the remaining 31 data fields were missing  information, including 17 data fields for which 50 to 100 percent of the  transactions had no data. ForeignAssistance.gov does not explain the  reasons for missing information. However, State and other agency  officials told us that a data field without any data may not necessarily  mean that the agency did not provide required information because (1)  the data field may not be relevant to the agencies’ reporting of foreign  assistance, or (2) such data are not yet available for U.S. foreign  assistance. For example, the data field for award interagency transfer  status may not be relevant for an agency if there are no interagency funds  to report. Additionally, other data fields may not be reported by any  agency because of the nature of U.S. foreign assistance. For example,  data fields for the budget period may not be populated because U.S.  agencies simply do not provide such information, according to State.  Table 6 below provides the data field name, data field value format, and  definition for each of the 55 data fields, as well as the number and  percentage of transactions that contained no data for each data field. In  the table, if all transactions for a data field were populated with data, then  the number of transactions (and the percentage of transactions) with no  data are zero.
section_title: Appendix IV: Comments from the Department of State
section_title: Appendix V: Comments from the U.S. Agency for International Development
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Mona Sehgal (Assistant  Director), Farahnaaz Khakoo-Mausel, Sada Aksartova, Melissa  Wohlgemuth, Bridgette Savino, Debbie Chung, Martin De Alteriis, Carl  Ramirez, Alexander Welsh, David W. Hancock, Julie Hirshen, David  Dayton, Neil Doherty, and Alexandra Jeszeck made key contributions to  this report. Kevin Walsh, Peter DelToro, and Shirley Hwang provided  technical assistance.