Iraq and Afghanistan: DOD, State, and USAID Cannot Fully Account for Contracts, Assistance Instruments, and Associated Personnel
Published: 2011-09-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DOD, State, and USAID have relied extensively on contracts and assistance instruments (grants and cooperative agreements) for a range of services in Iraq and Afghanistan. In the last 3 years, GAO has provided information on the agencies' contracts, assistance instruments, and associated personnel in the two countries, detailing the agencies' challenges tracking such information. Amendments from the National Defense Authorization Act for Fiscal Year 2011 now require the agencies to provide this and other information to Congress through annual joint reports. They also direct GAO to review those reports. In response, GAO reviewed the first joint report and assessed (1) data and data sources used to prepare the report; (2) use of data from the Synchronized Predeployment and Operational Tracker (SPOT) for management, oversight, and coordination; and (3) efforts to improve SPOT's tracking of statutorily required information. GAO compared data in the joint report to agency data GAO previously obtained, reviewed supporting documentation, and interviewed agency officials, including those in Iraq and Afghanistan, on how the data were collected and used.
section_title: What GAO Found
paragraphs: The Departments of Defense (DOD) and State and the U.S. Agency for International Development (USAID) designated SPOT as their system in 2010 for tracking statutorily required information on contracts, assistance instruments, and associated personnel in Iraq and Afghanistan. Citing limitations with SPOT's implementation, the agencies generally relied on data sources other than SPOT to prepare their 2011 joint report. Only State used SPOT but just for its contractor personnel numbers. However, GAO found that regardless of the data source used, the agencies' data had significant limitations, many of which were not fully disclosed. For example, while the agencies collectively reported $22.7 billion in fiscal year 2010 obligations, we found that they underreported the value of Iraq and Afghanistan contracts and assistance instruments by at least $4 billion, the majority of which was for DOD contracts. In addition, data presented in the joint report on personnel, including those performing security functions, are of limited reliability because of significant over- and undercounting. For example, DOD did not disclose that its contractor personnel numbers for Afghanistan were overreported for most of the reporting period because of double counting. Additionally, despite the reporting requirement, State did not provide information on its assistance instruments or the number of personnel working under them. As a result of such limitations, data presented in the joint report should not be used to draw conclusions or identify trends over time. DOD, State, and USAID have used SPOT to a limited extent, primarily to manage and oversee individual contracts and personnel. Agency officials cited instances of using SPOT to help identify contractors that should be billed for the use of government services, including medical treatment and dining facilities. State and DOD officials also identified instances of using SPOT to help inform operational planning, such as preparing for the drawdown of U.S. forces in Iraq. Officials from the three agencies indicated that shortcomings in data and reporting capabilities have limited their use of SPOT and, in some cases, led them to rely on other data systems to help manage and oversee contracts and assistance instruments. Further, the agencies cannot readily access each other's data in SPOT, which limits interagency coordination opportunities. Recent efforts have been made to improve SPOT's tracking of contractor and assistance personnel. SPOT now allows users to enter aggregate, rather than individual personal information into SPOT, which may overcome resistance to using the system based on security concerns. In addition, DOD and State report increased efforts to validate personnel data in SPOT. However, practical and technical challenges continue to affect SPOT's ability to track other statutorily required data. For example, SPOT cannot be used to reliably distinguish personnel performing security functions from other contractors. Also, while SPOT has the capability to record when personnel have been killed or wounded, such information has not been regularly updated. The agencies have identified the need for further modifications and new guidance to address some but not all of these limitations. It is unclear when SPOT will serve as a reliable source of data to meet statutory requirements and be used by the agencies for management, oversight, and coordination. As a result, the agencies still do not have reliable sources and methods to report on contracts, assistance instruments, and associated personnel in Iraq and Afghanistan. In 2009, GAO recommended that DOD, State, and USAID develop a plan for addressing SPOT's limitations. The agencies disagreed, citing ongoing coordination as sufficient. GAO continues to believe such a plan is needed and is not making new recommendations.

section_title: Letter
paragraphs: Over the last three years, Congress has required us to report on  Department of Defense (DOD), Department of State (State), and U.S.  Agency for International Development (USAID) contracts, grants, and  cooperative agreements with work performed in Iraq or Afghanistan. In  those reports, we also detailed challenges faced by the agencies in  tracking information on these contracts and assistance instruments, as  well as the personnel working under them in the two countries. Reliable,  meaningful data related to contractors, grantees, and cooperative  agreement recipients are a starting point for informing agency decisions  and ensuring proper management and oversight. Our prior work has  shown that the lack of complete and accurate information may inhibit  planning, increase costs, and introduce unnecessary risk. The importance  of such information is heightened given the three agencies’ extensive  reliance on contractors to provide a range of services, including security,  transportation, and base operations, relating to practically every facet of  U.S. efforts in Iraq and Afghanistan. Additionally, State and USAID have  relied on recipients of grants and cooperative agreements—two types of  assistance instruments—to implement infrastructure, governance, and  economic development projects in both countries.
Congress has taken a series of actions to increase the oversight and  availability of information on Iraq and Afghanistan contracts and  assistance instruments. Specifically, the agencies are now required to  produce their own report that provides much of the information that GAO  was previously required to report. Amendments from the Ike Skelton  National Defense Authorization Act for Fiscal Year 2011 (NDAA for  FY2011) require DOD, State, and USAID to submit annual joint reports to  Congress on their contracts and assistance instruments with work  performed in Iraq and Afghanistan. The reports are to address several  matters, such as the number and value of contracts and assistance  instruments, number of contractor and assistance personnel, number of  contractor personnel performing security functions, and any plans for  strengthening the collection and coordination of contract information. In  May 2011, the three agencies transmitted to congressional committees  the first Annual Joint Report on Contracting in Iraq and Afghanistan  (hereafter, referred to as the joint report), which covers fiscal year 2010.
The joint reporting requirement builds upon earlier requirements for the  three agencies to track information on contracts, assistance instruments,  and associated personnel. Specifically, the National Defense  Authorization Act for Fiscal Year 2008 (NDAA for FY2008) directed DOD,  State, and USAID to sign a memorandum of understanding (MOU)  regarding contracting in Iraq and Afghanistan. The law specified several  matters to be covered in the MOU, including identifying common  databases to serve as repositories of information on contracts and  contractor personnel in the two countries. In a July 2008 MOU, the  agencies designated the Synchronized Predeployment and Operational  Tracker (SPOT) database as their common database for the statutorily  required contract and contractor personnel information. With the passage  of the National Defense Authorization Act for Fiscal Year 2010 (NDAA for  FY2010), Congress expanded the requirement to cover grants,  cooperative agreements, and associated personnel. In response, the  agencies revised the MOU in April 2010 to specify that SPOT was also  their system of record to track statutorily required assistance instrument  and personnel information.
Amendments from the NDAA for FY2011 also direct us to report annually  on the agencies’ joint reports. Pursuant to that mandate, we have  reviewed the agencies’ 2011 joint report and are providing our  assessments of (1) the data and data sources used by the agencies to  develop their report; (2) the agencies’ use of data from SPOT to manage,  oversee, and coordinate their Iraq and Afghanistan contracts, assistance  instruments, and associated personnel; and (3) progress and plans for  improving SPOT to track statutorily required information.
We used the following methodologies to conduct our review.     To assess the data and data sources used, we obtained the  underlying data the agencies used to prepare the joint report and  performed several analyses to identify limitations, omissions,  duplicates, and other errors that would affect the reliability of the data.  This included comparing data in the joint report with data we collected  from the agencies for the first half of fiscal year 2010 to prepare our  2010 report, as well as a comparison to Federal Procurement Data  System – Next Generation (FPDS-NG) data from the second half of  fiscal year 2010. We also interviewed agency officials to discuss steps  they took to validate and verify the data presented in the report and  reviewed supporting documentation that the agencies provided at our  request. For our 2010 report, we obtained multiple data sets from  DOD, State, and USAID, including data from FPDS-NG, agency- specific databases, and manually compiled lists of contracts and  assistance actions. We determined those data were sufficiently  reliable to identify the minimum number of active or awarded contracts  and assistance instruments, associated obligation amounts, and  extent of competition for the first half of fiscal year 2010. For that  report, we also obtained data on contractor and assistance personnel  in the two countries; these data were generally obtained by the  agencies through surveys and periodic reports submitted by  contractors and assistance recipients.    We reviewed the joint report to determine the current status of and  future plans for the agencies’ use of SPOT data. We conducted  interviews with DOD, State, and USAID officials, including those in  Iraq and Afghanistan, responsible for maintaining SPOT and other  data sources and for managing, overseeing, and coordinating  contracts to identify examples of how the agencies have used the  data.    We obtained relevant documentation, including agency policies and  guidance, and interviewed agency officials to assess the progress and  plans for improving SPOT, particularly as they pertain to shortcomings  identified in our prior reports.
We conducted this performance audit from May 2011 through September  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The first joint report on contracting in Iraq and Afghanistan required under  amendments from the NDAA for FY2011 was to be issued by February 1,  2011, with subsequent reports due in 2012 and 2013. In the reports,  DOD, State, and USAID are to provide the following for each 12-month  reporting period:  total number and value of contracts and assistance instruments  awarded,  total number and value of active contracts and assistance  instruments,  the extent to which such contracts and assistance instruments used  competitive procedures,  total number of contractor and assistance personnel at the end of  each quarter of the reporting period,  total number of contractor and assistance personnel performing  security functions at the end of each quarter of the reporting period,  and  total number of contractor and assistance personnel killed or  wounded.
The joint reports are also to include the sources of information and data  used to compile the required information; a description of any known  limitations of the data reported, including known limitations of the  methodology and data sources used; and plans for strengthening  collection, coordination, and sharing of information on contracts and  assistance instruments in Iraq and Afghanistan through improvements to  common databases. The first joint report submitted by the agencies in  May 2011 provides an overview of the reporting requirements, an  introduction, and a section for each agency to present its data. Each  agency was responsible for collecting its fiscal year 2010 data from  relevant sources and compiling its section of the report.
The reporting requirements in the NDAA for FY2011 build upon prior  national defense authorization act requirements. Specifically, Section 861  of the NDAA for FY2008 directed the Secretaries of Defense and State  and the USAID Administrator to sign an MOU related to contracting in  Iraq and Afghanistan. The law, as amended by the NDAA for FY2010,  specified a number of issues to be covered in the MOU. These include  specifying each agency’s roles and responsibilities in matters related to  contracting in the two countries, determining responsibility for establishing  procedures for and coordination of movement of contractor personnel in  the two countries, and identifying common databases to serve as  information repositories on contracts and assistance instruments with  more than 30 days of performance in Iraq or Afghanistan and the  personnel working in either country under those contracts and assistance  instruments. The common databases are to include a brief description of  each contract and assistance instrument, its total value, and whether it  was awarded competitively; for personnel working under contracts or  assistance instruments, the databases will include the total number  employed, total number performing security functions, and total number  killed or wounded. Tracking this information should provide much of the  information the agencies are to include in the joint reports.
In July 2008, DOD, State, and USAID agreed in an MOU that SPOT  would serve as their common database and be the system of record for  the statutorily required contract and personnel information. The agencies  revised their MOU in April 2010, making SPOT their system for also  tracking assistance instruments and associated personnel. SPOT is a  web-based system initially developed by the U.S. Army to track detailed  information on a limited number of contractor personnel deployed with  U.S. forces. The 2010 MOU specified that SPOT would include  information on DOD, State, and USAID contracts and assistance  instruments with more than 30 days of performance in Iraq or Afghanistan  or valued at more than $100,000, as well as information on the personnel  working under those contracts and assistance instruments. SPOT is  configured so that it can track individuals by name and unique identifier,  such as Social Security number, and record information, including the  contracts they are working under, deployment dates, and next of kin. The  agencies agreed that contract-related information, such as value and  extent of competition, are to be imported into SPOT from FPDS-NG, the  federal government’s system for tracking information on contracting  actions. According to the MOU, DOD is responsible for all basic  maintenance, upgrades, training, and systems operations costs, but the  agencies agreed to negotiate funding arrangements for any agency- unique requirements. Within DOD, a program management office has  responsibility for the development, integration, testing, training, and  deployment of SPOT and as such, oversees the contractor that operates,  maintains, and sustains the system.
DOD, State, and USAID have phased in SPOT’s implementation, with  each developing its own policies and procedures governing the system’s  use.     DOD designated SPOT in January 2007 as its primary system for  collecting data on contractor personnel deployed with U.S. forces. At  that time, it implemented a contract clause directing firms to enter  data into SPOT on U.S., third country, and local nationals working  under its contracts in Iraq or Afghanistan that meet reporting  thresholds.    State issued a policy in March 2008 that included language to be  incorporated in applicable contracts requiring contractors to enter data  into SPOT on U.S., third country, and local nationals working in either  Iraq or Afghanistan. State expanded this requirement in January  2009 to cover personnel working under certain assistance instruments  in the two countries. As amended, State’s assistance policy directed  that U.S. and third country nationals working under grants must be  entered into SPOT but allowed for discretion in determining whether  local nationals were entered given safety and security concerns. In  January 2011, State revised its assistance guidance and related  provision to allow grantees with locally hired Iraqi or Afghan personnel  to report aggregate numbers of local nationals without providing  personally identifying information when safety concerns exist.    USAID issued a directive in April 2009 that required the use of  contract clauses and assistance provisions requiring contractors and  assistance recipients in Iraq to enter personnel data into SPOT. The  directive explicitly excluded Iraqi entities and nationals from having to  be entered into SPOT until a classified system is in place. In July  2010, USAID issued a directive establishing a similar requirement for  Afghanistan. However, the policy notes that procedures will be  provided separately for entering information on Afghan nationals, but  to date, such procedures have not been issued.
section_title: Agencies Relied on Sources Other Than SPOT, but Data Used Had Significant Limitations
subsections: section_title: Variety of Data Sources Used to Prepare Joint Report
paragraphs: DOD, State, and USAID’s joint report cited a number of limitations  associated with SPOT’s implementation, and as a result, the agencies  relied on a variety of other data sources to develop the report. The only  exception was State’s use of SPOT as the basis for its contractor  personnel numbers. Whereas GAO previously collected and compiled  data from numerous sources including manually compiled lists of  contracts and assistance instruments and personnel data obtained  through surveys, officials from the three agencies told us they decided to  rely on existing databases and sources to the greatest extent possible.  Table 1 summarizes the data sources used to prepare the joint report and  the reasons cited by the agencies for not using SPOT.
The data presented in the agencies’ joint report had significant limitations,  many of which were not fully disclosed. As a result, the data should not  be used to draw conclusions about contracts, assistance instruments, and  associated personnel in Iraq or Afghanistan for fiscal year 2010 or to  identify trends over time.
While the agencies collectively reported $22.7 billion in fiscal year 2010  obligations, the joint report understates the three agencies’ obligations on  contracts and assistance instruments with work performed in Iraq and  Afghanistan by at least $4 billion, nearly all for DOD contracts. We  identified this minimum amount by comparing the underlying data the  agencies used to prepare the joint report with data we obtained from the  agencies during our prior review of contracts and assistance instruments  with work in either country during the first half of fiscal year 2010. The  level of underreporting we identified does not fully account for new  awards or obligations that the agencies made in the second half of fiscal  year 2010.
DOD and State underreported their contracts and obligations in the joint  report because they relied solely on FPDS-NG to identify contracts with  work performed in Iraq or Afghanistan. FPDS-NG allows agencies to only  report one principal place of contract performance. However, contracts  can have performance in multiple countries, and the reporting  requirement applies to contracts with performance in Iraq or Afghanistan,  even if neither country is the principal place of performance. Further, not  all DOD contracts with performance in Iraq and Afghanistan were entered  into FPDS-NG. Neither DOD nor State disclosed any limitations with their  FPDS-NG queries or that there could be additional contracts with  associated obligations with work in the two countries.
Using FPDS-NG to identify contracts with a principal place of  performance in Iraq and Afghanistan, DOD reported $18.4 billion in fiscal  year 2010 obligations but underreported its contract obligations by at  least $3.9 billion. Specifically, we identified an additional 20,810 contracts  and orders that totaled to about $3.5 billion in fiscal year 2010 obligations  that DOD had reported to us last year but were not included in the joint  report because the principal place of performance was not Iraq or  Afghanistan. For example, DOD previously reported to us two contracts  for translation and interpretation services with performance in Iraq and/or  Afghanistan with $1.5 billion in fiscal year 2010 obligations, but these  contracts were not included in the joint report because FPDS-NG  identified the principal place of performance as the United States. We  also identified additional contracts that were previously reported to us but  not included in the joint report because they were not in FPDS-NG.  Among those, we identified 13 contracts with $418 million in obligations  during the first half of fiscal year 2010, including combat support contracts  for information technology services and linguist support in the two  countries. DOD did not report any assistance instruments with  performance in Iraq or Afghanistan. This is consistent with our 2010  report for which we found DOD had no assistance instruments with  performance in either country during fiscal year 2009 or the first half of  fiscal year 2010.
For the joint report, State relied on FPDS-NG and reported $1.8 billion in  contract obligations in Iraq and Afghanistan for fiscal year 2010. We  found, however, that State underreported its fiscal year 2010 contract  obligations by at least $62 million by not including 49 contracts and orders  that were reported to us last year. Specifically, we identified a State  delivery order for facility management with about $54.3 million in  obligations in fiscal year 2010 that was not in the joint report because the  United States was identified as the principal place of performance in  FPDS-NG, as opposed to either Iraq or Afghanistan. We also identified  another 48 contracts and orders that State reported to us last year as  having performance in either country that were not identified through  State’s FPDS-NG query. These include 23 contracts and orders awarded  by the embassies in Iraq and Afghanistan with about $1 million in  obligations in the first half of fiscal year 2010, even though the joint report  states that it includes all procurement activities contracted for by State’s  missions in the two countries.
While the reporting requirement applies to both contracts and assistance  instruments, State did not report any assistance instruments with  performance in Iraq or Afghanistan or provide any explanation in the joint  report as to why such information was not included. Based on data  provided by State last year, we identified 155 assistance instruments with  work performed in Iraq and/or Afghanistan with $120 million obligated  during the first half of fiscal year 2010. These assistance instruments  covered a wide range of activities, such as media workshops, small  business development, and capacity building for nongovernmental  organizations. State officials informed us that they did not include  information on assistance instruments as they were not including  information on personnel working under assistance instruments because  of limitations, as discussed below. They told us, however, that they plan  to include assistance instrument information in next year’s joint report.
Unlike DOD and State, USAID did not rely on FPDS-NG as its data  source for the number and value of contracts. As explained in the joint  report, USAID knew gaps existed in its FPDS-NG data, particularly for  Afghanistan, so it used data from its financial management system, which  contains information on the number and value of both contracts and  assistance instruments. USAID reported $2.6 billion in contract and  assistance instrument obligations in Iraq and Afghanistan for fiscal year  2010. However, by comparing the data from the financial management  system to data USAID provided us last year, we found that the agency  underreported its obligations by about $3.9 million. These obligations  were for 16 contracts and 8 assistance instruments in the first half of  fiscal year 2010 that were not included in the joint report. Almost all of the  contracts that were not reported were personal services contracts.  USAID officials told us they did not report personal services contracts  because they consider such contractor personnel to be USAID  employees, but this was not disclosed in the joint report. Further, unlike  DOD and State, which provided competition information for nearly all  contracts included in the joint report, USAID provided competition data on  fewer than half the active contracts and assistance instruments included  in the joint report. Other than acknowledging FPDS-NG data gaps, USAID  provided no specific explanation for why the competition data presented  in the report are incomplete.
We identified a number of limitations and methodological challenges that  resulted in both over- and underreporting of contractor and assistance  personnel and call into question the overall reliability of the data in the  joint report. However, we were not able to determine the full magnitude of  the discrepancies.
For the joint report, DOD relied on quarterly censuses as its source of  data on contractor personnel, including personnel performing security  functions. DOD provided the numbers of contractor personnel, broken out  by nationality, in Iraq and Afghanistan at the end of each quarter.  However, the numbers for local nationals working under contracts in  Afghanistan were generally overreported. According to the U.S. Central  Command (CENTCOM) official who oversees the compilation of the  census, a methodological error resulted in double counting of local  nationals in Afghanistan for the first three fiscal year 2010 quarters. The  error was discovered as the fourth quarter census was being compiled,  which resulted in a significant reduction in the number of local national  contractor personnel in Afghanistan for that quarter. To illustrate the  magnitude of the double counting, DOD reported 73,392 local national  contractor personnel in Afghanistan for the third quarter of fiscal year  2010 and only 34,222 in the fourth quarter—a difference of 39,170  personnel. No adjustments were made to the prior three quarters to  correct for the double counting. Furthermore, this error and an  explanation as to what occurred are not provided in the joint report,  except to note that there are challenges associated with counting local  national personnel in Afghanistan. Officials from the Office of the Deputy  Assistant Secretary of Defense for Program Support and CENTCOM told  us they have a high level of confidence in the census numbers for all  contractor personnel except local nationals in Afghanistan. However, as  we noted in October 2010, DOD officials overseeing the census  characterized the census as providing rough approximations of the actual  numbers of contractor personnel in either country. They explained that  several challenges pertaining to counting local nationals and validating  contractor-reported data have hindered their ability to collect accurate and  reliable personnel data.
State relied on SPOT as its source for data on contractor personnel,  which led to several omissions and discrepancies. Based on our analysis  of State’s reported personnel data and the contract data reported from  FPDS-NG, we identified 50 contracts that met SPOT reporting  requirements but were not in the system. Therefore, personnel working  on those contracts in Iraq and Afghanistan were not included in the joint  report. For example, we identified 5 contracts for construction with about  $525 million in fiscal year 2010 obligations with no contractor personnel  reported in SPOT. Further, at the end of the second quarter of fiscal year  2010, there were 1,336 fewer contractor personnel in SPOT than were  reported to us last year from State’s surveys of contractor personnel in  the two countries. Such omissions are consistent with what State officials  told us in 2010—that manually compiled surveys of contractor personnel  in either country have some limitations but provide more accurate  information than SPOT. Additionally, while the joint report presents the  numbers as “contractor personnel,” and we confirmed with State officials  that the numbers were only to include contractor personnel, we found that  about 13 percent of the personnel State reported as contractor personnel  were actually working under assistance instruments.
In addition, State did not include in the joint report the number of  personnel working under assistance instruments in Iraq and Afghanistan  or explain why assistance personnel were not included. State officials  informed us that although State’s policy required assistance personnel to  be entered into SPOT since January 2009, assistance recipients had  been reluctant to enter information into the system. As a result, for fiscal  year 2010, officials told us that little information regarding personnel  working under assistance instruments had been entered into the system.  However, State could have relied on other data sources to provide the  required personnel information. Last year, based on surveys State  conducted of its assistance recipients, we reported that there were at  least 8,074 personnel working under State’s assistance instruments in  Iraq and Afghanistan at the end of the second quarter of fiscal year 2010.  We cautioned that the number was likely understated because of several  factors. State officials informed us that response rates to their requests  for personnel numbers from assistance instrument recipients were low;  they also stated that local nationals were not always captured in  personnel counts because it was not feasible or it was too difficult to  obtain accurate information.
In reporting the number of personnel performing security functions, State  relied exclusively on SPOT and did not disclose any limitations with that  source. As we reported last year, SPOT cannot be used to reliably  distinguish personnel performing security functions from other contractor  personnel, as each of the three available methods has limitations. State  officials responsible for compiling the joint report told us they queried  SPOT based on security-related job titles. Upon review of the data,  officials from the Bureau of Diplomatic Security noticed that the numbers  appeared low. An analyst from the Bureau of Diplomatic Security  identified five large security contracts with numerous personnel who did  not have the word “security” in their job titles and as a result were not  included in the query results, a risk we noted in our prior report. The  SPOT query indicated that there were 3,924 State contractor personnel  performing security functions in Iraq and Afghanistan at the end of the  fourth quarter of fiscal year 2010. State revised this number and reported  8,034 personnel performing security functions for that quarter. Despite the  fact that the SPOT data were incomplete and had to be manually  adjusted, the joint report provides no explanation and does not identify  limitations with the SPOT data for determining the number of personnel  providing security functions.
In presenting personnel numbers in the joint report, USAID was the only  agency that used estimates as opposed to actual counts for the total  number of contractor and assistance personnel, as allowed by the  reporting requirement. USAID also used estimates for the number of  personnel performing security functions, which is not provided for in the  reporting requirement. Specifically, USAID estimated the number of  personnel for Afghanistan. However, the full extent to which estimates  were used is not disclosed in the joint report. Further, the estimates are  based on unreliable data. USAID officials explained to us that the  estimates were based on data from several sources including databases  used to track aid effectiveness metrics, quarterly reports submitted by its  contractors and grantees, and data submitted to us for last year’s report.  All of these sources have limitations. For example,     while contractors and assistance recipients in Iraq report their  personnel numbers on a regular basis, a USAID official informed us  that only about 70 percent of their contractors and assistance  recipients in Afghanistan provide personnel information;    a USAID official told us they have a limited ability to verify the  accuracy or completeness of the data that are reported, especially for  Afghanistan where they operate far more projects than in Iraq;   the USAID official responsible for preparing the joint report raised  concerns about possible inconsistent reporting of security personnel  that could result in double counting; and  the data provided to us by USAID for our 2010 report did not include  personnel working under several contracts and assistance  instruments, such as four cooperative agreements for food security  programs in Afghanistan.
USAID officials also told us that the numbers in the joint report do not  include the number of personnel working under certain support service  contracts, such as facilities maintenance, or personal services  contractors. For example, a USAID official told us that at least  109 contractor personnel supporting the Iraq mission were not counted in  the joint report because a decision was made not to include support  services and personal services contractors.
Although all three agencies are required to track the number of personnel  killed or wounded while working on contracts and assistance instruments  in Iraq or Afghanistan, DOD still does not have a system that reliably  tracks killed and wounded contractor personnel. For the joint report, DOD  relied on data maintained by the Department of Labor (Labor) regarding  Defense Base Act (DBA) claims. While DOD acknowledged in the joint  report that claims data from this workers’ compensation program do not  provide a true reflection of how many DOD contractor personnel were  killed or wounded while working in either country, DOD did not fully  disclose the limitations associated with DBA claims data. First, the claims  data presented in the joint report are for death and injury claims filed in  fiscal year 2010 for all U.S. government contractors and civilians— including those employed by State and USAID—and not just DOD  contractors. Further, as we concluded in 2009, DBA claims data do not  provide an appropriate basis for determining the number of contractor  personnel killed or wounded in either country. Most notably, not all deaths  and injuries for which claims are filed under DBA would be regarded as  contractors killed or wounded within the context of the NDAA for FY2011  reporting requirement. For example, we previously identified DBA claims  filed for occupational injuries and medical conditions such as sprains and  appendicitis. Also, Labor officials previously explained to us that injuries  to local and third country contractor personnel, in particular, may be  underreported.
To provide their data on personnel killed and wounded, State and USAID  relied on data collected by State bureaus and USAID missions in Iraq and  Afghanistan. These data were based on reports submitted to State by  contractors and to USAID by contractors and assistance recipients.  Without alternative sources of data, we could not verify whether State’s  and USAID’s data were complete, except to note that State did not  include assistance personnel who were killed or wounded. However,  there are indications of underreporting by contractors and assistance  recipients. For example, a May 2010 report from the USAID Inspector  General indicated that not all contractors and assistance recipients in  Afghanistan were reporting incidents that result in personnel being injured  or killed. In addition, a USAID official in Afghanistan acknowledged that  for fiscal year 2010, it was voluntary for contractors and assistance  recipients to file serious incident reports, which would provide information  on personnel killed or wounded. Earlier this year, USAID began modifying  contracts in Afghanistan to require its contractors to file serious incident  reports.
section_title: Use of SPOT Hindered by Data and Reporting Shortcomings
subsections: section_title: Agencies Have Used SPOT to a Limited Extent
paragraphs: Officials from the three agencies told us they have used SPOT in some  instances to obtain information on individual contracts and contractor  employees. For example, an official from State’s Bureau of Diplomatic  Security said they have used SPOT during investigations to verify  whether the individuals involved were deployed in theater at the time of  the incidents being investigated. A USAID contracting officer in Iraq told  us that when a security incident involving a contractor employee occurs,  she uses SPOT to determine if the individual involved has a letter of  authorization, which should provide personal information including  whether the individual is authorized to carry a weapon. A senior official  with DOD’s CENTCOM Contracting Command in Iraq explained that he  used SPOT to obtain information on specific contracts, such as the name  of the contracting officer or contracting officer’s representative, in  response to questions about contracts that were not awarded or managed  by his office. State and DOD officials have also reported using SPOT to  better manage contractor personnel. For example, DOD officials from the  SPOT program management office told us that SPOT has been used in  conjunction with information from other systems to identify contractors  that should be billed for the use of government services, including  medical treatment and dining facilities. Additionally, State Diplomatic  Security officials told us they have used SPOT to confirm that contractor  personnel are authorized to be in Iraq and determine to what government  services those personnel are entitled.
DOD and State officials also identified instances of using SPOT data to  inform operational planning for contractor support. Officials from the  SPOT program management office told us they have received requests  from U.S. Forces-Iraq commanders to identify the universe of contractors  and contractor capabilities in Iraq to assist with the drawdown of U.S.  forces. They also stated that base commanders in Iraq are receiving  contractor population reports to obtain insight into which contractors are  on their bases. Additionally, officials in the Office of the Deputy Assistant  Secretary of Defense for Program Support told us that data from SPOT  are being used to help prepare future operational plans. For example,  SPOT data have been analyzed to help determine what services  contractors have provided and what level of life support the U.S.  government has provided to them, which can aid combatant commanders  in developing operational plans. State officials also told us that the U.S.  Embassy in Iraq has requested SPOT data to help it determine the  number of contractors in country and to assist with planning for the future  U.S. presence in Iraq once the U.S. military withdraws at the end of this  year. However, USAID officials including those we spoke with in Iraq and  Afghanistan told us that they do not use SPOT data to manage, oversee,  or coordinate contracts aside from obtaining information on specific  contractor employees.
section_title: Data and Reporting Limitations Affect Opportunities for Using SPOT to Manage, Oversee, and Coordinate
paragraphs: DOD, State, and USAID officials informed us that shortcomings in SPOT  data and reporting capabilities limit their ability to use the system in  managing, overseeing, and coordinating contracts with work performed in  Iraq and Afghanistan. In some cases, officials have relied on other data  sources for such purposes. For example, DOD officials with the  Contracting Fusion Cell in Iraq told us that because SPOT is designed to  track contractor personnel on an individual basis rather than to support  the operational management of contractors, they developed a new,  separate database containing aggregate-level data on contractor  personnel at each base to help manage the drawdown of personnel and  equipment from the country. While the new database includes  information not available from SPOT, such as information on contractor  equipment, some of the basic contract information overlaps with SPOT  and was added to the database from sources other than SPOT. Similarly,  officials from State’s Bureau of Diplomatic Security told us that SPOT  does not provide the level of detail needed to manage their security  contractor employees and that they rely on their own data system for the  day-to-day management of their contractors.
Officials from all three agencies also raised concerns about the reports  that can be generated from SPOT. USAID officials in Iraq explained that  one reason they do not rely on SPOT to help manage contractors and  assistance recipients is that the types of reports they need are not easily  available from the system. State officials also indicated that the standard  reports available through SPOT do not meet their needs and they have to  request ad hoc reports from the SPOT program management office’s help  desk. CENTCOM Contracting Command officials in Iraq also told us that  for a large data run they cannot obtain data from SPOT in a timely  manner, with it taking up to a week to receive the data. SPOT program  management officials acknowledged that agency personnel are not fully  aware of SPOT’s reporting capabilities and may not have confidence in  the system given its data reliability challenges. As a result, the program  management officials are seeking to expand their outreach to potential  users of the data, focusing on improving customer service, and exploring  the development of training on how SPOT data could be used for  management and operations, as opposed to the current training that has  been focused on entering data into the system. Also, the SPOT program  management office told us that they have taken steps to facilitate agency  officials’ ability to query SPOT for contracts awarded by their agencies, a  process they described as cumbersome, to allow for better coordination  and leveraging of existing contracts within an agency. Staff from the  Office of the Senior Contracting Official in Afghanistan told us that they  recently began using this query functionality and they expect it to better  enable their use of SPOT in responding to future data requests.
The agencies’ ability to use SPOT for interagency coordination purposes  has been limited by the fact that they cannot easily access each other’s  data. SPOT program management officials told us that SPOT could be  used by the agencies to identify and leverage contracts being performed  for common services so that agencies could minimize duplication, share  price information, and obtain cost savings. However, agency officials are  currently not able to access information on other agencies’ contracts  unless DOD grants them permission to have full access to the information  in SPOT. SPOT program management officials informed us that they are  developing a separate reporting and analysis functionality to allow users  to more easily share, analyze, and use data available in SPOT. However,  this functionality is currently being tested and there are no time frames for  when it will be available to all users. While USAID officials agreed that  coordination among the agencies is important, they did not share the  perspective that the agencies needed access to each others’ information  in SPOT. They explained that this is partly due to the fact that interagency  coordination before the award of a contract or assistance instrument is  occurring without using SPOT.
section_title: Recent Efforts Have Been Made to Improve SPOT, but Past Problems Have Not Been Fully Addressed
subsections: section_title: Recent Efforts to Improve Tracking of Personnel in SPOT
paragraphs: We previously reported that a significant challenge associated with  SPOT’s implementation was ensuring that Iraqi and Afghan nationals  working under contracts and assistance instruments were consistently  and accurately entered in SPOT. Last year we reported that local  nationals were not always entered into the system because of agency  policies as well as practical and technical limitations. For example, many  local nationals work at remote locations, which limits agencies’ ability to  track these personnel and verify the completeness of reported  information. Also, DOD, State, and USAID officials have told us that some  local national contractors refuse to submit information on their personnel  because of safety concerns. Additionally, some information required for  SPOT data fields, such as first and last names and dates of birth, may not  be known due, in part, to cultural norms specific to each country.
The agencies have taken some steps to improve the reliability of the  personnel data in SPOT. DOD and State officials informed us that they  have increased efforts to validate SPOT data. In DOD’s case, this is  done, in part, through the SPOT-Plus process, which began in January  2010. This process is used to reconcile contractor personnel numbers in  SPOT with the quarterly contractor census and identify information that  needs to be updated or entered into SPOT. DOD officials informed us that  they will continue comparing SPOT and census data until there is  confidence that 85 percent of the personnel reported through the census  are reported in SPOT, at which point the plan is to discontinue the census  and fully rely on SPOT. According to DOD officials, their analyses indicate  that for some categories of contractor personnel they may have achieved  the 85 percent confidence level, but that for other categories—particularly  local nationals in Afghanistan—they are still below that level. The officials  could not provide an estimate as to when they will discontinue the  census. However, they noted that once the 85 percent confidence level is  achieved, DOD plans to conduct random samplings to ensure it is  maintained. Similarly, State officials informed us that program and  contracting officials have begun reviewing SPOT data on a quarterly or  even monthly basis in an effort to improve SPOT data entry. Given this  emphasis, State officials told us that they are increasingly confident in the  reliability of personnel data in SPOT. However, a USAID official  responsible for preparing the joint report told us that the agency does not  validate SPOT data and does not intend to do so, noting it has  experienced high staff turnover in Iraq and Afghanistan and has other  reporting priorities.
In April 2011, SPOT was modified to address concerns cited by State and  USAID officials, as well as by contractors and assistance recipients, that  the safety of local nationals could be at risk should SPOT, with its detailed  personal information, be compromised. The system now allows users to  enter the aggregate number of personnel working under a contract or  assistance instrument, rather than requiring personnel to be entered  individually with personally identifiable information. This provides a means  of counting local nationals working under contracts and assistance  instruments who previously were not entered into the system. USAID  officials said that while guidance on the use of the aggregate count  function has not yet been issued, they have begun entering aggregate  data on local nationals in Afghanistan into SPOT. In January 2011, State  revised its assistance policy to allow grantees with locally hired Iraqi or  Afghan personnel to report their aggregate numbers of local nationals into  SPOT. State officials told us the modification appears to have satisfied  assistance recipients’ concerns, as they are now providing State officials  with aggregate numbers for inclusion in SPOT. DOD officials informed us  that they will not be issuing guidance regarding the aggregate count  function, as DOD’s policy continues to require personnel working under  contracts that meet reporting thresholds to be individually entered into  SPOT.
Additional measures have been undertaken to help address the challenge  of tracking local nationals in SPOT. For example, the SPOT program  management office developed procedures for establishing unique  identification numbers for local nationals who are entered into the system  by name but whose personal identifying information does not conform to  the required SPOT data fields. Similarly, DOD officials told us they have  developed work-arounds for Iraqi and Afghan firms that lack reliable  Internet connections to submit their personnel information via templates,  which are then uploaded by DOD personnel into SPOT. In an effort to  improve the collection of data on personnel working at remote locations,  DOD officials informed us that the department is also piloting a handheld  device that does not require an Internet connection and can be used to  collect information on personnel that is then uploaded into SPOT.
section_title: Practical and Technical Challenges Continue to Affect SPOT’s Ability to Track Statutorily Required Data
paragraphs: In 2009, we recommended that the three agencies develop a joint plan  with associated time frames to address SPOT’s limitations, but agencies  responded that a plan was not needed as their ongoing coordination  efforts were sufficient. However, we concluded last year and our work  continues to demonstrate that coordination alone is not sufficient to  ensure that statutory requirements are met. Specifically, SPOT still  cannot be used to reliably track statutorily required contract, assistance  instrument, and personnel data as agreed to in the agencies’ MOU  because of a number of longstanding practical and technical limitations.  SPOT program management officials and the agencies have identified  plans for further modifications and new guidance needed to address  some but not all of these limitations.
SPOT still is not linked with FPDS-NG or other agency systems for  obtaining information on contracts and assistance instruments.  Consequently, SPOT cannot be used to obtain financial and competition  information on contracts and assistance instruments as agreed to in the  MOUs. According to the joint report, the link to FPDS-NG to obtain  contract information is scheduled to occur in early fiscal year 2012—this  functionality was previously planned to be available in 2010. As we  reported in 2009, one reason for this delay is that contract numbers,  which are the unique identifiers that would be used to match records in  SPOT to those in FPDS-NG, are entered into SPOT using different  formats. To help resolve this, the SPOT program management office  modified SPOT earlier this year to require DOD users to enter contract  numbers in a standardized manner that can be matched with FPDS-NG  information. SPOT program management officials told us that a similar  modification has not been made for State or USAID contracts. Once the  link is made between SPOT and FPDS-NG, information from the two  systems can only be merged if the contract number has been entered into  SPOT. If the contract is not in SPOT, because, for example, no contractor  personnel working on that particular contract have been entered, its  information cannot be linked with the information in FPDS-NG.  Conversely, current information on the contract has to be in FPDS-NG,  which does not always occur as we found in our analyses of the  information presented in the joint report. Most notably, officials told us that  information on USAID contracts awarded in Afghanistan must still be  manually entered into FPDS-NG, which has resulted in known information  gaps. USAID is planning to deploy a new system to Afghanistan—already  in place in Iraq and other countries—that will automatically upload  contract information into FPDS-NG by the end of 2011. Once the link  between SPOT and FPDS-NG is established and the necessary data are  in both systems, then SPOT could be relied on to provide more complete  information on contracts with performance in either country, as opposed  to relying only on the FPDS-NG principal place of performance. SPOT  program management officials informed us that there are currently no  plans to establish links with the State or USAID systems that contain  assistance instrument information. Officials stated that, therefore,  information on those instruments needs to be manually entered into  SPOT.
SPOT does not provide a reliable means of obtaining information on  orders and subawards. The statutory requirement to track information  on contracts and assistance instruments includes a requirement to track  comparable information on task and delivery orders as well as  subcontracts and subgrants. However, SPOT does not have a specific  data field for this information. Instead, contractors and assistance  recipients are instructed by the agencies to enter information on their  subawards into a data field designed to track information on task orders.  As a result, it has not been possible to obtain accurate counts of orders  and subawards using SPOT. SPOT program management officials told us  that they expect to address this issue by creating a new subaward data  field in a September 2011 SPOT upgrade.
SPOT does not reliably distinguish personnel performing security  functions. As discussed in our 2010 report, there are three methods to  distinguish personnel performing security functions from others in SPOT.  Each method has limitations and yields different results, none of which  are fully consistent with the statutory definition of contractor personnel  performing security functions. SPOT program officials acknowledge this  limitation but informed us that they have not yet developed a corrective  action to ensure that security personnel are consistently and reliably  distinguished for statutory tracking and reporting purposes.
SPOT is not being used to track the number of personnel killed and  wounded. As we reported last year and as noted in the joint report,  contractors and assistance recipients generally have not been recording  information on killed or wounded personnel in SPOT. According to the  joint report, the SPOT program management office is working with users  to explore ways of improving compliance by clarifying the terminology and  expanding data fields. For example, there have been questions about  whether deaths or injuries resulting from car accidents should be  recorded in SPOT or if SPOT should only be used to track those killed or  wounded while performing their contractual duties. SPOT program  officials informed us that there has been some discussion of expanding  the data fields in SPOT to include information like the date of injury or  death and details surrounding the incident. However, officials told us  these actions are still being discussed internally and no plans are in place  to include such changes in upcoming versions of SPOT. Instead, DOD  and State officials said they are helping contractors and assistance  instrument recipients gain a better understanding of the requirement to  report killed or wounded personnel using SPOT. Additionally, State  officials told us that they have begun entering information into SPOT on  killed and wounded personnel based on information provided by  contractors and assistance recipients and anticipate using the data in  SPOT to prepare future joint reports.
section_title: Concluding Observations
paragraphs: In 2008, DOD, State, and USAID designated SPOT as their system of  record for tracking statutorily required information on contracts and  contractor personnel in Iraq and Afghanistan, a designation they  reaffirmed in 2010 when the requirement was expanded to include  assistance instruments and personnel. Yet the agencies still do not have  reliable sources and methods to report on contracts, assistance  instruments, and associated personnel in Iraq and Afghanistan. This is  evidenced by the fact that the agencies could not reliably use data from  SPOT to prepare their first joint report and instead relied on other data  sources and methods that had significant limitations. Over the years, we  have reported on the limitations associated with SPOT’s implementation  and the agencies’ resulting decisions to rely on other methods of  collecting and reporting data that have their own shortcomings. We  recommended in 2009 that the agencies develop a joint plan with  associated time frames to address limitations and ensure SPOT’s  implementation to fulfill statutory requirements. The agencies disagreed  with the need for the plan, citing ongoing coordination efforts as sufficient.
While the agencies’ recent modifications to SPOT help address some  limitations, such as those related to tracking local nationals, other  limitations persist that undermine SPOT’s ability to fulfill statutory  reporting requirements. Further, while agency officials have recognized  some benefits of using SPOT to help manage, oversee, and coordinate  contracts, assistance instruments, and associated personnel, their ability  to do so has been hindered by SPOT’s shortcomings. Our prior  recommendation for a joint plan was intended to provide an opportunity  for the agencies to work with potential users of the data to better  understand their information needs and determine how best to proceed  with defined roles, responsibilities, and associated time frames that could  help hold the agencies accountable and ensure timely implementation.  We were concerned that without such a plan, SPOT’s implementation  would continue to languish with the agencies not collecting statutorily  required information in a reliable manner, either using SPOT or other  sources. Based on our review of the agencies’ joint report, we continue to  have this concern and are uncertain when SPOT will be fully implemented  and serve as a reliable source of data for management, oversight, and  coordination. We have, therefore, concluded that the recommendation  from our 2009 report still applies, and we are not making any new  recommendations.
section_title: Agency Comments
paragraphs: We requested comments on a draft of this report from DOD, State, and  USAID. The three agencies informed us that they had no comments on  the draft’s findings or concluding observations. DOD and State provided  us with technical comments that we incorporated into the final report, as  appropriate.
We are sending copies of this report to the Secretary of Defense, the  Secretary of State, and the Administrator of the U.S. Agency for  International Development, as well as interested congressional  committees. The report also is available at no charge on the GAO website  at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or huttonj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix I.
section_title: Appendix I: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Johana R. Ayers, Assistant  Director; E. Brandon Booth; Virginia Chanley; Julia Kennon; Gilbert Kim;  Angie Nichols-Friedman; Anne McDonough-Hughes; Margaret McKenna;  Robert Swierczek; Michael Rohrback; and Alyssa Weir made key  contributions to this report.