Contingency Contracting: Contractor Personnel Tracking System Needs Better Plans and Guidance
Published: 2015-02-18T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: SPOT-ES contains data on almost 1 million contractor personnel who have supported DOD, State, and USAID in contingency operations. Also, USAID's GLAAS provides data, such as award value, for reports to Congress on contract support. The National Defense Authorization Act for Fiscal Year 2013 mandated that GAO review the data systems of DOD, State, and USAID related to certain contract support.
This report evaluates the extent to which, among other things, (1) DOD and USAID have assessed resources needed to sustain the systems used to track contracts and contractor personnel; (2) DOD has developed business rules and processes to help ensure the timeliness and reliability of SPOT-ES data; and (3) DOD has completed interoperability testing and registered and approved data for SPOT-ES. GAO reviewed DOD and USAID documents, such as cost schedules, business rules, and user manuals, and interviewed cognizant officials.
section_title: What GAO Found
paragraphs: The U.S. Agency for International Development (USAID) has assessed resources that it needs to sustain its contract data system, the Global Acquisition and Assistance System (GLAAS), but the Department of Defense (DOD) has not assessed all resources that it will need to sustain the Synchronized Predeployment and Operational Tracker–Enterprise Suite (SPOT-ES). DOD, the Department of State (State) and USAID use SPOT-ES as a repository of information on contracts and contractor personnel in contingency operations; USAID also uses GLAAS to record information about contracts. DOD uses the budget process to identify resources it projects it will need in the next budget year to modernize and operate its systems, but DOD has not updated its life-cycle cost estimate or fully defined and assessed its plans to determine all resources needed to sustain SPOT-ES. For example, DOD has not updated its life-cycle cost estimate since 2010, despite changes to costs due to schedule delays, because officials said the system has proven stable. Also, DOD has not defined some of its plans that involve cost elements that need to be included in the estimate because it accepted the system's previous program management estimates as reported. GAO's Cost Estimating and Assessment Guide states that cost estimates should be current and comprehensive. Without regularly updating life-cycle costs and defining and assessing plans to provide a full accounting for the systems' costs, management will have difficulty planning program resource requirements and making decisions.
DOD has business rules for the entry of contract and contractor personnel data in SPOT—the database component of SPOT-ES—but lacks reasonable assurance that SPOT provides personnel data that are consistently timely and reliable because the department does not use its available mechanisms for assessing contractor performance to track whether contractors enter data in accordance with the business rules. The business rules, DOD guidance, and an applicable Defense Federal Acquisition Regulation Supplement clause describe how contractors and contracting officers are to enter data in SPOT. Using existing mechanisms for tracking contractor performance could provide DOD reasonable assurance that contractors have abided by business rules to enter and provide timely and reliable data.
DOD has completed SPOT-ES interoperability testing, but has not fully registered or approved the system's data. DOD Instruction 8320.02 directs heads of DOD components to register authoritative data sources and metadata in DOD's Data Services Environment (DSE), its primary online repository for technical descriptions related to information technology and systems for all authorized users, and provides policy that data will be visible and trusted. GAO found that registration for SPOT-ES data was not completed, although program officials thought they had completed all the steps needed to register the system. Full registration and approval in the DSE would help ensure that data are visible and trusted.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other things, that DOD regularly update its lifecycle cost estimate for SPOT-ES to include defining and assessing its plans for SPOT-ES; use mechanisms to track contractor performance of SPOT-ES data entry; and complete SPOT-ES registration in the DSE. DOD concurred with these recommendations, and described planned steps to address them.

section_title: Letter
paragraphs: The Department of Defense (DOD), Department of State (State), and  U.S. Agency for International Development (USAID) relied on contractor  personnel to augment military and civilian personnel in recent operations  in Iraq and Afghanistan. For example, DOD reported that there were more  contractor personnel than military personnel at the peak of operations in  Iraq and Afghanistan. Since 2008, the three agencies have used a DOD  information system, called the Synchronized Predeployment and  Operational Tracker-Enterprise Suite (SPOT-ES), as a repository of  information on contracts and contractor personnel in Iraq and  Afghanistan. USAID uses SPOT-ES to track contractor personnel, but  records information about contracts in its own system, called the Global  Acquisition and Assistance System (GLAAS). As of November 2014,  SPOT-ES contained data on almost 1 million contractor personnel, of  whom about 22 percent are U.S. citizens, who have supported DOD,  State, USAID, and other federal agencies in contingency operations.
According to DOD, success in future contingency operations will depend  heavily on contracted support as part of the total force. DOD joint doctrine  projects that the continual introduction of high-tech equipment, coupled  with military force structure reductions, mission specific force cap  restrictions, and high operating tempo, mean that contract support will  augment military forces in most operations. Expanded use of SPOT-ES  is a key aspect of improving the accountability and visibility of contractors  supporting the U.S. government in contingency operations, according to  senior DOD and Joint Staff leaders. SPOT-ES may also cover contractor  personnel supporting humanitarian assistance, peacekeeping, and other  missions or scenarios.
Section 844 of the National Defense Authorization Act for Fiscal Year  2013 requires DOD, State, and USAID to each issue guidance regarding  data collection on contract support for future contingency operations  outside the United States that involve combat operations. The guidance  must ensure that the agencies take the steps necessary to ensure the  capability to collect and report, at a minimum, on eight specific data  elements related to contracts and contractor personnel.
See § 844(c). developed guidance to meet statutory data-collection and reporting  requirements for future contingency operations.
To determine the extent to which DOD and USAID have assessed  resources needed to sustain the systems used to track contracts and  contractor personnel, we reviewed and compared agencies’ funding  information, cost estimates, systems plans, and schedule for GLAAS and  SPOT-ES to accepted cost-estimating guidance and internal-control  standards. We reviewed funding information for SPOT-ES and GLAAS  for fiscal years 2013 through 2015 because the SPOT-ES current  program office assumed full operational control and management of  SPOT-ES in fiscal year 2013 and GLAAS became operational at all  USAID offices in fiscal year 2013. For USAID, we compared GLAAS’  business cases for fiscal years 2013 and 2014, GLAAS’ Earned Value  Management Metrics, and GLAAS’ cost estimates to determine how costs  were assessed and updated. For DOD, we reviewed and analyzed  SPOT-ES schedules for upgrades, its program acquisition baseline, and  the SPOT-ES fiscal year 2013 business case to determine how costs and  cost estimates were assessed and updated. We also interviewed officials  from the agencies’ system program offices about how they assessed  costs and resources needed to sustain their systems.
To determine the extent to which DOD has developed business rules to  help ensure the timeliness and reliability of data, we obtained and  reviewed current (January and November 2014) and past versions of  DOD’s Business Rules for SPOT-ES, and examined whether the data  had been entered in accordance with the business rules by comparing  certain data to the corresponding business rule. We obtained the data from the SPOT-ES program-management office based on database  queries that the program-management office conducted from September  through November 2014 and determined that the data were sufficiently  reliable for the purposes for which we used them. We also reviewed user  manuals for the Joint Asset Movement Management System (JAMMS)  personnel-location tracking tool; circulated questions to the six  geographic combatant commands about their experiences with SPOT-ES  and analyzed the responses; and interviewed officials in the Office of the  Under Secretary of Defense for Acquisition, Technology and Logistics  about their efforts to enhance SPOT-ES.
To determine the extent to which DOD has completed interoperability  testing and registered and approved SPOT-ES data, we reviewed the  interoperability certification for the system and circulated a common set of  questions to the SPOT-ES program office to obtain technical details on  the system. Further, we interviewed officials with the SPOT-ES program  offices about the system’s capabilities, including interoperability  certification testing, and with DOD’s Defense Information Systems  Agency to discuss registration of the system’s data. We compared DOD’s  actions in these areas with requirements outlined in DOD guidance.
To determine the extent to which agencies have developed guidance to  meet statutory data-collection and reporting requirements for future  contingency operations, we obtained and reviewed documents, including  relevant provisions from State’s Foreign Affairs Manual, USAID’s  Automated Directives System and Acquisition and Assistance Policy  Directives, and DOD guidance related to operational contract support. We  analyzed these sections to determine whether they addressed each of the  eight specific data elements related to contracts and contractor personnel  that are in Section 844 of the National Defense Authorization Act for  Fiscal Year 2013. We also interviewed policy officials at DOD and  management officials at State and USAID to learn about how they related  departmental or agency guidance to statutory requirements. Appendix I  provides further information on our scope and methodology.
We conducted this performance audit from April 2014 to February 2015 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: SPOT-ES comprises five systems: unclassified and classified versions of  the SPOT database, unclassified and classified versions of the Total  Operational Picture Support System (TOPSS) analytic and reporting tool,  and the JAMMS personnel-location tracking tool. Figure 1 illustrates the  five systems.
DOD guidance provides that, in applicable contingency operations,  contractor visibility and accountability shall be maintained through a  common joint database—SPOT, or its successor.
The SPOT database  contains information about contracts—such as company name, contract  number, and task order if any—and information about contractor  personnel, such as contact information and next of kin, blood type, and  the government-furnished support to which they are entitled. Government  contracting officers use the information in SPOT to generate letters of  authorization, which contractor personnel must obtain and carry in order  to process through a deployment center or to travel to, from, or within the  designated operational area. The letter of authorization also identifies any  additional authorizations, privileges, or government support that  contractor personnel are entitled to under the contract. Examples of such  services could include access to dining facilities, transportation, or  medical care beyond emergency treatment. The letter of authorization  also identifies contractor personnel whose contracts permit eligible  personnel to carry weapons, although arming approval rests with the  combatant command to which personnel deploy.
DOD guidance indicates that the department intends to use SPOT to  facilitate integration of contingency contractors and other personnel as  directed by the Under Secretary of Defense for Acquisition, Technology  and Logistics or the combatant commander, and to ensure that  accountability, visibility, force protection, medical support, personnel  recovery, and other related support can be accurately forecasted and  provided. According to the guidance, SPOT data elements are intended  to provide planners and combatant commanders an awareness of the  nature, extent, and potential risks and capabilities associated with  contracted support.
TOPSS is the reporting and analysis component of SPOT-ES. This tool  generates a variety of standard reports that provide information on users,  accounting and compliance, specific contracts or task orders, contractor  deployments, and specific individuals. In addition, this tool can provide  summary-level data in geospatial format.
The JAMMS movement tracker is an information technology application  developed to capture movement and location information about contractor  personnel in specified operational theaters. Also, this tracker can capture  information about operating forces and government civilians. JAMMS  data-collection points are established at locations such as: dining  facilities, aerial ports of debarkation, and medical locations; and at U.S.  embassies and other State locations. JAMMS terminals can scan a wide  range of identification credentials, such as common access cards, SPOT- generated letters of authorization, and some driver’s licenses and  passports. These credentials provide identity information about the  cardholder, which is retained and made available to integration partners.
subsections: section_title: Global Acquisition and Assistance System (GLAAS)
paragraphs: GLAAS is USAID’s worldwide web-based procurement system that  manages awards throughout USAID’s acquisition and assistance life  cycle. GLAAS adapted commercial off-the-shelf software to  accommodate USAID’s procurement-management needs. GLAAS  integrates with the USAID financial-management system and other  external government systems to provide reports to the Office of  Management and Budget, Congress, and other stakeholders. GLAAS  also provides data, such as award value and whether awards were  competed, for reports to Congress on contract support for contingency  operations outside the United States. GLAAS contained information on  approximately 19,000 awards in fiscal year 2013, of which 382 contracts  were related to contingency operations in Iraq and Afghanistan. GLAAS is  an independent system that does not interoperate with any DOD systems,  including SPOT-ES.
USAID has developed business rules and processes to explain to users  how to operate GLAAS. The GLAAS user guides illustrate USAID’s  acquisition and award process and instruct users on how to complete  various tasks such as procurement planning, and creating and modifying  purchase orders, solicitations, and assistance awards. USAID also  supplements the GLAAS user guides with training sessions and other  reference guides.
section_title: Federal Procurement Data System
paragraphs: The Federal Procurement Data System (FPDS) provides a  comprehensive web-based tool for agencies to report information related  to contracts. Executive agencies are to use FPDS to maintain publicly  available information about all unclassified contract actions exceeding  certain monetary thresholds. Generally, contracting officers must submit  complete and accurate contract information to FPDS within 3 business  days after contract award. Agencies can report data to FPDS using  either of two methods: an Internet web portal or by means of contract- writing systems. DOD, State, and USAID report to FPDS through their  respective contract-writing systems—Standard Procurement System,  Global Financial Management System, and GLAAS.
section_title: Data Services Environment
paragraphs: According to DOD, the Data Services Environment (DSE) is DOD’s  primary resource for registering, sharing, and publishing different types of  metadata about systems, services, and data resources, such as  authoritative data sources, to support DOD and the needs of all  authorized users. DOD components, according to DOD guidance, must  register all authoritative data sources, information technology services,  and required metadata in the DSE. DOD guidance also provides policy  that data will be made visible and trusted, among other things, for all  authorized users. According to the guidance, data is made visible by  creating and associating metadata. Data is considered trusted when  there is sufficient pedigree (source and lineage) and descriptive metadata  for users to rely on it as an authoritative data source. According to the  DOD DSE Concept of Operations, once a system is registered in the  DSE, the appropriate authoritative body is to review the data source and  either approve or not approve the source as an authoritative data source.  The DSE is linked to DOD’s broader efforts for managing data in a net- centric environment, the key attributes of which include ensuring that data  are visible, among others things; associating all data with metadata; and  posting all data to shared space to provide access to users except when  limited by security, policy, or regulation.
section_title: Section 2222 of Title 10, U.S. Code, and SPOT-ES Certification with Relevant Requirements
paragraphs: Section 2222 of Title 10, U.S. Code, contains provisions regarding DOD  development of a defense Business Enterprise Architecture (BEA),  associated enterprise transition plan, and investment management  structures and review processes. Among other requirements, section  2222 prohibits DOD from obligating funds for a defense business system  program with a total cost in excess of $1 million over the period of the  future years defense program unless the precertification authority  certifies that the defense business system meets specified conditions.  Specifically, the appropriate precertification authority must determine that  the defense business system program is in compliance with the defense  BEA and that appropriate business process reengineering efforts have  been undertaken.
For fiscal years 2013 through 2015, the precertification authority for  SPOT-ES determined that the system was in compliance with the BEA  and that appropriate business process reengineering efforts were  undertaken. Subsequent Office of the Deputy Chief Management Officer  investment decision memorandums from the Defense Business Systems  Management Committee certified SPOT-ES funds. We have previously  reported on the need to improve the quality of BEA compliance  assertions. In May 2013, we found that compliance assertions continued  to lack adequate validation; we recommended that DOD implement and  use the BEA compliance assessments more effectively to support  organizational transformation efforts by, among other things, establishing  milestones by which selected validations of BEA compliance assertions  are to be completed. DOD partially agreed with this recommendation. In  our follow-up report in May 2014, we found that DOD needed to continue  working to ensure the quality of the BEA assessments, as we had  previously recommended.
section_title: Our Prior Work on Contractor Management Systems
paragraphs: We have issued several recent reports about systems that DOD, State,  and USAID use to manage information regarding contracts and contractor  personnel and to prepare statutorily required reports on contracts,  assistance instruments, and related personnel in Iraq and Afghanistan.
In September 2012, we found that although SPOT was designated as the  common database for the statutorily required information, officials from  DOD, State, and USAID generally relied on other data sources they  regarded as more reliable to prepare the 2011 joint report to  congressional committees. We also found that the agencies generally  did not use SPOT to help manage, oversee, and coordinate contracting in  Iraq and Afghanistan. Instead, we found that the agencies primarily used  the system to generate authorizations for contractor personnel to use  U.S. government services. We recommended that the Secretaries of  Defense and State and the Administrator of USAID work together to  standardize the methodologies used to obtain and present information  contained in the annual joint report on contracting in Iraq and Afghanistan  to the greatest extent possible. The agencies agreed with our  recommendation and indicated they would work together to implement it.
Additionally, in February 2014, we reported that State and USAID had  taken, or were planning to take, a number of actions to better track the  number of contracts and contractor personnel in contingency  environments. For example, we reported that, according to State  officials, State developed additional guidance in fall 2012 that outlined the  process of inputting contracts, how contractors should enter contractor  personnel, how to request letters of authorization and approvals, and  how to enter data through the contract close-out. We reported that,  according to the department, State also established a new office that will  be responsible for overseeing contractors’ input of Iraq contractors’ data  into SPOT. We also reported that USAID was in the early stages of  developing a proposal to use SPOT solely as a tool to track contractor  personnel in contingency environments rather than the number and value  of contracts. USAID officials stated that other data systems, such as  FPDS-NG and GLAAS, provided more reliable information on the number  and value of contracts. We recommended that the Secretary of State  develop plans to assess whether planned initiatives are achieving their  intended objectives and that the Administrator of USAID ensure that its  nonpermissive working group consider procedures and practices  developed by missions and offices with contingency-related  responsibilities. The agencies concurred with our recommendations and  provided information on actions taken or planned to address them.
section_title: USAID Has Assessed Resources Needed to Sustain GLAAS, but DOD Has Not Fully Assessed All Resources Needed to Sustain SPOT-ES
paragraphs: USAID has assessed resources that it needs to sustain GLAAS, but DOD  has not fully assessed all future resources that it needs to sustain SPOT- ES. DOD and USAID use the budget process to identify resources they  project they will need in the next budget year to modernize and operate  their systems. USAID assessed its plans and updated GLAAS’s cost  estimates with the actual costs of system upgrades and the latest cost  estimates for future upgrades; however, DOD has not updated its life- cycle cost estimates or fully defined and assessed its plans to determine  all resources needed to sustain SPOT-ES.
subsections: section_title: USAID Has Assessed Resources Needed to Sustain GLAAS
paragraphs: USAID has assessed the resources it needs to sustain GLAAS and  regularly updates GLAAS’s life-cycle cost estimate to reflect its plans.  USAID uses historical cost data and cost estimates to develop the annual  funding request to operate, maintain, and modernize GLAAS. Table 1  shows GLAAS’s cost estimates and funds that USAID requested for the  operation and development of GLAAS from fiscal years 2013 through  2015 and amounts USAID identified as funded in fiscal years 2013 and  2014.
We found that USAID assessed its plans and associated costs for  modernizing GLAAS and has regularly updated its life-cycle cost  estimate. Our review of USAID’s business cases for GLAAS’s  modernization projects found that, in 2012 and 2013, USAID estimated  the costs to implement system upgrades. For the upgrade completed in  2013, USAID assessed additional costs and analyzed the effect on  project scope of a 3-month delay in releasing a system upgrade, known  as version 7.1. In addition, USAID regularly updated GLAAS’s life-cycle  cost estimate, which was originally calculated in 2009. Officials said they  update cost estimates for the following budget year based on prior actual  costs and upcoming projects. For example, in July and September 2014  USAID updated the life-cycle cost estimate with actual costs, resulting in  a decrease in the projected costs to operate and maintain GLAAS from  fiscal year 2015 through 2020. Further, USAID updated the modernization  estimates of the life-cycle estimate for fiscal year 2015 through 2016 in  September 2014.
section_title: DOD Has Not Updated Its Cost Estimates or Fully Defined and Assessed Its Plans for SPOT-ES
paragraphs: DOD has identified resources for fiscal years 2013 through 2015 that it  needs to operate SPOT-ES; however, since 2010, DOD has not updated  its life-cycle cost estimate or fully defined and assessed its plans to  identify all the resources it needs to achieve the system’s objectives.  GAO’s Cost Estimating and Assessment Guide states that approved cost  estimates are often used to create budget spending plans and they  should be updated with actual costs so that they are always relevant and  current. In addition, Standards for Internal Control in the Federal  Government state that management must continually assess and  evaluate its plans to ensure that the control activities being used are  effective and updated when necessary.to the defense acquisition system states that effective life-cycle  sustainment for information systems requires continuous monitoring to   Further, DOD guidance related  ensure investments are maintained at the right size, cost, and condition,  to support business missions and objectives.
According to Under Secretary of Defense (Comptroller) figures, the  SPOT-ES program received about $27.1 million to operate and maintain  its system and $2.8 million to develop and modernize in fiscal year 2014.  Table 2 shows the SPOT-ES cost estimates, and the funds that DOD  officials identified as requested for fiscal years 2013 through 2015 and the  amount they identified as funded for fiscal years 2013 and 2014.
DOD prepared a SPOT-ES life-cycle cost estimate in 2010; however, the  department has not updated the SPOT-ES life-cycle cost estimate since  then to reflect any changes in costs due to schedule delays or program  changes. The SPOT-ES life-cycle cost estimate projected that SPOT and  TOPSS would undergo upgrades that were not completed or not  completed as scheduled. For example, program officials projected that  TOPSS versions 1.3 to 1.7 upgrades would be completed in fiscal year  2011, but as of October 2014, a subversion of 1.1 was the latest upgrade;  however, SPOT-ES program officials have not updated the life-cycle cost  estimate to reflect the costs of implementing the upgrades in the future.  Figure 2 illustrates SPOT-ES’s schedule delays.
In addition, DOD has not fully defined and assessed its plans for SPOT- ES to ensure a comprehensive life-cycle cost estimate, representative of  all resources it needs to sustain the system. GAO’s Cost Estimating and  Assessment Guide states that a comprehensive life-cycle cost estimate  should include both government and contractor costs of the program over  its full life cycle, from inception of the program through retirement of the  program. The estimate should also completely define the program, reflect  the current schedule, be technically reasonable, and be structured in  sufficient detail to ensure that cost elements are neither omitted nor  double counted. However, DOD has not defined some of its plans that  involve cost elements that need to be incorporated into SPOT-ES life- cycle cost estimate. For example, SPOT-ES program officials identified  the need to network and to further develop the JAMMS movement  tracking application that feeds movement data to other SPOT-ES  systems. SPOT-ES program officials had scheduled to network JAMMS  to the SPOT database in the JAMMS version 4.0 release in fiscal year  2011, but as of November 2014, had not done so, as figure 2 illustrates.  SPOT-ES program officials prepared a concept paper in September 2014  that proposed ideas for JAMMS’ further development; however, the paper  does not define the scope of capabilities for a networked JAMMS.  Further, even though the SPOT-ES life-cycle estimate reflected the costs  of networking JAMMS in fiscal year 2011, it has not been updated to  include the plans for networking or developing JAMMS in future years.
Officials said they have not updated the life-cycle cost estimate since  2010 because the system has proven to be stable, and they will update  the estimate to reflect additional development and modernization funds  they may require to improve the system. However, while the system may  be stable, the cost elements and assumptions that DOD used to develop  the life-cycle cost estimate have changed. Specifically, the host site  supporters and the contractor that provided software maintenance for  SPOT-ES have changed. SPOT-ES program officials had projected  operation and maintenance life-cycle costs based on the actual costs of  the previous host site supporters and the actual costs of software  maintenance under the previous contract. For example, the life-cycle cost  estimate projected an average yearly software maintenance cost of $16  million, but the software maintenance costs for fiscal years 2013 and  2014 were $2.7 million and $2.5, million respectively. Further, SPOT-ES  program officials had requested additional development and  modernization funds for fiscal years 2013 and 2014 when compared to  the 2010 cost estimate, but they did not update the life-cycle cost  estimate.
SPOT-ES program officials did not update the life cycle cost estimate or  fully assess and define plans to ensure a comprehensive and accurate  cost estimate because they accepted the system’s previous program- management estimates as reported. Further, the Under Secretary of  Defense for Acquisition, Technology and Logistics and the Under  Secretary of Defense for Personnel and Readiness, whose offices both  have oversight responsibilities for the SPOT-ES program, have not  ensured that the current program management assess and define all  plans for the system’s further development or update the life-cycle cost  estimate. According to GAO’s Cost Estimating and Assessment Guide, a  life-cycle cost estimate should encompass all past (or sunk), present, and  future costs for every aspect of the program, regardless of funding  source, including all government and contractor costs. Without defining  and assessing plans to provide a full accounting for the system, thereby  fully accounting for life-cycle costs, management will have difficulty  planning program resource requirements and making decisions. Also, to  ensure accuracy, GAO’s Cost Estimating and Assessment Guide, states  that cost estimates should be updated regularly to reflect significant  changes in the program—such as when schedules or other assumptions  change—and actual costs, so that it is always reflecting current status. If  the estimate is not regularly updated, it will be difficult to analyze changes  in program costs, and collecting cost and technical data to support future  estimates will be hindered. Moreover, the cost estimate cannot provide  decision makers with accurate information for assessing alternative  decisions.
section_title: DOD Has Developed Business Rules, but Lacks Reasonable Assurance SPOT and JAMMS Data Are Timely and Reliable
paragraphs: DOD has developed business rules governing data entry about contracts  and contractor personnel into SPOT. However, DOD cannot assure that  SPOT or JAMMS—which provides data to SPOT—provide either contract  or contractor personnel data that are consistently timely and reliable.
subsections: section_title: DOD Has Developed Business Rules for Entering Data about Contracts and Contractor Personnel, but Lacks Reasonable Assurance that SPOT is Timely and Reliable
paragraphs: DOD has developed SPOT business rules for entering data about  contracts and contractor personnel; however, DOD’s process does not  provide reasonable assurance that the business rules are followed and  SPOT data are timely and reliable. In the context of operational contract  support, various provisions in DOD guidance and the Defense Federal  Acquisition Regulation Supplement (DFARS) relate to the accuracy and  timeliness of information in SPOT. For example:
DOD guidance provides that contracting officers, through the terms of  contracts, shall require contractors to enter data before employee  deployment and to maintain and update the information for relevant  employees.
An applicable DFARS clause specifies that the contractor shall use  SPOT to enter and maintain data for all contractors authorized to  accompany the force and, as designated, other personnel. The  contractor is to enter the required information about its personnel prior  to deployment and to continue to use SPOT to maintain accurate, up- to-date information throughout the deployment. Under the clause,  changes to personnel status relating to arrival date and duty location,  including closing out deployment with proper status (such as mission  complete, killed, wounded) must be annotated in SPOT in accordance  with timelines established in SPOT business rules.
Defense Manpower Data Center, DOD Business Rules for the Synchronized  Predeployment and Operational Tracker (SPOT) (Jan. 21, 2014); and Defense Manpower  Data Center, DOD Business Rules for the Synchronized Predeployment and Operational  Tracker (SPOT) (Nov. 21, 2014). arrived at the primary duty station.they did not know how often company administrators entered such  information within a day. However, they reported that nearly 4,000  contractors with active deployments on active contracts lacked in- theater arrival dates in SPOT at least 7 days after the scheduled date,  as of September 2014.
SPOT program officials said that
Armed Contractor Personnel—The SPOT business rules indicate that  company administrators are to enter information into SPOT on  equipment used by their personnel who perform security functions,  such as the serial numbers of each weapon issued specifically to an  individual. However, program officials told us that they sometimes  could not link weapon serial numbers to individual contractor  personnel because companies did not always assign specific  weapons to specific individuals. As a result of our inquiries, program  officials also discovered and corrected a software deficiency in SPOT  wherein no serial numbers were displayed if personnel had more than  one assigned weapon.
In addition, we found that the business rules may be insufficient to allow  contractors to provide accurate information on foreign contractor  personnel. According to the business rules, for contractor personnel who  lack a foreign identification number, the contractor must create a number  according to instructions in the SPOT User Guide. For U.S. Central  Command, in the case of Afghanistan or non-Iraqi citizens, the guide  directs personnel to use the first five letters of the last name, plus date of  birth in the format “mmddyyyy.” However, SPOT program officials found  that some individuals lack birth certificates and do not know their birth  date. Out of about 769,000 foreign nationals in SPOT, 213,348 have a  birth date of January 1 recorded in SPOT, and about 100,000 of these  have identical surname and birth date information. This has created  challenges in situations where people have identical names, according to  DOD officials, including officials from Pacific Command. DOD has  developed a process to begin assigning unique identification numbers,  but has not made corresponding changes to the user guide to show how  the process will apply to U.S. Central Command.
DOD has established a goal for SPOT of 85 percent accuracy. In 2011,  DOD directed contracting activities in the Central Command area of  responsibility to continue conducting a quarterly manual census, or  physical count of contractor personnel, until SPOT contains at least 85  percent of the data revealed by the manual count. DOD officials have  conducted this census at Central Command, where most contractors who  are now required to register in SPOT are located, every quarter since  2008. While some contracting activities have recently exceeded that  threshold—for example, our review of census data for the third quarter of  fiscal year 2014 found that 69 out of 102 contracting activities had  exceeded 85 percent—officials said that they would continue to conduct  the census until all activities did so.
DOD’s process also cannot provide reasonable assurance that  contractors and contracting officers enter data into SPOT according to the  business rules for two primary reasons. First, the department does not  use its available mechanisms for tracking contractor performance to  promote contractor accountability for entering data correctly and within  prescribed time frames into SPOT. These mechanisms include the  Contractor Performance Assessment Reporting System (CPARS), which  contracting officers can use to report on contractors’ performance.  Office of the Under Secretary of Defense for Acquisition, Technology, and  Logistics officials said that they thought using such mechanisms would be  helpful. Second, Office of the Under Secretary of Defense for Acquisition,  Technology and Logistics officials said that contracting officers  sometimes did not oversee contractors’ data input or enter their own  required data.officers in its Operational Contract Support Action Plan for Fiscal Years  2014-2017, which calls for the military departments to issue policy  mandating that their contracting activities populate SPOT with required  data.
DOD has taken steps that may address contracting  By not having timely and reliable contract and contractor personnel data,  DOD does not have complete visibility into the number of contractors  present in the contingency environment, and for whom the department  may have to develop plans for such services as force protection,  sustenance, and repatriation of injured or deceased contractor personnel.  For example, one official at a DOD combatant command stated that DOD  can more quickly repatriate injured or deceased contractors when their  information is up-to-date and accurate in SPOT. Not using the  mechanisms it already has in place to track contractor performance limits  DOD’s assurance that contractors have abided by business rules to  provide timely and reliable data, and requires DOD officials to devote  additional resources to conducting the quarterly manual census.
section_title: SPOT-ES Does Not Receive Timely and Reliable Data from JAMMS Terminals about the Location of Contractor Personnel
paragraphs: JAMMS is a major data source for SPOT-ES to track contractor personnel  in key deployment sites such as dining halls and military airfields;  however, SPOT-ES does not receive timely and reliable data from  JAMMS. DOD policy states that information solutions shall provide,  among other things, reliable, timely, and accurate information.  Moreover, DOD has noted that the purpose of SPOT is to provide the  combatant commander with accurate, real-time information on all  personnel within specified geographic combatant command operations  areas and to enable the department to keep track of all persons deployed  in contingency zones.
We found examples in which deployed contractors provided three types  of unreliable data to JAMMS: not scanning at all, scanning incorrect  documents, or scanning documents into a terminal that reported back to  SPOT-ES as if it was in a different location. First, a review of data at one  deployed operational area from August 2013 through March 2014 found  that contractor scans in the week preceding the date the data were pulled  represented less than half of the contractors believed to be in the area.  Second, JAMMS terminals can scan incorrect documents because they  can accept bar-coded items that are not identification documents, such as  supermarket loyalty cards, and other identification documents, such as  letters of authorization, that do not contain photos. For example, in the  last two weeks of September 2014, there were almost 15,000 JAMMS  scans in Afghanistan that could not be linked to either a deployment in  SPOT or a Defense Enrollment Eligibility Reporting System  record. Third, officials told us that they sometimes receive erroneous data from  JAMMS terminals about the locations of contractor personnel. For  example, officials at Central Command said that they have received  reports that contractor personnel were at sites that they never visited. We  also reviewed documentation about one site in Afghanistan that reported  that more than 20 percent of records represented individuals who either  never scanned their identification documents or whose most recent scan  was in a country other than Afghanistan.
We also found that JAMMS data may not always be transmitted to SPOT  in a timely manner. Although DOD officials have indicated an intention to  develop a networking capability that would allow JAMMS data to be  transmitted to SPOT in real time, data are currently transmitted by  uploading compact discs from stand-alone JAMMS terminals to  networked computers. The JAMMS User Manual and the System of  Record Notice for SPOT-ES indicate that JAMMS uploads personnel  movement records to SPOT-ES daily. However, personnel at operating  locations vary in the frequency with which they upload JAMMS location  data into SPOT. For example, SPOT program office officials told us that  data usually are uploaded every several days. In contrast, officials at  Central Command said their data are uploaded about every week, but  that at one site, data for 20 percent of contractor personnel were not  uploaded for 30 days.
Personnel at operating locations are not consistently ensuring that  JAMMS data are transmitted accurately or in a timely manner into SPOT- ES because they lack comprehensive guidance from DOD that describes  the purpose of JAMMS and its role in supporting plans for different types  of missions. The lack of guidance stems from lack of consensus about  requirements for visibility and accountability—such as what constitutes  “near real-time” movement tracking—within DOD. Officials at the Joint  Staff and in combatant commands told us that in the absence of such  guidance, they do not know what resources or emphasis to allocate to  correcting problems that they encounter. For example, they could  minimize inaccurate scans by posting a monitor at each JAMMS terminal  to verify that everyone scans a valid identification document, but such an  action has resource implications that could conflict with other command  priorities. Without clear guidance about the purpose of JAMMS, such as  direction about what types of missions it is to support, the department is  not able to help Joint Staff and combatant command planners to  determine where to locate JAMMS terminals and what resources to  allocate to minimize inaccurate scans. For example, combatant  commanders would need more precise data if they expected to use  JAMMS to plan a short-notice operation such as a noncombatant  evacuation than they would to calculate how much to bill contractors for  their employees’ use of on-base dining or medical facilities. Such  guidance could also assist program management officials as they develop  cost estimates and associated plans, as more frequent data uploads or  the potential networking capability described earlier would be more  applicable to some missions than others.
section_title: DOD Has Completed Interoperability Certification Testing but Has Not Fully Registered or Approved SPOT-ES Data
paragraphs: DOD has completed interoperability certification testing between SPOT- ES and its data sources, but has not fully registered or approved the  system’s data to ensure data are visible and trusted.
subsections: section_title: DOD Has Completed Interoperability Certification Testing for SPOT-ES
paragraphs: DOD’s SPOT-ES program office completed interoperability certification  testing of SPOT-ES with the systems that provide it with data. In March  2014, the Defense Information Systems Agency Joint Interoperability Test  Command certified that the latest version of SPOT-ES meets all joint  critical interoperability requirements, such as supporting military  operations and effectively exchanging information with other systems.  Our review of documentation for SPOT-ES found that the system began  as a rapid capability (i.e., a capability to be delivered quickly), was  conditionally approved for full deployment in November 2010, and did  not receive joint interoperability certification until March 2014. In addition,  we found that three different DOD offices provided support to SPOT-ES  before the Defense Manpower Data Center assumed management and  operation of the system in November 2013. However, DOD’s SPOT-ES  program-management office completed SPOT-ES interoperability  certification testing and associated steps, such as developing and  submitting an Information Support Plan for DOD approval.
section_title: DOD Has Not Fully Registered or Approved SPOT-ES Data
paragraphs: The SPOT-ES program office has not ensured that the system’s data  are visible and trusted because it has not fully registered the system’s  data in the Data Services Environment (DSE). In May 2003, DOD’s Chief  Information Officer issued a memorandum that provided guidance for  managing data in a net-centric environment and highlighted key  attributes of the department’s net-centric strategy, such as ensuring that  data are visible, among other things; all data are associated with  metadata to enable discovery by users; and all data are posted to shared  spaces to provide access to users except when limited by security, policy,  or regulation. According to DOD, the DSE is DOD’s primary resource for  registering, sharing, and publishing different types of metadata about  systems, services, and data resources to support DOD operational  capabilities and data standards and needs for all authorized users. DOD  Instruction 8320.02 requires heads of DOD components to register all  authoritative data sources, information technology services, and required  metadata in the DSE and further states as policy that data will be made  visible and trusted, among other things, for all authorized users.According to the guidance, data is made visible by creating and  associating metadata. Data is considered trusted when there is sufficient  pedigree and descriptive metadata for users to rely on it as an  authoritative data source.related to authoritative data sources: (1) data collection, which supports  the registration of new data needs, data producers, systems, and  databases; (2) data association, which defines all the systems, data  producers, and data needs that make up the proposed structure of the  authoritative data source; and (3) authoritative data source approval,  which involves the authoritative body reviewing and making a  determination to approve or not approve the proposed authoritative data  source. Figure 3 provides a high-level overview of the key processes  related to authoritative data sources.
Department of Defense, Deputy Under Secretary of Defense for Logistics and Materiel  Readiness and Deputy Under Secretary of Defense for Program Integration, “Designation  of Synchronized Predeployment and Operational Tracker (SPOT) as Central Repository  for Information on Contractors Deploying with the Force (CDF),” memorandum  (Washington, D.C.: Jan. 25, 2007).
section_title: State Has Issued Guidance to Meet Statutory Requirements, but DOD and USAID Guidance May Not Ensure Complete Data Collection
paragraphs: State issued guidance in February 2014 to address the statutory  requirements for data collection on contract support for future contingency  operations. Section 844 of the National Defense Authorization Act for  Fiscal Year 2013 requires DOD, State, and USAID to each issue  guidance regarding data collection on contract support for future  contingency operations outside the United States that involve combat  operations. The guidance is to ensure that each agency takes the steps  necessary to possess the capability to collect and report on at least eight  data elements that the statute enumerates. These elements are:  total number of contracts entered into as of the date of any report, total number of such contracts that are active as of such date,total value of contracts entered into as of such date, total value of such contracts that are active as of such date, identification of the extent to which the contracts entered into as of  such date were entered into using competitive procedures, total number of contractor personnel working under contracts entered  into as of the end of each calendar quarter during the 1-year period  ending on such date, total number of contractor personnel performing security functions  under contracts entered into as of the end of each calendar quarter  during the 1-year period ending on such date, and total number of contractor personnel killed or wounded under any  contracts entered into.
We found that a provision in State’s Foreign Affairs Manual lists each  element and stipulates which data source the department will use to  gather information on each of the eight data elements. For example, State  obtains data on contract value and the use of competitive procedures  from FPDS-NG, according to the provision.
DOD published guidance about operational contract support in 2011, which officials in the Office of the Under Secretary of Defense for  Acquisition, Technology and Logistics told us meets the statutory  requirement. However, unlike the State guidance, the DOD guidance  does not clearly cover each data element listed in section 844 and may  not ensure that the department has the capability to collect and report on  the required data. This guidance identifies as DOD policy that contractor  visibility and accountability shall be maintained in applicable contingency  operations through a common joint database–SPOT or its successor.  Also, the guidance provides some information on contract visibility and  contractor accountability responsibilities and procedures, including  responsibilities related to several of the data elements specified in section  844. For example, the guidance indicates that SPOT or its successor  shall contain, or link to, minimum contract information, such as contract  number, contract category, period of performance, contracting agency,  and contracting office, necessary to establish and maintain accountability  and visibility of certain contractor personnel, maintain information on  specific equipment related to private security contracts, and the contract  capabilities in relevant operations.
Operational Contract Support, 32 C.F.R. pt. 158; Department of Defense Instruction  3020.41, Operational Contract Support (OCS) (Dec. 20, 2011). collect information on certain contractor personnel, but it is not clear that it  would ensure the collection of all personnel information required by  section 844. For example, it is not clear whether the collection of  employee data based on the guidance would ensure the identification of  contractor personnel performing security functions or the number killed or  wounded. DOD officials said that they have not updated their guidance to  specifically address each data element and source because the 2011  instruction meets their needs, but commented that the instruction is  undergoing a revision and it would be useful to update SPOT provisions  during that process.
Without current and comprehensive guidance that identifies the data  elements to collect, the systems with which to collect and report them,  and relevant responsibilities and procedures, DOD lacks assurance that  its 2011 guidance provides the department the ability to ensure that it can  take the steps necessary to collect and report on the eight required data  elements. Moreover, DOD may find it more difficult to reconcile  information when there are multiple sources for the same data element.  For example, SPOT receives some information about contract numbers  and about whether a contract was awarded competitively both from the  government-wide FPDS-NG and from contractors’ manual entry.  Additionally, an update would also clarify current procedures to help  ensure the collection of the total number and total value of contracts  active as of a reporting date, as well as the number and value of contracts  entered into as of the reporting date.
USAID has published guidance about the use of GLAAS and SPOT to  record data about contracts and contractor personnel, but this guidance  may also not ensure that the agency has the capability to collect and  report on the required data. USAID officials told us in May 2014 that they  were not aware of guidance to implement the statutory requirement, and  our review of agency guidance revealed no provisions specifically tied to  section 844. However, in November 2014, officials added that USAID  believed that its current policy and procedures related to data collection  on contract support for contingency operations were sufficient. According  to a USAID management official, the agency reports on data related to  number, value, and competition of contracts through the reporting  functionality of GLAAS, but it is not clear that the corresponding guidance  identified by the official would ensure the collection of each data element.  The official also cited two Acquisition and Assistance Policy Directives  (AAPD), dated in 2009 and 2010. These directives require USAID  contracting officers to include a provision in certain contracts with  performance in Iraq and Afghanistan that identifies SPOT as the required  system to use for personnel data.contract provisions do not specifically address data collection on  contractors who are killed or wounded, and do not address any future  contingency operations beyond Iraq or Afghanistan. As with DOD, until  USAID develops current and comprehensive guidance that identifies the  data elements to collect, the systems with which to collect and report  them, and relevant responsibilities and procedures, it cannot ensure that  it is providing itself with the ability to take the steps necessary to collect  and report on the eight required data elements.
section_title: Conclusions
paragraphs: DOD, State, and USAID expect to continue to rely on contractor  personnel to augment military and civilian personnel as they did in recent  operations in Iraq and Afghanistan. While SPOT-ES is a central  repository in which the three agencies have stored information about  nearly a million contractor personnel, the data system is not  comprehensive and reliable. Additionally, DOD has not updated its life- cycle cost estimate with actual costs or to reflect changes in costs due to  SPOT-ES schedule delays and program changes. Also, DOD has not  fully defined and assessed some of its plans to incorporate as cost  elements in its life-cycle cost estimate. Consistent with cost-estimating  guidance and standards for internal control, without updating its cost  estimates to include defined plans for SPOT-ES further development,  DOD may not be able to fully determine future resources needed to  sustain SPOT-ES. Ensuring that the SPOT-ES program updates its cost  estimate to include defined program plans could help the program to  better plan program resource requirements and make decisions.
DOD has developed business rules to enter data about contracts and  contractor personnel in SPOT, but neither SPOT nor JAMMS can provide  contractor personnel data that are consistently timely and reliable.  Without a reasonable assurance of timely and reliable contract and  contractor personnel data, DOD has incomplete visibility into the number  of contractors present in the contingency environment, and for whom the  department may have to develop plans for such services as force  protection, sustenance, and repatriation of wounded or deceased  contractor personnel. DOD has not used existing mechanisms for tracking  contractor performance that could help provide reasonable assurance  that contractors have abided by business rules to provide timely and  reliable data, and relieve DOD officials of the need to continue to devote  resources to conducting the quarterly manual census. Finally, DOD has  not developed comprehensive guidance for JAMMS, which could provide  the combatant commands with better information on how to allocate their  resources to maximize JAMMS’ utility as a tracking and planning tool.
The SPOT-ES program office has completed interoperability certification  testing for the system, but the program office did not fully register the  system’s data or complete the steps required of authoritative data  sources in the DSE. Full registration and approval of SPOT-ES data in  the DSE would ensure data are visible and trusted and provide authorized  users seeking authoritative data on contracts and contractor personnel  confidence in the information, which may prevent the need to develop  their own data and potential duplication of efforts. In addition, full  registration and approval may preclude data producers from continuing to  rely on individual data solutions for their systems rather than leveraging  shared common data across the DOD enterprise.
State has issued guidance to address section 844 of the National  Defense Authorization Act for Fiscal Year 2013, which requires DOD,  State, and USAID to each issue guidance regarding data collection on  contract support for future contingency operations. The guidance is to  ensure that the agencies have the capability to collect and report at least   DOD, however, issued guidance in 2011 that may  eight data elements.not guarantee that the department takes the steps necessary to ensure  the capability to collect and report on the eight required data elements,  and USAID guidance may also not ensure complete data collection. For  both DOD and USAID, including information on each of the eight  elements in departmental guidance could better ensure the ability to  collect and report on those elements, possibly including better timeliness  and accuracy of data entry.
section_title: Recommendations for Executive Action
paragraphs: To help improve DOD, State, and USAID’s ability to track contracts and  contractor personnel in contingency operations, we are making the  following five recommendations to DOD:
To ensure SPOT-ES cost estimates are accurate and comprehensive,  we recommend that the Under Secretary of Defense for Personnel  and Readiness in coordination with the Under Secretary of Defense  for Acquisition, Technology and Logistics direct the system’s program  office to regularly update its life-cycle cost estimate to include defining  and assessing its plans for SPOT-ES.
To help improve timeliness and reliability of data in SPOT-ES, the  Secretary of Defense should direct Defense Procurement and  Acquisition Policy officials, through the Under Secretary of Defense  for Acquisition, Technology and Logistics, to ensure that contracting  officers use available mechanisms to track contractor performance of  SPOT data entry, such as its Contractor Performance Assessment  Reporting System or other appropriate performance systems or  databases.
To provide clarity about expectations for JAMMS that can help  improve the timeliness and reliability of data for SPOT-ES from  JAMMS uploads, the Secretary of Defense should direct the  Chairman of the Joint Chiefs of Staff, in coordination with the  combatant commanders, to develop comprehensive guidance  regarding the purpose of JAMMS and its role in supporting plans for  different types of missions. Such guidance could include direction on  the number and location of JAMMS terminals and how frequently  JAMMS’s data should be uploaded into SPOT-ES to meet DOD’s  information needs.
To enhance the value of SPOT-ES data, we recommend that the  Secretary of Defense direct the Under Secretary of Defense for  Personnel and Readiness to fully register SPOT-ES data in the DSE  to make data visible and trusted, including taking the necessary steps  related to authoritative data sources.
To help ensure that DOD possesses the capability to collect and  report statutorily required information and to clarify responsibilities and  procedures, the Secretary of Defense should direct the Under  Secretary of Defense for Acquisition, Technology and Logistics to  update SPOT provisions during the process of updating operational  contract support guidance.
In addition, we are making the following recommendation to USAID:
To help ensure that USAID possesses the capability to collect and  report statutorily required information, the Administrator of USAID  should issue current and comprehensive guidance regarding data  collection on contract support for future contingency operations  outside the United States that involve combat operations.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD, USAID, and State for their  review and comment. DOD and USAID provided written comments, which  are summarized below and reprinted in appendixes II and III, respectively.  State did not provide comments on the report. DOD concurred with four of  the five recommendations directed to it and partially concurred with the  fifth.  DOD also described actions underway or plans to address the  recommendations. USAID agreed with the recommendation directed to it  and described plans to address it.
DOD concurred with the first recommendation, that the SPOT-ES  program office regularly update the system’s life-cycle cost estimate to  include defining and assessing its plans for SPOT-ES. DOD stated that  the Under Secretary of Defense for Personnel and Readiness would  direct the system program office to regularly update its life-cycle cost  estimate to include defining and assessing its plans for SPOT-ES. We  believe that these actions, if fully implemented, would address the  recommendation and better ensure that SPOT-ES cost estimates are  accurate and comprehensive.
DOD concurred with the second recommendation, that the Secretary of  Defense direct Defense Procurement and Acquisition Policy officials,  through the Under Secretary of Defense for Acquisition, Technology and  Logistics, to ensure that contracting officers use available mechanisms to  track contractor performance of SPOT data entry, such as DOD’s  Contractor Performance Assessment Reporting System or other  appropriate performance systems or databases. DOD stated that the  Under Secretary of Defense for Acquisition, Technology, and Logistics  would direct Defense Procurement and Acquisition Policy officials to  ensure that DOD contracting officers use available mechanisms to  improve compliance with contract requirements for SPOT data entry. If  DOD uses available mechanisms to improve compliance with contract  requirements, that would address the intent of the recommendation and  improve timeliness and reliability of data in SPOT-ES.
DOD partially concurred with the third recommendation, that the  Secretary of Defense direct the Chairman of the Joint Chiefs of Staff, in  coordination with the combatant commanders, to develop comprehensive  guidance regarding the purpose of JAMMS and its role in supporting  plans for different types of missions. Such guidance could include  direction on the number and location of JAMMS terminals and how  frequently JAMMS’s data should be uploaded into SPOT-ES to meet  DOD’s information needs. DOD stated that it agreed to provide clarity  regarding the purpose and use of JAMMS to improve the timeliness and  reliability of JAMMS data, but it did not agree that such guidance could  include direction on the number and location of JAMMS terminals and  how frequently JAMMS’s data should be uploaded into SPOT-ES.  DOD  stated that it would revise language in DOD Instruction 3020.41,  Operational Contract Support, to reflect in policy the requirement to use  the entire SPOT Enterprise Suite (SPOT-ES), which includes JAMMS.  DOD also stated that the combatant commander should establish the  requirements for terminal quantities and locations and for data upload  schedules based on operational needs in the relevant theater. We agree  with DOD that the combatant commands need flexibility based on  operational requirements. The recommendation was intended to allow for  such flexibility by suggesting the types of direction that could be included  in the guidance, which would facilitate consistency in the use of terminals  and data across the commands. Further, as we recommended, this  guidance should be developed in coordination with the combatant  commanders.
DOD concurred with the fourth recommendation, that the Under Secretary  of Defense for Personnel and Readiness fully register SPOT-ES data in  the DSE to make data visible and trusted, including taking the necessary  steps related to authoritative data sources. DOD stated that it agreed to  complete the process of registering the system’s data, including validation  of authoritative data sources. We believe that, if fully implemented, these  steps would address the recommendation and enhance the value of the  system’s data for authorized users seeking authoritative data on contracts  and contractor personnel.
DOD concurred with the fifth recommendation, that the Secretary of  Defense direct the Under Secretary of Defense for Acquisition,  Technology and Logistics to update SPOT provisions during the process  of updating operational contract support guidance. DOD stated that DOD  Instruction 3020.41, Operational Contract Support, is scheduled to be  republished in 2016 and will be updated to specifically identify the  statutorily required data elements to ensure collection and reporting can  be accomplished. We believe this action, if fully implemented, would  address the recommendation and help to ensure that DOD possesses the  capability to collect and report statutorily required information and to  clarify responsibilities and procedures.
USAID agreed with the recommendation that the Administrator of USAID  issue current and comprehensive guidance regarding data collection on  contract support for future contingency operations outside the United  States that involve combat operations. USAID stated that such guidance  would explicitly describe the Section 844 statutory requirement, list each  of the eight data elements required by Section 844, and stipulate the data  source that USAID will use to collect and report data on each specific  element.  USAID stated that the guidance would refer back to its policies  and procedures as appropriate and supplement them as needed to  ensure that responsibilities and procedures are adequately defined. We  believe that issuing such guidance would address the recommendation  and help to ensure that USAID possesses the capability to collect and  report statutorily required information.
We are sending copies of this report to appropriate congressional  committees, the Secretary of Defense, the Chairman of the Joint Chiefs of  Staff, the Secretary of State, and the Administrator of the U.S. Agency for  International Development. In addition, the report is available at no  charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-5431 or russellc@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine the extent to which the Department of Defense (DOD) and  United States Agency for International Development (USAID) have  assessed resources needed to sustain the systems used to track  contracts and contractor personnel, we reviewed and compared agencies’  funding information, cost estimates, systems plans and schedule for  DOD’s Synchronized Predeployment and Operational Tracker–Enterprise  Suite (SPOT-ES) and USAID’S Global Acquisition and Award System  (GLAAS) to accepted cost-estimating guidance and internal- control  standards. We did not review resources State needs because State uses  SPOT-ES and does not contribute any funds to its operation or  development. We limited our review of funding information for SPOT-ES  and GLAAS to fiscal years 2013 through 2015 because SPOT-ES’s  current program office assumed full operational and management control  of SPOT-ES in fiscal year 2013 and GLAAS became operational at all  USAID offices worldwide in fiscal year 2013. Also, we defined resources  to include the costs of maintaining and updating each system. For  GLAAS, we reviewed funding information provided by the system’s  program office, funding estimates found in the President’s Budget  Requests, and Office of Management and Budget information on  informational-technology spending. We acquired and reviewed funding  information for SPOT-ES from the DOD Comptroller, the program office,  and the President’s Budget Requests. We accepted values as reported  by the Office of the Under Secretary of Defense (Comptroller). For  USAID, we compared GLAAS’s business cases for fiscal years 2013 and  2014, GLAAS’s Earned Value Management Metrics, and GLAAS’s cost  estimates for its modernization projects and operational activities to  determine how costs were assessed or adjusted. For DOD, we reviewed  and analyzed SPOT-ES schedules for upgrades, the program acquisition  baseline, and the SPOT-ES fiscal year 2013 business case to determine  whether costs were assessed and estimates were updated. We also  interviewed officials from the agencies’ system program offices about how  they developed, and decided whether to update, cost estimates.
To determine the extent to which DOD has developed business rules and  processes to ensure the timeliness and reliability of data, we obtained  and reviewed documents on system usage and business rules and user  guides for SPOT-ES. These included: the January and November 2014  versions of DOD’s Business Rules for SPOT-ES; DOD’s User Guides for  SPOT, the Joint Asset Movement Management System (JAMMS), and  the Total Operational Picture Support System (TOPSS); and State’s and  USAID’s Business Rules for SPOT-ES. We sent a structured set of  questions on data reliability and use and evaluated the responses. These  responses included details about data quality checks that the program  office performed. We also sent a series of requests for specific data from  both SPOT and JAMMS records to the SPOT-ES program management  office during August and September 2014, and received data pulls based  on those requests in installments from September through November  2014. We interviewed officials at the three agencies about their efforts to  improve SPOT-ES timeliness and reliability. We determined that the data  were not reliable for determining exact numbers of contractor personnel  or their exact locations at a point in time. However, we used them  primarily to illustrate that data accuracy depended on contractors and  contracting officers entering data according to the business rules, and  secondarily to provide approximate contractor personnel totals, and they  were sufficiently reliable for those purposes. At DOD, we interviewed  officials in the Office of the Under Secretary of Defense for Acquisition,  Technology and Logistics (Deputy Assistant Secretary of Defense for  Program Support and Directorate for Defense Procurement and  Acquisition Policy); at the Joint Staff, Directorates for Manpower and  Personnel (J-1) and Logistics (J-4); and U.S. Central Command  headquarters and U.S. Forces-Afghanistan. We also circulated questions  to officials at the other five geographic combatant commands about their  experiences with SPOT-ES, particularly with SPOT and JAMMS, and  analyzed their responses. U.S. Africa Command did not provide written  responses but addressed some of the questions during other meetings  related to operational contract support. At State, we met with officials from  the Offices of Logistics Management and Acquisitions Management in  Arlington, Virginia. In the interest of producing an unclassified report, we  excluded the classified versions of both SPOT and TOPSS from our  scope.
To determine the extent to which DOD has completed interoperability  testing and registered and approved SPOT-ES data, we reviewed DOD  guidance on sharing data and information in the department, as well as  guidance on interoperability. We reviewed relevant documents, including  DOD’s Defense Information Systems Agency Joint Interoperability  Certification of SPOT-ES; Interim Certificates to Operate granted to  SPOT-ES by the Interoperability Steering Group; the DOD Data Services  Environment (DSE) Concept of Operations; and SPOT-ES’s Tailored  Information Support Plan and DSE profile. We obtained access to DOD’s  DSE and reviewed SPOT-ES’s profile to identify whether the program  office had completed all appropriate steps to register the system’s data in  the DSE: this included comparing SPOT-ES’s profile against another  profile for an approved authoritative data source. Also, we interviewed  DOD officials from the SPOT-ES program office about the system’s  capabilities; and conducted a telephone interview with officials from the  DOD Defense Information Systems Agency’s DSE to discuss SPOT-ES’s  profile and confirm what requirements were and were not completed for  the system in the DSE. We also obtained and reviewed responses  provided by officials with the Defense Information Systems Agency’s DSE  regarding SPOT-ES’s DSE profile and completion of requirements in the  registry; and responses provided by the SPOT-ES program office  regarding technical information on the system.
We also circulated a standard set of questions to DOD and USAID on the  systems used to track contract and contractor personnel data and  analyzed the results, and determined that the information was sufficiently  reliable for the purposes for which we used it. That is, we determined that  DOD collects and reports on contract and contractor personnel data and  USAID collects and reports on contract data through its own system. We  did not verify figures about the total value of contracts, total number of  contractor personnel, or other attributes of the data. The standard set of  questions we circulated to DOD and USAID asked detailed and technical  questions about the systems. For example, for system architecture we  asked about system interfaces and for DOD to identify the systems that  provide data to SPOT-ES. Similarly, we asked USAID to identify what  systems interface with GLAAS. We also asked about data quality controls  and limitations: for example, we asked DOD and USAID to provide their  perception regarding the data quality of their respective systems for  collecting and reporting contract and contractor personnel data. In the  case of USAID, we also asked for its perception regarding the data  reliability of DOD’s SPOT. We collected responses from DOD and USAID  regarding their contract and contractor personnel data collection systems  and conducted follow-up on their responses when needed.
To determine the extent to which agencies have developed guidance to  meet statutory data collection and reporting requirements related to  contract support for future contingency operations, we obtained and  reviewed documents, including relevant provisions in State’s Foreign  Affairs Manual; USAID’s Automated Directives System and Acquisition  and Assistance Policy Directives; and DOD Instruction 3020.41,  Operational Contract Support. We analyzed these provisions to determine  whether they addressed each of the eight specific data elements related  to contracts and contractor personnel that are in section 844 of the  National Defense Authorization Act for Fiscal Year 2013. We also  interviewed officials to learn about how they related departmental or  agency guidance to statutory requirements. At DOD, these included:  officials at the Office of the Under Secretary of Defense for Acquisition,  Technology and Logistics (Program Support); deputy director of the Joint  Staff for Logistics; and SPOT-ES program office. At State, we met with  officials from the Offices of Logistics Management and Acquisitions  Management in Arlington, Virginia; at USAID, we met with officials from  the Bureau of Management (Office of Management Policy, Budget and  Performance).
We conducted this performance audit from April 2014 to February 2015 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: Comments from the U.S. Agency for International Development
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, the following individuals made  contributions to this report: Carole F. Coffey, Assistant Director (retired);  Tim DiNapoli; Jamarla Edwards; Rebecca Guerrero; Michael Holland;  Amie Steele Lesser; Sally Newman; Richard Powelson; Michael  Shaughnessy; and Michael Silver.