Contingency Contracting: Contractor Personnel Tracking System Needs Better Plans and Guidance
Released: 2015-02-18T00:00:00

Letter

Background

SPOT-ES comprises five systems: unclassified and classified versions of the SPOT database, unclassified and classified versions of the Total Operational Picture Support System (TOPSS) analytic and reporting tool, and the JAMMS personnel-location tracking tool. Figure 1 illustrates the five systems.

DOD guidance provides that, in applicable contingency operations, contractor visibility and accountability shall be maintained through a common joint database—SPOT, or its successor.

The SPOT database contains information about contracts—such as company name, contract number, and task order if any—and information about contractor personnel, such as contact information and next of kin, blood type, and the government-furnished support to which they are entitled. Government contracting officers use the information in SPOT to generate letters of authorization, which contractor personnel must obtain and carry in order to process through a deployment center or to travel to, from, or within the designated operational area. The letter of authorization also identifies any additional authorizations, privileges, or government support that contractor personnel are entitled to under the contract. Examples of such services could include access to dining facilities, transportation, or medical care beyond emergency treatment. The letter of authorization also identifies contractor personnel whose contracts permit eligible personnel to carry weapons, although arming approval rests with the combatant command to which personnel deploy.

DOD guidance indicates that the department intends to use SPOT to facilitate integration of contingency contractors and other personnel as directed by the Under Secretary of Defense for Acquisition, Technology and Logistics or the combatant commander, and to ensure that accountability, visibility, force protection, medical support, personnel recovery, and other related support can be accurately forecasted and provided. According to the guidance, SPOT data elements are intended to provide planners and combatant commanders an awareness of the nature, extent, and potential risks and capabilities associated with contracted support.

TOPSS is the reporting and analysis component of SPOT-ES. This tool generates a variety of standard reports that provide information on users, accounting and compliance, specific contracts or task orders, contractor deployments, and specific individuals. In addition, this tool can provide summary-level data in geospatial format.

The JAMMS movement tracker is an information technology application developed to capture movement and location information about contractor personnel in specified operational theaters. Also, this tracker can capture information about operating forces and government civilians. JAMMS data-collection points are established at locations such as: dining facilities, aerial ports of debarkation, and medical locations; and at U.S. embassies and other State locations. JAMMS terminals can scan a wide range of identification credentials, such as common access cards, SPOT- generated letters of authorization, and some driver’s licenses and passports. These credentials provide identity information about the cardholder, which is retained and made available to integration partners.

Global Acquisition and Assistance System (GLAAS)

GLAAS is USAID’s worldwide web-based procurement system that manages awards throughout USAID’s acquisition and assistance life cycle. GLAAS adapted commercial off-the-shelf software to accommodate USAID’s procurement-management needs. GLAAS integrates with the USAID financial-management system and other external government systems to provide reports to the Office of Management and Budget, Congress, and other stakeholders. GLAAS also provides data, such as award value and whether awards were competed, for reports to Congress on contract support for contingency operations outside the United States. GLAAS contained information on approximately 19,000 awards in fiscal year 2013, of which 382 contracts were related to contingency operations in Iraq and Afghanistan. GLAAS is an independent system that does not interoperate with any DOD systems, including SPOT-ES.

USAID has developed business rules and processes to explain to users how to operate GLAAS. The GLAAS user guides illustrate USAID’s acquisition and award process and instruct users on how to complete various tasks such as procurement planning, and creating and modifying purchase orders, solicitations, and assistance awards. USAID also supplements the GLAAS user guides with training sessions and other reference guides.

Federal Procurement Data System

The Federal Procurement Data System (FPDS) provides a comprehensive web-based tool for agencies to report information related to contracts. Executive agencies are to use FPDS to maintain publicly available information about all unclassified contract actions exceeding certain monetary thresholds. Generally, contracting officers must submit complete and accurate contract information to FPDS within 3 business days after contract award. Agencies can report data to FPDS using either of two methods: an Internet web portal or by means of contract- writing systems. DOD, State, and USAID report to FPDS through their respective contract-writing systems—Standard Procurement System, Global Financial Management System, and GLAAS.

Data Services Environment

According to DOD, the Data Services Environment (DSE) is DOD’s primary resource for registering, sharing, and publishing different types of metadata about systems, services, and data resources, such as authoritative data sources, to support DOD and the needs of all authorized users. DOD components, according to DOD guidance, must register all authoritative data sources, information technology services, and required metadata in the DSE. DOD guidance also provides policy that data will be made visible and trusted, among other things, for all authorized users. According to the guidance, data is made visible by creating and associating metadata. Data is considered trusted when there is sufficient pedigree (source and lineage) and descriptive metadata for users to rely on it as an authoritative data source. According to the DOD DSE Concept of Operations, once a system is registered in the DSE, the appropriate authoritative body is to review the data source and either approve or not approve the source as an authoritative data source. The DSE is linked to DOD’s broader efforts for managing data in a net- centric environment, the key attributes of which include ensuring that data are visible, among others things; associating all data with metadata; and posting all data to shared space to provide access to users except when limited by security, policy, or regulation.

Section 2222 of Title 10, U.S. Code, and SPOT-ES Certification with Relevant Requirements

Section 2222 of Title 10, U.S. Code, contains provisions regarding DOD development of a defense Business Enterprise Architecture (BEA), associated enterprise transition plan, and investment management structures and review processes. Among other requirements, section 2222 prohibits DOD from obligating funds for a defense business system program with a total cost in excess of $1 million over the period of the future years defense program unless the precertification authority certifies that the defense business system meets specified conditions. Specifically, the appropriate precertification authority must determine that the defense business system program is in compliance with the defense BEA and that appropriate business process reengineering efforts have been undertaken.

For fiscal years 2013 through 2015, the precertification authority for SPOT-ES determined that the system was in compliance with the BEA and that appropriate business process reengineering efforts were undertaken. Subsequent Office of the Deputy Chief Management Officer investment decision memorandums from the Defense Business Systems Management Committee certified SPOT-ES funds. We have previously reported on the need to improve the quality of BEA compliance assertions. In May 2013, we found that compliance assertions continued to lack adequate validation; we recommended that DOD implement and use the BEA compliance assessments more effectively to support organizational transformation efforts by, among other things, establishing milestones by which selected validations of BEA compliance assertions are to be completed. DOD partially agreed with this recommendation. In our follow-up report in May 2014, we found that DOD needed to continue working to ensure the quality of the BEA assessments, as we had previously recommended.

Our Prior Work on Contractor Management Systems

We have issued several recent reports about systems that DOD, State, and USAID use to manage information regarding contracts and contractor personnel and to prepare statutorily required reports on contracts, assistance instruments, and related personnel in Iraq and Afghanistan.

In September 2012, we found that although SPOT was designated as the common database for the statutorily required information, officials from DOD, State, and USAID generally relied on other data sources they regarded as more reliable to prepare the 2011 joint report to congressional committees. We also found that the agencies generally did not use SPOT to help manage, oversee, and coordinate contracting in Iraq and Afghanistan. Instead, we found that the agencies primarily used the system to generate authorizations for contractor personnel to use U.S. government services. We recommended that the Secretaries of Defense and State and the Administrator of USAID work together to standardize the methodologies used to obtain and present information contained in the annual joint report on contracting in Iraq and Afghanistan to the greatest extent possible. The agencies agreed with our recommendation and indicated they would work together to implement it.

Additionally, in February 2014, we reported that State and USAID had taken, or were planning to take, a number of actions to better track the number of contracts and contractor personnel in contingency environments. For example, we reported that, according to State officials, State developed additional guidance in fall 2012 that outlined the process of inputting contracts, how contractors should enter contractor personnel, how to request letters of authorization and approvals, and how to enter data through the contract close-out. We reported that, according to the department, State also established a new office that will be responsible for overseeing contractors’ input of Iraq contractors’ data into SPOT. We also reported that USAID was in the early stages of developing a proposal to use SPOT solely as a tool to track contractor personnel in contingency environments rather than the number and value of contracts. USAID officials stated that other data systems, such as FPDS-NG and GLAAS, provided more reliable information on the number and value of contracts. We recommended that the Secretary of State develop plans to assess whether planned initiatives are achieving their intended objectives and that the Administrator of USAID ensure that its nonpermissive working group consider procedures and practices developed by missions and offices with contingency-related responsibilities. The agencies concurred with our recommendations and provided information on actions taken or planned to address them.

USAID Has Assessed Resources Needed to Sustain GLAAS, but DOD Has Not Fully Assessed All Resources Needed to Sustain SPOT-ES

USAID has assessed resources that it needs to sustain GLAAS, but DOD has not fully assessed all future resources that it needs to sustain SPOT- ES. DOD and USAID use the budget process to identify resources they project they will need in the next budget year to modernize and operate their systems. USAID assessed its plans and updated GLAAS’s cost estimates with the actual costs of system upgrades and the latest cost estimates for future upgrades; however, DOD has not updated its life- cycle cost estimates or fully defined and assessed its plans to determine all resources needed to sustain SPOT-ES.

USAID Has Assessed Resources Needed to Sustain GLAAS

USAID has assessed the resources it needs to sustain GLAAS and regularly updates GLAAS’s life-cycle cost estimate to reflect its plans. USAID uses historical cost data and cost estimates to develop the annual funding request to operate, maintain, and modernize GLAAS. Table 1 shows GLAAS’s cost estimates and funds that USAID requested for the operation and development of GLAAS from fiscal years 2013 through 2015 and amounts USAID identified as funded in fiscal years 2013 and 2014.

We found that USAID assessed its plans and associated costs for modernizing GLAAS and has regularly updated its life-cycle cost estimate. Our review of USAID’s business cases for GLAAS’s modernization projects found that, in 2012 and 2013, USAID estimated the costs to implement system upgrades. For the upgrade completed in 2013, USAID assessed additional costs and analyzed the effect on project scope of a 3-month delay in releasing a system upgrade, known as version 7.1. In addition, USAID regularly updated GLAAS’s life-cycle cost estimate, which was originally calculated in 2009. Officials said they update cost estimates for the following budget year based on prior actual costs and upcoming projects. For example, in July and September 2014 USAID updated the life-cycle cost estimate with actual costs, resulting in a decrease in the projected costs to operate and maintain GLAAS from fiscal year 2015 through 2020. Further, USAID updated the modernization estimates of the life-cycle estimate for fiscal year 2015 through 2016 in September 2014.

DOD Has Not Updated Its Cost Estimates or Fully Defined and Assessed Its Plans for SPOT-ES

DOD has identified resources for fiscal years 2013 through 2015 that it needs to operate SPOT-ES; however, since 2010, DOD has not updated its life-cycle cost estimate or fully defined and assessed its plans to identify all the resources it needs to achieve the system’s objectives. GAO’s Cost Estimating and Assessment Guide states that approved cost estimates are often used to create budget spending plans and they should be updated with actual costs so that they are always relevant and current. In addition, Standards for Internal Control in the Federal Government state that management must continually assess and evaluate its plans to ensure that the control activities being used are effective and updated when necessary.to the defense acquisition system states that effective life-cycle sustainment for information systems requires continuous monitoring to Further, DOD guidance related ensure investments are maintained at the right size, cost, and condition, to support business missions and objectives.

According to Under Secretary of Defense (Comptroller) figures, the SPOT-ES program received about $27.1 million to operate and maintain its system and $2.8 million to develop and modernize in fiscal year 2014. Table 2 shows the SPOT-ES cost estimates, and the funds that DOD officials identified as requested for fiscal years 2013 through 2015 and the amount they identified as funded for fiscal years 2013 and 2014.

DOD prepared a SPOT-ES life-cycle cost estimate in 2010; however, the department has not updated the SPOT-ES life-cycle cost estimate since then to reflect any changes in costs due to schedule delays or program changes. The SPOT-ES life-cycle cost estimate projected that SPOT and TOPSS would undergo upgrades that were not completed or not completed as scheduled. For example, program officials projected that TOPSS versions 1.3 to 1.7 upgrades would be completed in fiscal year 2011, but as of October 2014, a subversion of 1.1 was the latest upgrade; however, SPOT-ES program officials have not updated the life-cycle cost estimate to reflect the costs of implementing the upgrades in the future. Figure 2 illustrates SPOT-ES’s schedule delays.

In addition, DOD has not fully defined and assessed its plans for SPOT- ES to ensure a comprehensive life-cycle cost estimate, representative of all resources it needs to sustain the system. GAO’s Cost Estimating and Assessment Guide states that a comprehensive life-cycle cost estimate should include both government and contractor costs of the program over its full life cycle, from inception of the program through retirement of the program. The estimate should also completely define the program, reflect the current schedule, be technically reasonable, and be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted. However, DOD has not defined some of its plans that involve cost elements that need to be incorporated into SPOT-ES life- cycle cost estimate. For example, SPOT-ES program officials identified the need to network and to further develop the JAMMS movement tracking application that feeds movement data to other SPOT-ES systems. SPOT-ES program officials had scheduled to network JAMMS to the SPOT database in the JAMMS version 4.0 release in fiscal year 2011, but as of November 2014, had not done so, as figure 2 illustrates. SPOT-ES program officials prepared a concept paper in September 2014 that proposed ideas for JAMMS’ further development; however, the paper does not define the scope of capabilities for a networked JAMMS. Further, even though the SPOT-ES life-cycle estimate reflected the costs of networking JAMMS in fiscal year 2011, it has not been updated to include the plans for networking or developing JAMMS in future years.

Officials said they have not updated the life-cycle cost estimate since 2010 because the system has proven to be stable, and they will update the estimate to reflect additional development and modernization funds they may require to improve the system. However, while the system may be stable, the cost elements and assumptions that DOD used to develop the life-cycle cost estimate have changed. Specifically, the host site supporters and the contractor that provided software maintenance for SPOT-ES have changed. SPOT-ES program officials had projected operation and maintenance life-cycle costs based on the actual costs of the previous host site supporters and the actual costs of software maintenance under the previous contract. For example, the life-cycle cost estimate projected an average yearly software maintenance cost of $16 million, but the software maintenance costs for fiscal years 2013 and 2014 were $2.7 million and $2.5, million respectively. Further, SPOT-ES program officials had requested additional development and modernization funds for fiscal years 2013 and 2014 when compared to the 2010 cost estimate, but they did not update the life-cycle cost estimate.

SPOT-ES program officials did not update the life cycle cost estimate or fully assess and define plans to ensure a comprehensive and accurate cost estimate because they accepted the system’s previous program- management estimates as reported. Further, the Under Secretary of Defense for Acquisition, Technology and Logistics and the Under Secretary of Defense for Personnel and Readiness, whose offices both have oversight responsibilities for the SPOT-ES program, have not ensured that the current program management assess and define all plans for the system’s further development or update the life-cycle cost estimate. According to GAO’s Cost Estimating and Assessment Guide, a life-cycle cost estimate should encompass all past (or sunk), present, and future costs for every aspect of the program, regardless of funding source, including all government and contractor costs. Without defining and assessing plans to provide a full accounting for the system, thereby fully accounting for life-cycle costs, management will have difficulty planning program resource requirements and making decisions. Also, to ensure accuracy, GAO’s Cost Estimating and Assessment Guide, states that cost estimates should be updated regularly to reflect significant changes in the program—such as when schedules or other assumptions change—and actual costs, so that it is always reflecting current status. If the estimate is not regularly updated, it will be difficult to analyze changes in program costs, and collecting cost and technical data to support future estimates will be hindered. Moreover, the cost estimate cannot provide decision makers with accurate information for assessing alternative decisions.

DOD Has Developed Business Rules, but Lacks Reasonable Assurance SPOT and JAMMS Data Are Timely and Reliable

DOD has developed business rules governing data entry about contracts and contractor personnel into SPOT. However, DOD cannot assure that SPOT or JAMMS—which provides data to SPOT—provide either contract or contractor personnel data that are consistently timely and reliable.

DOD Has Developed Business Rules for Entering Data about Contracts and Contractor Personnel, but Lacks Reasonable Assurance that SPOT is Timely and Reliable

DOD has developed SPOT business rules for entering data about contracts and contractor personnel; however, DOD’s process does not provide reasonable assurance that the business rules are followed and SPOT data are timely and reliable. In the context of operational contract support, various provisions in DOD guidance and the Defense Federal Acquisition Regulation Supplement (DFARS) relate to the accuracy and timeliness of information in SPOT. For example:

DOD guidance provides that contracting officers, through the terms of contracts, shall require contractors to enter data before employee deployment and to maintain and update the information for relevant employees.

An applicable DFARS clause specifies that the contractor shall use SPOT to enter and maintain data for all contractors authorized to accompany the force and, as designated, other personnel. The contractor is to enter the required information about its personnel prior to deployment and to continue to use SPOT to maintain accurate, up- to-date information throughout the deployment. Under the clause, changes to personnel status relating to arrival date and duty location, including closing out deployment with proper status (such as mission complete, killed, wounded) must be annotated in SPOT in accordance with timelines established in SPOT business rules.

Defense Manpower Data Center, DOD Business Rules for the Synchronized Predeployment and Operational Tracker (SPOT) (Jan. 21, 2014); and Defense Manpower Data Center, DOD Business Rules for the Synchronized Predeployment and Operational Tracker (SPOT) (Nov. 21, 2014). arrived at the primary duty station.they did not know how often company administrators entered such information within a day. However, they reported that nearly 4,000 contractors with active deployments on active contracts lacked in- theater arrival dates in SPOT at least 7 days after the scheduled date, as of September 2014.

SPOT program officials said that

Armed Contractor Personnel—The SPOT business rules indicate that company administrators are to enter information into SPOT on equipment used by their personnel who perform security functions, such as the serial numbers of each weapon issued specifically to an individual. However, program officials told us that they sometimes could not link weapon serial numbers to individual contractor personnel because companies did not always assign specific weapons to specific individuals. As a result of our inquiries, program officials also discovered and corrected a software deficiency in SPOT wherein no serial numbers were displayed if personnel had more than one assigned weapon.

In addition, we found that the business rules may be insufficient to allow contractors to provide accurate information on foreign contractor personnel. According to the business rules, for contractor personnel who lack a foreign identification number, the contractor must create a number according to instructions in the SPOT User Guide. For U.S. Central Command, in the case of Afghanistan or non-Iraqi citizens, the guide directs personnel to use the first five letters of the last name, plus date of birth in the format “mmddyyyy.” However, SPOT program officials found that some individuals lack birth certificates and do not know their birth date. Out of about 769,000 foreign nationals in SPOT, 213,348 have a birth date of January 1 recorded in SPOT, and about 100,000 of these have identical surname and birth date information. This has created challenges in situations where people have identical names, according to DOD officials, including officials from Pacific Command. DOD has developed a process to begin assigning unique identification numbers, but has not made corresponding changes to the user guide to show how the process will apply to U.S. Central Command.

DOD has established a goal for SPOT of 85 percent accuracy. In 2011, DOD directed contracting activities in the Central Command area of responsibility to continue conducting a quarterly manual census, or physical count of contractor personnel, until SPOT contains at least 85 percent of the data revealed by the manual count. DOD officials have conducted this census at Central Command, where most contractors who are now required to register in SPOT are located, every quarter since 2008. While some contracting activities have recently exceeded that threshold—for example, our review of census data for the third quarter of fiscal year 2014 found that 69 out of 102 contracting activities had exceeded 85 percent—officials said that they would continue to conduct the census until all activities did so.

DOD’s process also cannot provide reasonable assurance that contractors and contracting officers enter data into SPOT according to the business rules for two primary reasons. First, the department does not use its available mechanisms for tracking contractor performance to promote contractor accountability for entering data correctly and within prescribed time frames into SPOT. These mechanisms include the Contractor Performance Assessment Reporting System (CPARS), which contracting officers can use to report on contractors’ performance. Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics officials said that they thought using such mechanisms would be helpful. Second, Office of the Under Secretary of Defense for Acquisition, Technology and Logistics officials said that contracting officers sometimes did not oversee contractors’ data input or enter their own required data.officers in its Operational Contract Support Action Plan for Fiscal Years 2014-2017, which calls for the military departments to issue policy mandating that their contracting activities populate SPOT with required data.

DOD has taken steps that may address contracting By not having timely and reliable contract and contractor personnel data, DOD does not have complete visibility into the number of contractors present in the contingency environment, and for whom the department may have to develop plans for such services as force protection, sustenance, and repatriation of injured or deceased contractor personnel. For example, one official at a DOD combatant command stated that DOD can more quickly repatriate injured or deceased contractors when their information is up-to-date and accurate in SPOT. Not using the mechanisms it already has in place to track contractor performance limits DOD’s assurance that contractors have abided by business rules to provide timely and reliable data, and requires DOD officials to devote additional resources to conducting the quarterly manual census.

SPOT-ES Does Not Receive Timely and Reliable Data from JAMMS Terminals about the Location of Contractor Personnel

JAMMS is a major data source for SPOT-ES to track contractor personnel in key deployment sites such as dining halls and military airfields; however, SPOT-ES does not receive timely and reliable data from JAMMS. DOD policy states that information solutions shall provide, among other things, reliable, timely, and accurate information. Moreover, DOD has noted that the purpose of SPOT is to provide the combatant commander with accurate, real-time information on all personnel within specified geographic combatant command operations areas and to enable the department to keep track of all persons deployed in contingency zones.

We found examples in which deployed contractors provided three types of unreliable data to JAMMS: not scanning at all, scanning incorrect documents, or scanning documents into a terminal that reported back to SPOT-ES as if it was in a different location. First, a review of data at one deployed operational area from August 2013 through March 2014 found that contractor scans in the week preceding the date the data were pulled represented less than half of the contractors believed to be in the area. Second, JAMMS terminals can scan incorrect documents because they can accept bar-coded items that are not identification documents, such as supermarket loyalty cards, and other identification documents, such as letters of authorization, that do not contain photos. For example, in the last two weeks of September 2014, there were almost 15,000 JAMMS scans in Afghanistan that could not be linked to either a deployment in SPOT or a Defense Enrollment Eligibility Reporting System record. Third, officials told us that they sometimes receive erroneous data from JAMMS terminals about the locations of contractor personnel. For example, officials at Central Command said that they have received reports that contractor personnel were at sites that they never visited. We also reviewed documentation about one site in Afghanistan that reported that more than 20 percent of records represented individuals who either never scanned their identification documents or whose most recent scan was in a country other than Afghanistan.

We also found that JAMMS data may not always be transmitted to SPOT in a timely manner. Although DOD officials have indicated an intention to develop a networking capability that would allow JAMMS data to be transmitted to SPOT in real time, data are currently transmitted by uploading compact discs from stand-alone JAMMS terminals to networked computers. The JAMMS User Manual and the System of Record Notice for SPOT-ES indicate that JAMMS uploads personnel movement records to SPOT-ES daily. However, personnel at operating locations vary in the frequency with which they upload JAMMS location data into SPOT. For example, SPOT program office officials told us that data usually are uploaded every several days. In contrast, officials at Central Command said their data are uploaded about every week, but that at one site, data for 20 percent of contractor personnel were not uploaded for 30 days.

Personnel at operating locations are not consistently ensuring that JAMMS data are transmitted accurately or in a timely manner into SPOT- ES because they lack comprehensive guidance from DOD that describes the purpose of JAMMS and its role in supporting plans for different types of missions. The lack of guidance stems from lack of consensus about requirements for visibility and accountability—such as what constitutes “near real-time” movement tracking—within DOD. Officials at the Joint Staff and in combatant commands told us that in the absence of such guidance, they do not know what resources or emphasis to allocate to correcting problems that they encounter. For example, they could minimize inaccurate scans by posting a monitor at each JAMMS terminal to verify that everyone scans a valid identification document, but such an action has resource implications that could conflict with other command priorities. Without clear guidance about the purpose of JAMMS, such as direction about what types of missions it is to support, the department is not able to help Joint Staff and combatant command planners to determine where to locate JAMMS terminals and what resources to allocate to minimize inaccurate scans. For example, combatant commanders would need more precise data if they expected to use JAMMS to plan a short-notice operation such as a noncombatant evacuation than they would to calculate how much to bill contractors for their employees’ use of on-base dining or medical facilities. Such guidance could also assist program management officials as they develop cost estimates and associated plans, as more frequent data uploads or the potential networking capability described earlier would be more applicable to some missions than others.

DOD Has Completed Interoperability Certification Testing but Has Not Fully Registered or Approved SPOT-ES Data

DOD has completed interoperability certification testing between SPOT- ES and its data sources, but has not fully registered or approved the system’s data to ensure data are visible and trusted.

DOD Has Completed Interoperability Certification Testing for SPOT-ES

DOD’s SPOT-ES program office completed interoperability certification testing of SPOT-ES with the systems that provide it with data. In March 2014, the Defense Information Systems Agency Joint Interoperability Test Command certified that the latest version of SPOT-ES meets all joint critical interoperability requirements, such as supporting military operations and effectively exchanging information with other systems. Our review of documentation for SPOT-ES found that the system began as a rapid capability (i.e., a capability to be delivered quickly), was conditionally approved for full deployment in November 2010, and did not receive joint interoperability certification until March 2014. In addition, we found that three different DOD offices provided support to SPOT-ES before the Defense Manpower Data Center assumed management and operation of the system in November 2013. However, DOD’s SPOT-ES program-management office completed SPOT-ES interoperability certification testing and associated steps, such as developing and submitting an Information Support Plan for DOD approval.

DOD Has Not Fully Registered or Approved SPOT-ES Data

The SPOT-ES program office has not ensured that the system’s data are visible and trusted because it has not fully registered the system’s data in the Data Services Environment (DSE). In May 2003, DOD’s Chief Information Officer issued a memorandum that provided guidance for managing data in a net-centric environment and highlighted key attributes of the department’s net-centric strategy, such as ensuring that data are visible, among other things; all data are associated with metadata to enable discovery by users; and all data are posted to shared spaces to provide access to users except when limited by security, policy, or regulation. According to DOD, the DSE is DOD’s primary resource for registering, sharing, and publishing different types of metadata about systems, services, and data resources to support DOD operational capabilities and data standards and needs for all authorized users. DOD Instruction 8320.02 requires heads of DOD components to register all authoritative data sources, information technology services, and required metadata in the DSE and further states as policy that data will be made visible and trusted, among other things, for all authorized users.According to the guidance, data is made visible by creating and associating metadata. Data is considered trusted when there is sufficient pedigree and descriptive metadata for users to rely on it as an authoritative data source.related to authoritative data sources: (1) data collection, which supports the registration of new data needs, data producers, systems, and databases; (2) data association, which defines all the systems, data producers, and data needs that make up the proposed structure of the authoritative data source; and (3) authoritative data source approval, which involves the authoritative body reviewing and making a determination to approve or not approve the proposed authoritative data source. Figure 3 provides a high-level overview of the key processes related to authoritative data sources.

Department of Defense, Deputy Under Secretary of Defense for Logistics and Materiel Readiness and Deputy Under Secretary of Defense for Program Integration, “Designation of Synchronized Predeployment and Operational Tracker (SPOT) as Central Repository for Information on Contractors Deploying with the Force (CDF),” memorandum (Washington, D.C.: Jan. 25, 2007).

State Has Issued Guidance to Meet Statutory Requirements, but DOD and USAID Guidance May Not Ensure Complete Data Collection

State issued guidance in February 2014 to address the statutory requirements for data collection on contract support for future contingency operations. Section 844 of the National Defense Authorization Act for Fiscal Year 2013 requires DOD, State, and USAID to each issue guidance regarding data collection on contract support for future contingency operations outside the United States that involve combat operations. The guidance is to ensure that each agency takes the steps necessary to possess the capability to collect and report on at least eight data elements that the statute enumerates. These elements are: total number of contracts entered into as of the date of any report, total number of such contracts that are active as of such date,total value of contracts entered into as of such date, total value of such contracts that are active as of such date, identification of the extent to which the contracts entered into as of such date were entered into using competitive procedures, total number of contractor personnel working under contracts entered into as of the end of each calendar quarter during the 1-year period ending on such date, total number of contractor personnel performing security functions under contracts entered into as of the end of each calendar quarter during the 1-year period ending on such date, and total number of contractor personnel killed or wounded under any contracts entered into.

We found that a provision in State’s Foreign Affairs Manual lists each element and stipulates which data source the department will use to gather information on each of the eight data elements. For example, State obtains data on contract value and the use of competitive procedures from FPDS-NG, according to the provision.

DOD published guidance about operational contract support in 2011, which officials in the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics told us meets the statutory requirement. However, unlike the State guidance, the DOD guidance does not clearly cover each data element listed in section 844 and may not ensure that the department has the capability to collect and report on the required data. This guidance identifies as DOD policy that contractor visibility and accountability shall be maintained in applicable contingency operations through a common joint database–SPOT or its successor. Also, the guidance provides some information on contract visibility and contractor accountability responsibilities and procedures, including responsibilities related to several of the data elements specified in section 844. For example, the guidance indicates that SPOT or its successor shall contain, or link to, minimum contract information, such as contract number, contract category, period of performance, contracting agency, and contracting office, necessary to establish and maintain accountability and visibility of certain contractor personnel, maintain information on specific equipment related to private security contracts, and the contract capabilities in relevant operations.

Operational Contract Support, 32 C.F.R. pt. 158; Department of Defense Instruction 3020.41, Operational Contract Support (OCS) (Dec. 20, 2011). collect information on certain contractor personnel, but it is not clear that it would ensure the collection of all personnel information required by section 844. For example, it is not clear whether the collection of employee data based on the guidance would ensure the identification of contractor personnel performing security functions or the number killed or wounded. DOD officials said that they have not updated their guidance to specifically address each data element and source because the 2011 instruction meets their needs, but commented that the instruction is undergoing a revision and it would be useful to update SPOT provisions during that process.

Without current and comprehensive guidance that identifies the data elements to collect, the systems with which to collect and report them, and relevant responsibilities and procedures, DOD lacks assurance that its 2011 guidance provides the department the ability to ensure that it can take the steps necessary to collect and report on the eight required data elements. Moreover, DOD may find it more difficult to reconcile information when there are multiple sources for the same data element. For example, SPOT receives some information about contract numbers and about whether a contract was awarded competitively both from the government-wide FPDS-NG and from contractors’ manual entry. Additionally, an update would also clarify current procedures to help ensure the collection of the total number and total value of contracts active as of a reporting date, as well as the number and value of contracts entered into as of the reporting date.

USAID has published guidance about the use of GLAAS and SPOT to record data about contracts and contractor personnel, but this guidance may also not ensure that the agency has the capability to collect and report on the required data. USAID officials told us in May 2014 that they were not aware of guidance to implement the statutory requirement, and our review of agency guidance revealed no provisions specifically tied to section 844. However, in November 2014, officials added that USAID believed that its current policy and procedures related to data collection on contract support for contingency operations were sufficient. According to a USAID management official, the agency reports on data related to number, value, and competition of contracts through the reporting functionality of GLAAS, but it is not clear that the corresponding guidance identified by the official would ensure the collection of each data element. The official also cited two Acquisition and Assistance Policy Directives (AAPD), dated in 2009 and 2010. These directives require USAID contracting officers to include a provision in certain contracts with performance in Iraq and Afghanistan that identifies SPOT as the required system to use for personnel data.contract provisions do not specifically address data collection on contractors who are killed or wounded, and do not address any future contingency operations beyond Iraq or Afghanistan. As with DOD, until USAID develops current and comprehensive guidance that identifies the data elements to collect, the systems with which to collect and report them, and relevant responsibilities and procedures, it cannot ensure that it is providing itself with the ability to take the steps necessary to collect and report on the eight required data elements.

Conclusions

DOD, State, and USAID expect to continue to rely on contractor personnel to augment military and civilian personnel as they did in recent operations in Iraq and Afghanistan. While SPOT-ES is a central repository in which the three agencies have stored information about nearly a million contractor personnel, the data system is not comprehensive and reliable. Additionally, DOD has not updated its life- cycle cost estimate with actual costs or to reflect changes in costs due to SPOT-ES schedule delays and program changes. Also, DOD has not fully defined and assessed some of its plans to incorporate as cost elements in its life-cycle cost estimate. Consistent with cost-estimating guidance and standards for internal control, without updating its cost estimates to include defined plans for SPOT-ES further development, DOD may not be able to fully determine future resources needed to sustain SPOT-ES. Ensuring that the SPOT-ES program updates its cost estimate to include defined program plans could help the program to better plan program resource requirements and make decisions.

DOD has developed business rules to enter data about contracts and contractor personnel in SPOT, but neither SPOT nor JAMMS can provide contractor personnel data that are consistently timely and reliable. Without a reasonable assurance of timely and reliable contract and contractor personnel data, DOD has incomplete visibility into the number of contractors present in the contingency environment, and for whom the department may have to develop plans for such services as force protection, sustenance, and repatriation of wounded or deceased contractor personnel. DOD has not used existing mechanisms for tracking contractor performance that could help provide reasonable assurance that contractors have abided by business rules to provide timely and reliable data, and relieve DOD officials of the need to continue to devote resources to conducting the quarterly manual census. Finally, DOD has not developed comprehensive guidance for JAMMS, which could provide the combatant commands with better information on how to allocate their resources to maximize JAMMS’ utility as a tracking and planning tool.

The SPOT-ES program office has completed interoperability certification testing for the system, but the program office did not fully register the system’s data or complete the steps required of authoritative data sources in the DSE. Full registration and approval of SPOT-ES data in the DSE would ensure data are visible and trusted and provide authorized users seeking authoritative data on contracts and contractor personnel confidence in the information, which may prevent the need to develop their own data and potential duplication of efforts. In addition, full registration and approval may preclude data producers from continuing to rely on individual data solutions for their systems rather than leveraging shared common data across the DOD enterprise.

State has issued guidance to address section 844 of the National Defense Authorization Act for Fiscal Year 2013, which requires DOD, State, and USAID to each issue guidance regarding data collection on contract support for future contingency operations. The guidance is to ensure that the agencies have the capability to collect and report at least DOD, however, issued guidance in 2011 that may eight data elements.not guarantee that the department takes the steps necessary to ensure the capability to collect and report on the eight required data elements, and USAID guidance may also not ensure complete data collection. For both DOD and USAID, including information on each of the eight elements in departmental guidance could better ensure the ability to collect and report on those elements, possibly including better timeliness and accuracy of data entry.

Recommendations for Executive Action

To help improve DOD, State, and USAID’s ability to track contracts and contractor personnel in contingency operations, we are making the following five recommendations to DOD:

To ensure SPOT-ES cost estimates are accurate and comprehensive, we recommend that the Under Secretary of Defense for Personnel and Readiness in coordination with the Under Secretary of Defense for Acquisition, Technology and Logistics direct the system’s program office to regularly update its life-cycle cost estimate to include defining and assessing its plans for SPOT-ES.

To help improve timeliness and reliability of data in SPOT-ES, the Secretary of Defense should direct Defense Procurement and Acquisition Policy officials, through the Under Secretary of Defense for Acquisition, Technology and Logistics, to ensure that contracting officers use available mechanisms to track contractor performance of SPOT data entry, such as its Contractor Performance Assessment Reporting System or other appropriate performance systems or databases.

To provide clarity about expectations for JAMMS that can help improve the timeliness and reliability of data for SPOT-ES from JAMMS uploads, the Secretary of Defense should direct the Chairman of the Joint Chiefs of Staff, in coordination with the combatant commanders, to develop comprehensive guidance regarding the purpose of JAMMS and its role in supporting plans for different types of missions. Such guidance could include direction on the number and location of JAMMS terminals and how frequently JAMMS’s data should be uploaded into SPOT-ES to meet DOD’s information needs.

To enhance the value of SPOT-ES data, we recommend that the Secretary of Defense direct the Under Secretary of Defense for Personnel and Readiness to fully register SPOT-ES data in the DSE to make data visible and trusted, including taking the necessary steps related to authoritative data sources.

To help ensure that DOD possesses the capability to collect and report statutorily required information and to clarify responsibilities and procedures, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics to update SPOT provisions during the process of updating operational contract support guidance.

In addition, we are making the following recommendation to USAID:

To help ensure that USAID possesses the capability to collect and report statutorily required information, the Administrator of USAID should issue current and comprehensive guidance regarding data collection on contract support for future contingency operations outside the United States that involve combat operations.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD, USAID, and State for their review and comment. DOD and USAID provided written comments, which are summarized below and reprinted in appendixes II and III, respectively. State did not provide comments on the report. DOD concurred with four of the five recommendations directed to it and partially concurred with the fifth. DOD also described actions underway or plans to address the recommendations. USAID agreed with the recommendation directed to it and described plans to address it.

DOD concurred with the first recommendation, that the SPOT-ES program office regularly update the system’s life-cycle cost estimate to include defining and assessing its plans for SPOT-ES. DOD stated that the Under Secretary of Defense for Personnel and Readiness would direct the system program office to regularly update its life-cycle cost estimate to include defining and assessing its plans for SPOT-ES. We believe that these actions, if fully implemented, would address the recommendation and better ensure that SPOT-ES cost estimates are accurate and comprehensive.

DOD concurred with the second recommendation, that the Secretary of Defense direct Defense Procurement and Acquisition Policy officials, through the Under Secretary of Defense for Acquisition, Technology and Logistics, to ensure that contracting officers use available mechanisms to track contractor performance of SPOT data entry, such as DOD’s Contractor Performance Assessment Reporting System or other appropriate performance systems or databases. DOD stated that the Under Secretary of Defense for Acquisition, Technology, and Logistics would direct Defense Procurement and Acquisition Policy officials to ensure that DOD contracting officers use available mechanisms to improve compliance with contract requirements for SPOT data entry. If DOD uses available mechanisms to improve compliance with contract requirements, that would address the intent of the recommendation and improve timeliness and reliability of data in SPOT-ES.

DOD partially concurred with the third recommendation, that the Secretary of Defense direct the Chairman of the Joint Chiefs of Staff, in coordination with the combatant commanders, to develop comprehensive guidance regarding the purpose of JAMMS and its role in supporting plans for different types of missions. Such guidance could include direction on the number and location of JAMMS terminals and how frequently JAMMS’s data should be uploaded into SPOT-ES to meet DOD’s information needs. DOD stated that it agreed to provide clarity regarding the purpose and use of JAMMS to improve the timeliness and reliability of JAMMS data, but it did not agree that such guidance could include direction on the number and location of JAMMS terminals and how frequently JAMMS’s data should be uploaded into SPOT-ES. DOD stated that it would revise language in DOD Instruction 3020.41, Operational Contract Support, to reflect in policy the requirement to use the entire SPOT Enterprise Suite (SPOT-ES), which includes JAMMS. DOD also stated that the combatant commander should establish the requirements for terminal quantities and locations and for data upload schedules based on operational needs in the relevant theater. We agree with DOD that the combatant commands need flexibility based on operational requirements. The recommendation was intended to allow for such flexibility by suggesting the types of direction that could be included in the guidance, which would facilitate consistency in the use of terminals and data across the commands. Further, as we recommended, this guidance should be developed in coordination with the combatant commanders.

DOD concurred with the fourth recommendation, that the Under Secretary of Defense for Personnel and Readiness fully register SPOT-ES data in the DSE to make data visible and trusted, including taking the necessary steps related to authoritative data sources. DOD stated that it agreed to complete the process of registering the system’s data, including validation of authoritative data sources. We believe that, if fully implemented, these steps would address the recommendation and enhance the value of the system’s data for authorized users seeking authoritative data on contracts and contractor personnel.

DOD concurred with the fifth recommendation, that the Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology and Logistics to update SPOT provisions during the process of updating operational contract support guidance. DOD stated that DOD Instruction 3020.41, Operational Contract Support, is scheduled to be republished in 2016 and will be updated to specifically identify the statutorily required data elements to ensure collection and reporting can be accomplished. We believe this action, if fully implemented, would address the recommendation and help to ensure that DOD possesses the capability to collect and report statutorily required information and to clarify responsibilities and procedures.

USAID agreed with the recommendation that the Administrator of USAID issue current and comprehensive guidance regarding data collection on contract support for future contingency operations outside the United States that involve combat operations. USAID stated that such guidance would explicitly describe the Section 844 statutory requirement, list each of the eight data elements required by Section 844, and stipulate the data source that USAID will use to collect and report data on each specific element. USAID stated that the guidance would refer back to its policies and procedures as appropriate and supplement them as needed to ensure that responsibilities and procedures are adequately defined. We believe that issuing such guidance would address the recommendation and help to ensure that USAID possesses the capability to collect and report statutorily required information.

We are sending copies of this report to appropriate congressional committees, the Secretary of Defense, the Chairman of the Joint Chiefs of Staff, the Secretary of State, and the Administrator of the U.S. Agency for International Development. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-5431 or russellc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Scope and Methodology

To determine the extent to which the Department of Defense (DOD) and United States Agency for International Development (USAID) have assessed resources needed to sustain the systems used to track contracts and contractor personnel, we reviewed and compared agencies’ funding information, cost estimates, systems plans and schedule for DOD’s Synchronized Predeployment and Operational Tracker–Enterprise Suite (SPOT-ES) and USAID’S Global Acquisition and Award System (GLAAS) to accepted cost-estimating guidance and internal- control standards. We did not review resources State needs because State uses SPOT-ES and does not contribute any funds to its operation or development. We limited our review of funding information for SPOT-ES and GLAAS to fiscal years 2013 through 2015 because SPOT-ES’s current program office assumed full operational and management control of SPOT-ES in fiscal year 2013 and GLAAS became operational at all USAID offices worldwide in fiscal year 2013. Also, we defined resources to include the costs of maintaining and updating each system. For GLAAS, we reviewed funding information provided by the system’s program office, funding estimates found in the President’s Budget Requests, and Office of Management and Budget information on informational-technology spending. We acquired and reviewed funding information for SPOT-ES from the DOD Comptroller, the program office, and the President’s Budget Requests. We accepted values as reported by the Office of the Under Secretary of Defense (Comptroller). For USAID, we compared GLAAS’s business cases for fiscal years 2013 and 2014, GLAAS’s Earned Value Management Metrics, and GLAAS’s cost estimates for its modernization projects and operational activities to determine how costs were assessed or adjusted. For DOD, we reviewed and analyzed SPOT-ES schedules for upgrades, the program acquisition baseline, and the SPOT-ES fiscal year 2013 business case to determine whether costs were assessed and estimates were updated. We also interviewed officials from the agencies’ system program offices about how they developed, and decided whether to update, cost estimates.

To determine the extent to which DOD has developed business rules and processes to ensure the timeliness and reliability of data, we obtained and reviewed documents on system usage and business rules and user guides for SPOT-ES. These included: the January and November 2014 versions of DOD’s Business Rules for SPOT-ES; DOD’s User Guides for SPOT, the Joint Asset Movement Management System (JAMMS), and the Total Operational Picture Support System (TOPSS); and State’s and USAID’s Business Rules for SPOT-ES. We sent a structured set of questions on data reliability and use and evaluated the responses. These responses included details about data quality checks that the program office performed. We also sent a series of requests for specific data from both SPOT and JAMMS records to the SPOT-ES program management office during August and September 2014, and received data pulls based on those requests in installments from September through November 2014. We interviewed officials at the three agencies about their efforts to improve SPOT-ES timeliness and reliability. We determined that the data were not reliable for determining exact numbers of contractor personnel or their exact locations at a point in time. However, we used them primarily to illustrate that data accuracy depended on contractors and contracting officers entering data according to the business rules, and secondarily to provide approximate contractor personnel totals, and they were sufficiently reliable for those purposes. At DOD, we interviewed officials in the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (Deputy Assistant Secretary of Defense for Program Support and Directorate for Defense Procurement and Acquisition Policy); at the Joint Staff, Directorates for Manpower and Personnel (J-1) and Logistics (J-4); and U.S. Central Command headquarters and U.S. Forces-Afghanistan. We also circulated questions to officials at the other five geographic combatant commands about their experiences with SPOT-ES, particularly with SPOT and JAMMS, and analyzed their responses. U.S. Africa Command did not provide written responses but addressed some of the questions during other meetings related to operational contract support. At State, we met with officials from the Offices of Logistics Management and Acquisitions Management in Arlington, Virginia. In the interest of producing an unclassified report, we excluded the classified versions of both SPOT and TOPSS from our scope.

To determine the extent to which DOD has completed interoperability testing and registered and approved SPOT-ES data, we reviewed DOD guidance on sharing data and information in the department, as well as guidance on interoperability. We reviewed relevant documents, including DOD’s Defense Information Systems Agency Joint Interoperability Certification of SPOT-ES; Interim Certificates to Operate granted to SPOT-ES by the Interoperability Steering Group; the DOD Data Services Environment (DSE) Concept of Operations; and SPOT-ES’s Tailored Information Support Plan and DSE profile. We obtained access to DOD’s DSE and reviewed SPOT-ES’s profile to identify whether the program office had completed all appropriate steps to register the system’s data in the DSE: this included comparing SPOT-ES’s profile against another profile for an approved authoritative data source. Also, we interviewed DOD officials from the SPOT-ES program office about the system’s capabilities; and conducted a telephone interview with officials from the DOD Defense Information Systems Agency’s DSE to discuss SPOT-ES’s profile and confirm what requirements were and were not completed for the system in the DSE. We also obtained and reviewed responses provided by officials with the Defense Information Systems Agency’s DSE regarding SPOT-ES’s DSE profile and completion of requirements in the registry; and responses provided by the SPOT-ES program office regarding technical information on the system.

We also circulated a standard set of questions to DOD and USAID on the systems used to track contract and contractor personnel data and analyzed the results, and determined that the information was sufficiently reliable for the purposes for which we used it. That is, we determined that DOD collects and reports on contract and contractor personnel data and USAID collects and reports on contract data through its own system. We did not verify figures about the total value of contracts, total number of contractor personnel, or other attributes of the data. The standard set of questions we circulated to DOD and USAID asked detailed and technical questions about the systems. For example, for system architecture we asked about system interfaces and for DOD to identify the systems that provide data to SPOT-ES. Similarly, we asked USAID to identify what systems interface with GLAAS. We also asked about data quality controls and limitations: for example, we asked DOD and USAID to provide their perception regarding the data quality of their respective systems for collecting and reporting contract and contractor personnel data. In the case of USAID, we also asked for its perception regarding the data reliability of DOD’s SPOT. We collected responses from DOD and USAID regarding their contract and contractor personnel data collection systems and conducted follow-up on their responses when needed.

To determine the extent to which agencies have developed guidance to meet statutory data collection and reporting requirements related to contract support for future contingency operations, we obtained and reviewed documents, including relevant provisions in State’s Foreign Affairs Manual; USAID’s Automated Directives System and Acquisition and Assistance Policy Directives; and DOD Instruction 3020.41, Operational Contract Support. We analyzed these provisions to determine whether they addressed each of the eight specific data elements related to contracts and contractor personnel that are in section 844 of the National Defense Authorization Act for Fiscal Year 2013. We also interviewed officials to learn about how they related departmental or agency guidance to statutory requirements. At DOD, these included: officials at the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (Program Support); deputy director of the Joint Staff for Logistics; and SPOT-ES program office. At State, we met with officials from the Offices of Logistics Management and Acquisitions Management in Arlington, Virginia; at USAID, we met with officials from the Bureau of Management (Office of Management Policy, Budget and Performance).

We conducted this performance audit from April 2014 to February 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: Comments from the U.S. Agency for International Development

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, the following individuals made contributions to this report: Carole F. Coffey, Assistant Director (retired); Tim DiNapoli; Jamarla Edwards; Rebecca Guerrero; Michael Holland; Amie Steele Lesser; Sally Newman; Richard Powelson; Michael Shaughnessy; and Michael Silver.