Military Personnel: DOD and the Services Need to Take Additional Steps to Improve Mobilization Data for the Reserve Components
Published: 2006-09-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: GAO has previously reported on the Department of Defense's (DOD) ability to track reservists deployed to the theater of operations and made recommendations. Reliable mobilization and deployment data are critical for making decisions about reserve force availability and medical surveillance. Because of broad congressional interest, GAO initiated a review under the Comptroller General's authority to conduct evaluations on his own initiative to determine (1) what DOD data indicate are the number of reservists mobilized and deployed in support of the Global War on Terrorism (GWOT) and the selected demographic and deployment characteristics of those deployed and (2) whether DOD's reserve deployment and mobilization data and analyses are reliable. GAO analyzed data and data analyses from DOD's Contingency Tracking System (CTS) and interviewed agency officials.
section_title: What GAO Found
paragraphs: GAO's analysis of DOD data indicates that more than 531,000 reservists have been mobilized in support of GWOT as of June 30, 2006, and more than 378,000 reservists, or 71 percent of the number mobilized, have been deployed. The number of reservists deployed increased through fiscal year 2003 and remained stable through fiscal year 2005. The majority of reservists have been deployed once. GAO's analysis further indicates that of the more than 378,000 reservists who have deployed in support of GWOT, 81 percent have spent a year or less deployed and 17 percent of reservists have spent more than 1 year but less than 2 years deployed. Of those who deployed, almost 98 percent were U.S. citizens. Since GWOT began, about 78 percent of reservists who were deployed were White, about 14 percent were Black or African American, and almost 90 percent identified themselves as non-Hispanic and 8 percent as Hispanic. Of those who were deployed, 89 percent were male and 11 percent were female. There were three variables--volunteer status, location deployed, and unit deployed--required by DOD policy for which the Defense Manpower Data Center (DMDC) could not provide data because the data either did not exist or were not reliable enough for the purposes of GAO's report. GAO found the deployment and mobilization data used to be reliable for providing descriptive information. However, the mobilization data, some deployment data fields, and DMDC's processes for data analyses need improvement. DMDC and the services have recently taken steps to improve the reliability of mobilization data; however, additional steps are needed to make mobilization data more reliable. DMDC and the services have undertaken a large-scale, challenging effort to replace all previous service-provided mobilization data in DMDC's CTS database with new data from the services, referred to as "rebaselining." To date, the Air Force has certified that it has rebaselined its data and Navy officials say they have validated their personnel files and established a common baseline of data with DMDC. The Army, which has mobilized the largest number of reservists, has not completed its rebaselining effort and has not set a deadline for completion. Also, DOD has not fully addressed other data issues that could affect the accuracy and completeness of the data, such as standardizing the use of key terms and ensuring that the services address data issues identified by DMDC as well as provide data for all required data fields, such as location, to DMDC. Also, because the data analyses DMDC provided had numerous errors, GAO questions the effectiveness of its verification procedures and other supporting procedures, all of which DMDC has not documented. Until DOD addresses data issues and DMDC documents the internal control procedures it uses to analyze data and verify its analyses of the data, the information provided to decision makers within Congress and DOD may be unreliable and decision makers will not be in the best position to make informed decisions about reserve force availability and reservists' exposure to health hazards.

section_title: Letter
paragraphs: Since President Bush signed an Executive Order establishing the Global  War on Terrorism (GWOT) on September 14, 2001, hundreds of thousands  of National Guard and Reserve servicemembers have been activated,  mobilized, and deployed in support of efforts in, among other places,  Afghanistan and Iraq. As the Department of Defense (DOD) continues to  rely on about 1.3 million reservists to carry out its military operations  domestically and abroad, there continues to be congressional interest in  the impact of GWOT on reserve employment, income change, medical and  health status of reservists, and other issues.
For decades, DOD has been collecting and reporting information on active  duty and reserve component servicemembers. However, it was not until  October 2001, less than a month after the terrorist attacks, that DOD  emphasized the need for the services to specifically report information  about mobilized and deployed reservists who support contingencies.
While DOD has been collecting this information, several reports have  emphasized information about reservists who have been mobilized, not  deployed. Further, some of our prior reports have raised concerns about  DOD’s ability to effectively track reservists who are being deployed to the  theater of operation. Information about reservists’ deployments is needed  to assess reserve force availability and to link reservists’ locations with  exposure to medical hazards. Our past work has also confirmed that it is  critical that DOD collect, maintain, and report reliable information on  deployed reservists. In our experience, the data that DOD has reported in  the past about the number of reservists who have been mobilized and  deployed have not been consistent because, for example, the data used  came from different or varied sources and the analyses performed were  based on different analytical assumptions.
This report, initiated under the Comptroller General’s authority to conduct  evaluations on his own initiative, addressed the following issues: (1) what  DOD data indicate are the number of reservists mobilized and deployed in  support of GWOT and the selected demographic and deployment  characteristics of those deployed and (2) whether DOD’s reserve  deployment and mobilization data and analyses are reliable.
For this report, we used data provided by the Defense Manpower Data  Center (DMDC), which is DOD’s repository for departmentwide data. We  outline the major assumptions we used to analyze the data in the scope  and methodology section of this report (see app. I). Specifically, to  address our objectives, we obtained and analyzed data from DMDC’s  Contingency Tracking System (CTS). CTS is DMDC’s system that brings  together data about GWOT from many sources and, according to a senior  DMDC official, is the only source of these data within DOD. The Joint  Staff’s Manpower and Personnel office is working toward using only CTS  data to determine reserve force availability for future operations. We also  performed reliability assessments on the data after obtaining an  understanding of the data file structure and the sources of the data.
Specifically, we (1) performed electronic testing of the data files for  completeness (that is, missing data), out-of-range values, and dates outside  of valid time frames; (2) assessed the relationships among data elements;  and (3) worked with agency officials to identify data problems, such as  which variables may be unreliable. We also analyzed the extent to which  data provided by DMDC changed during this review as a result of DMDC’s  data cleaning effort, known as a rebaselining. Using applicable guidance,  we interviewed knowledgeable officials at DMDC about internal control  procedures and other matters. In addition to the officials at DMDC, we  also interviewed knowledgeable officials from the services, the Office of  the Assistant Secretary of Defense for Reserve Affairs, and the Joint Chiefs  of Staff Manpower and Personnel office. The data we report are  sufficiently reliable for our purposes (that is, providing descriptive  information) with one caveat. Since the Army is in the process of updating  its mobilization data, we could not assess the reliability of the Army’s  mobilization data to the same extent as those of the other services. In  comparing our analyses of the data with the analyses reported by DMDC,  we determined that DMDC’s analyses were not sufficiently reliable for this  report. We performed our audit work from December 2005 through August  2006 in accordance with generally accepted government auditing  standards. A more thorough description of our scope and methodology is  provided in appendix I.
subsections: section_title: Background
paragraphs: In 1975, DOD implemented the Reserve Components Common Personnel  Data System (RCCPDS) to collect information on current and past  members of the six reserve components—Army National Guard, Air  National Guard, Army Reserve, Navy Reserve, Marine Corps Reserve, and  Air Force Reserve. This information included data on reservists’ personal  characteristics, such as name, Social Security number, date of birth,  gender, home address, and education, as well as data on their military  characteristics, such as service, reserve component, prior service status,  and date of initial entry into the reserve forces. According to the director  of DMDC, the services send daily, weekly, and monthly updated data  submissions to DMDC in accordance with applicable guidance.
After the first Gulf War, in a May 15, 1991, memorandum, DOD identified  16 recommendations requiring action by many offices within DOD  regarding Desert Storm personnel data issues. For example, the  memorandum said that DOD should consistently report on who  participated in the operations and cites examples of key terms, such as in  theater, that were being interpreted differently by DMDC, the Office of the  Secretary of Defense, and the services. In December 1991, DOD reported  on how DMDC provided information about operations Desert Shield and  Desert Storm. This report cited areas for improvement. For example, the  report indicated that DMDC created makeshift procedures to establish and  maintain the new data sources and to accommodate varied data requests.  The report cited that these procedures sometimes resulted in inconsistent  or incomplete data being provided in response to a request. On May 2,  2001, DOD updated guidance to the military services, among others, to  maintain a centralized database of active duty personnel. In this  guidance, DOD requires the services to report personnel information  about all active duty military servicemembers as well as reservists who are  ordered to active duty. While this instruction called for the services to  report information about servicemembers on active duty in support of a  contingency, the requirements for reporting contingency data were not  specific.
On October 4, 2001, the Under Secretary of Defense, Personnel and  Readiness (USD (P&R)), issued a memorandum that required the  services to report personnel information to DMDC on all active and  reserve component personnel mobilized or deployed in support of GWOT,  in accordance with DOD guidance. The purpose of GWOT data was,  among other things, to establish eligibility for benefits and entitlements as  a result of participation in the named contingencies. The information is  critical because it provides a historical database with which to assess the  impact of policies and processes, events, and exposures on the health of  deployed reserve component servicemembers. DMDC was tasked with  providing reporting guidance to the services for these data submissions.  DMDC sent this guidance to the services on October 12, 2001.
DMDC is a civilian-led agency with a mission to deliver timely and quality  support to its customers, and to ensure that data received from different  sources are consistent, accurate, and appropriate when used to respond to  inquiries. DMDC reports to the Deputy Under Secretary of Defense for  Program Integration, who is in the Office of the USD (P&R) (see fig. 1).
In February 2002, USD (P&R) reminded the services in another  memorandum of its earlier requirement for reporting personnel data to  DMDC and informed the services that they had 2 weeks to provide plans to  DMDC on how they were going to correct any personnel data reporting  problems. On August 6, 2004, DOD updated prior guidance regarding  RCCPDS to include an enclosure that set out specific requirements for  the services to report personnel information for all reserve component  servicemembers supporting a named contingency, unlike previous  guidance. The purpose of the new enclosure was to ensure more accurate  reporting on a named contingency, such as GWOT missions, as well as to  establish eligibility for benefits and entitlements, and to develop a registry  of participants for tracking in support of research and evaluation of DOD  programs and policies. According to DOD officials, the services, in general,  were still reporting data according to previous guidance for a few years  after the new guidance was issued.
In August 2004, DMDC began operation of its CTS database to address  DOD’s reporting requirements, including those in the new enclosure (that  is, enclosure 11). The CTS database is DOD’s repository for collecting  activation, mobilization, and deployment data for reservists who have  served and continue to serve in support of GWOT. The CTS database  contains both an activation file, which contains mobilization data, and a  deployment file. Both files are updated monthly by service submissions  and cover GWOT from September 11, 2001, to the present. The purpose of  the activation file is to account for and provide medical and educational  benefits for all reservists called to active duty in support of GWOT  contingencies, and it allows DOD to provide data on the number of  reservists who have been mobilized in support of GWOT. The purpose of  the CTS deployment file is to account for a deployed servicemember’s  deployment date and location during each deployment event in support of  deployment health surveillance and DOD guidance. The database is also  used to track and report the number of reservists who have been deployed  in support of GWOT since September 11, 2001.
section_title: DOD Data Show Demographic and Deployment Characteristics of Hundreds of Thousands of Reservists Deployed in Support of GWOT
paragraphs: Our analysis of DOD data indicates that more than 531,000 reservists have  been mobilized in support of GWOT and more than 378,000 reservists, or  about 71 percent of the number mobilized, have been deployed in support  of GWOT through June 30, 2006 (see fig. 2). The Army National Guard  deployed the greatest number of reservists in support of GWOT from  September 2001 through June 30, 2006, and, of those, the majority were  deployed once. The data also indicate that the vast majority of reservists  who deployed in support of GWOT were U.S. citizens, White, and male.  Further, the data indicate that most of the reservists spent 1 year or less  deployed.
DOD guidance requires the services to report timely, accurate, and  complete activation, mobilization, and deployment data. DOD guidance  also requires DMDC to collect and maintain mobilization and deployment  data from the services about the reservists. DOD is required by policy to  report personnel data about reservists, such as service, service  component, reserve component category, race, ethnicity, gender,  citizenship status, occupation, unit, and volunteer status regarding a  current mobilization. In addition, DOD is required by policy to capture  deployment information such as the location a reservist is deployed to and  the dates the reservist was deployed to that location.
subsections: section_title: DOD Data on Selected Demographic and Deployment Variables
subsections: section_title: Number of Reservists Mobilized and Deployed
paragraphs: Our analysis of DOD data indicates that more than 531,000 reservists have  been mobilized in support of GWOT and more than 378,000 reservists, or  71 percent of the number mobilized, have been deployed in support of  GWOT through June 30, 2006 (see fig. 2). The number of mobilizations and  deployments peaked in fiscal year 2003 with about 206,000 reservists  mobilized and about 127,000 reservists deployed (see figs. 3 and 4). Since  fiscal year 2003, the total number of mobilizations has declined, while the  number of deployments remained stable through fiscal year 2005. The  Army National Guard has mobilized and deployed the greatest number of  reservists—more than 230,000 mobilized and more than 163,000 deployed.  The Navy Reserve had the least number of reservists mobilized—with  about 29,000—while the Marine Corps Reserve had the fewest number  deployed with about 19,000 reservists (see fig. 2). The percentage of the  total reservists mobilized or deployed varies across the fiscal years (see  figs. 3 and 4). For example, looking at the percentage of mobilizations by  component each year, Navy Reserve, Air Force Reserve, and Air National  Guard mobilizations occurred early in GWOT and have generally declined  over time. Conversely, the percentage of Army National Guard and Army  Reserve mobilizations has generally increased over time. The greatest  number of Army National Guard deployments—more than 60,000— occurred in fiscal year 2005 (see table 5 totals in app. II), while also in  fiscal year 2005, the Army National Guard represented the largest  deploying component, with 52 percent of deployments belonging to it (see  fig. 4).
Although reservists usually deployed only once, some experienced  multiple deployments (see fig. 5). For example, compared to the other  reserve components, the Air National Guard and the Air Force Reserve  had nearly half of their reservists deploying two and three or more times,  but they tend to have shorter deployment cycles according to the Air  Expeditionary Force cycle. Under this cycle, reservists deploy for about  120 days in a 20-month cycle. However, servicemembers assigned to  stressed specialties deploy for longer periods of time and in greater  frequency. At the unit level, some deployment rules have been modified to  increase volunteerism or to add stability to key missions. The Army  National Guard and the Marine Corps Reserve had the lowest percentage  of reservists deploying two and three or more times, but they tend to have  longer deployment cycles. In general, DOD policy stipulates that Army  units spend 1 year “boots on the ground” in theater. This policy also  states that Marine Corps units below the regimental or group level deploy  for 7 months while regimental and group headquarters units and above  deploy for 12 months. This policy also states that the Chief of Naval  Operations’ goal is for servicemembers to have a 6-month deployment with  12 months in a nondeployed status.
Our analysis of DOD data indicates that across the services, the majority  of reservists have been deployed once, and of those deployed in support of  GWOT, most—about 307,000 reservists, or 81 percent—have spent a year  or less deployed. Alternatively, more than 65,000 reservists, or 17 percent,  have spent more than 1 year but less than 2 years deployed, and about  6,000 reservists, or fewer than 2 percent, have spent more than 2 years  deployed. The data also indicate that the Marine Corps Reserve had the  highest percentage of reservists serving more than 2 years. In addition, the  data also indicate that very few—less than 1 percent—of Air National  Guard reservists served more than 2 years (see fig. 6).
Our analysis of DOD data indicates that most reservists who have  deployed in support of GWOT through June 30, 2006, were members of the  Selected Reserve (see fig. 7 and table 5 in app. II). The majority of units  and individuals in each reserve component are part of the Selected  Reserve. These units and individuals have been designated as so essential  to the initial wartime mission that they have priority for training,  equipment, and personnel over all categories of reservists. Congress  authorizes end strength for Selected Reserve personnel each year. The  authorized end strength for the Army National Guard has been about  350,000 for the past several years. For fiscal year 2005, data provided by  the services to DMDC indicate that the Army National Guard deployed  more than 60,000 Selected Reserve servicemembers, which represents the  highest number of Selected Reserve servicemembers deployed in a single  fiscal year by a single reserve component since GWOT began.
Although the services are authorized a maximum number of selected  reservists, the actual number of reservists will fluctuate when additional  reservists are recruited or others leave the reserve component. In addition,  reservists such as those in the Individual Ready Reserve, are also available  for deployment. In general, reservists are trained to have specific skills  and specialties and may not be suited to deploy for a specific mission until  additional training is provided. In addition, some reservists may not be  available for deployment because they are in training, on medical leave, or  awaiting training.
section_title: Citizenship Status of Reservists at the Time of Their Most Current Deployment in Support of GWOT
paragraphs: Our analysis of DOD data indicates that almost 98 percent of reservists  who have deployed in support of GWOT through June 30, 2006, were U.S.  citizens at the time of their most current deployment (see fig. 8). The data  indicate that about 1 percent of reservists were non-U.S. citizens or non- nationals at the time of their most current deployment. The citizenship  status of more than 1,400 reservists was unknown. DOD data also indicate  that 168 reservists’ citizenship status changed. Table 1 shows the  citizenship status of reservists by reserve component by fiscal year.
Our analysis of DOD data indicates that about 78 percent of those  deployed for GWOT were White; about 14 percent were Black or African  American; about 2 percent were Asian, Native Hawaiian, or Other Pacific  Islander; and about 1 percent were American Indian or Alaskan Native  (see table 2). Overall, about 5 percent of the deployed reservists declined  to indicate their race. The Army National Guard, the Air National Guard,  and the Air Force Reserve had the highest percentages of the reservists  who identified themselves as White. Further, about 90 percent of those  who responded identified themselves as non-Hispanic and 8 percent as  Hispanic (see table 3).
Our analysis of DOD data indicates that about 338,000 reservists, or about  89 percent of the number deployed, were male (see table 4). About 11  percent of those deployed in support of GWOT were female. Of the  approximately 163,500 Army National Guard servicemembers who have  been deployed through June 30, 2006, more than 92 percent were male.  Almost 98 percent of those deployed in support of GWOT through June 30,  2006, for the Marine Corps Reserve were male, representing the highest  percentage of males compared with females for all of the reserve  components.
Our analysis of DOD data indicates that California, Texas, Pennsylvania,  and Florida had the highest numbers of reservists who have deployed in  support of GWOT through June 2006 (see table 6 in app. II for the number  of reservists deployed by state of residence by reserve component by  fiscal year). The 4 states combined had more than 76,000 reservists in  residence at the time of their deployments. Eleven states deployed more  than 10,000 reservists each, accounting for more than 160,000 reservist  deployments. Of those deployed, about 39 percent came from states in the  southern United States, about 23 percent from the midwest, about 18  percent from states in the western United States, and about 15 percent  came from states in the northeast part of the country. More than 20,000  reservists indicated California or Texas as their state of residence at the  time they were deployed (see fig. 9). Nineteen states and 5 territories had  fewer than 5,000 reservists in residence at the time of their deployment  and 20 states and 1 territory had from 5,000 to 9,999 reservists in residence  at the time of their deployment.
Our analysis of DOD data indicates that since GWOT began, the  occupational areas of enlisted reservists deployed in support of GWOT  have stayed somewhat consistent across all services. For example, the  Army National Guard, the Air Force Reserve, and the Marine Corps  Reserve have deployed reservists mostly in infantry occupational areas  including such groups as infantry, air crew, and combat engineering. All  six reserve components have deployed electrical and mechanical  equipment repairers, such as automotive, aircraft, and armament and  munitions. Three of the six reserve components—the Army National  Guard, the Army Reserve, and the Marine Corps Reserve—have deployed  reservists who are service and supply handlers, such as law enforcement  and motor transport.
Since GWOT began, the occupational areas most deployed for reserve  component officers have varied, but all reserve components primarily  deployed tactical operations officers, to include ground and naval arms,  helicopter pilots, and operations staff subgroups. The Army National  Guard, the Air National Guard, and the Navy Reserve have deployed  engineering and maintenance officers, such as the communications and  radar and aviation maintenance occupational subgroups. The Air National  Guard, the Army Reserve, and the Air Force Reserve have deployed  reservists in the health care officer occupational areas, including  physicians and nurses. The Army Reserve and the Marine Corps Reserve  have deployed supply and procurement occupational areas that include  transportation, general logistics, and supply occupational subgroups. The  Air Force Reserve has also deployed intelligence officers in occupational  subgroups such as general intelligence and counterintelligence.
section_title: Data for the Volunteer Status, Location Deployed, and Unit Deployed Variables Were Either Not Available or Not Reliable
paragraphs: We were unable to analyze the volunteer status variable because the data  do not exist for all of the reserve components. Similarly, we were unable  to analyze the deployment location and deployment unit variables because  we determined, in agreement with DMDC officials, that the data in these  fields were not reliable. This issue is discussed further below.
section_title: DOD and the Services Have Taken Steps to Improve the Reliability of Reserve Component Data, but More Needs to Be Done
paragraphs: While we found selected deployment and mobilization data to be  sufficiently reliable for our purposes (that is, providing descriptive data),  some of the data were not reliable enough for us to report, even for  descriptive purposes. DMDC and the services, as required by DOD policy,  have taken steps to improve the reliability of the mobilization data;  however, more action is needed to improve the reliability of CTS data and  DMDC’s analyses of those data. For example, (1) the rebaselining effort  resulted in substantial changes being made to the mobilization data, and  the Army—which has mobilized and deployed the largest number of  reservists for GWOT—has not completed this rebaselining effort, which  the Joint Staff tasked DMDC and the services to do in November 2005;   (2) we identified data issues that DOD has not addressed that could  further improve the reliability of the data, such as standardizing the use of  key terms like deployment; and (3) DMDC does not have effective controls  for ensuring the accuracy of its data analyses used to produce reports as  required by federal government internal control standards. Although  DMDC and DOD have undertaken a major data cleaning—or  rebaselining—effort to improve the reliability of mobilization data, the  effort does not address some fundamental data quality issues. While we  recognize that such a large-scale effort, although replete with challenges,  is a positive step toward better quality data, if data reporting requirements  and definitions are not uniform, and if there are no quality reviews of  DMDC’s analyses, some data elements and DMDC’s analyses of those data  may continue to be unreliable. A senior DMDC official stated that it  emphasizes getting data to customers in a timely manner rather than  documenting the internal control procedures needed to improve the  reliability of the data and the data analyses produced. However, with  proper internal controls, DMDC could potentially achieve both timeliness  and accuracy. Without reliable data and analyses, DOD cannot make  sound data-driven decisions about reserve force availability. Moreover,  DOD may not be able to link reservists’ locations with exposure to medical  hazards.
subsections: section_title: DMDC and the Services Are Updating the Mobilization Data in CTS, but Concerns Remain
paragraphs: We have found the deployment and mobilization data we used to be  sufficiently reliable for our purposes (that is, providing descriptive data),  and DMDC and the services have recently taken steps to improve the  reliability of mobilization data. However, additional steps are needed to  make mobilization data more reliable. As previously noted, DOD  guidance requires the services to report timely, accurate, and complete  activation, mobilization, and deployment data. DMDC officials  responsible for overseeing the CTS database stated that a rebaseline of the  deployment data was not necessary because the deployment data matched  the data in the Defense Finance and Accounting Service’s (DFAS)  systems by more than 98 percent. Although DMDC and the services  rebaselining of the mobilization data in CTS has resulted in improvements,  the Army, which has mobilized the greatest number of reservists for  GWOT, has not completed its rebaselining effort. A senior-level DMDC  official responsible for overseeing the CTS database said that the  mobilization data in the CTS database prior to the rebaselining effort were  less than 80 percent accurate for the Army, the Navy, and the Air Force,  but that the Marine Corps’ data were generally considered to be accurate  prior to the rebaselining effort. The official also stated that DMDC expects  that the mobilization data within the CTS database will be 90 percent  accurate because of this rebaselining effort, which was still ongoing  through August 2006. While we recognize that this is a considerable  undertaking, to date, only the Navy and the Air Force have validated or  certified their mobilization data files. Navy officials said that the Navy has  validated its personnel records and established a common baseline of data  with DMDC. Air Force Reserve officials said that their data within CTS are  now 99 to 100 percent accurate. The Chief of the Personnel Data Systems  Division for the Air National Guard certified that although file  discrepancies are still being reconciled, the data that were processed by  DMDC on June 11, 2006, were the most accurate activation data and that  data accuracy will improve with each future file sent to DMDC. The DMDC  official said that the Marine Corps had only partially completed its  rebaselining effort and would not be finished until the Marine Corps  provided its August 2006 data file in September 2006. The Army National  Guard and Army Reserve are still working to rebaseline their mobilization  data, and the Army has not provided a time frame for completing the  effort.
However, we still have concerns regarding the reliability of the  mobilization data, because the scope of the rebaselining effort changed  and the data changed substantially as a result of the rebaselining. At the  beginning of our review, DMDC and the services referred to the  rebaselining effort as a “reconciliation,” which, according to a DMDC  official and a Reserve Affairs official, would have resulted in all data  (current and past) being reviewed and corrected as needed. We  acknowledge that some degree of change is expected in any data cleaning  effort, especially with large-scale, multisource collection methods such as  DMDC’s data collection process. However, our experience has shown that  cleaning efforts that result in a large degree of change would suggest  systematic error. Such error raises concerns about the reliability of both  the original data and the “cleaned” data. If both the source data and the  cleaned data are populated with the same assumptions and information,  any reconciliation of data points should result in relatively small change  that correct simply for random error, such as from keypunch or data  source errors. However, for some variables, the data changed substantially  as a result of DMDC and the services’ rebaselining or data cleaning effort.  Our analysis shows that data from the period of September 2001 through  December 2005 have changed by about 4 percent to as much as 20 percent.  For example:    The number of reservists mobilized for GWOT through December 2005  went from about 478,000 to about 506,000—an increase of more than  27,000 reservists or a change of more than 5 percent.
The Army Reserve data sustained the greatest change during this time with  a more than 19 percent increase in the number of reservists mobilized. The  number of mobilized Army National Guard reservists increased more than  7 percent. According to a senior DMDC official, the Army data are  expected to continue to change, perhaps substantially enough to require  the rebaselining of the data again in the future.
The number of Air National Guard reservists mobilized decreased by more  than 13 percent.
The Navy Reserve, the Marine Corps Reserve, and the Air Force Reserve  data all changed about 5 percent.
DOD officials stated that the rebaselining effort occurred because the  Joint Staff tasked DMDC and the services with ensuring that the data the  Joint Staff’s Manpower and Personnel office was using in CTS were the  same data as the services were using to determine reserve force  availability. According to a senior-level DMDC official responsible for  overseeing CTS, the rebaselining effort’s scope changed because all of the  services agreed that starting over and replacing all of the data would make  more sense than trying to correct transactions already in CTS, because the  services found errors in the CTS files initially used for the reconciliation.  Service officials said that some of the data discrepancies developed  because of a DMDC quality check procedure that sometimes resulted in  DMDC replacing the service-submitted data with data from other sources.  DMDC officials said that they did this because the services were unable to  report some of the required CTS data. According to DMDC officials,  service submissions have become more complete over time, resulting in  DMDC now using the quality check procedures only to check the data  rather than to populate the CTS database. This DMDC official stated that  DMDC expected the data to change substantially based on the issues  identified with service data during the initial reconciliation effort and the  subsequent rebaselining effort.
Because the rebaselining effort is not complete and the Army—which has  mobilized and deployed the largest number of reservists for GWOT—has  not finished the rebaselining, we do not know how much the data will  continue to change as DMDC and the services work to finish this effort.  DOD data on reservists’ mobilizations and deployments are important  because decision makers at DOD and in Congress need the data to make  sound decisions about personnel issues and for planning and budgeting  purposes.
section_title: DOD Has Not Fully Addressed Data Issues That Could Improve Data Reliability
paragraphs: Prior to the rebaselining effort, some services recognized that there were  data issues that needed to be addressed and took steps to do so, as DOD  guidance requires the services to report accurate and complete  mobilization and deployment data. However, some data issues that would  ensure more accurate, complete, and consistent mobilization and  deployment data across the services in the future have not been fully  addressed by DOD. Some examples of data issues being addressed include  the following:    The Air Force and the Navy were having difficulty tracking mobilizations  based on reservists’ mobilization orders, which has resulted in both  services independently working to develop and implement systems that  write reservists’ orders.
The Army Reserve recently began to modify its mobilization systems,  which Army officials expect will improve the collection of reservists’  mobilization data.
The Air Force identified problems with the way in which the Defense  Eligibility Enrollment Reporting System (DEERS) processed end dates for  reservists’ mobilizations, which resulted in some reservists not receiving  appropriate benefits (for example, dental benefits). Air Force officials  worked with officials from the Office of the Secretary of Defense and  DMDC to identify and address the data processing logic issues.
Despite these positive steps, service process improvements are not all  complete, and further, there has been no comprehensive review across  DOD to identify data issues that if addressed, could result in more  complete, accurate, and consistent mobilization and deployment data  across and within the services. Reserve Affairs officials in the office of  Reserve Systems Integration said that a more sustainable fix to the  processes of collecting data is needed to ensure that data captured in the  future are accurate and more efficiently collected. We agree and have  identified some issues that may continue to affect data reliability, such as  the following:    The use of terms, such as activated, mobilized, and deployed, has not been  standardized across the services. Although the department has defined  these terms in the Department of Defense Dictionary of Military and  Associated Terms, the terms are used differently by the individual  services. In the Air Force, “activation” can refer to the time when a  reservist either volunteers or is involuntarily mobilized; however, the term  “mobilized” refers only to someone who is not a volunteer. Even within a  single service, these words can have different meanings. For example, an  Army National Guard official who participated in the rebaselining effort  said that Army National Guard servicemembers who backfill active duty  servicemembers are not considered deployed since they have not left the  United States. However, according to this official, some staff in the Army  National Guard use “deployed” to include reservists who are mobilized  within the United States.
There is no single data entry process that would minimize the potential for  contradictory data about reservists in multiple systems. Currently, data  about reservists are entered separately into multiple systems.
There is no mechanism for DMDC to ensure that the services are  addressing the data inconsistencies DMDC identifies during its ongoing,  monthly validation process, such as Social Security numbers that are  duplicated in two reserve components.
DOD has taken an ad hoc, episodic approach to identifying data reporting  requirements and to addressing data issues. DOD has periodically issued  policies regarding its need to collect and report specific data, such as  volunteer status and location deployed, about active duty servicemembers  and reservists. As a result of changing requirements, many of these  policies have addendums that include these additional data requirements,  which are not immediately supported by the services’ existing systems that  are used to collect the data. Over time, this has led to disjointed policies  that overlap and that require the services to modify their existing systems  and processes, which can take months to complete.
There are incomplete data submissions across the services. Specifically,  data for volunteer status was not available in CTS for all service  components, and the location deployed and deploying unit data were not  reliable enough for the purposes of this report. Only three of the six  reserve components—the Air National Guard, the Marine Corps Reserve,  and the Air Force Reserve—provide information on a reservist’s volunteer  status, which neither we nor DMDC report because it is not available for  all six components. Further, DMDC officials said that they consider CTS  location data incomplete although the data are improving with each fiscal  year. DMDC officials said that most unit information is based on the unit a  reservist is assigned to and may not represent the unit the reservist is  currently deployed with in theater. For this reason, we did not consider  these data reliable enough to report.
A DMDC official stated that DMDC does not have the authority to direct  the services to correct data errors or inconsistencies or to address data  issues. DMDC does, however, work with the services and tries to identify  and address data challenges. According to some service officials, the  department plans to implement a new, integrated payroll and personnel  system—Defense Integrated Military Human Resource System  (DIMHRS)—and that the services have been diverting resources needed to  modify their existing systems and relevant processes to support DIMHRS.  However, our past work has shown that DOD has encountered a number  of challenges with DIMHRS, which is behind schedule, and the current  schedule has it available no sooner than April 2008, when the Army is  scheduled to begin implementing the system.
In general, service officials said that they are working to collect data on  volunteer status, location deployed, and deploying unit; however, Air  Force officials stated that they do collect data on location deployed and  deploying unit and that these data are accurate and are being provided to  DMDC. Army Reserve officials stated that they currently do not have plans  to collect data on volunteer status.
section_title: DMDC Does Not Have Effective Controls for Ensuring the Accuracy of Its Data Analyses Used to Produce Reports
paragraphs: DMDC has not documented (1) its procedures for verifying that the data  analyses it performs are correct and (2) the procedures for monthly  validation of service data or the procedures used to perform analyses of  data. Either of these issues could, if documented as part of DMDC’s  verification process, address some of our concerns about internal  controls. DMDC is required by policy to develop and produce reports  about mobilization data and respond to requests for information about  deployed personnel. DOD policy requires DMDC and the reserve  components to ensure the accuracy of files and the resulting reports.  Federal government internal control standards require that data control  activities, such as edit checks, verifications, and reconciliations, be  conducted and documented to help provide reasonable assurance that  agency objectives are being met.
DMDC officials said that they have internal verification procedures that  require supervisors to review all data analyses used to generate reports,  although these procedures are not documented. Specifically, the  supervisors are to review (1) the statistical programming code used to  generate the data analyses to ensure that the code includes the customer’s  data analyses parameters (that is, the assumptions used to produce the  analyses) and (2) the “totals” generated to ensure that these totals match  the control totals that show the number of reservists currently or ever  mobilized or deployed in support of GWOT. DMDC officials acknowledge  the importance of verifying the accuracy of the data analyses prior to  providing the reports to customers, and they stated that they had verified  the accuracy of the analyses provided to us. However, we found numerous  errors in the initial and subsequent analyses we received of the GWOT  data through May 2006, causing us to question whether DMDC verified the  data analyses it provided to us and, if it did, whether the current process is  adequate. For example, we found that DMDC had done the following:    Counted reservists with more than one deployment during GWOT also  among those who deployed only once during GWOT, which resulted in  overcounting the number of reservists’ deployments.
Used ethnicity responses to identify race despite having told us that the  internal policy was changed in 2006 and that this was no longer an  acceptable practice.
Counted reservists whose ethnicity was “unknown” as “non-Hispanic”  although “unknown” does not necessarily mean someone’s ethnicity is  “non-Hispanic” and there was a category for unknowns.
Repeatedly categorized data based on a reservist’s first deployment (when  there was more than one) despite agreeing to modify this analytical  assumption so that we could present data by the reservist’s most current  deployment.
Reported thousands of reservists as having changed citizenship status  during GWOT although, in our analyses, we found that only 168 reservists  had changed status.
Analyzed data by reserve component categories (for example, Selected  Reserve and Individual Ready Reserve) rather than by reserve component  as we had asked. By analyzing the number of days a reservist was  deployed by reserve component category, a reservist could be counted  multiple times within one component if he or she changed category. This  error affected the way in which the total number of days a reservist was  deployed was calculated. For example, if the same reservist served 350  days as an Army National Guard Selected Reserve member and an  additional 350 days as an Army National Guard Individual Ready Reserve  member, he or she would be counted as two reservists who were each  deployed for less than a year. However, our intent was to report that the  same individual had been deployed for a total of 700 days. In our analysis,  all of a reservist’s days deployed were totaled and counted once for each  reserve component, regardless of which category he or she belonged to  when deployed.
Miscoded the end date for the analysis of how many days reservists were  deployed for GWOT. This resulted in up to an additional 90 days of  deployment being counted for reservists who were still deployed at the  time the data were submitted to DMDC.
In our discussions with DMDC officials, they readily acknowledged that  errors had been made, although they stated that the analyses had  undergone supervisory review prior to our receiving them. During these  discussions, we also discovered that many of these errors occurred  because DMDC had not used all of our data analyses parameters, although  these officials had stated that this was one of the verification process steps  followed. Although we were able to work with DMDC officials and identify  the analytical assumptions they were going to use to complete our  analyses, without documented analytical procedures, it is unclear to what  degree the analyses DMDC provides to other users of the data also contain  errors since many may not similarly verify the analyses provided to them  by DMDC.
In addition, DMDC officials have not documented additional processes  that would further support a verification process, such as (1) the ongoing,  monthly validation process of service-provided data and (2) the  procedures to perform analyses and generate reports, including the  assumptions DMDC uses when producing periodic and special reports for  customers. In the past, according to the services, the ongoing, monthly  validation process DMDC used resulted in two sets of data—one set of  service data and one set of DMDC data—that may not have been the same.  For example, we were told by the Air Force that, in some cases, service  data were replaced with default values because of a business rule that  DMDC applied to the data and that this change resulted in errors to the  service-provided data. These inconsistent data caused the Joint Staff to  request that the services and DMDC reconcile the data. As stated above,  there were errors in the analyses performed to generate the reports DMDC  provided to us, including DMDC’s not using many of the assumptions we  agreed to for the analyses. DMDC also made errors that contradicted its  own undocumented policy.
A senior DMDC official said DMDC has not documented these procedures  because the organization emphasizes getting data and reports to its  customers in a timely manner rather than preparing this documentation.  This official said that documentation is not a top priority because  situations change rapidly, and it would be hard to keep these documents  up-to-date. The official also said that the errors made in the analyses  provided to us were caused by human error and the need to provide data  quickly. Further, the DMDC official said that while there are standard data  requests that are generated frequently, GAO’s request was an ad hoc  request, and the procedures for addressing such requests, in practice, are  not as well defined. While we agree that our requests met DMDC’s  definition of an ad hoc request, we disagree that sufficient time was not  allowed for DMDC to prepare the analyses. For the initial request, we  worked with DMDC over the course of about 5 business days to define the  analytical assumptions that would be used during the analysis. DMDC then  took about 8 business days to complete the analysis and provide it to us.
DOD data analyses are important because decision makers at DOD and in  Congress need the data to make sound decisions about reserve force  availability, medical surveillance, and planning and budgeting. In the  absence of documented procedures and the necessary controls to ensure  that they are implemented, it is difficult for an organization to ensure that  it has established a robust process that is being consistently applied and  that accurate results are being achieved. Joint Staff and Reserve Affairs  officials are emphasizing the need to use one data source for most  analyses to further reduce the inconsistencies in data analyses because  service-produced analyses and DMDC-produced analyses could differ if  both are not using the same set of data and assumptions. Otherwise, it is  possible that the data analyses provided to decision makers at DOD or in  Congress will be incomplete and inconsistent. If the data analyses are  incorrect, users could draw erroneous conclusions based on the data,  which could lead to policies that affect reservists in unanticipated ways.
section_title: Conclusions
paragraphs: DOD recognizes the need for accurate, complete, and consistent data and  data analyses, and it has taken some preliminary, ad hoc steps to improve  its data, including undertaking a considerable effort to rebaseline its  mobilization data. It has not, however, addressed some of the  inconsistencies in data and data analyses departmentwide, such as when  terms are used differently from one service to the next. Further, service  officials stated that it is anticipated that a lot of these problems will be  addressed when DIMHRS is implemented. However, the schedule for  DIMHRS continues to slip, so it is unclear when this solution will be  available. We recognize that the need for accurate, complete, and  consistent data and data analyses about reservist mobilization and  deployment is always important, and even more so during higher levels of  mobilization and deployment, such as is the case now with GWOT. This is  especially true since, in general, there are restrictions on the maximum  length of time a reservist can be involuntarily activated. Thus, having  accurate and complete data on a reservist’s status is critical for  determining availability for future deployments. This is especially true of  the CTS data since the Manpower and Personnel office in the Joint Staff  and the Office of the Assistant Secretary of Defense for Reserve Affairs  mostly use the data found in CTS. These data also help DOD and Congress  to understand the potential impacts of policy decisions as they relate to  reservists who are eligible for TRICARE Reserve Select and educational  benefits based on the number of days a reservist is deployed. DOD has not  provided guidance to the services to better define and standardize the use  of key terms. DOD also has not collected and maintained all essential data  nor has it established a process for ensuring that data inconsistencies are  resolved. Further, DOD has not documented key procedures and  processes for verifying the data analyses it provides to its customers, thus  compromising its ability to ensure the accuracy, completeness, and  consistency of these analyses. Until decision makers in DOD and Congress  have accurate, complete, and consistent data and analyses, they will not be  in the best position to make informed decisions about the myriad of  reserve deployment matters.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Defense take the following four  actions:    Direct the Under Secretary of Defense, Personnel and Readiness, to  provide guidance to the services to better define and standardize the use  of key terms, like activation, mobilization, and deployment, to promote the  completeness, accuracy, and consistency of the data within CTS.
Direct the service secretaries to (1) take the steps necessary to provide all  required data to DMDC, such as volunteer status and location deployed,  and (2) have the services address data inconsistencies identified by  DMDC.
Direct the service secretaries to establish the needed protocols to have the  services report data consistent with the guidance above.
Direct the Under Secretary of Defense, Personnel and Readiness, to  require DMDC to document its internal procedures and processes,  including the assumptions it uses in data analyses. In doing this, the Under  Secretary of Defense, Personnel and Readiness, should collaborate on the  reasonableness of the assumptions established and used by DMDC in its  data analyses with the Office of the Assistant Secretary of Defense for  Reserve Affairs and the Joint Staff.
section_title: Agency Comments and Our Evaluation
paragraphs: The Under Secretary of Defense, Personnel and Readiness, provided  written comments on a draft of this report and stated that we changed one  of our original audit objectives and did not inform the department of this  change. We disagree. While the scope of our audit did change after our  initial notification letter of June 17, 2005, was sent to DOD, we notified the  proper officials of this change in a December 2, 2005, email to the agency- designated liaison within the DOD Inspector General’s office. In this email,  we specifically said that we would be contacting DMDC and that we would  be focusing on data for reserve component activation, mobilization, and  deployment for GWOT. In accordance with generally accepted government  auditing standards (GAGAS), GAO analysts are expected, as appropriate,  to review an agency’s internal controls as they relate to the scope of the  performance audit. Specifically, we are required by GAGAS to review the  reliability of the data and the data analyses provided to us. To assess the  reliability of data and data analyses, we often review an agency’s internal  controls that are put in place to ensure the accuracy of the data and  analyses. As we discuss in our report, we found the data to be sufficiently  reliable for our purposes. However, over the course of the work, the  analyses of the data DMDC provided to us continued to have errors. This  raised concerns about the adequacy of DMDC’s internal controls for  preparing and verifying these analyses, which DMDC stated were not  documented. In accordance with GAGAS, when reporting on the results of  their work, auditors are responsible for disclosing all material or  significant facts known to them which, if not disclosed, could mislead  knowledgeable users or misrepresent the results. Consistent errors in  DMDC’s analyses led us to include an audit objective on the reliability of  the data and the data analyses.
In its written comments, DOD generally concurred with three of our  recommendations and did not concur with one of our recommendations.  DOD also provided technical comments, which we have incorporated in  the report, as appropriate. Regarding our recommendation that DOD  provide guidance to the services to better define and standardize the use  of key terms, DOD stated that this requirement has already been addressed  because these terms are defined. We acknowledged in our draft report that  these key terms are defined in the Department of Defense Dictionary of  Military and Associated Terms. However, as we state in our report, our  audit work indicates that the services are not operationalizing the use of  the terms in a consistent manner. The intent of our recommendation is to  have DOD standardize the use of the key terms across the services.
DOD generally concurred with our recommendation that the services  provide all required data to DMDC and address data inconsistencies, and  stated that the services have been directed to provide all necessary data  and are working to address data inconsistencies. While we agree that the  services are working with DMDC to address data inconsistencies with  regard to the rebaselining of mobilization data, we also identified other  data inconsistencies that DOD has not addressed, such as Social Security  numbers that are duplicated in more than one reserve component. We  agree with DOD that some requirements cannot be immediately supported  by service data systems and modifications to them can take time to  complete. However, as our report notes, some service officials stated that  resources are being diverted from these efforts to the DIMHRS program,  which we reported is behind schedule. We continue to observe the need  for the services to provide all necessary data, to address these data  inconsistencies, and to establish needed protocols to have the services  report data consistent with DOD guidance, especially since the data are  used to determine reserve force availability and for medical surveillance.
DOD also generally concurred with our recommendation that DMDC  document its internal procedures and processes, including the  assumptions it uses in data analyses. In its written comments, DOD stated  that DMDC is in the process of developing documentation on its internal  procedures and processes and has a draft that addresses the processes  used from receipt of the data from the service components to the final  quality control of the consolidated file. DOD also stated that DMDC has a  draft product regarding many of the data analyses procedures used.  During this engagement, we asked if these procedures and processes were  documented. As we say in the report, DMDC stated that they were  undocumented and that documenting them was not a priority. Although  DOD stated that it is in the process of drafting these procedures and  processes, we were never provided a draft of these documents. DOD also  stated that while DMDC attempts to document the assumptions made in  resulting report titles and footnotes, the disclosure of assumptions used in  data analyses remain the responsibility of the requester of the data  analyses. Although we agree that the requesters of the data bear  responsibility to disclose the analytical assumptions used in the data  analyses, our audit work indicates that there are basic assumptions that  DMDC establishes and uses that, if documented and discussed with those  who request data analyses, would allow the users to understand how the  information can be used, as well as the limitations of the data analyses.  For example, during a discussion with a Reserve Affairs official, who uses  the data analyses provided by DMDC to provide information to senior  DOD officials, we stated that DMDC defaults to using a servicemember’s  first deployment rather than the most current deployment when preparing  data analyses. This official was unaware that DMDC used this assumption  and stated that the expectation was that DMDC was using the most  current deployment to generate the analyses. This official planned to  discuss this issue with DMDC in the future.
In its written comments, DOD did not concur with what it characterized as  our fourth recommendation. Specifically, DOD separated a single  recommendation into two recommendations. In the draft report we sent to  DOD, the recommendation read: “We recommend that the Secretary of  Defense direct the Under Secretary of Defense, Personnel and Readiness,  to require DMDC to document its internal procedures and processes,  including the assumptions it uses in data analyses. In doing this, the Under  Secretary of Defense, Personnel and Readiness, should collaborate on the  reasonableness of the assumptions used by DMDC in its data analyses  with the Office of the Assistant Secretary of Defense for Reserve Affairs  and the Joint Staff.” DOD stated that DMDC is a support organization that  generates reports for a multitude of organizations and that each  organization that requests reports provides the assumptions that DMDC  uses to develop the reports. However, our audit work showed that DMDC  has established and uses some basic assumptions in analyzing data and  that DMDC may not always discuss these assumptions with other DOD  offices, such as Reserve Affairs. As a result, we continue to emphasize the  need for DMDC to document these assumptions and to collaborate with  these offices to ensure a common understanding of these assumptions.  Although DOD organizations can request data analyses using multiple  assumptions, without written documentation other organizations may not  be fully aware of the analytical assumptions used by DMDC and this may  lead to miscommunication and, ultimately, the data analyses may not be  valid in that it does not report what the user intended. We continue to  believe that the assumptions used need to be documented and discussed  with other DOD offices as we recommended. Based on DOD’s comments,  we modified this recommendation to clarify our intent.
We are sending copies of this report to the Secretary of Defense; the  Secretaries of the Army, the Navy, and the Air Force; the Commandant of  the Marine Corps; the Under Secretary of Defense, Personnel and  Readiness; and other interested parties. We will also make copies available  to others upon request. In addition, the report will be available at no  charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions on this report, please contact me at  (202) 512-5559 or stewartd@gao. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page   of this report. GAO staff who made major contributions to this report are  listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: Our objectives were to determine (1) what Department of Defense (DOD)  data indicate are the number of reservists mobilized and deployed in  support of the Global War on Terrorism (GWOT), and the selected  demographic and deployment characteristics of those deployed and   (2) whether DOD’s reserve deployment and mobilization data and analyses  are reliable.
We identified, based on congressional interest and our knowledge of DOD  issues, selected demographic and deployment variables to review. We then  worked with the Defense Manpower Data Center (DMDC) to identify the  data fields within DMDC’s Contingency Tracking System (CTS) that best  provided information about the selected demographic and deployment  variables we wanted to analyze. Although we wanted to analyze the  locations to which reservists were deployed and the units with which  reservists were deployed, DMDC officials said, and we agreed based on  our review of the data, that the data were not reliable enough for those  purposes. Our selected variables included  the number of deployed reservists who volunteered for at least one  deployment;  the number of deployed reservists who have served one, two, or three or  more deployments;  the race and ethnicity of the deployed reservists;  the gender of the deployed reservists;  the state of residence of the deployed reservists;  the number of deployed reservists who were Selected Reserve, Individual  Ready Reserve, Standby Reserve, or Retired Reserve;  the number of deployed reservists who were citizens at the time of their  deployment;  the number of days the reservists were deployed; and  the top occupational areas for reservists deployed in support of GWOT.
To address objective 1, we obtained and analyzed data for September 2001  through June 2006 from DMDC’s CTS. CTS consists of two files—the  activation file, which tracks activations and mobilizations, and the  deployment file, which tracks deployments. Using CTS data from both  files, we analyzed the number of National Guard and Reserve  servicemembers mobilized and deployed in support of GWOT, as well as  selected demographic and deployment variables, using statistical analysis  software.
To address objective 2, we performed a data reliability assessment on the  data provided by DMDC from CTS’ activation and deployment files. We  requested DMDC reports that replicated our analyses and then compared  those report results to our analyses, and we reviewed the programming  code DMDC used to generate those reports. To assess the reliability of  CTS data, we obtained an understanding of the data, the file structure, the  sources of the data, and relevant DOD guidance. Specifically, we   (1) performed electronic testing of the data files for completeness (that is,  missing data), out-of-range values, and dates outside of valid time frames;  (2) assessed the relationships among data elements (for example,  determining whether deployment dates were overlapping since each  record in the deployment file is intended to represent one deployment);  (3) reviewed existing information about the data and the systems that  produced them; (4) interviewed department officials to identify known  problems or limitations in the data, as well as to understand the  relationship between the two files and how data are received from the  services, cleaned (“rebaselined”), and processed by DMDC; and   (5) compared “prerebaselined” mobilization data to “postrebaselined”  mobilization data to determine the extent to which the data changed as a  result of the cleaning effort. When we found discrepancies (for example,  overlapping deployment dates), we worked with DMDC to understand the  discrepancies.
In our interviews with DMDC officials, we discussed the purpose and uses  of CTS, the service data rebaselining effort and the internal controls for  verifying data analyses, monthly validation of data, and performing data  analyses. Similarly, we discussed data collection, processing, and  reliability issues as well as service-specific data issues and the rebaselining  effort with officials from the Office of the Assistant Secretary of Defense  for Reserve Affairs and from each of the reserve components, including  the U.S. Army National Guard, the U.S. Air National Guard, the U.S. Army  Reserve, the U.S. Navy Reserve, the U.S. Marine Corps Reserve, and the  U.S. Air Force Reserve. We also discussed the reliability of the services’  data, the rebaselining effort, and the results of a previous Joint Staff  review of the quality of service data within CTS with officials in the Joint  Chiefs of Staff Manpower and Personnel office. Finally, we interviewed  officials from the Deployment Health Surveillance Directorate and the  Army Medical Surveillance Activity about the quality of the deployment  data and how they use the data.
In the course of our review, we determined that some data fields were  highly unreliable. For example, electronic testing indicated that data on  location and reservist unit information were missing in many cases. Based  on our conversations with DMDC and our understanding of the data  system, we decided not to conduct lower level analyses (for example,  analyses of reservists’ assigned units) because the results would be less  reliable than aggregate level analyses. Although we are reasonably  confident in the reliability of most CTS data fields at the aggregate level,  because we could not compare source documentation from each of the  services to a sample of DMDC data, we could not estimate precise margins  of error. Consequently, we used the data for descriptive purposes, and we  did not base any recommendations on the results of our analyses. In  addition, we presented only higher level, aggregate data from fields that  we determined were sufficiently reliable for our reporting purposes. For  these purposes, and presented in this way, the CTS data we use are  sufficiently reliable with the following caveat: The Army had not  completed its rebaselining effort for mobilization data before the  completion of our review, and we could not, therefore, assess the  reliability of Army mobilization data to the same extent as those of the  other services. However, based on our electronic testing, data  comparisons, and interviews with officials, we believe that the data are  sufficiently reliable to present as descriptive information.
To assess the reliability of DMDC’s reports (that is, its own analyses) of  CTS data, we compared our independent analyses of National Guard and  Reserve servicemembers’ mobilization and deployment statistics with  results that DMDC provided from its own analyses of the same data. To  pinpoint differences in analytical assumptions, we reviewed the statistical  code DMDC used to produce its reports and compared it with our  programming code. Through an iterative process, we noted errors in  DMDC’s programs and requested changes and reruns of the data. We  worked with DMDC to ensure that discrepancies were not caused by  differences in our analytical assumptions. Where there were discrepancies,  we reached the following consensus on how to address them:    Removed the Coast Guard entries from our analyses of the CTS database  since, as we state in this report, the Coast Guard Reserve is under the day- to-day control of the Department of Homeland Security rather than DOD.
Combined a reservist’s Social Security number with his or her reserve  component to create a unique identifier. DMDC officials said they do this  because they are unsure where the source of the error is when they find  that a Social Security number corresponds with two reserve components  for a deployment during approximately the same time period. DOD’s  policy, when there is a duplicate Social Security number for more than  one reserve component, is to count both transactions. However, the use of  duplicate Social Security numbers results in overcounting. Specifically, the  June 2006 file had 38 reservists with overlapping mobilizations, 20  reservists with overlapping deployments, and more than 800 deployed  reservists who appeared to have legitimately changed components. To  compensate for the 58 “errors” where DMDC did not know which  mobilization or deployment to count, it double-counted all 58 reservists.  Likewise, the 800 deployed reservists who changed reserve components  during GWOT were also double-counted.
Removed reservists from all analyses when their reserve component  category is unknown, so that the numeric totals across analyses would be  consistent. DMDC officials said that this is an undocumented standard  operating procedure.
Utilized the reservists’ information for most recent deployment to provide  the most current information possible in cases where a reservist deployed  more than once.
Calculated the length of a reservist’s deployment by including both the day  the deployment began and the day on which the deployment ended. Thus,  the number of days deployed is inclusive of the beginning and end dates.
Combined the race categories for Asian, Native Hawaiian, and Other  Pacific Islander because, prior to 2003, the distinction between these two  groups was not captured in the data.
After clarifying and agreeing on the analytical assumptions, we again  reviewed DMDC’s code and compared its results with our own to  determine whether and why there were remaining discrepancies. We also  requested written documentation of DMDC’s internal control procedures  for the CTS data and, when no documentation was available, interviewed  knowledgeable officials about existing internal control procedures. Using  the framework of standards for internal control for the federal  government, we compared the information from those documents and  interviews with our numerous, iterative reviews of DMDC’s statistical  programs used to generate comparative reports to assess the reliability of  DMDC-generated reports from CTS. We determined that the reports  DMDC generated for our review were not sufficiently reliable for our  reporting purpose. Thus, we completed our own data analyses.
We performed our work from December 2005 through August 2006 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Data on Reservists Deployed in Support of GWOT through June 30, 2006
paragraphs: Our analysis of DOD data indicates that most reservists who deployed in  support of GWOT through June 30, 2006, were part of the Selected Reserve  (see table 5). In addition, California, Texas, Pennsylvania, and Florida had  the highest numbers of reservists who have deployed in support of GWOT  through June 30, 2006 (see table 6).
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, Cynthia Jackson, Assistant  Director; Crystal Bernard; Tina Kirschbaum; Marie A. Mak; Ricardo  Marquez; Julie Matta; Lynn Milan; Rebecca Shea; and Cheryl Weissman  made key contributions to this report.