Defense Health Care: Post-Deployment Health Reassessment Documentation Needs Improvement
Released: 2009-11-19T00:00:00

Letter

Background

DOD policy states that each military service must administer the PDHRA to active component, Reserve component, and separated servicemembers 90 to 180 days following deployment if they meet the following conditions: deployed for greater than 30 days, deployed to locations outside the continental United States, and deployed to locations without permanent military treatment facilities.

The military services, using service-specific databases, identify active and Reserve component servicemembers to whom the PDHRA requirement applies and notify these servicemembers via various methods, such as postcards, e-mail, telephone, and face-to-face contact.

The DOD policy that initiated the PDHRA program stated that the military services should contact servicemembers who separate before or during the reassessment period to offer them the opportunity to fill out the PDHRA questionnaire. However, there is no mechanism in place to offer separated servicemembers the opportunity to fill out the PDHRA questionnaire. Once these servicemembers have separated from military service, they have no further obligation to the military services and accordingly, cannot be required to fill out the PDHRA questionnaire. It can also be difficult to locate and contact servicemembers after they separate. Instead, the military services, with the exception of the Air Force, implemented policies to administer the PDHRA to active and Reserve component servicemembers as part of the separation process from the military. Although the Army formally implemented its policy to administer the PDHRA to servicemembers before they separate in April 2008, the practice had been in effect as early as January 2006. The Navy and Marine Corps implemented a similar policy in January 2009. According to Air Force officials, there are plans to implement a policy by December 2009.

The PDHRA involves two steps—active or Reserve component servicemembers fill out a PDHRA questionnaire and then discuss their questionnaire health concern responses with health care providers. The PDHRA questionnaire consists of a demographic section, a medical section, and a third section for a health care provider to fill out and sign. The demographic section asks for servicemembers to provide information such as date of birth, gender, and marital status. The medical section of the questionnaire asks servicemembers to self-report information on their current physical and mental health condition and concerns. DOD considers a PDHRA questionnaire complete when a health care provider reviews and signs the questionnaire, regardless of whether servicemembers fill out the entire PDHRA questionnaire or only the demographic section. (See app. II for a sample PDHRA questionnaire.)

Methods of PDHRA Administration

The Army, Air Force, Navy, and Marine Corps require active and Reserve component servicemembers to whom the PDHRA requirement applies to fill out at least the demographic section of the PDHRA questionnaire. Although filling out the medical section is voluntary, DOD officials estimate that less than 1 percent of servicemembers who fill out the PDHRA questionnaire decline to fill out the medical section, with the exception of the Air Force, which reported a higher declination rate.

Active component servicemembers typically fill out the PDHRA questionnaire online either prior to PDHRA on-site events or during such events, which are usually held at military installations. Similarly, Reserve component servicemembers may fill out the PDHRA questionnaire— administered by the contractor, LHI—online, prior to or during on-site events. In addition, LHI maintains call centers to administer the PDHRA to Reserve component servicemembers. Reserve component servicemembers who use LHI’s call centers may fill out the demographic and medical sections of the questionnaire online or, with the help of call center staff, may fill out both sections of the questionnaire through the call centers.

Health Care Provider Review and Referral for Further Evaluation

DOD requires that a health care provider review and discuss active and Reserve component servicemembers’ health concern responses on the PDHRA questionnaire, including any physical and mental health concerns that servicemembers self-identify on their questionnaires. Health care providers use professional judgment to decide whether a further evaluation is needed, based on servicemembers’ responses and other information revealed from the discussions with servicemembers. If referrals for a further evaluation are recommended, health care providers offer servicemembers information on obtaining a referral appointment. For example, a health care provider may provide information on obtaining an appointment at a military treatment facility or a Vet Center.

Once a referral for a further evaluation is made, DOD does not require the military services to follow up to determine if active and Reserve component servicemembers have made or attended an appointment generated as a result of the health care provider’s assessment. However, for Reserve component servicemembers, referral follow-up is part of LHI’s administration of the PDHRA. LHI staff follow up within 72 hours with Reserve component servicemembers who have been issued a referral for a further evaluation through the call centers to ensure that these servicemembers have the information needed to obtain an appointment and to encourage servicemembers to schedule an appointment. In addition, LHI staff attempt to contact Reserve component servicemembers 30 days after medical referrals are issued, whether issued through PDHRA on-site events or through the call centers, to ask whether they scheduled and attended an appointment. If servicemembers did not, LHI staff ask if assistance is needed in scheduling an appointment.

Each of the military services is required to electronically submit PDHRA questionnaires for both active and Reserve component servicemembers to DOD’s central repository. The central repository is the single source of DOD-level health surveillance information. The central repository contains data on diseases and medical events and longitudinal data on personnel and deployments, including information from DOD’s various deployment health assessments, such as the PDHRA. For Reserve component servicemembers who fill out PDHRA questionnaires through LHI, LHI staff are responsible for verifying that the questionnaires are submitted to the appropriate military service’s database. The military services are then responsible for submitting the questionnaires from their respective databases to DOD’s central repository.

DOD’s Deployment Health Quality Assurance Program

DOD established a deployment health quality assurance program in January 2004 to assess whether DOD’s deployment health assessments, including the PDHRA, are conducted as required. The deployment health quality assurance program relies on data from DOD’s central repository, data from the military services, and site visits to military installations to monitor and report on the extent of compliance among the military services with DOD’s deployment health requirements, such as the number of active and Reserve component servicemembers that filled out the PDHRA questionnaire. According to DOD’s deployment health quality assurance program manager, the program performed a site visit to Reserve units to validate data provided by the military services for the first time in October 2008, as part of its oversight of deployment health assessments, including the PDHRA. Prior to that, the quality assurance program only performed site visits to active component sites. On a quarterly basis, the quality assurance program reports to the military services on each service’s compliance with deployment health requirements. The quality assurance program also reports annually to the Armed Services Committees of the House of Representatives and Senate on site visit findings and on deployment health assessment data, including the number and percentage of servicemembers with PDHRA questionnaires in DOD’s central repository. Through its monitoring and reporting, DOD’s quality assurance program helps ensure that DOD’s deployment health assessments are conducted for active and Reserve component servicemembers as required.

DOD’s Contract with LHI to Administer the PDHRA to Reserve Component Servicemembers

DOD contracts with LHI to administer the PDHRA to Reserve component servicemembers. Although LHI administers the PDHRA to Reserve component servicemembers on behalf of DOD, the military services are responsible for identifying and notifying servicemembers to whom the PDHRA requirement applies and for submitting questionnaires to DOD’s central repository. DOD’s RHRP office is responsible for monitoring DOD’s contract with LHI, which includes the administration of the PDHRA to Reserve component servicemembers, as well as the provision of other health services for this population, such as immunizations, physical examinations, and dental examinations and X-rays.

DOD’s contract with LHI is a performance-based contract and, as such, establishes performance standards that the RHRP office uses in monitoring and assessing LHI’s performance in providing services to Reserve component servicemembers, including the administration of the PDHRA. For example, under its contract with DOD, LHI call center staff are required to answer 80 percent of incoming calls within 120 seconds. In monitoring LHI’s performance, DOD’s RHRP office helps ensure that the objective of the PDHRA program—to identify and address servicemembers’ health concerns, including mental health concerns, that emerge over time following deployments—is achieved for Reserve component servicemembers.

DOD’s Central Repository Did Not Contain PDHRA Questionnaires for a Substantial Percentage of Servicemembers Who Returned from Deployment to Iraq or Afghanistan

On the two occasions we queried DOD’s central repository, we did not find PDHRA questionnaires for a substantial percentage of the active and Reserve component servicemembers in our population of interest. DOD policy requires that the military services electronically submit questionnaires to DOD’s central repository, which DOD uses as a key source of health surveillance information. The first of our two queries of DOD’s central repository occurred on April 15, 2009. On this date, we found that the central repository contained PDHRA questionnaires for only 77 percent of the roughly 319,000 active component, Reserve, and National Guard servicemembers who, according to DOD DMDC deployment data, returned from deployment to Iraq or Afghanistan between January 1, 2007, and May 31, 2008.

We could not identify PDHRA questionnaires in DOD’s central repository on April 15, 2009, for a large number of servicemembers in our population—about 74,000 servicemembers—which represents the remaining 23 percent of our population of interest. We made our query nearly 1 year after the last servicemembers in our population returned from deployment. The percentage of PDHRA questionnaires absent from DOD’s central repository for our population of interest varied by military service and component. For example, among military service components, this percentage ranged from a low of about 10 percent to a high of about 61 percent. (For more information on the extent to which servicemembers in our population of interest did not have PDHRA questionnaires in DOD’s central repository as of April 15, 2009, see app. III.)

After determining that about 74,000 servicemembers in our population of interest did not have questionnaires in DOD’s central repository based on our first query, we asked the military services whether these servicemembers had PDHRA questionnaires that could be identified in the services’ own databases. With the help of the services, we found that approximately 7,000 servicemembers—or about 9 percent of the 74,000 servicemembers—had questionnaires in their respective military services’ databases, but not in DOD’s central repository. The number of questionnaires identified in the military services’ databases that were not in the central repository varied by military service—ranging from over 300 questionnaires for Air Force and Navy servicemembers to about 3,000 for Army and Marine Corps servicemembers.

On September 4, 2009, we queried DOD’s central repository again to update our April 2009 data and determine whether any progress had been made in reporting questionnaires to the central repository. On this second query—15 months after the last servicemembers in our population of interest returned from deployment—we found that DOD’s central repository was missing the same percentage of PDHRA questionnaires as had been missing in April. As a result of our September 2009 query, we still found PDHRA questionnaires for only 77 percent of the approximately 319,000 servicemembers in our original population of interest. While we identified slightly more questionnaires than we identified in our April query, we were still unable to identify questionnaires in the central repository for about 72,000 servicemembers. (See table 1.)

The absence of 72,000 PDHRA questionnaires from DOD’s central repository for servicemembers who should have filled them out hinders DOD’s deployment health quality assurance program from effectively assessing the military services’ compliance with PDHRA requirements. The program, which DOD established to assess whether DOD’s deployment health assessments are conducted as required, relies in part on the presence of PDHRA questionnaires in the central repository. These questionnaires document the extent to which servicemembers were given the opportunity to fill out the questionnaire, as required under DOD policy. DOD officials specifically cited the importance of this documentation for helping the quality assurance program ensure that servicemembers have the opportunity to have their health concerns identified and addressed. However, the absence of questionnaires from the central repository for servicemembers who should have filled them out suggests either that not all of these servicemembers filled out the questionnaire or that questionnaires were filled out, but were not incorporated into DOD’s central repository. When questionnaires for servicemembers from our population of interest are not in the central repository, DOD does not have reasonable assurance that all members of this vulnerable population of active component, Reserve, and National Guard servicemembers that deployed to Iraq or Afghanistan were administered the PDHRA questionnaire, which is intended to help identify deployment-related health concerns that emerge over time and facilitate the opportunity for servicemembers to address these concerns.

DOD Uses Four Methods to Monitor the Administration of the PDHRA to Reserve Component Servicemembers, but Does Not Clearly Document Its Monitoring

DOD’s Reserve Health Readiness Program (RHRP) office uses four methods to monitor LHI’s administration of the PDHRA to Reserve component servicemembers. However, in using these methods, DOD’s RHRP office does not always clearly document its monitoring of the PDHRA program. The office’s documentation does not allow DOD to have reasonable assurance that potential problems that may relate to the welfare and safety of servicemembers have been addressed and resolved.

DOD Uses Four Methods to Monitor the Administration of the PDHRA to Reserve Component Servicemembers

DOD’s RHRP office uses four methods to monitor LHI’s performance in administering the PDHRA to Reserve component servicemembers. More broadly, the RHRP office also uses the four methods to monitor whether the objective of the PDHRA program—to identify and address servicemembers’ health concerns that emerge over time following deployments—is being met for Reserve component servicemembers. The four methods, which are identified in DOD’s contract with LHI, are the following: Reviews of periodic reports. The RHRP office receives several periodic reports that DOD requires from LHI on LHI’s administration of the PDHRA to Reserve component servicemembers. DOD requires LHI to report aggregate information on LHI’s administration of the PDHRA, including reports on its compliance with performance standards. For instance, the RHRP office receives a report on LHI’s compliance with the performance standard requiring that LHI call center staff answer 80 percent of incoming calls within 120 seconds. In addition, LHI provides descriptive information on the number of Reserve component servicemembers administered the PDHRA, referred for a further evaluation, and contacted by LHI staff 30 days after receiving referrals. An RHRP official told us that the office reviews LHI’s reports to examine data related to the administration of the PDHRA and identifies potential problems that could pose risks to servicemembers and the objective of the PDHRA program. For example, the number of servicemembers referred each month is compared against historical data to monitor any changes in the rate at which servicemembers receive referrals for physical and mental health concerns.

Inspections of the administration of PDHRA. The RHRP office also conducts inspections related to LHI’s administration of the PDHRA to Reserve component servicemembers. The inspections have included observing PDHRA on-site events, at which servicemembers are administered the PDHRA, to assess the quality of LHI services delivered during the events. For instance, an RHRP official told us that he checks whether the events are staffed with a sufficient number of health care providers and administrative staff. In another instance, an RHRP official listened to LHI call center discussions between LHI staff and Reserve component servicemembers to examine how LHI staff administer the PDHRA.

Feedback on the administration of the PDHRA from military service officials. The RHRP office obtains informal feedback about how the PDHRA is being administered to Reserve component servicemembers through e-mail correspondence, telephone conversations, and in-person discussions with military service officials who are responsible for managing the PDHRA for their respective services. These officials told us that the RHRP office generally maintains open, informal communication channels through which they can and do express their concerns.

Weekly telephone discussions with LHI staff. An RHRP official said that weekly telephone discussions with LHI staff are held to obtain their feedback on the administration of the PDHRA to Reserve component servicemembers. In addition, during these discussions, an RHRP official and LHI staff discuss and address potential problems identified through the periodic reports, inspections, and military service feedback. The potential problems staff discuss may include those that could pose a risk to the PDHRA program objective and to the welfare and safety of Reserve component servicemembers. For example, some problems concerned how servicemembers were responding to questions on the PDHRA questionnaire. According to an RHRP official, these discussions serve as a forum to determine the actions DOD officials or LHI staff need to take to address identified problems and to verify that problems raised during previous discussions have been properly resolved.

DOD Generally Does Not Clearly Document Its Monitoring of the PDHRA Program

When monitoring the administration of the PDHRA to Reserve component servicemembers, DOD does not maintain clear documentation that is consistent with good management practices outlined in federal internal control standards. According to these standards, internal control activities such as monitoring should be clearly documented in a manner that is accurate, timely, and helps provide reasonable assurance that program objectives are being achieved. Further, such documentation should be properly managed and maintained so that it is readily accessible and should allow someone other than the assigned officials to understand the identified potential problem, the actions taken to address the problem, and whether these actions have resolved the problem.

Instead of adopting an approach that generated documentation consistent with management practices outlined in federal internal control standards, the RHRP office created an unsystematic, improvised approach for documenting potential problems that were identified through review of periodic reports, inspections, and feedback from military service officials and LHI staff. These problems may pose a risk to the PDHRA program objective. The RHRP office’s approach solely relies on agendas and e-mail correspondence to document these potential problems and the actions taken to resolve them.

Weekly agendas with related notes. Prior to the weekly discussions with LHI staff, a typed agenda is prepared for discussion that lists the potential problems identified through monitoring activities. During these discussions, annotated brief handwritten notes are made on an agenda item and these notes are used to indicate that certain actions need to be taken by DOD officials or LHI staff to address an identified problem. When it appears that problems have been resolved, instead of documenting how the problem was resolved, the problem is not included on the agenda for the next week’s discussion. A hard copy of the agenda is retained and filed in the RHRP office after each weekly discussion with LHI staff.

E-mail correspondence. The RHRP office receives and generates e-mail correspondence with LHI staff and military service officials that discusses potential problems identified through monitoring the administration of the PDHRA to Reserve component servicemembers.

We found that the RHRP office’s sole reliance on these agendas and e-mail correspondence did not always result in clear documentation—that is, documentation that is understandable and readily accessible to others outside the RHRP office. In particular, the agendas and e-mail correspondence we reviewed did not always clearly describe the decisions made and the actions taken to address identified problems in a manner that provides reasonable assurance that the problems have been resolved. An RHRP official acknowledged that he could not consistently rely on the agendas and e-mail correspondence to reconstruct information obtained through the office’s internal control responsibility to monitor and address problems associated with the administration of the PDHRA to Reserve component servicemembers. Instead, this official relies on memory to recall such information.

In our review of the documentation related to 15 potential problems we selected, we found that the RHRP office’s documentation generally did not clearly describe the problem, the actions taken to address the problem, and whether these actions have resolved the problem. For example, on two September 2008 agendas for the weekly discussions with LHI staff, the agendas include a potential problem identified by DOD officials that 25 to 30 percent of servicemembers were not responding to questions on the PDHRA questionnaire about the number of alcoholic drinks they consumed on a typical day, and 18 percent were not responding to the questions about whether they were depressed. The RHRP office asked LHI staff if they were finding similar nonresponses to these questions from Reserve component servicemembers. However, the subsequent agendas— which the RHRP office relies on to track problems—do not contain any additional information about how this potential problem was resolved. There is no further information, including any related e-mail correspondence, on how this potential problem was addressed. This example raises questions about whether Reserve component servicemembers’ mental health concerns—specifically those related to alcohol use and depression—are being consistently identified or whether any follow-up actions are needed to address this problem.

We also reviewed e-mail correspondence related to the 15 selected potential problems, dated between March 4, 2008, and May 19, 2009, and found that it often contained vague information about the identified problems and the actions taken to resolve them. Some of these problems relate to the welfare and safety of servicemembers and require more information than is present in the available documentation to understand whether or how the problems were resolved. For example, we asked about an instance in which a military service official reported that an LHI health care provider failed to document on a Reserve component servicemember’s PDHRA questionnaire why the servicemember with reported suicidal ideations did not get an immediate referral. The e-mail correspondence about this potential problem had gaps. For instance, while the e-mail correspondence indicates that the RHRP office asked LHI staff to look into the incident and contains a reply from LHI staff stating that they would investigate further, the e-mail correspondence does not document the final results of the LHI staff’s investigation, including whether or how this potential problem was resolved. Although this potential problem is listed on several agendas, the agendas do not provide any more information about the problem or how it was resolved. Instead, an RHRP official told us that he relied on his memory to explain to us what happened and how it was resolved. This official told us he requested that LHI staff remind its health care providers to fully document the results of their physical and mental health assessments on Reserve component servicemembers’ PDHRA questionnaires in the future. However, there is no documentation of this request to LHI staff or any documentation of plans to follow up to ensure that LHI staff carried out the request. (For more information on the documentation related to the 15 potential problems, see app. IV.)

In addition to being generally incomplete and unclear, the e-mail correspondence related to the potential problems is not readily accessible. E-mail correspondence about identified problems is placed into labeled folders within the office’s e-mail system, but the labels are general and e- mail correspondence could appropriately be placed in a number of folders. An official told us that, as a result, he struggles in remembering into which folder a specific e-mail was placed. For example, e-mail correspondence about an identified problem about an Army Reserve servicemember’s PDHRA questionnaire may be placed in either an Army PDHRA folder, a PDHRA complaints folder, or a standard operating procedures folder. Not having documentation that is readily accessible hinders the RHRP office’s ability to promptly ensure that identified problems have been properly addressed. In addition, should LHI’s performance diminish—for example, if LHI was not resolving identified problems—the lack of readily available documentation could compromise DOD’s ability to take appropriate action.

Conclusions

DOD established the PDHRA program in order to identify and address servicemembers’ health concerns—including mental health concerns— that emerge over time following deployments. The PDHRA questionnaire is a key tool in DOD’s efforts to assess the physical and mental health condition of servicemembers who have returned from deployments to Iraq and Afghanistan, where exposure to intense combat can place servicemembers at risk for developing conditions, such as post-traumatic stress disorder. DOD’s deployment health quality assurance program assesses the military services’ compliance with the requirement to administer the PDHRA to active and Reserve component servicemembers to help ensure that servicemembers have the opportunity to have their health concerns identified and addressed through the PDHRA.

However, our current findings show that the concerns we previously raised about DOD’s quality assurance program remain. Specifically, the absence of PDHRA questionnaires in DOD’s central repository for thousands of eligible active and Reserve component servicemembers continues to hinder the program’s ability to assess the extent to which these servicemembers fill out the PDHRA questionnaire and have the opportunity to identify any health concerns that emerge over time following deployments. DOD officials have identified the presence of questionnaires in the central repository as important for the efforts of DOD’s quality assurance program. Without this information, the program may not be able to accurately determine completion rates among the military services and thus provide reasonable assurance to DOD or to Congress that one of DOD’s key health assessments is being administered as required.

Ensuring that Reserve component servicemembers fill out the PDHRA questionnaire may be particularly important, as some evidence suggests that these servicemembers may be more likely to develop mental health conditions after returning from deployment when compared with their active component counterparts. Although DOD monitors the administration of the PDHRA to Reserve component servicemembers, documentation associated with this monitoring needs to be strengthened. The RHRP office has not always clearly documented—in a way that is consistent with federal internal control standards—information related to problems that may pose risks to the objective of the PDHRA program. Some of these potential problems also involve Reserve component servicemembers’ welfare and safety. The office’s improvised, unsystematic approach to documenting potential problems results in documentation that does not provide DOD with reasonable assurance that these problems have been resolved.

Recommendations for Executive Action

To help DOD obtain reasonable assurance that all active and Reserve component servicemembers to whom the PDHRA requirement applies are provided the opportunity to have their health concerns identified, we recommend that the Assistant Secretary of Defense for Health Affairs and the military services take steps to ensure that PDHRA questionnaires are included in DOD’s central repository for each of these servicemembers.

To ensure adequate documentation of problems that may pose risks to the objective of the PDHRA program for Reserve component servicemembers, we recommend that the Assistant Secretary of Defense for Health Affairs require the RHRP office to document the information obtained through monitoring the PDHRA program in a manner consistent with federal internal control standards.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD concurred with our two recommendations. Specifically, DOD agreed with our recommendation that it take steps to ensure that PDHRA questionnaires are included in DOD’s central repository for each servicemember to whom the PDHRA requirement applies. DOD stated that to implement this recommendation, it will take the following actions: work to correctly identify servicemembers who need to receive the PDHRA and work to identify and resolve any obstacles to transmission of data from the military services to AFHSC. Additionally, DOD stated that it sent a memorandum on October 15, 2009, to the military services’ Surgeons General re- emphasizing the importance of deployment health requirements. DOD also agreed with our recommendation concerning documentation of problems that may pose risks to the objective of the PDHRA program for Reserve component servicemembers. DOD stated that during our engagement, the RHRP office recognized a need to improve documentation of its monitoring of the PDHRA program. DOD also stated in its response that the RHRP office established a more distinct and clear electronic filing system and began documenting not only potential problems with the PDHRA program, but also their resolution in a manner that DOD reports is sufficiently comprehensive, accessible, and understandable. We did not assess this new approach to documentation. DOD’s written comments are included in their entirety in appendix V. DOD did not provide technical comments.

We are sending copies of this report to the Secretary of the Department of Defense. The report will be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have questions about this report, please contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI.

Appendix I: Scope and Methodology

To determine the extent to which post-deployment health reassessment (PDHRA) questionnaires are contained in the Department of Defense’s (DOD) central repository for active and Reserve component servicemembers who returned from deployment to Iraq or Afghanistan, we conducted a quantitative analysis using DOD data from two sources—the Defense Manpower Data Center’s (DMDC) Contingency Tracking System (CTS) and the Armed Forces Health Surveillance Center’s (AFHSC) Defense Medical Surveillance System (DMSS). DMDC’s CTS contains data on servicemembers deployed in support of the Overseas Contingency Operations—including data on servicemembers’ deployment dates and location of deployment. We used CTS data to identify a selected population of interest: active component, Reserve, and National Guard servicemembers who had returned from deployments of greater than 30 days to Iraq or Afghanistan between January 1, 2007, and May 31, 2008.

DMDC officials identified this population for us using servicemembers’ deployment dates and locations. If a servicemember had multiple deployments during this period, we received data on a servicemember’s most recent return from deployment during this time period. Although DOD initiated the PDHRA program in March 2005, the military services implemented the program at different times, with full implementation across all services in late 2006. In addition, servicemembers are not eligible to fill out a PDHRA questionnaire until 90 to 180 days after they have returned from deployment. Thus, when we requested data from DMDC in late fall 2008, we needed to focus on a population of servicemembers who had returned from deployment at least 180 days prior to our data request. Therefore, we focused our analysis on servicemembers who returned from deployment on or after January 1, 2007, and on or before May 31, 2008. As a secondary check that this population of interest had, in fact, deployed, we sent our population of interest to AFHSC officials, who compared our population to two other deployment rosters. The results of this match identified our final population of interest of approximately 319,000 servicemembers.

AFHSC’s DMSS is DOD’s central repository for PDHRA questionnaires and the military services are required to submit questionnaires to this central repository. To determine the extent to which servicemembers in our population of interest had PDHRA questionnaires in DOD’s central repository, we sent personal identifying information and beginning and end deployment dates from CTS for the servicemembers in our population of interest to AFHSC. AFHSC officials matched our population of interest to the PDHRA questionnaires in DMSS using a servicemember’s personal identifying information and the beginning and end deployment dates as recorded in CTS and as reported by the servicemember on the PDHRA questionnaire. AFHSC officials then sent us data from identified questionnaires that had been incorporated into DMSS as of April 15, 2009.

As the military services collect PDHRA questionnaires in their own databases before transmitting them to AFHSC’s DMSS, we then asked officials from the military services to query their own databases to identify PDHRA questionnaires for servicemembers in our population of interest for which a PDHRA questionnaire could not be identified in DMSS. We provided the military services with a servicemember’s personal identifying information and deployment end date from CTS and asked them to query their databases for any questionnaires that they would consider to be ready to transmit to AFHSC that were filled out after the deployment end date listed in CTS. The military services then queried their own databases and, for each servicemember that we sent to them for which they could identify a PDHRA questionnaire, returned to us the end deployment date listed on the PDHRA questionnaire and date of questionnaire completion. If a servicemember filled out multiple PDHRA questionnaires after the deployment end date listed in CTS, we received information from each of these questionnaires. We received these data from the military services in late May and June 2009. We then matched our population of interest to the information from the PDHRA questionnaires received from the military services’ databases using a servicemember’s personal identifying information and the end deployment date as recorded in CTS and as reported by the servicemember on the PDHRA questionnaire.

Finally, in September 2009, we obtained additional data from AFHSC to update our April 2009 data. AFHSC officials matched information from the servicemembers in our population of interest without a questionnaire in DMSS as of April 15, 2009, to the PDHRA questionnaires in DMSS as of September 4, 2009, using a servicemember’s personal identifying information and the beginning and end deployment dates as recorded in CTS and as reported by the servicemember on the PDHRA questionnaire. AFHSC officials then sent us data from identified questionnaires that had been incorporated into DMSS as of September 4, 2009.

We conducted data reliability assessments for each DOD and military service data source we used by reviewing related documentation, interviewing knowledgeable agency officials, and performing electronic data testing for missing data, outliers, and obvious errors. We determined that these data sources were sufficiently reliable for our purposes. We determined only the extent to which questionnaires could be identified in AFHSC’s DMSS or the military services databases for our population of interest, and not the extent to which the servicemembers actually filled out a PDHRA questionnaire.

We interviewed officials from DMDC and AFHSC, the DOD deployment health quality assurance program, and Army, Air Force, Navy, and Marine Corps officials involved in the collection and transfer of PDHRA questionnaires from the military services’ databases to AFHSC. We also reviewed DOD policies, as well as those of the Army, Air Force, Navy, and Marine Corps, for submitting PDHRA questionnaires to DOD’s central repository. Finally, we reviewed our prior work on DOD’s deployment health quality assurance program.

To determine how DOD monitors the administration of the PDHRA to Reserve component servicemembers, we reviewed DOD’s policies for monitoring its contract with Logistics Health, Inc. (LHI), the contractor that administers the PDHRA to Reserve component servicemembers. We also interviewed officials with DOD’s Reserve Health Readiness Program (RHRP)—the DOD office responsible for monitoring LHI—along with military service officials responsible for managing the administration of the PDHRA to Reserve component servicemembers. We reviewed DOD’s contract with LHI and obtained and analyzed contractually required reports and other documentation on LHI’s performance related to the PDHRA portion of DOD’s contract with LHI. We visited LHI’s headquarters in La Crosse, Wisconsin, where we interviewed LHI staff to confirm our understanding of how LHI staff administers the PDHRA to Reserve component servicemembers. We additionally interviewed officials with the U.S. Army Medical Research Acquisition Activity, which provides support to DOD for the contract with LHI. We reviewed DOD’s monitoring of the administration of the PDHRA to Reserve component servicemembers to determine the extent to which these monitoring efforts met or were consistent with GAO’s Standards for Internal Control in the Federal Government. Internal controls include components of an organization’s management that provide reasonable assurance that program objectives are being achieved.

The RHRP office receives contractually required reports from LHI on its administration of the PDHRA and we obtained and reviewed monthly reports from May 2008 through April 2009 for the following four required reports: (1) the PDHRA monthly activity report, (2) the call center access report, (3) the data entry report, and (4) the PDHRA customer satisfaction survey report. We selected the PDHRA monthly activity and the call center access reports to review because the RHRP office told us that these reports provide information that is needed to monitor LHI’s activities in administering the PDHRA to Reserve component servicemembers. The PDHRA monthly activity report provides aggregate information on LHI’s PDHRA administration, such as the number of Reserve component servicemembers administered the PDHRA, referred for a further evaluation, and contacted by LHI 30 days after receiving referrals. The call center access report provides information on LHI’s performance in operating its call center. In addition, we selected the data entry and PDHRA customer satisfaction survey reports to review because both reports provide information on performance standards associated with the PDHRA that LHI must meet to comply with the contract. The data entry report provides information on LHI’s performance in entering PDHRA data into military services’ databases, and the PDHRA customer satisfaction survey report provides information on whether LHI meets a performance standard associated with servicemember feedback on LHI’s administration of the PDHRA. We also reviewed the documentation associated with the RHRP office’s inspections of LHI’s administration of the PDHRA from October 1, 2007, through May 6, 2009. We reviewed a list, provided by the RHRP office, of the feedback received from military service officials between January 1, 2008, and April 30, 2009.

To obtain more in-depth information on how DOD documents the information it obtains through its monitoring efforts, we examined documentation maintained by the RHRP office on its monitoring. Specifically, we reviewed the RHRP office’s agendas from the weekly telephone discussions with LHI staff and the office’s e-mail correspondence, which the RHRP office uses to document its monitoring. We reviewed approximately 70 of the RHRP office’s agendas for the period between October 1, 2007, and May 11, 2009. From the agendas and the provided list of military service feedback, we judgmentally selected 15 potential problems to review in further detail and asked the RHRP office to provide us with all of the documentation available on these problems, including any e-mail correspondence that related to the problem. We selected these 15 potential problems because the subject of each of these may have involved welfare and safety concerns for Reserve component servicemembers. The RHRP office provided us with e-mail correspondence dated between March 4, 2008, and May 19, 2009, related to the 15 potential problems. We reviewed the agendas and the available e- mail correspondence for the 15 selected problems to determine the actions taken to resolve the problems and how this information is documented and maintained. Our findings related to the documentation of the 15 potential problems are for illustrative purposes only and are not generalizable to other RHRP office documentation.

We conducted this performance audit from October 2008 to October 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Sample Post-Deployment Health Reassessment Questionnaire (DD 2900), January 2008

Servicemembers fill out the post-deployment health reassessment (PDHRA) questionnaire electronically on form DD 2900, which was originally issued in June 2005. The Department of Defense (DOD) issued a revised form DD 2900 in January 2008.

Appendix III: Post-Deployment Health Reassessment Questionnaires in DOD’s Central Repository and Services’ Databases

Department of Defense (DOD) policy requires that the military services electronically submit post-deployment health reassessment (PDHRA) questionnaires to DOD’s central repository, which DOD uses as a key source of health surveillance information. We queried DOD’s central repository on two occasions—April 2009 and September 2009. On April 15, 2009, we found that for approximately 23 percent of the roughly 319,000 servicemembers who, according to DOD deployment data, returned from deployment to Iraq or Afghanistan between January 1, 2007, and May 31, 2008, we could not identify questionnaires in the central repository (see table 2).

After determining that about 74,000 servicemembers in our population of interest did not have questionnaires in DOD’s central repository based on our first query, we asked the military services whether these servicemembers had PDHRA questionnaires that could be identified in the services’ own databases. With the help of the services, we found that approximately 7,000 servicemembers had questionnaires in their respective military services’ databases, but not in DOD’s central repository (see table 3). On our second query of the central repository, which occurred on September 4, 2009, questionnaires for about 1,000 of these 7,000 servicemembers were in the central repository.

Appendix IV: DOD’s Documentation of Its Monitoring of the Administration of the Post- Deployment Health Reassessment

The Department of Defense (DOD) contracts with Logistics Health, Inc. (LHI) to administer the post-deployment health reassessment (PDHRA) to Reserve component servicemembers. DOD’s Reserve Health Readiness Program (RHRP) is responsible for monitoring LHI’s administration of the PDHRA. To obtain more in-depth information on how the RHRP office documents the information it obtains through its monitoring of the administration of the PDHRA to Reserve component servicemembers, we examined documentation maintained by the RHRP office on its monitoring efforts. Specifically, from the potential problems that the RHRP office had identified as possibly posing a risk to the objective of PDHRA program, we judgmentally selected 15 to review in detail and obtained all available RHRP documentation from the RHRP office on those problems. We selected these 15 potential problems because the subject of each of these may have involved welfare and safety concerns for Reserve component servicemembers.

We reviewed RHRP’s documentation to determine the extent to which RHRP maintains documentation in a manner consistent with GAO’s federal internal control standards. In particular, we examined the extent to which RHRP’s documentation clearly documented any actions taken to address a problem and indicated whether the problem had been resolved. In general, the selected RHRP documentation we reviewed did not meet these standards. Nine of the 15 selected potential problems lacked documentation on the actions taken to address the problems and/or lacked documentation of the problems’ resolutions. Four of the 15 problems had documentation of the actions taken and their resolutions, however, RHRP’s documentation was not sufficiently clear to allow us to independently understand what actions had been taken to address the problem or the problems’ resolutions. Rather, an RHRP official had to explain to us what had occurred. Two of the 15 problems had documentation that allowed us to understand the actions taken to address the problems and the problems’ resolutions. Table 4 summarizes the results of our analysis of RHRP’s documentation.

For these potential problems, documented the nature of a servicemember’s suicidal ideations on the PDHRA questionnaire. documentation was lacking on the actions taken to address the problem and/or the problem’s resolution.

For these potential problems, the RHRP call center to be administered the PDHRA may have been denied LHI PDHRA services and not administered the PDHRA. office had some available documentation, but this documentation was not sufficiently clear to allow us to independently determine the actions taken to address each problem and to determine the ultimate resolution of each problem.

11. Servicemembers who should not have been called by LHI may have been inappropriately called by LHI. 12. A PDHRA event may have lacked sufficient staff. 13. Health care providers may not have been documenting needed referrals for further evaluations when servicemembers declined the referrals.

For these potential problems, the RHRP through the LHI call center were not receiving the same informational brochures as servicemembers administered the PDHRA at PDHRA on-site events. office had documentation of the actions taken to address each problem and the problem’s resolution.

15. Reserve component servicemembers did not know what to do to set up an appointment for a further evaluation.

Appendix V: Comments from the Department of Defense

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Mary Ann Curran, Assistant Director; Katherine L. Amoroso; Helen T. Desaulniers; Michael Erhardt; Martha A. Fisher; Krister Friday; Martha Kelly; Carolyn Kirby; Carolina Morgan; Lisa A. Motley; Julie E. Pekowski; William Woods; and Suzanne Worth made key contributions to this report.

Related GAO Products

Federal Contractors: Better Performance Information Needed to Support Agency Contract Award Decisions. GAO-09-374. Washington, D.C.: April 23, 2009.

Military Operations: DOD Needs to Address Contract Oversight and Quality Assurance Issues for Contracts Used to Support Contingency Operations. GAO-08-1087. Washington, D.C.: September 26, 2008.

DOD Systems Modernization: Maintaining Effective Communication Is Needed to Help Ensure the Army’s Successful Deployment of the Defense Integrated Military Human Resources System. GAO-08-927R. Washington, D.C.: September 8, 2008.

Defense Health Care: Oversight of Military Services’ Post-Deployment Health Reassessment Completion Rates Is Limited. GAO-08-1025R. Washington, D.C.: September 4, 2008.

Electronic Health Records: DOD and VA Have Increased Their Sharing of Health Information, but More Work Remains. GAO-08-954. Washington, D.C.: July 28, 2008.

VA and DOD Health Care: Administration of DOD’s Post-Deployment Health Reassessment to National Guard and Reserve Servicemembers and VA’s Interaction with DOD. GAO-08-181R. Washington, D.C.: January 25, 2008.

Defense Health Care: Comprehensive Oversight Framework Needed to Help Ensure Effective Implementation of a Deployment Health Quality Assurance Program. GAO-07-831. Washington, D.C.: June 22, 2007.

Military Personnel: DMDC Data on Officers’ Commissioning Programs is Insufficiently Reliable and Needs to be Corrected. GAO-07-372R. Washington, D.C.: March 8, 2007.

Military Personnel: Actions Needed to Strengthen Management of Imminent Danger Pay and Combat Zone Tax Relief Benefits. GAO-06-1011. Washington, D.C.: September 28, 2006.

Post-Traumatic Stress Disorder: DOD Needs to Identify the Factors its Providers Use to Make Mental Health Evaluation Referrals for Servicemembers. GAO-06-397. Washington, D.C.: May 11, 2006.

Military Pay: Inadequate Controls for Stopping Overpayments of Hostile Fire and Hardship Duty Pay to Over 200 Sick or Injured Army National Guard and Army Reserve Soldiers Assigned to Fort Bragg. GAO-06-384R. Washington, D.C.: April 27, 2006.

Contract Management: Opportunities to Improve Surveillance on Department of Defense Service Contracts. GAO-05-274. Washington, D.C.: March 17, 2005.

VA and Defense Health Care: More Information Needed to Determine If VA Can Meet an Increase in Demand for Post-Traumatic Stress Disorder Services. GAO-04-1069. Washington, D.C.: September 20, 2004.

Military Pay: Army Reserve Soldiers Mobilized to Active Duty Experienced Significant Pay Problems. GAO-04-911. Washington, D.C.: August 20, 2004.

Defense Health Care: Quality Assurance Process Needed to Improve Force Health Protection and Surveillance. GAO-03-1041. Washington, D.C.: September 19, 2003.