Defense Health Care: 2008 Access to Care Surveys Indicate Some Problems, but Beneficiary Satisfaction Is Similar to Other Health Plans
Released: 2010-03-31T00:00:00

Letter

Background

In fiscal year 2009, over 9 million beneficiaries were eligible to receive health care and mental health care through DOD’s TRICARE program. Under TRICARE, beneficiaries have choices among various benefit options and may obtain care from either military treatment facilities or civilian providers.

Composition of TRICARE’s Beneficiary Population

TRICARE beneficiaries fall into different categories: (1) active duty personnel and their dependents, (2) National Guard and Reserve servicemembers and their dependents, and (3) retirees and their dependents or survivors. Retirees and certain dependents and survivors who are entitled to Medicare Part A and enrolled in Part B, and who are generally age 65 and older, are eligible to obtain care under a separate program called TRICARE for Life. As shown in figure 1, active duty personnel and their dependents represented 32 percent of the beneficiary population, while National Guard and Reserve servicemembers and their dependents represented 14 percent.

Ntionl Guard nd Reerve ervicememer nd dependentActive dty peronnel nd dependentRetiree nd their dependent or survivor(generlly nder ge 65)

TRICARE beneficiaries under 65 years of age who are eligible for Medicare Part A on the basis of disability or end-stage renal disease are eligible for TRICARE for Life if they enroll in Medicare Part B.

TRICARE’s Benefit Options

TRICARE provides its benefits through several options for its non- Medicare-eligible beneficiary population. These options vary according to TRICARE beneficiary enrollment requirements, the choices TRICARE beneficiaries have in selecting civilian and military treatment facility providers, and the amount TRICARE beneficiaries must contribute toward the cost of their care. Table 1 provides additional information about these options.

Beneficiaries’ Use of TRICARE

Claims data from fiscal years 2006 to 2009 show that the percentage of claims paid for using TRICARE Prime and TRICARE Extra has gradually increased, while the percentage of claims paid for using TRICARE Standard has declined 2 to 3 percentage points each year. (See fig. 2.) Moreover, in 2006 we reported that in fiscal year 2005 about 1.8 million beneficiaries who were eligible for TRICARE Standard or Extra elected not to use their benefits. In fiscal year 2009, about 926,000 beneficiaries who were eligible for TRICARE Standard, TRICARE Extra, and TRICARE Reserve Select elected not to use their benefits.

Network and Nonnetwork Providers under TRICARE

In order for network and nonnetwork civilian providers to be authorized to provide care and be reimbursed under TRICARE, they must be licensed by their state, accredited by a national organization (if one exists), and meet other standards of the medical community. Individual TRICARE- authorized civilian providers can include health care providers, such as primary care physicians and specialists, as well as mental health care providers, including clinical psychologists.

There are two types of authorized civilian providers—network and nonnetwork providers.

Network providers are TRICARE-authorized providers who enter a contractual agreement with a managed care support contractor to provide health care to TRICARE beneficiaries. By law, TRICARE maximum allowable reimbursement rates must generally mirror Medicare rates, but network providers may agree to accept lower reimbursements as a condition of network membership. However, network civilian providers are not obligated to accept all TRICARE beneficiaries seeking care. For example, a network civilian provider may decline to accept TRICARE beneficiaries as patients because the provider’s practice does not have sufficient capacity.

Nonnetwork providers are TRICARE-authorized providers who do not have a contractual agreement with a managed care support contractor to provide care to TRICARE beneficiaries. Nonnetwork civilian providers have the option of charging up to 15 percent more than the TRICARE reimbursement rate for their services on a case-by-case basis. The beneficiary is responsible for paying the extra amount billed in addition to required co-payments.

NDAA 2008 Requirements for Beneficiary and Provider Surveys to Determine Access to Care for Nonenrolled TRICARE Beneficiaries

The NDAA 2008 directed DOD to determine the adequacy of the number of health care and mental health care providers that currently accept nonenrolled beneficiaries as patients under TRICARE. Specifically, the mandate directed DOD to conduct surveys of beneficiaries and providers in at least 20 Prime Service Areas and 20 non-Prime Service Areas in each of fiscal years 2008 through 2011. The mandate also directed DOD to give a high priority to locations having high concentrations of Selected Reserve servicemembers. Additionally, the NDAA 2008 required DOD to consult with representatives of TRICARE beneficiaries and health care and mental health care providers to identify locations where nonenrolled beneficiaries have experienced significant access-to-care problems, and to survey health care and mental health care providers in these areas. The NDAA 2008 also required that specific types of information be requested in the surveys. For example, the mandate stated that the provider survey must include questions to determine whether providers are aware of TRICARE. Within DOD, TMA has primary responsibility for designing and implementing the beneficiary and provider surveys.

Implementation of DOD’s 2008 Beneficiary and Provider Surveys Followed OMB Survey Standards and Generally Addressed Requirements Outlined in the NDAA 2008

In implementing the first year of the beneficiary and provider surveys, TMA followed the OMB standards for statistical surveys that we reviewed. However, TMA did identify an error in the geographic categorization of its participants for both surveys and has taken steps to correct it. In addition, TMA generally addressed the requirements outlined in the NDAA 2008 for both of its surveys but did not give a high priority to selecting areas with a high concentration of Selected Reserve servicemembers. Instead, for both surveys, TMA randomly selected geographic areas to produce results that can be generalized to the populations from which the survey samples were drawn. TMA plans to cover the entire United States at the end of the 4-year survey period, which will include any locations with higher concentrations of Selected Reserve servicemembers.

TMA Followed the OMB Standards We Reviewed in Designing and Implementing Its Beneficiary and Provider Surveys

In implementing its first round of beneficiary and provider surveys, TMA’s methodology for both of the multiyear surveys is consistent with the OMB standards for statistical surveys that we reviewed. (See app. I for our list of selected OMB standards.) These standards document the professional principles and practices that federal agencies are required to follow and the level of quality and effort expected in statistical activities. For example, OMB standards recommend that agencies develop a survey design that includes a methodology for identifying the target population.

OMB standards also suggest that federal agencies ensure that the list of eligible survey participants representing the target population is evaluated for accuracy. In conducting its 2008 beneficiary and provider surveys, TMA representatives noted that they identified an error in the geographic categorization of eligible survey participants for both surveys. Specifically, TMA erroneously categorized some beneficiaries and providers as being located in 11 non-Prime Service Areas when in fact they were in Prime Service Areas. According to a TMA representative, this occurred as a result of a computer programming error. TMA identified this error after it began fielding the beneficiary survey in June 2008 but prior to fielding the provider survey, which began in December 2008. As a result, TMA surveyed approximately 9,000 fewer beneficiaries than intended in those 11 non-Prime Service Areas. TMA plans to correct this error in future years by surveying additional beneficiaries in the affected non-Prime Service Areas.

TMA Generally Addressed Requirements Outlined in the Mandate for Its Beneficiary and Provider Surveys

TMA generally addressed the requirements outlined in the mandate during the implementation of its 2008 beneficiary and provider surveys, but because of methodological considerations TMA used a different approach for its selection of survey areas. (See app. II for a more detailed description of DOD’s survey methodology.) Overall, the mandate outlined specific survey requirements, including the number and priority of areas to be surveyed each year, the content for each type of survey, and the use of benchmarks, which can be used to assess survey results. (See table 2.)

According to a TMA official responsible for implementing the surveys, TMA did not give a high priority to areas where higher concentrations of Selected Reserve servicemembers live, as specified in the mandate, because it decided to randomly select the areas to be surveyed in order to produce results that can be generalized to the populations from which the survey samples are selected. Moreover, at the end of the 4-year survey period for the beneficiary and provider surveys, TMA will have surveyed all areas of the United States, thereby including any locations with a higher concentration of Selected Reserve servicemembers. A TMA official also told us that TMA conducted additional analyses of the 2008 beneficiary survey results for TRICARE Reserve Select beneficiaries to obtain additional information about the Selected Reserve servicemembers, and plans to do so in the remaining 3 years of the survey. TMA found that its 2008 results indicate that TRICARE access is no different for the Selected Reserve servicemembers than for other beneficiaries in the surveyed areas.

A Higher Percentage of Nonenrolled Beneficiaries Reported Problems Accessing Certain Providers in Prime Service Areas, Though in General Beneficiaries Rated Their Satisfaction Similarly to Users of Other Health Plans

The first year’s results of TMA’s 4-year beneficiary survey are representative of the nonenrolled beneficiary population in the combined Prime Service Areas and combined non-Prime Service Areas that were selected for the 2008 survey. Based on our analysis of these results, we estimated that a higher percentage of nonenrolled beneficiaries in surveyed Prime Service Areas experienced problems accessing care from network or nonnetwork primary care physicians or nurses than beneficiaries in surveyed non-Prime Service Areas. However, we could not determine whether beneficiary access problems were related to TRICARE network providers because the survey did not ask beneficiaries to link problems accessing care with network or nonnetwork providers. However, we did find that the types of access problems beneficiaries experienced, such as providers not accepting TRICARE payments, were similar in Prime Service Areas and non-Prime Service Areas. In addition, we found that nonenrolled beneficiaries in both the surveyed Prime Service Areas and the surveyed non-Prime Service Areas rated their health care satisfaction similarly to each other and to beneficiaries of commercial health care plans, but slightly lower than Medicare beneficiaries.

Despite a Low Response Rate, TMA’s 2008 Beneficiary Survey Results Are Representative of the Nonenrolled Beneficiary Population for the Combined Areas Surveyed

Despite an overall response rate of about 38 percent, TMA’s 2008 beneficiary survey results are representative of the nonenrolled beneficiary population for the combined Prime Service Areas and combined non-Prime Service Areas surveyed. (See fig. 3.) Because of the low response rate, TMA conducted a nonresponse analysis to determine whether the responses it received were representative of the surveyed population. The nonresponse analysis indicated that there were no differences in demographic characteristics and health coverage between beneficiary survey respondents and nonrespondents. As a result, the results can be generalized to the combined areas surveyed—that is, the survey produced results that are representative of all nonenrolled beneficiaries in the 20 surveyed Prime Service Areas and all nonenrolled beneficiaries in the 20 surveyed non-Prime Service Areas.

2008 Survey Results Indicate That a Higher Percentage of Nonenrolled Beneficiaries in Prime Service Areas Experienced Problems Accessing Care from Primary Care Physicians or Nurses Than Those in Non-Prime Service Areas

Based on our analysis of the 2008 survey results, we estimated that a higher percentage of nonenrolled beneficiaries in surveyed Prime Service Areas experienced problems accessing care from network or nonnetwork primary care physicians or nurses than nonenrolled beneficiaries in surveyed non-Prime Service Areas. Specifically, about 30 percent of beneficiaries in Prime Service Areas experienced problems finding a civilian primary care physician or nurse, compared to about 24 percent of beneficiaries in non-Prime Service Areas. (See table 3.) While we found differences in access to care problems between nonenrolled beneficiaries in the surveyed Prime Service Areas and non-Prime Service Areas for other types of providers, these differences were not statistically significant at the 95 percent confidence level.

We could not determine the extent to which beneficiaries’ access problems were related to TRICARE network or nonnetwork providers because the 2008 beneficiary survey did not ask beneficiaries to link their problems accessing care with network or nonnetwork providers. However, in our analysis of the 2008 beneficiary survey results, we found that the specific types of access problems experienced by beneficiaries in Prime Service Areas and non-Prime Service Areas are similar. (See table 4.) For example, we found that the problem most commonly reported by nonenrolled beneficiaries in both Prime and non-Prime Service Areas, regardless of the type of provider, was that their provider was not accepting TRICARE payments. Other commonly reported reasons varied by provider type.

Nonenrolled Beneficiaries in Prime Service Areas and Non-Prime Service Areas Surveyed in 2008 Rated Satisfaction with Their Health Care Similarly to Each Other and to Beneficiaries of Commercial Health Care Plans

Despite differences in the percentage of beneficiaries in surveyed Prime and non-Prime Service Areas reporting problems accessing care from primary care physicians or nurses, our analysis showed that nonenrolled beneficiaries’ ratings for several categories of health care were similar in the surveyed Prime Service Areas and non-Prime Service Areas. Specifically, our analysis of beneficiaries’ ratings for four categories of health care—their satisfaction with providers of primary care and specialty care, their health care, and their health plan—indicates no statistically significant difference between beneficiaries in surveyed Prime Service Areas and non-Prime Service Areas. For example, we estimated that 75 percent of beneficiaries in surveyed Prime Service Areas and 74 percent of beneficiaries in surveyed non-Prime Service Areas rated their health plan “7” or higher on a 0 to 10 scale (with 0 being the worst possible).

Estimated ratings for nonenrolled beneficiaries in surveyed areas are also similar to the estimated ratings of beneficiaries in commercial health plans, based on data we analyzed from HHS’s 2008 Consumer Assessment of Healthcare Providers and Systems survey. Specifically, in all four satisfaction categories there are no statistically significant differences in the estimated percentage of beneficiaries who rated their satisfaction “7” or higher. (See fig. 4.) However, estimated ratings for nonenrolled beneficiaries in surveyed areas were slightly lower than estimated ratings of Medicare beneficiaries across three of the satisfaction categories— primary care physician or nurse, specialist physician, and health plan.

2008 Provider Survey Results Are Not Representative of All Providers in Surveyed Areas but Provide Limited Information That Indicates Differences among Respondents’ Awareness and Acceptance of TRICARE

Although the first year’s results of TMA’s 4-year provider survey are not representative of all providers in the areas surveyed, the results we analyzed do provide information about access to care based on the specific views of the respondents. According to a TMA official, generalizability of provider survey results to the entire country will likely be possible at the end of the 4-year survey period. Our analysis of the 2008 provider survey results indicate that a lower percentage of respondents from Prime Service Areas were aware of TRICARE and were accepting new TRICARE patients than providers who responded from non-Prime Service Areas. The survey results also indicate that respondents from the additional areas TMA surveyed reported levels of awareness and acceptance of TRICARE that were similar to respondents in non-Prime Service Areas. Additionally, there were differences between the responding physicians (primary care physicians and specialists) and mental health providers (psychiatrists, certified clinical social workers, clinical psychologists, and others) regarding their awareness and acceptance of TRICARE.

2008 Provider Survey Results Are Not Representative of the Provider Population in the Areas TMA Surveyed

Unlike the 2008 beneficiary survey, the results of the 2008 provider survey are not representative of all physicians and mental health providers in the areas TMA surveyed. The 2008 provider survey was administered in the same 20 Prime Service Areas and 20 non-Prime Service Areas as the beneficiary survey, as well as 21 additional locations that were identified as having access-to-care problems, with an overall response rate of about 45 percent. (See fig. 5.) Because of the low response rate, TMA conducted a nonresponse analysis, and the results of this analysis indicated that there were differences among those who responded to the provider survey and those who did not. Specifically, those who did not respond to the 2008 provider survey were less likely to be aware of TRICARE and less likely to accept TRICARE reimbursement as payment for services. As a result of these differences—even though the survey sample was randomly selected—the survey results cannot be generalized to all physicians and mental health providers in the areas surveyed and can be presented only as the specific views of the respondents. Similarly, because the 2008 survey results cannot be generalized, we did not compare them with the 2008 beneficiary survey results.

A TMA official stated that because more responses will have been obtained by the end of the 4-year survey, generalizability of provider survey results to the entire country will likely be possible. Moreover, the TMA official noted that TMA has decided to redesign the method of selecting mental health providers in the 2009 survey to increase the number of responses and the likelihood that survey results could be generalized.

2008 Provider Survey Results Indicate Differences among Respondents’ Awareness and Acceptance of TRICARE

Our review of the 2008 provider survey results indicated differences in awareness and acceptance of TRICARE among respondents in Prime Service Areas, non-Prime Service Areas, and the additional Hospital Service Areas TMA surveyed. (See table 5.) Specifically, a lower percentage of responding providers—physicians and nonphysician mental health providers—from Prime Service Areas were aware of the TRICARE program and were accepting new TRICARE patients, if they were accepting any new patients or any new Medicare patients, than providers who responded from non-Prime Service Areas or Hospital Service Areas. For example, 64 percent of the respondents in the surveyed Prime Service Areas who reported that they are accepting any new patients reported that they would accept nonenrolled TRICARE beneficiaries as new patients, compared to 76 percent of respondents in the surveyed non-Prime Service Areas and 72 percent of respondents in the surveyed Hospital Service Areas. Additionally, survey results indicate that in Hospital Service Areas, respondents reported awareness and acceptance of TRICARE that was similar to that of respondents in non-Prime Service Areas.

The reason most often cited by respondents in both Prime Service Areas and non-Prime Service Areas for not accepting nonenrolled beneficiaries as new patients, if they were accepting any new patients at all, was that they were not aware of the TRICARE program. Other reasons included concerns about low reimbursement rates and that the provider did not participate in TRICARE’s provider network. Respondents in Hospital Service Areas reported similar reasons, such as concerns about low reimbursement rates and not being aware of the TRICARE program, with the most cited reason being that they were not participating in TRICARE’s provider network.

The physicians and mental health providers who responded to the survey differed in their awareness and acceptance of TRICARE. Specifically, a higher percentage of responding physicians reported awareness and acceptance of TRICARE than the mental health providers who responded. (See table 6.) For example, 81 percent of the physicians who responded reported that they would accept new TRICARE patients, if they were accepting any new patients at all, compared to 50 percent of the mental health providers who responded.

Agency Comments

We received comments on a draft of this report from DOD. (See app. VI.) DOD concurred with our overall findings and provided technical comments, which we incorporated where appropriate.

We are sending copies of this report to the Secretary of Defense and appropriate congressional committees. The report is also available at no charge on GAO’s Web site at http://www.gao.gov.

Appendix I: Selected Office of Management and Budget Standards for Statistical Surveys

The National Defense Authorization Act for Fiscal Year 2008 (NDAA 2008) directed the Department of Defense (DOD) to determine the adequacy of the number of health care and mental health care providers that currently accept nonenrolled beneficiaries as patients under TRICARE, DOD’s health care program. The NDAA 2008 also directed us to review the processes, procedures, and analyses used by DOD to determine the adequacy of the number of health care and mental health care providers that currently accept nonenrolled beneficiaries as patients. To evaluate the methodology DOD used to implement the beneficiary and provider surveys, we reviewed the Office of Management and Budget’s (OMB) Standards and Guidelines for Statistical Surveys (2006) to identify key aspects and best practices of statistical survey methodology that result in sound survey design and implementation. We focused our evaluation on standards that address, among other things, designing a survey, developing sampling frames, collecting survey data, and analyzing survey response rates. Table 7 provides a description of these standards.

Appendix II: DOD’s Methodology for the 2008 Beneficiary and Provider Surveys

The NDAA 2008 directed DOD to determine the adequacy of the number of health care and mental health care providers that currently accept nonenrolled beneficiaries as patients under TRICARE, DOD’s health care program. For the purpose of this report, we refer to beneficiaries who are not enrolled in TRICARE Prime—that is, those who use TRICARE Standard, TRICARE Extra, or TRICARE Reserve Select—as nonenrolled beneficiaries. The mandate also included specific requirements related to the number and priority of areas to be surveyed, including the populations to be surveyed each year, the content for each type of survey, and the use of benchmarks. Within DOD, the TRICARE Management Activity (TMA), which oversees the TRICARE program, had responsibility for designing and implementing the beneficiary and provider surveys. The following information describes TMA’s methodology, including its actions to address the requirements for each of the following: (1) survey area, (2) sample selection, (3) survey content, and (4) the establishment of benchmarks. It also provides information on TMA’s analyses of its 2008 beneficiary and provider surveys.

Beneficiary and Provider Survey Area Selection

The NDAA 2008 specified that DOD survey beneficiaries and providers in at least 20 TRICARE Prime Service Areas, and 20 geographic areas in which TRICARE Prime is not offered—referred to as non-Prime Service Areas—each fiscal year, 2008 through 2011. The NDAA 2008 also required DOD to consult with representatives of TRICARE beneficiaries and health care and mental health care providers to identify locations where nonenrolled beneficiaries have experienced significant access-to-care problems, and give a high priority to surveying health care and mental health care providers in these areas. Additionally, the NDAA 2008 required DOD to give a high priority to surveying areas in which higher concentrations of Selected Reserve servicemembers live.

In designing the 2008 beneficiary and provider surveys, TMA defined 80 Prime Service Areas and 80 non-Prime Service Areas that will allow it to survey the entire country over a 4-year period and to develop estimates of access to health care and mental health care at service area, state, and national levels. TMA identified the 80 Prime Service Areas by collecting zip codes where TRICARE Prime was offered from officials within each of the three TRICARE Regional Offices. TMA grouped these zip codes into 80 nonoverlapping areas so that each area had roughly the same number of TRICARE-eligible beneficiaries. Because non-Prime Service Areas had not previously been defined, TMA sought to define them by grouping all zip codes not in Prime Service Areas into one large area using Hospital Referral Regions, which are groupings of Hospital Service Areas. TMA divided the large area into 80 non-Prime Service Areas so that each area had roughly the same number of TRICARE-eligible beneficiaries.

To identify locations where beneficiaries and health care and mental health care providers have identified significant levels of access-to-care problems under TRICARE Standard and Extra, TMA spoke with groups representing beneficiaries and health care and mental health care providers as well as TRICARE Regional Offices. These groups suggested cities and towns where access should be measured, and Hospital Service Areas corresponding to each city and town were then identified. Based on the groups’ recommendations, a list was created and sorted in priority order, resulting in 21 Hospital Service Areas being included in the 2008 provider survey. Additionally, TMA plans to include these 21 Hospital Service Areas in its 2009 beneficiary survey along with additional areas that are identified for 2009. Although the NDAA 2008 required DOD to give a high priority to surveying areas in which higher concentrations of Selected Reserve servicemembers live, TMA decided to randomly select areas for the surveys in order to produce results that could be generalized to the populations in the areas surveyed.

Selection of Beneficiary and Provider Survey Sample

TMA selected its sample of beneficiaries who met its criteria for inclusion in the beneficiary survey using DOD’s Defense Enrollment Eligibility Reporting System (DEERS), a database of DOD beneficiaries who may be eligible for military health benefits. TMA determined a beneficiary’s eligibility to be included in the 2008 beneficiary survey if DEERS indicated that the individual met five criteria: (1) eligible for military health care benefits as of the date of the sample file extract; (2) age 18 years old or older; (3) not an active duty member of the military; (4) residing in one of the 20 randomly selected Prime Service Areas or 20 randomly selected non-Prime Service Areas; and (5) a user of TRICARE Reserve Select, or not enrolled in TRICARE Prime. From this database, TMA randomly sampled about 1,000 beneficiaries from each Prime Service Area and non- Prime Service Area—a sample size that would achieve TMA’s desired sample error. Specifically, TMA surveyed 48,548 TRICARE beneficiaries representing active duty, retired, and Reserve servicemembers, including the Selected Reserve. TMA began mailing the beneficiary survey in June 2008. After receiving the returned surveys, TMA identified the responses that it considered complete and eligible based on whether the beneficiary had answered at least half of TMA’s identified “key” questions and answered that he or she used “TRICARE Extra or Standard” or “TRICARE Reserve Select” in response to the following question: “Which health plan did you use for all or most of your health care in the last 12 months?”

TMA selected the provider sample within the same 20 Prime Service Areas and 20 non-Prime Service Areas that had been randomly selected for the 2008 beneficiary survey. In addition, TMA mailed surveys to physicians and mental health providers in the 21 Hospital Service Areas identified by beneficiary and provider groups as having significant levels of access-to- care problems under TRICARE Standard and Extra. TMA used the American Medical Association Physician Masterfile (Masterfile) to select a sample of 20,030 physicians who were licensed office-based civilian medical doctors or licensed civilian doctors of osteopathy within the specified locations who were engaged in more than 20 hours of patient care each week. The Masterfile is a database of physicians in the United States—Doctors of Medicine and Doctors of Osteopathic Medicine—that includes data on all physicians who have the necessary educational and credentialing requirements. The Masterfile did not differentiate between TRICARE’s network and nonnetwork civilian providers, which TMA deemed acceptable to avoid any potential bias in TMA’s sample selection. As such, TMA selected this file because it is widely recognized as one of the best commercially available lists of providers in the United States and it contains over 940,000 physicians along with their addresses; phone numbers; and information on practice characteristics, such as their specialty. According to TMA, the American Medical Association updates physicians’ addresses monthly and other elements through a rotating census methodology involving approximately one-third of the physician population each year. Although the Masterfile is considered to contain most providers, deficiencies in coverage and inaccuracies in detail remain. Therefore, TMA attempted to update providers’ addresses and phone numbers and to ensure that providers were eligible for the survey by also using state licensing databases, local commercial lists, and professional society and association lists. For its mental health provider sample, TMA selected a sample of 20,386 mental health providers from two sources: the National Plan and Provider Enumeration System database maintained by the Centers for Medicare & Medicaid Services, and from LISTS, Inc., a list of names with contact information assembled from state licensing boards. According to TMA, it selected these sources for mental health providers because they have been identified as the most comprehensive databases for these health care providers. TMA did not include all physician specialist types, such as epidemiologists and pathologists, in its survey. From these data sets, TMA planned to randomly sample about 800 providers (400 each of physicians and mental health providers) from each Prime Service Area, non-Prime Service Area, and Hospital Service Area—a sample size that would achieve TMA’s desired sample error. In those instances where there were not 800 providers in a single area, TMA selected all of the providers in that area to receive surveys. TMA began mailing the provider survey in December 2008. Upon receipt of the returned surveys, TMA identified the responses that it considered completed and eligible based on the following criteria for respondents: (1) if the provider answered “yes” to the questions that asked whether the provider offers care in an office-based location or private practice; (2) for the nonphysician mental health survey, if the provider responded that he or she was in one of the six TRICARE participating specialties—certified clinical social worker, certified psychiatric nurse specialist, clinical psychologist, certified marriage and family therapist, pastoral counselor, or mental health counselor; and (3) if the provider had completed three key questions on the physician survey instrument or three key questions on the nonphysician mental health provider survey instrument.

Beneficiary and Provider Survey Content

The NDAA 2008 required that the beneficiary survey include questions to determine whether TRICARE Standard and Extra beneficiaries have had difficulties finding physicians and mental health providers willing to provide services under TRICARE Standard or TRICARE Extra. TMA’s beneficiary survey included 91 questions that address, among other things, health care plans used; perceived access to care from a personal doctor, nurse, or specialist; the need for treatment or counseling; and ratings of health plans. TMA based some of its 2008 beneficiary survey questions on those included in the Department of Health and Human Services’ 2006 Consumer Assessment of Healthcare Providers and Systems, a national survey of beneficiaries of commercial health insurance, Medicare, Medicaid, and the Children’s Health Insurance Program. When TMA began mailing the beneficiary survey, it included a combined cover letter and a questionnaire to all beneficiaries in its sample—with the option of having beneficiaries complete the survey by mail or Internet. (See app. III for a copy of the 2008 beneficiary survey instrument.) The cover letter provided information on the options available for completing the survey, as well as instructions for completing the survey by Internet. If the beneficiary did not respond to the mailed questionnaire, TMA mailed a second combined cover letter and questionnaire 4 weeks later encouraging the beneficiary to complete the survey.

For the provider survey, the NDAA 2008 required questions to determine (1) whether the provider is aware of TRICARE; (2) the percentage of the provider’s current patient population that uses any form of TRICARE; (3) whether the provider accepts Medicare patients for health care and mental health care; and (4) if the provider accepts Medicare patients, whether the provider would accept new Medicare patients. TMA obtained clearance for its provider survey from OMB as required under the Paperwork Reduction Act. Subsequent to this review, OMB approved an 11-item questionnaire for physicians (including psychiatrists) and a 12-item questionnaire for nonphysician mental health providers to be administered in fiscal year 2008. (See app. IV for a copy of the 2008 provider survey instruments.) The mental health providers’ version of the survey includes an additional question about what type of mental health care the provider practiced.

When TMA began mailing the provider survey, it included a combined cover letter and a questionnaire to each provider in the sample. The providers had the option of completing the survey by mail, fax, or Internet. The cover letter provided information on the options available for completing the survey, as well as instructions for completing the survey by Internet. If the provider did not respond to the mailed questionnaire, TMA mailed a second combined cover letter and questionnaire about 4 weeks later encouraging the provider to complete the survey.

Beneficiary and Provider Survey Benchmarks

In accordance with the NDAA 2008, TMA identified benchmarks for analyzing the results of the beneficiary and provider surveys. Because TMA based some of its 2008 beneficiary survey questions on those included in the Department of Health and Human Services’ 2006 Consumer Assessment of Healthcare Providers and Systems survey, it was able to compare the results of those questions with its 2008 beneficiary survey results. To benchmark its provider survey, TMA compared the results of its 2008 survey with the results of its 2005, 2006, and 2007 provider surveys. A TMA official noted that TMA was unaware of any external benchmarks that would be applicable to its surveys of providers.

Analyses of Beneficiary and Provider Survey Results

In analyzing the results of the beneficiary survey, TMA representatives conducted a nonresponse analysis because the overall response rate to the survey was about 38 percent. To conduct this analysis, TMA did the following: (1) compared key beneficiary demographic characteristics of respondents to those of nonrespondents (e.g., beneficiary gender and age) and (2) interviewed a sample of 400 beneficiaries who did not respond to the original survey or the follow-up mailing and compared their responses with those of the original survey respondents. The results of TMA’s nonresponse analysis indicated no difference in demographic characteristics and health coverage between beneficiary survey respondents and nonrespondents within the combined Prime Service Areas and combined non-Prime Service Areas surveyed in fiscal year 2008. Therefore, TMA concluded that the survey respondents were representative of the combined Prime Service Areas and combined non- Prime Service Areas surveyed, and the results of the survey can be generalized to the population from which the sample was chosen. TMA weighted each response so that the sampled beneficiaries represented the population in terms of size for the respective Prime Service Area or non- Prime Service Area from which they were selected.

In analyzing the results of the provider survey, TMA conducted a nonresponse analysis because the overall response rate to the survey was about 45 percent. To conduct this analysis, TMA did the following: (1) compared key provider demographic characteristics of respondents to those of nonrespondents (for example, provider type and location) and (2) interviewed a sample of 247 providers (140 physicians and 107 mental health providers) who did not respond to the original survey, follow-up mailing, or follow-up telephone calls and compared their responses with those of the original survey respondents. The results of TMA’s nonresponse analysis indicated that there are differences between respondents and those who did not respond to the original 2008 provider survey. Specifically, among both types of providers (physicians and mental health providers), nonrespondents are less likely to be aware of TRICARE and less likely to accept TRICARE as a form of payment for services. Additionally, nonrespondents are less likely to be accepting new patients. Therefore, the survey results cannot be generalized to the population from which the sample was chosen and can only be presented in terms of those civilian providers who responded to the survey.

Appendix III: Beneficiary Survey Instrument

The NDAA 2008 directed DOD to determine the adequacy of the number of health care and mental health care providers that currently accept nonenrolled beneficiaries as patients under TRICARE, DOD’s health care program. For the purpose of this report, we refer to beneficiaries who are not enrolled in TRICARE Prime—that is, those who use TRICARE Standard, TRICARE Extra, or TRICARE Reserve Select—as nonenrolled beneficiaries. Specifically, the NDAA 2008 specified that DOD conduct surveys of beneficiaries each fiscal year, 2008 through 2011. The NDAA 2008 also required that the beneficiary survey include questions seeking information from nonenrolled beneficiaries to determine whether they have had difficulties finding health care and mental health care providers willing to accept them as patients. Following is the actual survey instrument that DOD used to obtain information from nonenrolled beneficiaries.

Appendix IV: Survey Instruments for Health Care and Mental Health Care Providers

The NDAA 2008 directed DOD to determine the adequacy of the number of health care and mental health care providers that currently accept nonenrolled beneficiaries as patients under TRICARE, DOD’s health care program. For the purpose of this report, we refer to beneficiaries who are not enrolled in TRICARE Prime—that is, those who use TRICARE Standard, TRICARE Extra, or TRICARE Reserve Select—as nonenrolled beneficiaries. Specifically, the NDAA 2008 directed DOD to survey providers each fiscal year, 2008 through 2011. The NDAA 2008 also required that the provider survey include questions seeking information to determine (1) whether the provider is aware of the TRICARE program; (2) the percentage of the provider’s current patient population that uses any form of TRICARE; (3) whether the provider accepts Medicare patients; and (4) if the provider accepts Medicare patients, whether the provider would accept new Medicare patients. DOD implemented two versions of its provider survey, one for physicians, including psychiatrists, and one for nonphysician mental health providers. Following are the actual survey instruments that DOD used to obtain information from physicians and nonphysician mental health care providers.

Appendix V: Areas Included in the Fiscal Year 2008 Beneficiary and Provider Surveys

The NDAA 2008 directed DOD to determine the adequacy of the number of health care providers and mental health care providers that currently accept nonenrolled beneficiaries as patients under TRICARE, DOD’s health care program. For the purpose of this report, we refer to beneficiaries who are not enrolled in TRICARE Prime—that is, those who use TRICARE Standard, TRICARE Extra, or TRICARE Reserve Select—as nonenrolled beneficiaries. The NDAA 2008 specified that DOD conduct surveys of TRICARE beneficiaries and health care and mental health care providers in 20 TRICARE Prime Service Areas and in 20 areas in which TRICARE Prime is not offered—referred to as non-Prime Service Areas— for each fiscal year, 2008 through 2011. Additionally, the NDAA 2008 required DOD to consult with representatives of TRICARE beneficiaries and health care and mental health care providers to identify locations where nonenrolled beneficiaries have experienced significant access-to- care problems, and give a high priority to surveying health care and mental health care providers in these areas. For the 2008 beneficiary and provider surveys, DOD selected 20 Prime Service Areas and 20 non-Prime Service Areas to determine the adequacy of the number of health care providers and mental health care providers who currently accept nonenrolled TRICARE beneficiaries as patients. For the 2008 provider survey, DOD also surveyed 21 additional areas identified by beneficiary and provider groups where nonenrolled beneficiaries are experiencing significant levels of access-to care problems—called Hospital Service Areas—to determine the adequacy of access to care in these areas. DOD’s selected Prime Service Areas and non-Prime Service Areas for 2008 are presented in table 8 and table 9, respectively. Table 10 lists those locations identified by representatives of TRICARE beneficiaries and health care and mental health care providers as having significant levels of access-to-care problems, which were included in the 2008 provider survey.

Appendix VI: Comments from the Department of Defense

Appendix VII: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Bonnie Anderson, Assistant Director; Martha Kelly, Assistant Director; Susannah Bloch; Peter Mangano; Jeff Mayhew; Lisa Motley; Jessica C. Smith; C. Jenna Sondhelm; and Suzanne Worth made key contributions to this report.