Background

VA and DOD’s history of sharing resources to provide integrated health  care services to their beneficiaries in and around North Chicago has  occurred in three phases. The third phase culminated in the  establishment of the FHCC, which was formalized by an Executive  Agreement signed by the Secretaries of VA and DOD in April 2010. The  FHCC is unique among other VA and DOD sharing relationships in its  level of collaboration, governance structure, and financial model.

		History of VA and DOD Integration of Health Services in North Chicago

The history of VA and DOD integrating health services in North Chicago  can be described in three distinct phases. (See fig. 1.) These phases  cover the time period from 2003 to 2011 and include the official  establishment of the FHCC in October 2010.    Phase I began in 2003 when VA and DOD began sharing health care  resources between NCVAMC and NHCGL. NHCGL, then known as  Naval Hospital Great Lakes, transferred its inpatient mental health  services to NCVAMC. In the same year, VA and DOD formed a  working group to address issues related to sharing resources between  the two sites. In 2004, the NHCGL blood donor processing center was  transferred to NCVAMC.    Phase II began in 2005 with the $13-million renovation and  modernization of the NCVAMC, including its operating rooms and an  expansion of the emergency department. In 2006, NHCGL’s inpatient  medical, surgical, pediatric, and intensive care units, operating rooms,  and emergency department were transferred to NCVAMC. With the  transfer of inpatient services, the naval hospital became a naval  health clinic, since the facility no longer provided inpatient services.    Phase III began in 2007 when construction began on new parking  areas, followed in 2008 by the groundbreaking for the construction of  a new ambulatory care center. The FHCC Advisory Board was  established in 2009 to help provide guidance for the integration and  future operation of the facility, which was authorized in the NDAA for  Fiscal Year 2010. In October 2010, the former NCVAMC and NHCGL  facilities merged to become the FHCC, following completion of a  $130-million DOD-funded construction project. The FHCC consists of  all the services, buildings, and locations formerly operated by either  NCVAMC or NHCGL including the new 201,000-square-foot  ambulatory care center and its parking lot and garage, a 45,000- square-foot renovation of the NCVAMC, and various outpatient and  recruit clinics formerly operated by either NCVAMC or NHCGL. (See  fig. 2 for photographs of newly constructed or renovated areas of the  FHCC.) The ambulatory care center, which is physically connected to  the NCVAMC, houses outpatient services including pediatrics,  women’s health, and mental health. In addition, it has on-site  laboratory, radiology, and pharmacy services, enabling patients to  access these ancillary services in the same location as their  outpatient services. Phase III continued into 2011 with the move into  the new ambulatory care center and the delivery of patient services  there.
In April 2010, the Secretaries of VA and DOD signed the Executive  Agreement that established the FHCC. The Executive Agreement defines  the relationship between VA and DOD for establishing and operating the  FHCC, in accordance with the NDAA for Fiscal Year 2010, and contains  provisions in 12 integration areas regarding specific aspects of FHCC  operations. These 12 areas are: (1) governance structure; (2) access to  health care at the FHCC; (3) research; (4) contracting; (5) information  technology (IT); (6) fiscal authority; (7) workforce management   and personnel; (8) quality assurance; (9) contingency planning;   (10) integration benchmarks; (11) property (i.e., construction and   physical plant management); and (12) reporting requirements.
Within the 12 areas are provisions describing how the FHCC should be  operated or what VA and DOD will do as part of their efforts to jointly  operate the facility. Some provisions relate to establishing and operating  the FHCC and have designated deadlines, such as implementing IT  strategies. Other provisions do not have specified deadlines or will not be  met until a certain point in the integration, or are contingent on other  conditions being met. For example, a provision in the reporting  requirements integration area calls for a final report at the end of the   5-year demonstration in 2015. Since the report is due at the end of the   5-year demonstration period, this particular provision cannot yet be  implemented.
The Executive Agreement also includes 15 integration benchmarks that  VA and DOD plan to use to determine the integration’s success.  Assessment of the integration benchmarks throughout the 5-year  demonstration project will help inform whether the FHCC partnership  should continue beyond the demonstration period authorized by the  NDAA for Fiscal Year 2010.

		Unique Features of the FHCC

The FHCC is unique among VA and DOD joint ventures in three key  ways. First, the FHCC’s integration of the provision of care and  operations represents the highest level of collaboration among the nine  existing VA and DOD joint ventures. VA and DOD have periodically  assessed their joint venture arrangements to determine the level of  collaboration between partners, which they measure on a continuum from  “separate” to “consolidated.” They use this continuum to assess program  elements of a joint venture’s partnership such as governance, education  and training, and research. VA and DOD officials reported to us that  overall, the FHCC has more program elements, such as its clinical  services and staffing, which fall on the “consolidated” end of the  collaboration continuum than any of the other joint venture sites. Second,  the FHCC operates under a single line of governance to manage medical  and dental care, and has an integrated workforce of approximately 3,000  civilian and active duty military employees from both VA and DOD—a  feature that is unique to the FHCC. Although the FHCC’s leadership and  workforce are integrated, they also remain directly accountable to both  VA and DOD through the Joint Executive Council and the Health  Executive Council. VA and DOD officials told us that none of the other  joint venture sites has an integrated governance structure and instead  maintain separate VA and DOD lines of authority. The third way in which  the FHCC is unique among VA and DOD joint ventures is its financial  model. The FHCC has a single funding source to which VA and DOD will  contribute, unlike the other joint venture sites which have separate VA  and DOD funding sources. The NDAA for Fiscal Year 2010 established  the Joint DOD-VA Medical Facility Demonstration Fund (Joint Fund) as  the funding mechanism for the FHCC, with VA and DOD both making  transfers to the Joint Fund from their respective appropriations.

	FHCC Officials Have Made Progress Implementing VA and DOD’s Executive Agreement, but Challenges May Impact Further Implementation Progress

FHCC officials (including both VA and DOD officials) have made progress  implementing the provisions of the 12 Executive Agreement integration  areas. Four of the 12 integration areas have been fully implemented,   7 are in progress and proceeding according to plan, and 1 area, IT, has  been delayed and continues to present challenges. In addition, the  FHCC’s lack of a military treatment facility (MTF) designation has  presented challenges for FHCC operations and health care providers.

		FHCC Officials Have Implemented Four Executive Agreement Integration Areas and Are Progressing on Seven Others, but IT Implementation Has Been Delayed

FHCC officials have addressed all provisions for 4 of the 12 Executive  Agreement integration areas, are progressing in their implementation of   7 other areas, and have experienced delays in the implementation of   1 area, IT. (See table 1.)
Officials have completed implementation of the governance structure,  access to health care at the FHCC, research, and contracting integration  areas.
Governance structure. The Executive Agreement defined the structure  of the FHCC’s governance structure and leadership, with a VA official  serving as the director and a naval captain serving as the deputy director.  (See fig. 3.) As of October 1, 2010, the former director of NCVAMC  became the director of the FHCC, and a command change brought in a  new naval captain to become deputy director, taking over command from  the former NHCGL commanding officer.
In addition, the advisory bodies described in the Executive Agreement are  in place. The FHCC has an Advisory Board—co-chaired by and  comprised of senior officials from VA and DOD—that monitors the  FHCC’s performance and advises on strategic direction, mission, vision,  and policy. The Advisory Board also provides input into the performance  evaluations of FHCC leadership and serves as a communication link  between VA and DOD executive leadership and the FHCC through the  Joint Executive Council and the Health Executive Council. Also, a  Stakeholders Advisory Council—comprised of members from various  regional and local organizations representing FHCC interests—provides  feedback on how well the FHCC is meeting customers’ needs and  whether the FHCC is meeting VA and DOD missions. VA and DOD  officials also designed the major operational components of the FHCC to  have shared VA and DOD leadership. The six clinical and administrative  divisions that report to the director and deputy director are led by an  associate director and an assistant director, one from VA and the other  from the Navy.
Access to health care at the FHCC. To address access to health care at  the FHCC for veterans and DOD beneficiaries, the NDAA for Fiscal Year  2010 and the Executive Agreement established a patient priority system  that is unique to the FHCC for use in the event of resource constraints.  FHCC’s patient priority system is based on the priority list for TRICARE,  DOD’s program to provide health care to its beneficiaries, and  incorporates VA beneficiaries. More specifically, the system prioritizes  active duty servicemembers above veterans and other DOD beneficiaries  as follows:  1.  members of the Armed Forces on active duty;  2.  veterans and non veteran VA beneficiaries, and TRICARE Prime- enrolled active duty dependents;  3.  TRICARE Prime enrolled retirees, their dependents and survivors;  4.  TRICARE Standard active duty dependents; and  5.  TRICARE Standard retirees, their dependents, and survivors,  including TRICARE for Life beneficiaries.
Officials told us that they do not anticipate needing to activate the patient  priority system because they are currently meeting the needs of FHCC  beneficiaries—health care providers at the FHCC currently serve all  patients based on medical need. Officials also told us that their monitoring  of Navy recruit medical readiness ensures they are able to maintain the  “pipeline to the fleet” of enlisted sailors.
Research. The Executive Agreement stated that the FHCC would comply  with VA policy for research efforts, but provided that when DOD  researchers or patients are involved in a study, the Navy’s rules on  protection of human subjects would apply in addition to VA’s. In addition  to implementing this provision, FHCC officials told us they decided to  integrate the research program at the FHCC. Since a majority of the  research conducted is VA research, it was easily incorporated into the  broader FHCC integration efforts. The FHCC has an Institutional Review  Board—a body responsible for reviewing and approving research  protocols involving human subjects—located at a hospital affiliated with  the FHCC that provides research management and operational oversight  to the FHCC, the Edward Hines, Jr. VA Hospital in Hines, Illinois. Also, in  accordance with the Executive Agreement, FHCC officials told us that a  DOD Institutional Review Board in San Diego, California, may also be  involved for research involving DOD researchers or active duty  servicemembers at the FHCC.
Contracting. The Executive Agreement stated that VA would be  responsible for providing contracting support at the FHCC. Similar to the  FHCC’s research efforts, officials chose to integrate the contracting  function as part of the broader integration. Five former Navy civilian  employees who were at NHCGL prior to the integration were converted to  VA civilian employees to help support the integrated contracting and  purchasing functions.
Officials’ efforts have progressed as planned to implement provisions for  the fiscal authority, workforce management and personnel, quality  assurance, contingency planning, integration benchmarks, property, and  reporting requirements areas.
Fiscal authority. The FHCC fiscal authority integration area included the  development of an integrated budgeting and financial reconciliation  process. For fiscal years 2011 through 2013, the FHCC plans to use  historical financial data to budget and determine the amount each  department will transfer to the Joint Fund and expects to manually  conduct the year-end reconciliation process. Officials told us that by fiscal  year 2014, the FHCC intends to have an automated year-end financial  reconciliation process. However, as of April 1, 2011, the integration area  on fiscal authority had not been fully implemented because appropriations  had not been made available for the Joint Fund. The NDAA for Fiscal  Year 2010 established the Joint Fund as the FHCC’s funding source, but  FHCC officials could not use it until funds had been authorized and  appropriated for VA and DOD to transfer into the Joint Fund, which  occurred in April 2011. Until that time, the FHCC was funded by an  alternative funding mechanism established by the Executive Agreement  for use in the event that Congress did not authorize and appropriate funds  to be transferred to the Joint Fund. As of April 2011, FHCC officials  planned to cease use of the alternative funding mechanism and begin use  of the Joint Fund at the start of the next quarter on July 1, 2011.
The delay in availability of funds may result in a delay in addressing an  NDAA for Fiscal Year 2010 requirement that is also one of the integration  benchmarks—the annual independent audit of the Joint Fund, which is  conducted at the end of the fiscal year. The audit will evaluate the  adequacy of VA’s and DOD’s proportional contributions to the Joint  Fund. In addition, the implementation of the automated financial  reconciliation process is contingent on a related IT capability, which does  not yet have an estimated completion date. Together, these delays may  impact the FHCC’s ability to address one of the measures of its  integration’s success.
Workforce management and personnel. In the workforce management  and personnel integration area, the NDAA for Fiscal Year 2010  authorized a transfer from DOD to VA of the positions and personnel  necessary to operate the FHCC. The Executive Agreement identified  533 DOD civilian positions that were eligible for transfer to VA, and FHCC  officials told us that VA made offers of employment to the individuals in  those positions. In total, 469 DOD civilian personnel were transferred to  VA as of October 10, 2010—the deadline established in the Executive  Agreement. The 533 converted civilian positions, along with 724 active  duty positions, 1577 VA civilian positions, 249 VA and DOD contract  positions, and 18 new housekeeping positions, comprise the  approximately 3,000 positions that initially staffed the FHCC. FHCC  officials are in the process of resigning affiliation agreements with health  care facilities and training institutions and plan to address another  provision regarding the development of criteria and assessment methods  to measure staff experiences with the integration at a later date. Officials  have also integrated their staff training through an integrated education  department.
Quality assurance. The Executive Agreement stated that the FHCC  would have one integrated quality assurance plan and would maintain  accreditation by the external accrediting bodies required by either VA or  DOD. It also outlined the FHCC’s credentialing and privileging process for  health care professionals. FHCC officials have an integrated quality  assurance plan for the facility in place, as well as policies addressing  credentialing and privileging of providers and the role of independent duty  corpsmen at the FHCC. While some bodies, such as the VA Office of  the Inspector General, have conducted reviews of the FHCC since it was  established, other accreditation and certification reviews, such as that of  The Joint Commission, are pending and the first reviews for the FHCC  will occur on the schedules that were in place for the NHCGL and  NCVAMC prior to the integration.
One component of quality assurance is the maintenance of clinical skills  for the FHCC’s Navy health care providers. Officials told us that one of  the benefits of the integration is that dental school graduates obtaining  advanced education in the Navy can see veteran patients while  completing their residencies and have opportunities to be exposed to  different dental conditions than those normally seen in the generally  younger and healthier recruit population. Some of these dentists will be  placed on ships, where they are often the only on-site dentist. FHCC  officials described a similar benefit for health care professionals providing  inpatient care.
Contingency planning. The Executive Agreement included contingency  planning provisions regarding the establishment of certain FHCC  emergency management positions, and stated which antiterrorism and  other security guidelines would inform the establishment of the FHCC’s  security plans. In addition, the FHCC must maintain training standards for  staff that meet the joint VA/DOD programs in this integration area. The  FHCC has the necessary emergency management personnel, training  standards, and programs in place; however, officials are in the process of  finalizing an agreement to outline the relationship between VA police and  DOD security personnel.
Integration benchmarks. The Executive Agreement established   15 integration benchmarks to define the degree of the integration’s  success. VA and DOD officials at the regional and headquarters levels  and FHCC officials worked together to develop these benchmarks that  cover such topics as patient and staff satisfaction, clinical and  administrative functions, and external evaluation. The integration  benchmarks are being used by the FHCC to assess provision of care and  operations and are discussed in more detail later in this report.
Property. The Executive Agreement and the NDAA for Fiscal Year 2010  describe the terms of the transfer of property ownership that may occur at  the end of the 5-year demonstration period from DOD to VA. A  determination to transfer ownership may occur upon the earlier of   (1) completion of the integration benchmarks or (2) 5 years from the date  the Executive Agreement was executed. If it is determined that the FHCC  should not continue to be an integrated facility, DOD will retain ownership  of the ambulatory care center and associated structures that were built  with DOD funds.
Reporting requirements. There are several reporting requirements  described in the Executive Agreement that were established by the NDAA  for Fiscal Year 2010, including submitting the Executive Agreement to the  appropriate committees of Congress 1 week before its execution, and a  final report from the Secretaries of VA and DOD that will be submitted   5 ½ years after the Executive Agreement was executed. The report is to  describe and assess the performance of the FHCC, and to provide a  recommendation as to whether the partnership should continue beyond  the demonstration period. Congress will make the final determination as  to whether to continue the partnership.
The Executive Agreement identified IT capabilities that VA and DOD were  to have in place by the opening day of the FHCC, October 1, 2010, to  facilitate interoperability between VA and DOD electronic health record  systems, as well as other capabilities for financial management and  outpatient appointments that are to be developed in the future. The three  capabilities that were to be in place upon the FHCC’s opening were   (1) medical single sign-on—which allows staff to use one screen to  access both the VA and DOD electronic health record systems; (2) single  patient registration—which allows staff to register patients in both  systems simultaneously; and (3) orders portability for laboratory,  radiology, pharmacy, and consults—which will allow VA’s and DOD’s  electronic health record systems to exchange information for these  medical orders. In addition, the Executive Agreement stated that all IT  capabilities developed for the FHCC will be exportable to other VA/DOD  joint ventures and medical sharing locations.
FHCC officials, working with VA and DOD officials, have implemented or  are in the process of implementing the IT provisions of the Executive  Agreement, including working with a strategic working group that supports  implementation efforts, as well as defining procedures for the reporting  of information security incidents. However, VA and DOD did not meet  designated deadlines for the three capabilities that were to be in place  upon opening and, as of May 2011, not all capabilities are fully  implemented at the FHCC. Single sign-on and single patient registration  were implemented on December 13, 2010. On March 3, 2011, FHCC  officials began limited use of orders portability for laboratory and  radiology. While full operational capability was expected on April 14,  2011, officials told us that both orders portability capabilities remained in  limited use through April, with radiology expected to have full operational  capability on June 1, 2011, and laboratory delayed until an undetermined  date. FHCC officials decided to delay implementation of these capabilities  in order to allow more time to correct problems, such as difficulty  managing large numbers of automated laboratory test orders, and to train  users on the system. Additionally, FHCC officials told us that  implementation of the remaining orders portability capabilities (pharmacy  and consults) are indefinitely delayed while decisions are made at the  department level regarding development of these capabilities. FHCC  officials have implemented an interim orders portability process for the  pharmacy while VA and DOD continue to develop the automated orders  portability capability. This interim process necessitated the hiring of five  full-time pharmacists to conduct manual checks of the VA and DOD  electronic health record systems to ensure that the FHCC is able to  ensure patients’ safety by identifying possible interactions between drugs  prescribed in the two separate systems.
The three IT capabilities were delayed in part because of a need for more  on-site testing before use, as well as a lack of an integrated and  comprehensive project plan from VA and DOD. During on-site testing,  FHCC officials found that some requirements for pharmacy and radiology  orders portability did not meet the FHCC’s needs. FHCC officials told us  that an earlier round of system testing was performed in an off-site  environment that did not effectively simulate the FHCC environment. In  addition, we reported in February 2011 that, although VA and DOD  performed various planning activities for the FHCC IT system, these  activities generally were not completed in accordance with effective  project planning practices including defining the scope, estimating the  cost, and establishing a budget and schedule for the project.  Additionally, we expressed concern that VA’s and DOD’s ineffective  planning jeopardized their ability to fully provide the IT system capabilities  the FHCC needs on a timely basis.
As a result of the need for more on-site testing and the ineffective project  planning, VA and DOD have not yet fully provided clinicians at the FHCC  with the IT capabilities the Executive Agreement identified as needed  upon opening. Further, the IT issues caused a 2-week delay in the start of  the move of clinical services into the new ambulatory care center.  Additionally, since the needed capabilities have not yet been fully  implemented, the departments are not in a position to export all the  planned capabilities to other locations, as provided for in the Executive  Agreement.

		Lack of MTF Designation Has Presented Challenges for FHCC Operations and Health Care Providers

According to DOD policy, an MTF is a military treatment facility owned  and operated by DOD that is established for the purpose of furnishing  medical and/or dental care to eligible individuals. Among other things,  designation as an MTF allows co-payments to be waived for services  received by DOD beneficiaries. The former NHCGL was an MTF, and  FHCC officials will continue to list it as such on DOD’s list of MTFs  through the FHCC demonstration period even though the NHCGL no  longer exists. The FHCC’s ambulatory care center, which is currently  owned by DOD, is the only part of the FHCC that has an MTF  designation. The NDAA for Fiscal Year 2010 provided that the facility,  defined for purposes of the statute as the new Navy ambulatory care  center, parking structure, and supporting structures and facilities, as well  as related medical personal property and equipment, may be treated as  an MTF for purposes of eligibility for DOD health care. However, DOD  has concluded that it does not have the authority to designate the FHCC  as an MTF since the FHCC generally, including those areas of the FHCC  that provide inpatient services, is owned by VA and that its authority to  consider the FHCC an MTF is limited to the purpose of confirming the  categories of beneficiaries eligible for DOD health care.
FHCC officials, working with VA and DOD, have implemented or are in  the process of implementing workarounds for three issues related to the  lack of an MTF designation:  1.  Certain DOD beneficiaries would have been responsible for co- payments for care received at the FHCC. DOD beneficiaries do not  have to pay co-payments at MTFs, such as the former NHCGL.  However, because the FHCC lacks the MTF designation, certain DOD  beneficiaries would have had to pay co-payments for services  received at the FHCC. This issue was temporarily resolved through a  demonstration project to waive co-payments for DOD beneficiaries at  the FHCC during the 5 years of the FHCC demonstration.  2.  The FHCC was unable to continue DOD’s personal services  contracts. FHCC officials told us that prior to the integration, the  naval facilities used personal service contracts for temporary health  care provider staffing needs, but DOD may only enter into personal  services contracts to fulfill health care needs at MTFs and in other  select circumstances. FHCC officials decided to convert the personal  services contract positions that were needed at the FHCC into VA  civilian employee positions. FHCC officials told us that personal  services contracts are a preferred method for accommodating  fluctuations in medical and dental workload resulting from changes in  the number of Navy recruits on site at any given time, but they  anticipate that the Navy’s plan to maintain more consistent recruit  numbers throughout the year will reduce the need for temporary staff.  3.  The FHCC’s lack of MTF designation resulted in uncertainty about the  FHCC’s ability to use DOD’s contracted drug prices for prescription  orders, including the extent to which DOD’s drug pricing information  could be shared with VA’s pharmacy vendor. VA and DOD have  contracts with pharmacy vendors to obtain drugs and with  manufacturers to obtain favorable drug pricing for their beneficiaries,  but the two departments have contracts with different vendors and  have different pricing arrangements with manufacturers. As part of the  broader integration efforts at the FHCC, VA and DOD signed an  agreement to use VA’s pharmacy vendor for the FHCC while  maintaining access to DOD’s contracted manufacturer prices for DOD  beneficiaries treated at the FHCC. However, FHCC officials told us  they were later denied access to manufacturer pricing arrangements  because the FHCC was not an MTF. DOD officials told us that the  TRICARE Management Activity’s Pharmacy Operations Department  determined in March 2011 that the FHCC was entitled to use DOD- contracted prices with respect to DOD beneficiaries and issued a  letter to manufacturers indicating that VA’s pharmacy vendor would  use these prices. DOD officials told us that they plan to specifically  include the FHCC in future manufacturer pricing arrangements, and in  the meantime DOD will evaluate any objections received from  manufacturers.
While workarounds have helped address certain MTF designation-related  issues, another MTF designation-related issue continues to pose  challenges for FHCC operations and health care providers. Navy Bureau  of Medicine and Surgery policy regarding the deployment readiness of  Navy and Marine Corps servicemembers requires the approval of a  medical screener assigned to an MTF. The commanding officer of an  MTF assigns specific providers at the facility (medical officers, physician  assistants, nurse practitioners, or Independent Duty Corpsmen) the  responsibility to conduct suitability and medical assignment screening.  For example, for a Navy or Marine Corps servicemember to be approved  for overseas duty, the servicemember must have a medical, dental, and  educational suitability screening, and MTF medical and dental screeners  must sign off on the form stating that the servicemember is suitable for  that assignment. FHCC officials told us that while the forms are being  signed, there is uncertainty as to whether providers continue to have the  authority to sign the forms as MTF medical screeners because the FHCC  is not an MTF, and they have not seen documentation that confirms  whether provider sign-off authority has changed with the establishment of  the FHCC. FHCC officials said this has created confusion among FHCC  providers about how to interpret DOD policies regarding these  documents.

	VA and DOD Use Integration Benchmarks to Assess Provision of Care and Operations at the FHCC, but the Performance Reporting Plan May Not Yield Transparent, Meaningful, and Accurate Results

VA and DOD, through FHCC staff, are using the 15 integration  benchmarks set forth in the Executive Agreement (and their  corresponding performance measures) to assess the provision of care  and operations at the FHCC. The plan is to report on these performance  measures using a tool developed by FHCC staff—a scorecard that  generates a monthly summary score. However, the summary score does  not account for data collection variation, FHCC staff have not specified  what target score(s) would indicate successful performance, and the  scorecard initially contained an error, all of which raise concerns about  the FHCC’s ability to report transparent, meaningful, and accurate  performance results.

		VA and DOD Are Assessing Provision of Care and Operations at the FHCC through 15 Integration Benchmarks

The 15 Executive Agreement integration benchmarks, chosen by VA and  DOD, are intended to assess the provision of care and operations at the  FHCC in three main areas of focus: patient and staff satisfaction,  including benchmarks that measure patient and staff feedback; clinical  and administrative functions, such as benchmarks aimed at assessing  patient access to care and clinical productivity; and external evaluation,  including our review among others. The benchmarks vary in several  aspects including whether they were created specifically for the FHCC or  whether they are compared to historical performance before the facilities  were integrated, as well as the frequency of data collection for each  individual benchmark and the specific performance measures each  includes.
The 15 integration benchmarks vary by time frame of establishment.  Most (9) of the 15 integration benchmarks were used by the former  NCVAMC and NHCGL prior to the establishment of the FHCC, while the  remaining benchmarks (6) were established specifically for the FHCC.  (See table 2.) Of the benchmarks that pre-date the establishment of the  FHCC, some have separate measurements for VA and DOD populations,  such as patient satisfaction surveys, as was the case before the FHCC  was established. FHCC officials said they have no short-term plans to  integrate patient satisfaction, because separate measurements allow  them to compare results from before and after the FHCC integration. In  addition, the other benchmarks that pre-date the FHCC integration are  measured for only VA or DOD populations. For example, DOD measures  Navy servicemember medical readiness for duty, which was previously in  place at the former NHCGL to help assess performance of DOD’s  “mission critical” operational readiness goals. VA measures health  profession trainee satisfaction, which had been measured at the former  NCVAMC and helps to assess VA’s clinical and administrative  performance. The 6 remaining benchmarks were created specifically for  the FHCC, some of which were designed to measure aspects of FHCC  integrated performance, such as whether the “information technology  solution timeline is met and has no negative impact on patient safety.”  Many of these benchmarks have no historical data to which performance  can be compared.
The 15 integration benchmarks are comprised of 38 individual  performance measures. Each of the integration benchmarks has  corresponding performance measures, for a total of 38 individual  performance measures for the 15 benchmarks. For most of the  integration benchmarks, there are at least two individual performance  measures. (See table 3.) FHCC staff have developed a Technical  Manual to document their plan for the measurement, data collection,  and reporting of these performance measures.
The 38 performance measures vary by frequency of data collection.  FHCC staff collect data for the performance measures at different time  intervals. (See fig. 4.) Depending on the individual measure, data  generally are collected weekly, monthly, quarterly, annually, or every 2 to  3 years. Data for one performance measure is collected both  semiannually and annually, and data may also be collected on a varied  time frame.
In some cases, data for different performance measures within a single  integration benchmark may be collected at different points in time. For  example, data for the performance measures within the benchmark  “satisfactory facility and clinical inspection, accreditation, and compliance  outcomes from external oversight/groups” vary as to the times that the  respective reviews are conducted; the Joint Commission, for example,  conducts full reviews every 3 years. In addition, data collection for some  measures has not yet begun. For example, the “validation of FHCC fiscal  reconciliation model by annual independent audit” cannot yet be  measured because audits have yet to be performed.

		Reporting a Summary Score of Monthly Results May Not Yield Transparent, Meaningful, and Accurate Performance Information

FHCC staff have developed a reporting tool in the form of a scorecard  that tracks and summarizes performance data for all 38 performance  measures. The scorecard is designed to calculate scores for each of the  performance measures as well as to generate a summary score every  month. Each of the 38 performance measure scores is determined by  multiplying: (1) a rating based on performance, (2) assigned weights  based on the level of importance, and (3) a fixed multiplier to adjust the  score to a scale of 100. These individual performance measure scores  are then combined into a monthly summary score, also measured on a  scale from 0 to 100. (See fig. 5.)
Specifically, each performance measure’s score is determined by the  following:  1.  Each performance measure is rated on a scale from 1 (lowest rating)  to 5 (highest rating) based on how well the measure meets its target  goal, according to definitions set in the FHCC Technical Manual.  2.  The Advisory Board weighted each performance measure, assigning  the greatest weight to measures they determined were most critical  for meeting NDAA for Fiscal Year 2010 requirements. Specifically,  those measures concerning DOD’s missions of servicemember  readiness and VA’s mission of clinical and administrative performance  received the greatest weights. The three weights used, from low to  high, are 1 (important), 2 (essential), and 5 (critical).  3.  In addition to tracking the trends for each of the performance  measures, the scorecard is designed to calculate a summary score on  a scale of 0 to 100 by adjusting each measure using a fixed  multiplier. (See text box below for an example of the calculation of  one performance measure score.)
Sample Performance Measure Score Calculation  Using the example of the health profession trainee satisfaction performance measure, a  score is determined using the following information:  1.  FHCC staff assigned it a performance rating of 4 – health professionals are  “somewhat satisfied” with training.  (This is a baseline rating made before the FHCC was integrated on   October 1, 2010.)  2.  The performance measure has a weight of 1 (important).  3.  All performance measures are adjusted to the 100 point scale using a fixed  multiplier of 0.2128.
The rating and weight are multiplied together and adjusted to 100 for a score of 0.8512   (4 x 1 x 0.2128).
FHCC officials told us they designed the scorecard to calculate a user- friendly, single summary score on a monthly basis and report  performance to the Advisory Board at their regular meetings, which are  typically quarterly. However, we identified three areas of concern with  the monthly summary score:  The monthly summary score does not account for varied data  collection time frames. Although the summary score is calculated  monthly, data for all performance measures are not collected on a  monthly basis. Specifically, FHCC staff told us they record no score when  no new monthly data are available for a given performance measure,  even when that measure was not expected to be collected on a monthly  basis. In fact, there may be no single month where complete performance  data are available to be factored into the summary score. For instance,  the health profession trainee satisfaction performance measure, collected  annually, would appear in the scorecard only in the month it was collected  and have no data listed and no rating given for the remaining 11 months  of the year. For more than half the performance measures, data are  collected less frequently than monthly and would be similarly affected.  In any given month, fluctuations in the summary score may be caused by  varied data collection, and not changes in performance, which is not  transparent in the scorecard methodology.
The summary score lacks a set target score(s) to indicate success.  The summary score’s ability to provide a meaningful indication of success  is unclear because neither FHCC staff nor the Advisory Board to whom  the scores are reported has established any specific target score(s) to  indicate that the FHCC has achieved success. FHCC officials told us that  the goal of the scorecard is to calculate a summary score for all 38  performance measures on a scale of 0 to 100 to indicate the level of  success of the integration, with a maximum (perfect) score of 100. While  there are specific targets for each of the 38 performance measures,  officials have not determined what score(s) will indicate overall success of  the integration at the end of the 5-year demonstration. Without  establishing a target summary score(s) to indicate successful FHCC  integration, FHCC staff do not have the ability to gauge progress, thus  diminishing the usefulness of calculating a summary score.
The calculation error raises concerns about accuracy in the  scorecard methodology. Upon review of the FHCC’s final version of the  scorecard, we discovered an error in the multiplier used to adjust the  scores to a 100 point scale. With the addition or deletion of performance  measures, the multiplier needs to be recalculated to ensure that the  summary score retains a 100 point scale. We found that when the FHCC  added performance measures, which now total 38, they had not adjusted  the multiplier accordingly. This resulted in a scorecard that calculated a  summary score with a possible total of 119 rather than 100, as the FHCC  intended. Although the FHCC has fixed the error and in March 2011  presented a corrected scorecard to use going forward, the lack of initial  awareness of the error raised concerns about the accuracy of the results  reported using the scorecard.

	Conclusions

The FHCC is a 5-year demonstration project that has the potential to be a  model for future VA and DOD integration efforts. However, IT  implementation delays, the lack of MTF designation, and concerns about  the use of a summary score to report on FHCC performance may impact  FHCC officials’ ability to provide the information necessary for Congress  to determine whether to continue the FHCC beyond the 5-year  demonstration and whether the model should be replicated elsewhere.  While the delays implementing the IT integration area of the Executive  Agreement have been largely outside the control of FHCC officials, they  may impact FHCC officials’ ability to operate the FHCC as a fully  integrated facility. As we recommended in our February 2011 report, we  continue to believe that the Secretaries of VA and DOD should strengthen  their ongoing efforts to establish the joint IT system capabilities for the  FHCC by developing plans that include scope definition, cost and  schedule estimation, and project plan documentation and approval.
The lack of an MTF designation and its related challenges may affect  further progress in implementing the FHCC demonstration. The  administrative burden and uncertainty resulting from the lack of MTF  status may hinder FHCC officials’ ability to efficiently operate the FHCC  until DOD clarifies the facility’s status relative to the rest of DOD’s health  care system or obtains a legislative change to designate the FHCC as an  MTF.
As a reporting tool, the FHCC scorecard has the potential to be useful in  tracking performance results over time. However, calculating a monthly  summary score for the FHCC scorecard raises concerns about FHCC  officials’ ability to convey transparent, meaningful, and accurate  performance information to VA and DOD officials and other stakeholders.  If the monthly summary score calculations do not account for data  collection variation, do not specify a target score(s) that would indicate  successful performance, or continue to have errors, then the scorecard’s  ability to gauge FHCC performance results is unclear. Until these  concerns are addressed, the Secretaries of VA and DOD and Congress  will be unable to make fully informed decisions as to whether the FHCC  model should continue and whether it should be replicated in other  locations.

	Recommendations for Executive Action

To ensure that FHCC officials are able to efficiently operate the FHCC  and uncertainty regarding the lack of MTF status is resolved, we  recommend that the Secretary of Defense seek a legislative change to  designate the FHCC as an MTF.
To ensure that the plan to report on FHCC performance results is  transparent and provides meaningful information that can assist VA and  DOD leadership and Congress in decision making with regard to the  future of the FHCC or other VA/DOD integration efforts, we recommend  that the Secretaries of Veterans Affairs and Defense direct FHCC  leadership to conduct further evaluation of the scorecard reporting tool  and its methodology and make revisions that will better ensure the  transparency and accuracy of the information reported.

	Agency Comments and Our Evaluation

DOD and VA each provided comments on a draft of this report. In its  comments, DOD concurred with one of our two recommendations to the  Secretary of Defense. (DOD’s comments are reprinted in app. I.) VA, in  its comments, generally agreed with our conclusions and concurred with  our recommendation to the Secretary of VA. (VA’s comments are  reprinted in app. II.) In addition, both VA and DOD provided technical  comments which we have incorporated as appropriate.
DOD agreed with our finding that the lack of an MTF designation for the  FHCC has posed some challenges and confusion; however, the  department did not concur with our recommendation to the Secretary of  Defense to seek a legislative change to designate the FHCC as an MTF.  Rather than seek a legislative change as we recommended, DOD stated  that it will consider seeking legislative authorization for the use of  personal services contracts at the FHCC—one of the challenges we  discuss in this report. In its response, DOD stated that it anticipates that  as the FHCC stabilizes and matures, the confusion among employees  and providers at the FHCC due to the lack of an MTF designation will  dissipate and that each of the challenges described in the draft report has  been addressed through workarounds. We disagree with DOD’s  reasoning and maintain that our recommendation should be implemented  in order to eliminate the need for the current workarounds and to address  any future problems arising from the lack of an MTF designation for the  FHCC. Since the FHCC is now providing the services that the NHCGL  once did, it should have the same MTF designation the NHCGL had in  order to carry out its work as efficiently as possible. Eliminating the need  for workarounds could free staff time and contribute to increased  efficiency of patient care and operations at the FHCC. In addition, if the  FHCC model of collaboration between VA and DOD is replicated  elsewhere, the same workarounds will have to be implemented in order to  overcome the lack of an MTF designation. If our recommendation were  implemented, it would set a precedent for future VA and DOD integrations  and help make the integration process smoother.
VA and DOD concurred with our recommendation to the Secretaries of  Veterans Affairs and Defense to direct FHCC leadership to conduct  further evaluation of the scorecard reporting tool and its methodology,  and make revisions that will better ensure the transparency and accuracy  of the information reported. In its comments, VA describes the  department’s recent actions to implement this recommendation. VA  acknowledged that the varying reporting timelines for performance  measures resulted in an artificially low monthly summary score in some  months when using the original methodology. VA stated, in its comments,  that it has changed the calculation process for the scorecard’s monthly  score to address this issue. Specifically, FHCC staff will populate the  scorecard with a score for each measure every month using either data  acquired that month, or the most recent available data for those  measures. VA states that this will allow for a more accurate comparison  of performance from month to month.
We are sending copies of this report to the Secretary of Defense,  Secretary of Veterans Affairs, and appropriate congressional committees.  In addition, the report will be available at no charge on the GAO Web site  at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  Debra A. Draper at (202) 512-7114 or draperd@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix III.

Appendix I: Comments from the Department of Defense

Appendix II: Comments from the Department of Veterans Affairs

Appendix III: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the contact named above, Marcia A. Mann, Assistant  Director; Jill K. Center; Kaycee M. Glavich; E. Jane Whipple; and   Malissa G. Winograd made key contributions to this report. Lisa A. Motley  provided legal support and Jennie F. Apter assisted in the message and  report development.

Summary:
Why GAO Did This Study

The National Defense Authorization Act (NDAA) for Fiscal Year 2010 authorized the Departments of Veterans Affairs (VA) and Defense (DOD) to establish a 5-year demonstration project to integrate VA and DOD medical care into a first-of-its- kind Federal Health Care Center (FHCC) in North Chicago, Illinois. Expectations for the FHCC are outlined in an Executive Agreement signed by VA and DOD in April 2010. The NDAA for Fiscal Year 2010 also directed GAO to annually evaluate various aspects of the FHCC integration. This report examines (1) what progress VA and DOD have made implementing the Executive Agreement to establish and operate the FHCC and (2) what plan, if any, VA and DOD have to assess FHCC provision of care and operations. GAO reviewed FHCC documents and conducted visits to the site; interviewed VA, DOD, and FHCC officials; and reviewed related GAO work.

What GAO Found

FHCC officials have made progress implementing provisions of the Executive Agreement's 12 integration areas. For some areas, all provisions have been addressed, including establishing the facility's governance structure and patient priority system. Progress continues to be made in other areas, such as workforce management and personnel and quality assurance. However, as previously reported by GAO, there have been delays implementing the information technology provisions, which present challenges for operating the FHCC as a fully integrated facility. In addition, while some workarounds are in place, the lack of an military treatment facility (MTF) designation that other DOD medical facilities have presents challenges for efficient FHCC operations and results in uncertainty regarding access to preferred drug prices and provider authority to sign medical readiness forms for active duty Navy servicemembers. Although VA and DOD are assessing the provision of care and operations at the FHCC, their plan to report on performance lacks transparency and may not provide a meaningful and accurate measure of success. Specifically, VA and DOD, through FHCC staff, are using 15 integration benchmarks set forth in the Executive Agreement to assess the integration. From these benchmarks, FHCC staff identified 38 corresponding performance measures to assess the integration's success. While FHCC staff plan to report on these performance measures through a reporting tool they developed--a scorecard that calculates a monthly summary score--the tool lacks transparency and may not provide a meaningful indicator of performance. The scorecard does not account for data collection variation, there is no designated target score(s) to indicate successful integration performance, and the scorecard initially contained a calculation error, all of which raise concerns about its ability to provide transparent, meaningful, and accurate information.

What GAO Recommends

GAO recommends that DOD seek a legislative change to designate the FHCC as a MTF--a DOD facility providing medical or dental care to eligible individuals, and that VA and DOD direct FHCC leadership to further evaluate its integration performance reporting tool. DOD did not agree with the recommendation regarding the MTF designation, but GAO continues to believe such designation is important. VA and DOD agreed with GAO's recommendation regarding the scorecard reporting tool.