VA set four goals for the Dialysis Pilot: (1) improve the quality of dialysis care veterans receive, (2) increase veterans’ access to dialysis care, (3) provide additional dialysis research opportunities, and (4) achieve cost savings for VA-funded dialysis treatments. VA is performing an internal evaluation of the performance of the four pilot locations by assessing the quality of care—including clinical quality, patient satisfaction, and access to care—and treatment costs at each pilot location. Results of this evaluation are reported in an annual performance review produced by VA each calendar year. To date, VA has completed reviews for calendar years 2012 and 2013. Clinical quality: To evaluate the clinical quality at the pilot locations, VA selected 11 clinical quality measures that were derived from measures and associated performance standards established by the Centers for Medicare & Medicaid Services (CMS)—the federal entity with primary responsibility for evaluating the quality of dialysis care in the United States—and endorsed by the National Quality Forum, a leader in evaluating clinical performance measurement for a variety of chronic clinical conditions. Patient satisfaction: To evaluate veterans’ satisfaction with the quality of care provided at the pilot locations, VA analyzed the CMS standard dialysis facility survey—the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems—for the veterans treated at pilot locations. Access to care: To evaluate veterans’ access to the pilot locations, VA is measuring the percentage of veterans whose actual travel distances are no more than 30 miles from their residence to the pilot location where they receive care. Treatment cost: To evaluate the cost of care, VA is comparing the pilot locations’ average cost per treatment and the average cost per treatment for non-VA dialysis providers in each pilot location’s Veterans Integrated Service Network (VISN). To calculate each pilot location’s average cost per treatment, VA divided each pilot location’s total health care costs by its total number of treatments. To calculate non-VA dialysis providers’ average cost per treatment, VA created a composite figure using the average per treatment contract rate from the two largest non-VA dialysis providers located in each pilot location’s VISN, a per treatment VA claims processing fee, a per treatment VA physician cost for nephrology oversight, and a per treatment VA financial administrative cost. In September 2012, VA awarded a 3-year contract to an evaluation contractor to provide an independent evaluation of its Dialysis Pilot— including a comparison of the clinical quality provided by the four pilot locations with non-VA dialysis providers and a dialysis per treatment cost comparison between the pilot locations and the Non-VA Medical Care Program. Clinical quality: To evaluate the clinical quality of care provided by the pilot locations, the evaluation contractor is assessing the results of several measures—such as mortality, hospitalization, and infection rates—for the pilot locations and comparing them to relevant state and national results. These results are presented for each pilot location in quarterly quality reviews and an annual dialysis facility report (DFR), a standard report produced by the evaluation contractor for CMS for each of the more than 6,000 private sector dialysis facilities nationwide. Treatment cost: To evaluate the treatment costs for each pilot location, the evaluation contractor is comparing the pilot locations’ average cost per treatment to that of private sector dialysis providers, including certain contracted non-VA dialysis providers. Results are presented for each pilot location in quarterly cost reviews. The evaluation contractor used a similar methodology as VA to determine each pilot location’s average cost per treatment. To determine the non-VA dialysis provider average cost per treatment in calendar year 2012, the evaluation contractor created a composite figure that included (1) the contract per treatment average price for the three non-VA dialysis providers with the largest market share in the VISN responsible for the Raleigh and Fayetteville pilot locations, (2) VA’s per treatment travel-related costs, and (3) VA’s per treatment medical oversight costs. Even though it is 5 years into the Dialysis Pilot, VA Central Office has not yet set a timetable for completing the pilot or documented how it will determine the success of the four pilot locations. We have previously outlined best practices for project scheduling that specify project timelines are critical for managing and measuring an agency’s performance on projects and that it is necessary for the agency to have a document that details the rationale used in developing a project timeline. In addition, we have outlined best practices for designing evaluations that state that choosing and documenting well-regarded criteria that are used to make comparisons can lead to strong, defensible conclusions. According to VA’s business analysis of the Dialysis Pilot conducted prior to opening the pilot locations, VA planned to conclude the Dialysis Pilot 5 years after the pilot locations opened and assumed all four pilot locations would begin operations at generally the same time. Due to delays in opening the pilot locations, the completion of this 5-year period would be in fiscal year 2018—5 years after the opening of the fourth pilot location. However, in March 2014, VA officials told us they are no longer operating under the 5-year timeline outlined in this plan, although they have not updated the timeline included in the Dialysis Pilot business analysis. VA officials told us that instead of ending the Dialysis Pilot based on the timeline outlined in VA’s business analysis, they believe the Dialysis Pilot will be successful and should conclude once the pilot locations have achieved the following: Creation of a replicable model. According to VA officials, the first achievement that indicates success is the creation of a model for a VA-operated, free-standing dialysis clinic that can be replicated by other VAMCs. Such a model would include clear documentation of the necessary staff, equipment, time, and resources required to establish a new clinic and serve as a guide to other VAMCs seeking to establish similar clinics. VA officials did not specify a timeline for the creation of this model; however, they did state that they believe the pilot locations have already achieved this milestone. Pilot location breakeven points. VA officials stated that the second achievement that indicates success is the confirmation of the time necessary for a pilot location to reach its breakeven point—which VA has defined as the point when a pilot location achieves cost savings in the Non-VA Medical Care Program for dialysis services for its sponsoring VAMC and repays its start-up funding to VA Central Office. Through the Dialysis Pilot, sponsoring VAMCs for the pilot locations were each provided about $2.5 million in start-up funding to establish a VA-operated, free-standing dialysis clinic and were to finish repaying this start-up funding to VA Central Office in fiscal year 2014. According to VA Central Office officials, all four pilot locations have repaid their start-up funding as of the beginning of fiscal year 2014 and two pilot locations—Raleigh and Fayetteville—have demonstrated that their cost per treatment is lower than comparable per treatment costs of non-VA dialysis providers in their areas. Achieving this breakeven point indicates that the pilot locations are no longer incurring additional costs for VA and are fulfilling VA’s goal of providing dialysis treatments to veterans at a lower cost than similar care provided through non-VA dialysis providers. While VA officials indicated they could conclude the Dialysis Pilot once the pilot locations had created a replicable model and realized their breakeven points, VA has not formally communicated these achievements in writing as criteria for concluding the pilot. Moreover, these criteria were not included in the performance work statement for the evaluation contractor, and according to a VA official, have not been formally recorded in any official document. By not clearly communicating these milestones as criteria for concluding the Dialysis Pilot, the transparency of VA’s management decisions on pilot location outcomes is compromised and the Department lacks accountability for ensuring the success of the Dialysis Pilot. VA and the evaluation contractor have been evaluating the Dialysis Pilot for 2 years; however, neither has concluded its evaluation of the Dialysis Pilot and data on two of the pilot locations—Philadelphia and Cleveland— is limited due to their delayed opening. Table 1 shows the data available for each pilot location as of June 2014. To date, VA has produced two annual performance reviews on quality of care and treatment costs for the Dialysis Pilot—the first for calendar year 2012 and the second for calendar year 2013—that include the results of pilot locations’ clinical quality, patient satisfaction, access to care, and treatment cost performance. The VA calendar year 2012 annual performance review included results for the Raleigh and Fayetteville pilot locations only. VA included results for all four pilot locations in its calendar year 2013 annual performance review; however, this review does not include a full year of data for the Cleveland pilot location and VA noted operational challenges that affected the data for the Philadelphia pilot location. VA noted several contributing factors to the limited data available on the Cleveland and Philadelphia pilot locations, including the following: The Cleveland pilot location experienced issues with laboratory data in veterans’ VA medical records that limited the results on one of the clinical quality measures in 2013. For both the Cleveland and Philadelphia pilot locations, VA reported that the dialysis machines did not operate properly, which resulted in a lower than expected number of veterans being treated at the Philadelphia pilot location between January and June 2013 and delayed the opening of the Cleveland pilot location until July 2013. As a result, VA recommended that another full 12 months of data from the Philadelphia and Cleveland pilot locations be analyzed before making conclusive comparisons of the treatment costs of these pilot locations with non-VA dialysis providers. In addition, the evaluation contractor has reviewed the Dialysis Pilot for 2 years and has one year remaining on its contract. At the time of our analysis, the evaluation contractor had completed 2 years of evaluations on the quality of care provided by the four pilot locations and 1 year of treatment cost evaluations for two of the pilot locations.