VA Dialysis Pilot: Increased Attention to Planning, Implementation, and Performance Measurement Needed to Help Achieve Goals
Released: 2012-05-23T00:00:00

Letter

Background

VHA oversees VA’s health care system, which includes 153 VAMCs organized into 21 VISNs. Each VISN is charged with the day-to-day management of the VAMCs within its network. VA Central Office decentralized its budgetary, planning, and decision-making functions to the VISN offices in an effort to improve accountability and oversight of daily facility operations. However, VA Central Office maintains responsibility for monitoring and overseeing both VISN and VAMC operations.

Providing Dialysis Treatments to Veterans

Veterans may elect to have their dialysis treatments through VA or Medicare but cannot receive dialysis benefits from both simultaneously. In 2008, there were over 18,000 veterans enrolled in VA’s health care system diagnosed with ESRD who required dialysis treatments. Of these enrolled veterans, about two-thirds elected to receive their dialysis treatments through VA. However, VA treated less than half of these veterans in VAMC-based dialysis clinics because of capacity limitations and other factors, such as long distances veterans may have to travel to the VAMC.most of these veterans receiving their dialysis treatments through VA’s fee basis program. Through VA’s fee basis program, veterans can select any dialysis provider, as long as their chosen provider accepts VA’s established payment rate for dialysis treatment. Currently, fee basis rates differ by VISN; however, several VISNs are currently paying for dialysis treatments through several multi-VISN-negotiated agreements with dialysis providers. Under these agreements, per-treatment costs for these VISNs currently range from $248 to $310. Veterans who elect to have their dialysis treatments provided through VA—either in VAMCs or through the fee basis program—may not incur any out-of-pocket expenses.

This limited internal capacity and other factors resulted in The remaining one-third of all veterans enrolled in VA’s health care system who were diagnosed with ESRD, in fiscal year 2008, elected to have their dialysis treatments paid for by Medicare. The veteran can select any Medicare-certified dialysis provider that accepts Medicare payment and there are few, if any, restrictions on this choice since all major dialysis providers accept individuals covered by Medicare. Medicare reimburses dialysis providers 80 percent of a specified per- treatment base bundled rate—about $230 in 2011—and beneficiaries or private insurance companies are responsible for the remaining 20 percent.for by Medicare, the remaining 20 percent may be an out-of-pocket expense that was about $7,600 per year in 2008, because VA is not authorized to pay these out-of-pocket expenses incurred by veterans covered by Medicare.

Development of the Dialysis Pilot

In 2009, VA began developing the Dialysis Pilot to build its in-house capacity to provide dialysis treatments to veterans in response to several issues, including the following:

Rising numbers of veterans needing dialysis. A VA-funded study found that the number of veterans requiring dialysis treatments was projected to increase 6 percent from fiscal year 2008 to fiscal year 2015 and the number of veterans receiving these dialysis treatments from community providers through the fee basis program was projected to increase 16 percent.

Rising costs of providing dialysis through the fee basis program.

The same VA-funded study also found that VA’s fee basis per- treatment costs were projected to increase about 59 percent from $337 per treatment in fiscal year 2008 to $535 per treatment in fiscal year 2015.

Unsuccessful efforts to achieve lower reimbursement rates with fee basis dialysis providers. Another VA-funded study found that if VA adopted Medicare rates, set by the Centers for Medicare & Medicaid Services (CMS), for outpatient dialysis treatments, it would reduce its dialysis fee basis expenditures by 39 percent resulting in projected cost reductions from fiscal year 2011 through fiscal year 2020 of about $2 billion. As a result of this study, VA Central Office instructed VISN directors to begin using Medicare rates as the prevailing reimbursement rate for fee basis dialysis treatments in 2009. However, the major fee basis dialysis providers did not agree to provide dialysis treatments to veterans through VA’s fee basis program at these reduced rates. This resulted in VA continuing to pay for these treatments according to previously established rates, which are typically higher than Medicare rates.

In response to these issues, VA Central Office charged VISN 6, a VISN with a significant volume of veterans who require dialysis treatments, with establishing a dialysis workgroup to evaluate VA’s options for dialysis treatment delivery. The Dialysis Workgroup—led by officials from VISN 6, with representatives from VA Central Office and the VHA Chief Business Office and others with financial and clinical expertise—met in 2009 to discuss various options for providing dialysis care for veterans. This workgroup identified several options VA could take to build its internal capacity to provide dialysis treatments to veterans and identified several ways VA could address the rising costs of dialysis care provided through the fee basis program, including (1) building dialysis units in leased space in communities surrounding select VAMCs, (2) purchasing modular dialysis units, (3) modifying existing space in selected VAMCs to expand or build dialysis units, and (4) negotiating pricing agreements with select dialysis providers to allow VAMCs to pay a lower rate for fee basis dialysis treatments.

In March 2010, after discussing these options and exploring potential solutions, the Dialysis Workgroup began designing the Dialysis Pilot as an effort to build VA’s capacity to provide dialysis treatments to veterans in VA-operated facilities and reduce fee basis costs. The Dialysis Workgroup projected that the Dialysis Pilot would result in a 5-year cost savings of about $33 million by operating four outpatient dialysis clinics that could each treat 48 veterans a week. The Dialysis Pilot was approved by the Under Secretary for Health in August 2010 and by the Secretary of Veterans Affairs in September 2010. The final four pilot locations selected by the Dialysis Workgroup were Durham and Fayetteville, North Carolina; Philadelphia, Pennsylvania; and Cleveland, Ohio. Each of these locations was provided approximately $2.5 million in start-up funding by VA Central Office to establish an outpatient dialysis clinic with 12 dialysis stations that could treat 48 veterans per week. VA Central Office expected this start-up funding would be repaid by the pilot locations. (See app. I for more information on the current status of each pilot location.)

The pilot locations in Durham and Fayetteville, North Carolina, began treating veterans in June 2011. The Philadelphia, Pennsylvania, pilot location is scheduled to open in May 2012, and the Cleveland, Ohio, pilot location is scheduled to open in September 2012.

Planning and Early Implementation Phases of the Dialysis Pilot Had Significant Weaknesses

There were a number of weaknesses in VA’s execution of the planning and early implementation phases of the Dialysis Pilot that collectively could limit the achievement of its goals. Specifically, weaknesses in pilot location selection, cost estimation practices, and cost savings calculations could hamper the Dialysis Pilot’s effectiveness.

VA’s Pilot Location Selection Process Was Not Transparent or Appropriately Documented

While the Dialysis Workgroup reported using several criteria to select the Dialysis Pilot locations and documented some of these criteria in the approval documents for the Dialysis Pilot, it did not document how these criteria were applied or whether it assessed all 153 VAMCs for potential inclusion in the Dialysis Pilot. According to GAO internal control standards, clearly documenting key information is necessary to ensure that appropriate internal controls for communicating and recording According to Dialysis Workgroup decision-making activities are in place.officials, the Dialysis Workgroup began its pilot location selection process by identifying 13 potential pilot locations using several criteria, including (1) the number of veterans receiving outpatient dialysis treatments living within a 30-mile radius or a 30-minute drive of a VAMC, (2) a VAMC’s potential to achieve cost savings by operating a pilot location, and (3) the perceived level of dialysis-related clinical expertise available at each VAMC. Dialysis Workgroup officials told us that the final four pilot locations in Durham, Fayetteville, Philadelphia, and Cleveland were all ultimately selected because they had a high number of veterans receiving dialysis treatments and, in some cases, had access to high-quality clinical expertise. However, no documentation was provided by VA discussing how these criteria were applied to all 153 medical centers and why Durham, one of the final four pilot locations, was omitted from the list of 13 potential pilot locations when it clearly met these selection criteria.addition, VA officials from the Dialysis Workgroup whom we spoke with could not recall complete details regarding the pilot location selection process that occurred in 2009, including whether additional VAMCs were assessed against the various criteria. As a result, it is not possible for VA or an external party to definitively determine if there were any other VAMCs that could have been viable pilot locations beyond the 13 considered by the Dialysis Workgroup.

The transparency of the pilot site selection process was further compromised by the manner in which the Durham VAMC was selected as a pilot location. Dialysis Workgroup officials did not document their rationale for selecting this VAMC—a site not included in the original 13 potential pilot locations—as one of the four final pilot locations. According to Dialysis Workgroup officials, the VAMC in Salisbury was originally selected as one of the four final pilot locations; however, this VAMC was undergoing managerial changes at that time and the VAMC in Durham, located in the same VISN, was selected as a replacement pilot location. In April 2012, Dialysis Workgroup officials reported that the Durham VAMC’s high number of veterans receiving dialysis and nephrology expertise also contributed to its selection as one of the four final pilot locations. However, VA did not document either the initial selection of the VAMC in Salisbury as a pilot location or the rationale for why the VAMC in Durham was a better final selection for the Dialysis Pilot than one of the other 13 potential pilot locations that was not selected. The lack of documentation on this particular selection further reduced the transparency of the decision-making process.

VA Central Office officials do not have complete information about how or why pilot locations were selected because key decisions and the rationale behind them were not documented. Such documentation of decision- making processes is necessary to ensure that VA decision makers have access to relevant, reliable, and timely information and could follow a rigorous and fair decision-making process for this critical aspect of the Dialysis Pilot. The lack of documentation related to a key planning decision—such as the complete process used to select pilot locations— limits VA’s ability to access this information in the future, evaluate the success of the Dialysis Pilot, and make decisions about how best to expand the pilot to additional locations.

VA Did Not Produce Consistent and Comparable Cost Estimates for Pilot Locations

It is not possible to determine whether pilot locations completed reliable cost estimations because these estimates are not consistent and comparable. This will limit VA’s ability to determine if the Dialysis Pilot has met its mission to reduce the cost of dialysis treatments paid for by VA. Reliable cost estimates are necessary to ensure that pilot location costs Generating reliable and are comparable across the four pilot locations.comparable cost estimates prior to opening the pilot locations was critical to the early implementation of the Dialysis Pilot in order to ensure that appropriate site-specific baseline cost estimates were generated that would allow VA to evaluate the cost of the Dialysis Pilot and ensure that any cost savings generated by the pilot locations could be accurately calculated. The importance of thorough and reliable cost estimation processes was included in VA’s own business analysis of the Dialysis Pilot, which stated that pilot locations were intended to use the same cost estimation methodology to facilitate uniformity and ensure that all pilot locations produced reliable information.

To its credit, the Dialysis Workgroup worked with VA systems redesign engineers to develop a sophisticated cost estimation model to help VISN and VAMC officials estimate costs for their pilot locations. VA systems redesign engineers built the cost estimation model using validated research as the foundation for its general baseline cost estimates. The cost estimation model included information on several aspects of establishing and operating an outpatient dialysis clinic—including equipment costs, leased-space costs, staff costs, and veteran travel costs. (See fig. 1.)

Despite this effort to build a robust model for estimating Dialysis Pilot costs, VA did not maintain proper control over the cost estimation model following its release for use by VISNs and VAMCs. While there were designated areas in the cost estimation model for each pilot location to enter its specific cost inputs, a VA systems redesign engineer we spoke with explained that the formulas in the cost estimation model should not be customized by pilot locations. These formulas were validated by VA systems redesign engineers and were meant to remain constant across all pilot locations to ensure that comparable and consistent cost estimates were produced. However, VA Central Office requested that the cost estimation model be fully customizable—including all cost inputs and formulas—in order to encourage pilot locations to use the model. Because the model was fully customizable, some pilot locations both appropriately altered their pilot location-specific inputs and inappropriately altered the formulas that were intended to remain constant. As a result of the inappropriately altered formulas, final cost estimates for the four pilot locations are inconsistent and do not include comparable information.

We found several inconsistencies in pilot locations’ use of the cost estimation model, including the following:

Not all pilot locations used validated formulas for developing cost estimates. We found that not all pilot locations used the validated formulas developed by VA systems redesign engineers to calculate their cost estimates. For example, Philadelphia’s cost estimation model did not use the validated formula for calculating the pilot location’s equipment costs. While the validated equipment cost formula included a patient transport cardiac monitor for each dialysis pilot location, the Philadelphia pilot location’s model omitted this equipment from its calculation. Also, instead of using the validated formulas, the Cleveland pilot location deleted some formulas related to annual patient demand and leased-space costs and replaced them with specific numeric values. As a result of these changes, some of Cleveland’s cost estimations cannot be confidently compared with those from the other three pilot locations because it is unclear what the location used to calculate these numeric values. Consistent use of the cost estimation model and its validated formulas is necessary to ensure that the cost estimations of each pilot location can be compared and evaluated.

Pilot location capacity changes. Two pilot locations increased the capacity of their outpatient dialysis clinics, despite the fact that the Dialysis Workgroup specifically established clear capacity limits and the cost estimation model was developed for these specific capacity limits. According to the Dialysis Workgroup and the Dialysis Pilot approval document signed by the Secretary of Veterans Affairs, each pilot location’s capacity was limited to 12 dialysis stations that could provide up to 48 veterans with dialysis treatments each week. However, the Fayetteville pilot location increased its capacity from 12 to 16 dialysis stations and the Cleveland pilot location increased its capacity from 12 to 20 dialysis stations. These capacity increases were not validated by VA systems redesign engineers, and as a result, it is unclear how these changes may affect the efficiency of these pilot locations. In addition, it is unclear whether these capacity increases were approved by VA Central Office since the size of these two pilot locations is larger than what was originally approved by the Secretary of Veterans Affairs.

According to Dialysis Workgroup officials, pilot location-specific baseline cost estimates were included in VA’s own business analysis of the Dialysis Pilot. However, the baseline cost estimates included in this document are unreliable for the following reasons:

The baseline cost estimates included in VA’s business analysis of the Dialysis Pilot are based on the assumption that all pilot locations will be limited to 12 dialysis stations. However, the Cleveland and Fayetteville pilot locations currently have considerably more dialysis stations, with 20 and 16 dialysis stations, respectively.

The baseline cost estimates included in VA’s business analysis of the Dialysis Pilot were generated prior to the cost estimation model’s distribution to pilot locations for customization. Therefore, these estimates do not account for the pilot location-specific customization of several model inputs, such as actual leased-space costs. During the site-specific customization process, several of the costs associated with these customizable inputs increased significantly due to either changes in pilot location size or other factors. For example, the Cleveland pilot location’s customized model includes about $400,000 in annual lease expenses, while the baseline cost estimate for Cleveland’s lease expenses from VA’s business analysis is only about $220,000.

VA Did Not Provide Clear and Timely Guidance to Pilot Locations on Start-up Fund Repayment and Cost Savings Calculations

VA Central Office officials did not provide VISN and VAMC officials with clear and timely written guidance or instructions on how to pay back start-up funds or how to calculate cost savings from the Dialysis Pilot. VA Central Office provided a total of approximately $10 million in start-up funding for the Dialysis Pilot to the three VISNs associated with the pilot locations. Each pilot location received about $2.5 million in start-up funding to establish its outpatient dialysis clinic. Pilot locations are expected to achieve cost savings through the outpatient dialysis clinics and to repay their start-up funding. Specifically, the memorandum approving the Dialysis Pilot signed by the Secretary of Veterans Affairs and the Under Secretary for Health states that pilot location start-up funds are to be repaid in two equal payments in fiscal year 2012 and fiscal year 2014.

A lack of communication—including ongoing discussions, reporting, and guidance—regarding the repayment of pilot location start-up funds could make it difficult for VA and external parties to determine if the pilot locations are making reasonable progress toward repaying these funds and realizing cost savings from the Dialysis Pilot.officials reported a lack of clarity about how Dialysis Pilot start-up funds must be repaid. Specifically, officials from two of the three VISNs associated with pilot locations—VISNs 4 and 10—told us that they have not discussed the repayment of their pilot locations’ start-up funding with VA Central Office. Similarly, officials from the VAMCs associated with the two operational pilot locations acknowledged their understanding that start-up fund repayment would likely be included as part of their cost VISN and VAMC savings calculations, but told us that they were not aware of any specific agreements or plans for repayment.

In addition, while the Dialysis Workgroup provided pilot locations with 5-year cost savings projections and articulated plans for calculating actual cost savings in a document that was published prior to the approval of the Dialysis Pilot, VISN officials we spoke with were uncertain about how cost savings would be calculated. According to VA’s own business analysis of the Dialysis Pilot, actual cost savings will be calculated by comparing the cost per treatment at each pilot location to actual fee basis per-treatment rates for each pilot location’s corresponding VAMC.states that cost saving calculations will be a collaborative effort between pilot location leadership, VA researchers, and the VHA Chief Business Office. However, VISN officials stated that while they expect that the cost savings from their pilot locations will be examined, they did not receive written guidance about how cost savings will be calculated. Given the lack of specific and timely guidance to VISN and VAMC officials on the calculation of cost savings, officials at the pilot locations may not use the same methodology to track these savings.

VA’s Dialysis Pilot Has a Stated Mission but Lacks Clearly Defined Performance Criteria and an Evaluation Plan

VA Central Office has not yet determined how it will define success for the Dialysis Pilot or created clear performance measures linked to the four Dialysis Pilot goals. When other leading public sector organizations are engaged in efforts to improve their performance and help their organizations become more effective—similar to VA’s goals for the Dialysis Pilot—we found that these organizations commonly take three steps: (1) define a clear mission and goals, (2) measure performance to gauge progress toward achieving goals, and (3) use performance information as a basis for decision making.

Step 1—Defining a clear mission and goals. VA has completed the first of these steps by defining a clear mission and goals for the Dialysis Pilot. Specifically, the Dialysis Workgroup noted that the Dialysis Pilot would allow VA to develop a cost-effective business model that could be used to optimize VA’s resources and increase its capacity to provide dialysis treatment to veterans. This workgroup also outlined four goals of the Dialysis Pilot: (1) improved quality of care, (2) increased access for veterans, (3) additional dialysis research opportunities, and (4) cost savings for VA-funded dialysis treatments. Through the participation of its membership in developing this mission and these goals, the Dialysis Workgroup was able to incorporate the input of several VA internal stakeholders—including VA Central Office representatives, VISN leadership, clinical experts with experience treating veterans with ESRD, and VA systems redesign engineers. This process included a thorough assessment of VA’s options for providing dialysis treatments to veterans—including the resources, equipment, and staffing needed to operate a cost-effective outpatient dialysis clinic.

Step 2—Measuring performance to gauge progress. Despite its success in defining a clear mission and goals for the Dialysis Pilot, VA has not developed a clear plan for evaluating the pilot. Specifically, while two pilot locations (Durham and Fayetteville) began treating veterans in June 2011, VA has not yet begun an evaluation of the establishment and management of the pilot locations—including causes for opening delays, operating challenges, or the sufficiency of start-up funding. Previously, we found that developing sound evaluation plans before a pilot program is implemented can increase confidence in results and facilitate decision making about broader applications of the pilot. In March 2012, the VHA Chief Business Office reported that VA is in the early stages of establishing an agreement with a leading university research center to conduct an evaluation of the Dialysis Pilot; however, no target dates were provided for when this evaluation would begin or what aspects of the Dialysis Pilot beyond cost-effectiveness it would evaluate.

In addition, VA Central Office has not developed a cohesive strategy for evaluating the Dialysis Pilot and has not yet formally defined its criteria for measuring the performance of the pilot locations or the success of the Dialysis Pilot in general. Several potential performance measures could be used to measure the pilot locations’ progress toward the achievement of each Dialysis Pilot goal: Improved quality of care. Officials from the Dialysis Workgroup told us that quality assurance outcomes, specifically those used by CMS to certify outpatient dialysis clinics, could be used to assess pilot locations. These metrics would likely help VA assess the quality of dialysis care provided by the pilot locations.

Increased access for veterans. Dialysis Workgroup officials told us that patient satisfaction information could be used to assess pilot locations. This potential metric could help determine if the pilot locations increased veterans’ access to dialysis care.

Additional dialysis research opportunities. In its business analysis of the Dialysis Pilot, the Dialysis Workgroup recommended that VA fund a 4-year research study to evaluate the quality of care at all pilot locations and identify best practices in veteran dialysis care. According to this business analysis, the findings of this study would enable VA to develop an evidence-based strategy for veteran dialysis care that ensures veterans receive the highest quality of care.

Cost savings for VA-funded dialysis treatments. In its business analysis of the Dialysis Pilot, the Dialysis Workgroup suggested that pilot locations could use the cost estimation model to calculate cost savings generated by the pilot locations by comparing the cost of providing dialysis at each pilot location to the cost of providing this treatment through fee basis providers. However, this potential performance metric may be limited by VA’s failure to maintain control over the cost estimation model or provide sufficient guidance to pilot locations about how to properly use it.

Because VA has not yet developed an evaluation plan or formally defined performance measures for pilot locations, it does not have access to consistent and reliable information on the performance of the pilot locations and may not have this information accessible when it is time to either make midcourse corrections for the Dialysis Pilot or decide whether and how to open additional VA-operated outpatient dialysis clinics.

Step 3—Using performance as a basis for decision making. Despite not having fully developed performance measures for assessing the pilot locations, VA has already begun planning for the expansion of the Dialysis Pilot, which should not occur until after VA has defined clear performance measures for the existing pilot locations and evaluated their success. Specifically, a member of the VHA Dialysis Steering Committee told us that the committee has already developed a limited plan for expansion of the Dialysis Pilot. However, this plan does not incorporate the results of a performance assessment for the existing four pilot locations. In addition, VA systems redesign engineers have begun developing three additional cost estimation models despite not having fully evaluated the effectiveness of the cost estimation model used in the Dialysis Pilot. Taken together, these two actions indicate that VA is beginning to make decisions about the future of the Dialysis Pilot and the cost estimation model, even though VA decision makers currently lack critical performance information on the existing four pilot locations.

Conclusions

To its credit, VA developed the Dialysis Pilot as a potential way of addressing the rising cost and utilization of fee basis dialysis treatments among veterans. Through the Dialysis Pilot, VA intends to test the viability of increasing its capacity to provide dialysis treatments in VA-operated outpatient dialysis clinics. VA set four goals for its Dialysis Pilot: (1) improve quality of care, (2) increase access for veterans, (3) provide additional dialysis research opportunities, and (4) achieve cost savings for VA-funded dialysis treatments. While these are commendable goals, there were weaknesses in VA’s planning and early implementation of the Dialysis Pilot that if not corrected, will make it difficult to determine whether the Dialysis Pilot has met its goals and will provide cost-effective care if expanded. Specifically, VA did not conduct a transparent and well- documented pilot location selection process, provide clear and timely guidance to participating VISNs and VAMCs on key financial aspects of the Dialysis Pilot, or articulate clear performance measures for pilot locations. We believe that VA can rectify these weaknesses, but must act prior to full implementation of the four pilot locations to ensure that the Dialysis Pilot is not compromised and can serve as an effective demonstration effort. Moreover, until VA has reliable data, we believe it would be unwise for VA to expand the Dialysis Pilot beyond the current four pilot locations, as doing so may risk investing resources inappropriately.

Moving forward, we believe four critical areas should be addressed. First, VA must clearly document the selection process it used to identify the existing four pilot locations and may use to identify any future pilot locations. VA relied on a decentralized and ad hoc selection process to choose the existing four pilot locations and failed to properly document the results of this key decision-making effort. Such inattention to documenting critical decisions results in a lack of transparency and weakens the credibility of the Dialysis Pilot.

Second, VA needs to ensure that changes to its cost estimation model are reviewed by knowledgeable staff. This step is necessary to ensure that this model produces comparable data for all pilot locations that can serve as an accurate basis for evaluating the financial success of the Dialysis Pilot. To date, pilot locations altered existing formulas and model assumptions, which resulted in cost estimation data with questionable reliability that may limit VA’s ability to compare results consistently across the pilot locations.

Third, VISN and VAMC officials need specific guidance for the repayment of Dialysis Pilot start-up funds and the calculation of cost savings realized from the pilot locations. To date, VA Central Office has not articulated its expectations regarding these two critical aspects of the Dialysis Pilot. Without a clear understanding of the terms for start-up fund repayment, it is difficult for VA or external entities to determine if the four pilot locations are making significant progress toward repaying these funds and generating cost savings that can be used to offset the cost of treating the projected increased number of veterans who will need dialysis treatment in the coming years.

Finally, VA must develop clear and measurable performance criteria that can be consistently applied to evaluate the Dialysis Pilot. Despite defining the mission and goals for the Dialysis Pilot, VA has not developed a plan for evaluating its success or developed performance measures to track pilot locations’ progress toward meeting its stated mission and goals. An effective evaluation plan and clear performance measures are needed to help ensure that the Dialysis Pilot operates in an environment of accountability.

Recommendations for Executive Action

In order to increase VA’s attention to planning, implementation and performance measurement of the Dialysis Pilot we are making five recommendations.

To improve VA’s communication related to the Dialysis Pilot, we recommend that the Secretary of Veterans Affairs direct the Under Secretary for Health to ensure that key decisions made regarding pilot location selection and efforts to continue or expand the Dialysis Pilot are clearly documented.

To ensure that reasonable cost estimates are created for the Dialysis Pilot and other similar programs, we recommend that the Secretary of Veterans Affairs direct the Under Secretary for Health to restrict or evaluate changes made to cost estimation models at the VISN and VAMC levels that affect pilot development and analysis.

To ensure that start-up funds are repaid and cost savings are accurately calculated for the Dialysis Pilot, we recommend that the Secretary of Veterans Affairs direct the Under Secretary for Health to develop written guidance about expectations for the repayment of start-up funds and how the cost savings generated by the four pilot locations should be calculated.

To ensure that VA Central Office effectively evaluates the Dialysis Pilot, we recommend that the Secretary of Veterans Affairs direct the Under Secretary for Health to take the following two actions:

Develop an evaluation plan that outlines how the Dialysis Pilot will be assessed and provides target dates for the completion of this assessment.

Develop clear measures for assessing the performance of the four Dialysis Pilot locations in key areas—including quality, access, and cost.

Agency Comments and Our Evaluation

VA provided written comments on a draft of this report, which we have reprinted in appendix II. In its comments, VA generally agreed with our conclusions, concurred with our recommendations, and described the department’s plans to implement each of our five recommendations. VA did not provide any technical comments.

In its general comments, VA noted that it has established a comprehensive strategic plan for chronic kidney disease and dialysis services; however, a copy of this strategic plan was not provided as part of VA’s response to the draft report. According to VA, this plan incorporates aspects of several of our recommendations. In addition, VA stated that it is in the process of developing longer-range plans for the expansion of dialysis services, including establishing additional freestanding outpatient dialysis clinics similar to the current four pilot locations. We support VA’s efforts to carefully analyze its delivery of dialysis services to veterans, including the most cost-effective method of providing these life-saving medical treatments, and make reasoned decisions based on a thorough evaluation of its current pilot locations. In this regard, we continue to believe that it is unwise to establish additional freestanding outpatient dialysis clinics until the current four pilot locations are fully evaluated and VA rectifies the weaknesses we identified in this report.

In its plan for addressing our recommendations, VA stated that it is developing a plan for the Dialysis Pilot that will address three of our recommendations related to (1) the documentation of Dialysis Pilot key decisions, including the selection of future pilot locations; (2) the creation of reasonable cost estimates for pilot locations; and (3) guidance for the repayment of start-up funding and cost savings calculations. According to VA, this plan will ensure better communication and documentation of decisions for future pilot site selections; more rigorous oversight of cost estimation tools and analysis of financial and clinical outcomes; and a thorough analysis of start-up fund repayment, including whether VA will reverse its decision to require repayment of these funds. VA’s anticipated completion date for these actions is July 1, 2012.

Finally, to address our remaining two recommendations, VA intends to develop a detailed evaluation plan for the Dialysis Pilot by July 1, 2012. According to VA, this plan will include specific criteria, target dates, and activities that must occur throughout the remainder of the pilot. VA intends to use this plan to periodically review and evaluate the Dialysis Pilot. In addition, VA described its efforts to significantly enhance the decision-making tools used for the Dialysis Pilot, including the cost estimation model. VA reported that these enhancements will include more rigorous accounting for facility costs, such as those for staffing and equipment. In addition, VA plans to task its systems redesign engineers with assessing pilot locations’ performance using metrics for cost, access, and quality.

We are sending copies of this report to the Secretary of Veterans Affairs, appropriate congressional committees, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Status of VA Dialysis Pilot Locations

Table 1 provides additional information on the current operating status of the four Dialysis Pilot locations at the Department of Veterans Affairs (VA) medical centers (VAMC) in Durham, North Carolina; Fayetteville, North Carolina; Philadelphia, Pennsylvania; and Cleveland, Ohio.

Appendix II: Comments from the Department of Veterans Affairs

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Marcia A. Mann, Assistant Director; Kathleen Diamond; Katherine Nicole Laubacher; Rebecca Rust; and Malissa G. Winograd made key contributions to this report. Lisa Motley provided legal support.