Nurse staffing is a critical part of health care because of the effects it can have on patient outcomes and nurse job satisfaction. According to VHA, its staffing methodology aims to maximize nurses’ productivity and efficiency, while providing safe patient care by ensuring appropriate nurse staffing levels and skill mix. VHA’s nurse workforce is primarily composed of RNs, licensed practical nurses (LPN), and nursing assistants (NA). These nurses provide care—ranging from primary care to complex specialty care—in inpatient, outpatient, and residential care settings at 151 VAMCs across the country. In addition to the size of the nursing workforce, the nursing skill mix—i.e., the share of each type of nurse (RNs, LPNs, or NAs) of the total—is an important component of nurse staffing. Units vary in their nursing skill mix, depending on the needs of their patients. For example, intensive care units require higher intensity nursing, and may have a skill mix that is primarily composed of RNs compared to other types of nursing units that may provide less complex care. (See table 1 for a general description of the types of nursing staff position, responsibilities, and educational requirements.) Although the number of nurses at VAMCs increased from FY 2009 to FY 2013, VHA ranked nurses as the second most challenging occupation to recruit and retain. Specifically, the total number of nurses at VAMCs increased 13 percent from 72,542 in FY 2009 to 81,940 in FY 2013, with similarly proportionate increases within each position type—RN, LPN, and NA. During the same time period, the annual nurse turnover rate at VAMCs—the percentage of nurses who left VHA through retirement, death, termination, or voluntary separation—increased from 6.6 percent to 8.0 percent. Although RNs had the lowest turnover rate among nurses, VHA noted particular difficulty recruiting and retaining for the position, particularly for RNs with advanced professional skills, knowledge, and experience, such as RNs that provide services in medical and surgical VHA projects that approximately 40,000 new nurses will be care units. needed through FY 2018 to maintain current staffing levels and to meet the needs of veterans. See Department of Veterans Affairs, Veterans Health Administration, 2013 Workforce Succession Strategic Plan (Washington, D.C.: 2013). To help ensure adequate and qualified nurse staffing at VAMCs, in July 2010, VHA issued VHA Directive 2010-034: Staffing Methodology for VHA Nursing Personnel. ONS, the VHA office responsible for providing national policies and guidelines for all VHA nursing personnel, led the development of the nurse staffing methodology, which began in 2007. (See fig. 1.) To implement the methodology, each VAMC is required to (1) develop a VAMC-wide staffing plan for its nurse workforce, comprised of individual unit-level staffing plans, and (2) execute that plan. (See figure 2 for an outline of the process for implementing VHA’s nurse staffing methodology.) Each VAMC unit is to develop a staffing plan outlining recommendations on the appropriate nurse staffing levels and skill mix needed in that unit to support high-quality patient care in the most effective manner possible. Specifically, staffing plans are to be developed using expert panels and a data-driven analysis of nursing hours per patient day (NHPPD). VAMC nurse executives—members of senior management within each VAMC— are responsible for implementing the staffing methodology in their respective VAMCs. Expert panels: advisory groups—at the unit and facility level—of VAMC staff with in-depth knowledge of nurse staffing needs. The use of expert panels is intended to apply principles of shared governance, which allows nurses to have influence over the delivery of patient care and involves stakeholders from across the VAMC. VAMC nurse executives are responsible for ensuring that the unit-based expert panels represent all nursing types (RN, LPN, NA) and developing the VAMC’s facility expert panel. Data-driven analysis of NHPPD: involves determining the number and skill mix of nurses needed for each unit by calculating the number of direct patient care nursing hours provided for all patients on that unit during a 24-hour period. The use of NHPPD represents a move away from the more traditional nurse-to-patient ratios that assign a certain number of patients to each nurse. Some research suggests that NHPPD can better capture changes in nurses’ workloads and case mix resulting from admissions and discharges, as well as patient acuity levels, which can impact the amount of time nurses spend with each patient. After developing the staffing plan, each unit-based expert panel presents its plan, which includes staffing recommendations, to the VAMC’s facility expert panel. Those staffing recommendations may include, for example, initiatives to change the number and skill mix of nurses needed for each shift; change the number of nurses required for coverage during predicted absences, such as annual and sick leave; and develop support services for nurses, such as designated individuals to transport patients to other areas of the facility as needed. The facility expert panel—comprised of staff from across the VAMC—reviews each unit-based panel’s staffing plan and aggregates all of the unit plans into one VAMC-wide staffing plan. The VAMC nurse executive reviews the VAMC-wide staffing plan and forwards it to the VAMC director for review and approval. Once approved, the VAMC then begins execution of the initiatives outlined in the VAMC-wide staffing plan. The directive requires each VAMC to conduct an ongoing staffing analysis to evaluate staffing plans annually, at a minimum, and for VAMC directors to incorporate projected staffing needs into their annual budget review. The staffing methodology is being implemented in three phases. In Phase I, VAMCs were to implement the staffing methodology in all inpatient units no later than September 30, 2011. In Phase II, VAMCs are to implement the staffing methodology for all other units, including the operating room, emergency department, and spinal cord injury units. ONS has completed the Phase II pilot for operating room units, and VAMCs are expected to implement the methodology in their operating room units by October 1, 2014. Deadlines for the implementation in other Phase II units have not been set. In Phase III, VAMCs are to use an automated system developed by VHA that (1) merges VHA staffing data used in the staffing methodology and other VHA data, such as human resource data, into one data system, and (2) incorporates the data into staffing-related reports, such as quality-of-care reports. A deadline for Phase III implementation has not been set. In May 2014, the VA OIG found that VAMCs in its review varied in their implementation of the staffing methodology. Specifically, the VA OIG reported that 8 of the 28 VAMCs reviewed had not fully implemented all components of the staffing methodology by September 2013, 2 years past the implementation date required by the VHA directive. As these findings were similar to those of its April 2013 report, the VA OIG stated in its 2014 report, “We re-emphasize the need for all facilities to fully implement the methodology and accurately address patient needs with safe and adequate staffing.” Adequate and qualified nurse staffing at VAMCs is required to provide effective and continuous patient care and to maintain a stable and engaged workplace. The importance of nurse staffing on patient outcomes and nurse job satisfaction has been emphasized by various entities, including The Joint Commission; American Nurses Association; Institute of Medicine; and Agency for Healthcare Research and Quality.and qualifications of nurse staffing to patient outcomes and nurse job satisfaction. For example, studies have shown: Additionally, research has linked the adequacy A link between the adequacy of nurse staffing and patient outcomes, particularly in inpatient units, such as intensive care and surgical units. For example, medication errors, pressure ulcers, hospital acquired infections, pneumonia, longer-than-expected stays, and higher mortality rates each have been associated with inadequate nurse staffing. A link between the qualifications of nursing staff and patient outcomes. For example, one study found that patients cared for in units utilizing more licensed and experienced nursing staff (RNs and LPNs) and fewer unlicensed aides (NAs) had shorter lengths of stay. Other studies linked baccalaureate-prepared nurses to lower mortality rates. A link between nurse staffing and job satisfaction. For example, some studies have linked low job satisfaction to heavy workloads and an inability to ensure patient safety. Other studies found that improving nurse staffing and working conditions may simultaneously reduce nurses’ burnout, risk of turnover, and the likelihood of medical errors, while increasing patients’ satisfaction with their care. Non-VA health care organizations use various approaches to ensure effective nurse staffing. For example, some use fixed nurse-to-patient ratios while others use adjustable, unit-specific minimum staffing levels, and there have been several efforts to address nurse staffing using these different approaches.requiring regulations that mandate specific nurse-to-patient ratios that limit the number of patients cared for by an individual nurse. Other states have passed legislation or adopted regulations addressing nurse staffing without mandating specific ratios or staffing levels. For example, some states require hospitals to have committees responsible for developing unit staffing plans or require public reporting of staffing. All seven VAMCs in our review developed staffing plans using VHA’s nurse staffing methodology and have taken steps to execute them. However, VAMCs experienced problems in both the development and execution of their staffing plans. Improvements in nurse staffing were reported by some of the VAMCs which had taken steps to execute the staffing plans. The seven VAMCs in our review have implemented VHA’s nurse staffing methodology; specifically, each of these VAMCs has developed a facility- wide staffing plan, comprised of unit-level staffing plans for inpatient units, and has taken steps to execute it. Although each of the seven VAMCs in our review developed a staffing plan for FY 2013, only one had developed a plan per VHA’s directive—that is, used both expert panels and analysis of NHPPD—by September 30, 2011, the deadline specified in the directive. (See table 2.) Across all 151 VAMCs, according to ONS officials, VAMCs’ implementation of the nurse staffing methodology varied, with, for example, some VAMCs completing the development of their staffing plans during FY 2013, and some only beginning the development process. In addition to developing staffing plans, all seven VAMCs in our review had taken steps to execute their respective staffing plans. For example, VAMCs had taken steps to execute initiatives to increase the number of unit nurses or change the skill mix of nurses to address patient care needs. (See table 3 for examples of VAMCs’ staffing plan initiatives.) VAMC officials told us there are many factors that could affect the execution of staffing plan initiatives, such as available resources, the amount of time needed, and other strategic priorities. Officials and nursing staff from the seven VAMCs in our review told us they experienced problems developing and executing staffing plans. (See table 4 for examples of problems.) Some VAMCs were able to devise solutions; however in many cases, the problems have persisted. Problems Developing Staffing Plans. Staff and officials from each of the seven VAMCs in our review reported facing problems developing staffing plans. Lack of necessary data resources. Staff and officials at six of the seven VAMCs in our review said they did not have the appropriate data resources to effectively calculate NHPPD as required by VHA’s staffing methodology directive. Specifically, the directive instructs VAMC staff to calculate NHPPD using a wide range of data, such as number of admissions, transfers, and discharges; hours used for planning and treatment; and human resources data. We found that staff and officials needed to use multiple sources to collect the necessary data, in some cases manually, a process they said was time-consuming and potentially error-prone, and required data expertise they did not always have. For example, at one VAMC, the staffing methodology coordinator—a VAMC official who assists with the administrative tasks associated with the implementation process— told us she struggled with some data analysis techniques, such as creating a spreadsheet to help track staffing data, but the VAMC did not have the financial resources to hire additional data analysts to support the methodology. In contrast, officials from two VAMCs in our review told us staffing methodology coordinators were assigned in part based on their data analysis expertise. Difficulty completing and understanding training. Staff from six of the seven VAMCs in our review said the ONS training on the methodology was time consuming to complete, and difficult to understand. In 2011, ONS switched from instructor-led, group training to individual, computer-based PowerPoint training. Many unit staff reported that because the computer-based training took many hours to complete, it was difficult to find the time to complete it, while also carrying out their patient care responsibilities. They told us they often had to start and stop the training to attend to patients, which diminished its effectiveness. Further, the course’s complex material was hard to absorb through an individual, computer-based course, with many staff suggesting their understanding would have been greatly improved with an instructor-led, group course where they could ask questions, ensure consistency of learning, and build camaraderie among unit expert panel members. To address the difficulties in completing and understanding the training, one VAMC developed its own instructor-led, group training provided to all its units. Time required. Staff and officials at all seven of the VAMCs in our review reported that developing staffing plans required a lot of staff time due to the complexity of the process. In particular, they said gaining an understanding of the methodology, collecting the necessary data, convening the unit expert panels, and preparing presentations for the facility expert panel were time-intensive tasks that, in some circumstances, took time away from patient care. For example, members from one unit expert panel estimated they spent, in total, about 160 hours (4 weeks) developing the unit staffing plan during the first year the staffing methodology was implemented in their unit. Some VAMCs’ staffing methodology coordinators developed specific processes designed to decrease the burden on nursing staff and improve efficiency. For example, they created templates for unit panel members to use in staffing plan development; such templates improved efficiency because unit panel members did not have to independently develop their presentation format. Further, facility expert panel members had to orient themselves to only one template, and were therefore able to more easily make facility-level comparisons and decisions. Lack of communication within VAMC. Unit expert panel members at four of the seven VAMCs in our review said there was a lack of communication between nurses and VAMC leadership regarding the status of the staffing plans, including plans for execution of the staffing plan initiatives. Staff at one of these VAMCs said they had not received any feedback on their FY 2012 or FY 2013 unit staffing plans; they added that developing the 2013 staffing plan without getting any feedback on the prior year’s plan felt “frustrating.” In contrast, at another VAMC, officials told us that all unit staff—not just staff involved in the unit panel—received regular updates on the nurse staffing process at their monthly unit staff meetings. Difficulty integrating unit staff into expert panels. Staff and officials at three VAMCs described challenges in integrating unit staff into expert panels. Some unit panel members told us that although they were considered members of their respective unit panels, they were not significantly involved in the development of their units’ staffing plans. For example, a unit panel member said the VAMC’s staffing methodology coordinator calculated the unit’s NHPPD, developed the corresponding unit staffing plan, and presented the unit staffing plan to the respective facility expert panel almost entirely without her unit’s input. As a result, there was limited involvement of the unit panel members in the expert panel and, consequently, limited shared governance. Officials at these VAMCs said that from their perspectives, there was interest in the methodology among unit panel members, but sometimes it was difficult for these staff to attend relevant meetings because of patient needs. In contrast, unit panel members at other VAMCs in our review described how they were fully integrated into the unit panels. They described in detail the data analyses they prepared, the meetings they participated in, and their experiences presenting their unit staffing plans to the facility expert panel. Members from one unit panel told us it was helpful to be able to use data to validate the unit’s staffing and share this data with the facility expert panel—VAMC staff “beyond the typical chain of command.” Officials at this VAMC noted that unit panel members felt “empowered” to present their work to the facility expert panels. Problems Executing Staffing Plans. Staff and officials from six of the seven VAMCs in our review noted problems executing staffing plans once approved by the VAMC director. Hiring delays. Staff and officials from six of the seven VAMCs in our review said they often faced hiring delays that impacted their ability to execute staffing plan initiatives. Some VAMC staff noted it could take more than 6 months to fill unit vacancies. Although staff from one VAMC said hiring was slowed by the dearth of qualified nurses in their community, staff from other VAMCs in our review said the supply of nurses was not the problem, but rather the problem was the VHA hiring process, which took months to complete for each candidate. Additionally, VAMC staff noted that new hires also needed to complete necessary internal trainings before joining a unit full time, which added to the delays, and that some new hires were hurried through this training process because their units were so desperate to have them on staff. Budget constraints. Staff and officials from five of the seven VAMCs in our review said their VAMCs were not able to fully execute their staffing plans due to budget constraints. For example, at one VAMC, one of the approved staffing plan initiatives was the hiring of a large number of nurses for its units, in part, to address the VAMC’s inability to increase their nursing staff over a period of years. The official told us that, due to budget constraints, the VAMC was going to phase in this hiring initiative over the next few years. Some VAMC staff reported improvements in the adequacy and qualifications of their units’ nursing staff when nurse staffing plan initiatives were executed. For example, at two VAMCs at which the number of nurses was increased or support services for nurses, such as patient transporters or sitters, were put in place,adequacy of the nursing staff had improved. Furthermore, improvements in the qualifications of unit nursing staff were noted by staff in VAMC units where, for example, skill-mix changes were made or the amount of floating of nurses from their home unit to an unfamiliar unit was decreased. Both VAMC officials and unit staff noted improvements in staffing when nurses’ qualifications were more appropriately matched to the right level of work (for example, having RNs rather than LPNs available to provide more complex patient care) and to the right units (for example, the units for which they were hired and trained). Some VAMC staff said they also had seen improvements in patient outcomes and nurse job satisfaction. For example, nursing staff at one VAMC said that after creating sitter positions—as indicated by their VAMC’s staffing plan—they saw a decrease in patient falls. The staff said sitters were able to monitor patients more closely, and as a result, patients were less likely to fall during walks to the bathroom, for example. Similarly, nursing staff in a mental health unit at another VAMC said that by having more staff they had decreased their restraint use because there were more staff available to meet veterans’ needs. Additionally, nursing staff we interviewed at one VAMC that had made staffing changes based on staffing plans said they were better able to provide the type of nursing care “veterans deserve,” and this made them feel more positive about their work. Some nurses at this VAMC also said the shared governance aspect of the methodology was empowering, which, combined with their enhanced understanding of staffing at their VAMC, helped improve their overall job satisfaction. However, some VAMC unit staff reported that unit nurse staffing continued to be inadequate and that nurse unit assignments and job duties were not always appropriate for their qualifications. For six of the VAMCs in our review, staff from at least one unit interviewed said their unit staffing levels were inadequate. Staff said ensuring adequate staffing was particularly challenging when there were unplanned staff absences and they had to “scramble” to provide coverage. Some unit staff noted that this situation often resulted in units forcing nurses to work overtime or nurses floating to other units where they did not always have the qualifications to provide care. At some VAMCs, staff said there were increased staff injuries due to inadequate staffing. Furthermore, staff at one VAMC reported that where there had not been any changes made based on the unit staffing plans, their units continued to be understaffed to the detriment of both patient care and their job satisfaction. Our review of VHA’s oversight of its nurse staffing methodology found that some internal controls—those related to environmental assessment, a plan for monitoring compliance, evaluation, timeliness of communication, and organizational accountability—are limited. The implementation of internal controls is necessary for ensuring initiatives achieve intended outcomes and for minimizing operational problems. Without these internal controls in place, VHA cannot ensure that its methodology meets department goals, such as establishing a standardized methodology for determining adequate and qualified nurse staffing at all VAMCs, and ultimately, having nurse staffing that is adequate to meet veterans’ health care needs. Environmental Assessment. VHA did not comprehensively assess each VAMC to ensure preparedness for implementing its methodology, including having the necessary technical support and resources, prior to the issuance of the methodology directive in 2010. Furthermore, as of August 2014, VHA did not have a plan for assessing whether VAMCs have the necessary resources to execute their approved nurse staffing plans. Under federal internal control standards, successful organizations monitor their internal and external environments continuously and systematically, and by building environmental assessments into the strategic planning process, are able to stay focused on long-term goals even as they make changes to achieve them. VHA did not assess VAMCs’ technical resources to determine if all VAMCs would be able to successfully implement the methodology. For example, the directive recommended that VAMCs use comparative data from external sources, such as the National Database of Nursing Quality Indicators (NDNQI) when analyzing unit-level staffing data. According to some VAMC officials, due to the costs and complexity of contracting, not all VAMCs had access to this data source. Each VAMC was responsible for establishing its own contract to purchase access to NDNQI data, which some VAMC officials said was expensive and time- consuming to set up, noting that it would have been helpful to have assistance in coordinating the contracting process. Officials from ONS reported that they are discussing the possibility of having a VHA-wide contract so that all VAMCs would have access to NDNQI data. In addition to access to comparative data, according to the directive, VAMCs need appropriate data system capabilities—in particular an automated staffing system for information such as patient admission, transfer and discharge data, and human resources data—to facilitate implementation of the data- driven methodology and calculation of NHPPD. However, not all VAMCs in our review had an automated staffing system in place even 3 years after the release of the directive. Officials at a VAMC without an automated staffing system told us staff were collecting and inputting data, in many cases manually, into a spreadsheet to calculate NHPPD, and that this process was extremely time-consuming and potentially error- prone. ONS officials said they knew VAMCs needed automated staffing systems when the directive was published in 2010. However, they thought Phase III—a national automated staffing system—would be forthcoming, and did not fully review whether VAMCs had alternative data capabilities to assist them in the interim. When we asked how they assessed the readiness of VAMCs for implementation of the methodology, ONS officials told us that they did not do this as well as they should have for Phase I implementation in inpatient units, despite its 2009 Phase I pilot evaluation to better understand the potential capabilities and weaknesses of VAMCs. According to ONS, it still has not conducted such an assessment of all VAMCs even though it has moved forward with planning the national rollout of Phase II in operating room, emergency department, and spinal cord injury units. ONS, however, has assessed some of the available resources of the sites that have participated in the pilots for Phase II in spinal cord injury units. For example, ONS officials told us that they asked these participating sites questions about their access to data and nurse turnover within the pilot units to determine their ability to fully and successfully participate in the pilot. According to ONS officials, all sites reported that they were able to fully participate in the pilot. By not comprehensively assessing the VAMCs’ technical support and resources to determine if they were prepared to implement the methodology, VHA had no assurance that the VAMCs would be successful. Plan for Monitoring Compliance. ONS did not develop a plan for monitoring VAMCs to ensure they were in compliance with the implementation and ongoing administration of Phase I of the methodology. Under federal internal control standards, plans should be designed to ensure that ongoing monitoring occurs in the course of normal program operations, and managers should identify performance gaps in compliance with program policies and procedures. ONS reported implementing two mechanisms for obtaining information from VAMCs—a 2013 questionnaire sent to all VAMCs and monthly methodology conference calls with VAMCs—but neither was an adequate mechanism for comprehensively assessing the compliance of each VAMC. The questionnaire, sent nearly 2 years after the deadline for implementation of Phase I of the methodology, asked VAMCs to report their status of staffing plan development, but because of lack of clarity in the questions asked, inconsistency in medical center responses, and lack of validation of the self-reported responses, it was not reliable for determining the extent to which VAMCs had developed staffing plans. ONS officials reported that they have no plans to survey VAMCs again on their status of developing staffing plans. Furthermore, the monthly methodology conference calls that started when the directive was published in 2010 did not provide an adequate mechanism for monitoring compliance because they too relied on VAMCs to self-report problems. A VAMC official told us that participants were reluctant to raise problems, such as not developing staffing plans on time, during these monthly calls. In addition, the directive requires VAMCs to evaluate their staffing plans for Phase I annually, or more frequently if needed, but ONS officials told us that they did not have a systematic plan for monitoring compliance with this evaluation beyond the 2013 questionnaire and the monthly methodology conference calls. Moving forward, ONS officials said they plan to review whether all VAMCs implemented both the unit and facility expert panels, but, as of August 2014, had no detailed plan or timeline for conducting this review or for monitoring VAMCs’ ongoing evaluation of their staffing plans. The lack of a plan for monitoring VAMCs’ compliance with the implementation and ongoing administration of the methodology hinders VHA from being able to ensure that all VAMCs are staffing their nurses using the same, standardized methodology. Evaluation. There have been limited evaluations of the methodology, and one of these evaluations has been significantly delayed. Under federal internal control standards, measuring performance allows organizations to track the progress they are making towards program goals and objectives, and provides managers important information on which to make management decisions and resolve any problems or program weaknesses. Evaluation of Phase I pilot (conducted in September 2009)—ONS identified VAMC challenges with implementing the methodology— such as difficulties accessing data, and staff nurses having an overall lack of knowledge of the methodology process. The evaluation contained recommendations, such as developing a training guidebook and providing guidelines on the role of the expert panels, to improve the methodology process. According to ONS, most of the recommendations from this 2009 evaluation have been addressed; however, we found that weaknesses identified in the 2009 evaluation still existed for all of the seven VAMCs included in our review. Evaluations of Phase I national implementation and training (began early 2014, preliminary results were expected August 2014). Similarly, ONS did not begin an evaluation of the national implementation of the methodology until January 2014, more than 2 years after VAMCs were required to have implemented it, and, as of August 2014, had still not been completed. According to ONS officials, the Phase I national evaluation was to review VAMCs’ experiences during implementation, including a review of the training provided to VAMCs during that phase. The lengthy delay in the evaluation of Phase I was potentially problematic because the ongoing difficulties that VAMCs have experienced during implementation may have been avoided or resolved more quickly if the evaluation results had been available and corrective actions put into place. VAMC staff we interviewed told us they have been struggling with components of the methodology since the directive was issued. For example, some VAMC staff expressed difficulty completing and understanding the data analysis process for calculating NHPPD. An earlier evaluation of the methodology could have helped identify this problem, as well as potential solutions to address it. Furthermore, the delay limited ONS’s time to apply lessons learned from Phase I evaluations to the implementation of Phase II, portions of which are already nearly complete. Phase II pilot training evaluation (began in early 2014 with results expected November 2014)—ONS is conducting an evaluation of the training that was provided to the VAMCs involved in the Phase II pilots in operating room, emergency department, and spinal cord injury units to determine if the training provided to these units needs to be changed in preparation for the national rollout. ONS officials told us that they have completed the operating room pilot; the national rollout of the methodology in operating room units in all VAMCs began in February 2014 and is expected to be completed by October 1, 2014. ONS officials said that it has completed the pilot for the emergency department units, but has not completed the pilot for spinal cord injury units; ONS has not scheduled deadlines for their national implementation. VHA’s delays in completing evaluations of the methodology limit its ability to identify and resolve VAMC implementation and administration problems, and thus help to ensure successful rollouts of subsequent phases of the methodology. Timeliness of Implementation and Communication. The long timeline for implementing the pilots and national rollouts of Phases I and II, as well as evaluating Phase I of the staffing methodology—more than 7 years— and for communicating methodology-related information to VAMCs may have hindered the ability of VAMCs to develop their staffing plans and to execute the initiatives contained in those plans. Under federal internal control standards, timeliness in the development of a program or implementation of a policy is needed to maintain relevance and value in managing operations and making decisions. When information regarding a policy or program is not provided in a timely manner, there can be a loss of stakeholder support, which can affect how stakeholders make decisions. For example, staff from some VAMCs involved in the Phase II pilot stated that they believed the data and reports generated from the methodology were only a paper exercise because they had not gotten any feedback from ONS on next steps. ONS officials told us they have communicated information on the Phase II pilot, such as the status of the pilot and feedback obtained from the training sessions, through their monthly conference calls with VAMCs; however, based on our interviews, this information did not reach many staff at the VAMCs in our review that participated in the Phase II pilot. Furthermore, ONS officials have not adequately communicated to VAMCs the status of Phase III of the methodology—development of a national automated staffing system. According to the directive, a national automated staffing system was to be developed to support VAMCs in the implementation of the methodology. Because this automated staffing system has yet to be developed as per the directive, officials from two VAMCs told us they bought their own systems, which helped to effectively administer the methodology. ONS officials told us at the time the directive was published in 2010, Phase III implementation was an aspirational goal. ONS officials said they had expected VHA data system teams to begin the process of developing a national automated system; however, it was not made a department goal, and is not currently on the list of projects under consideration for funding. Having a variety of staffing systems, and thus inconsistent data variables across VAMCs, inhibits ONS’s ability to adequately evaluate the effectiveness of the staffing methodology. If an automated staffing system is eventually developed under Phase III, VAMCs likely will have to dismantle the staffing systems they have created and restructure their data analysis processes, which likely will be time-consuming and costly. VHA’s long timelines for the implementation and communication of methodology-related information put stakeholder support of the methodology at risk and increase the potential for duplication of efforts. Organizational Accountability. VHA did not define areas of responsibility or establish the appropriate line of reporting within the framework of VA’s management structure for the ongoing administration and oversight of the methodology. Under federal internal control standards, an agency’s organizational structure should provide management with a framework for planning, directing, and controlling operations to achieve agency objectives; a good internal control environment requires that the agency clearly defines key areas of authority and responsibility. VHA does not require VAMCs to submit any information or reports on the implementation and ongoing administration of the methodology to ONS or the VISNs. Such information, if it were shared, may have been used to inform ONS of any systematic problems that necessitate changes to help ensure the continued viability of its methodology, as well as identify any best practices that have been implemented by VAMCs across the country. ONS officials told us that they did not require the VAMCs to submit any such documentation to ONS, because they made a conscious decision to not “micro-manage” the local process of nurse staffing. Furthermore, VHA has not sufficiently utilized the VISN-level management structure in the implementation or ongoing administration of the methodology. While the methodology directive described a role for the VISNs, that role was limited to ensuring that resources are available to VAMCs as they try to staff their units; the directive did not mention a role in the implementation or ongoing administration of the actual methodology. As a result, VISNs have not been consistently aware of problems experienced by VAMCs in their region, and have not provided support or education. In our interviews with VISN officials representing each of the seven VAMCs in our review, we found that three of the VISNs were not substantively involved in the implementation and ongoing administration of the methodology. According to ONS, in many VISNs, discussions of staffing methodology implementation were minimal, and rather than VISN leadership, the nurse executives, in addition to their responsibilities within their individual VAMCs, had the responsibility of disseminating staffing methodology-related information to the VAMCs within the VISN. Staff from three VISNs that were more substantially involved in the implementation of the methodology provided oversight for the nurse staffing methodology and acted as liaisons for VAMC nurse executives for network-level issues. One VISN official we interviewed was developing oversight mechanisms for VAMCs in the region, including a requirement for nurse executives to submit a quarterly staffing report. According to the official, having such a reporting requirement at the VISN level would give the right amount of emphasis to the process and provide support to nurse executives implementing the methodology in the VAMCs. The quarterly report could also help inform VISN officials about issues with the methodology. This official was developing these mechanisms independently of ONS, but they could be considered potential best practices to be shared across all VISNs. ONS officials told us they thought ideas or problems across VAMCs related to the methodology would be shared through the VAMC nurse executives. They also hoped that VISN leadership would be interested in the methodology and, as a result, schedule VISN-level briefings to aid in its implementation. VHA, however, did not specify either of these roles in the directive or take steps to ensure that they were occurring. Moving forward, ONS officials said they are considering developing a VISN-level staff position that would specifically focus on educating VAMCs within the region about the methodology, and assisting them with implementing it. Without clearly defined roles and responsibilities within VA’s organizational structure, VHA’s ability to improve its oversight of the implementation and administration of the staffing methodology and provide VAMCs with additional resources to assist with problems is compromised. As the number of veterans requiring care in VAMCs and the complexity of services needed by many of these veterans increase, the need for an adequate and qualified nurse workforce is increasingly critical. Although VHA’s nurse staffing methodology was intended to provide a nationally standardized methodology for determining and ensuring adequate and qualified nurse staffing at VAMCs, its ability to do so across all 151 VAMCs is not likely to be realized unless existing weaknesses are addressed. Although some improvements in nurse staffing were reported with the implementation of the staffing methodology, the seven VAMCs in our review experienced problems developing and executing the related staffing plans, including problems pertaining to data resources, training, and communication. Many of these problems persist as the seven VAMCs continue to administer the methodology. We also found that VHA’s oversight of the staffing methodology is limited and in many cases lacks sufficient internal controls, which could diminish VHA’s ability to ensure an adequate and qualified nurse workforce. In particular, VHA has not adequately assessed the needs or preparedness of VAMCs to effectively implement the methodology, does not have a formal mechanism to ensure VAMCs’ ongoing compliance with the methodology, has not clearly defined a role in oversight for VISNs, and does not regularly communicate with VAMCs or VISNs to cull and share best practices system-wide. Furthermore, delays in VHA’s evaluations of early phases of the staffing methodology have made them too late to be useful in designing future phases or helping VAMCs with implementation. Because the implementation and administration of the nurse staffing methodology is ongoing, it is critical that VHA improve its oversight to help ensure an adequate and qualified nurse workforce across all VAMCs. To help ensure adequate and qualified nurse staffing at VAMCs, we recommend that the Secretary of Veterans Affairs direct the Interim Under Secretary for Health to enhance VHA’s internal controls through the following five actions: 1. Provide support to all VAMCs to meet the objectives of the VHA a. training that more clearly aligns with the needs of VAMC staff and b. a systematic process for collecting and disseminating staffing 2. Conduct an environmental assessment of all VAMCs, including an assessment of their data analysis needs, to determine their preparedness to implement the remaining phases of the methodology, and use that information to help guide and provide the necessary support for the implementation of the remaining phases and for the ongoing administration of the methodology; 3. Develop and implement a documented process to assess VAMCs’ ongoing compliance with the staffing methodology, including assessing VAMCs’ execution of staffing plans and more clearly defining the role and responsibilities of all organizational components, including VISNs, in the oversight and administration of the methodology; 4. Complete evaluations of Phase I and Phase II and make any necessary changes to policies and procedures before national implementation of Phase II in all VAMCs; and 5. Improve the timeliness and regularity of communication with VAMCs, including unit-level staff, regarding the status of the various phases of the methodology. We provided a draft of this report to VA for its review and comment. VA provided written comments, which are reprinted in appendix I. In its written comments, VA generally agreed with our conclusions and concurred with all five of the report’s recommendations. To address the recommendations, VA indicated that VHA will take a number of actions, such as developing a written document specifying its process for assessing ongoing compliance with the staffing methodology and improving the timeliness and regularity of communication with VAMCs through face-to-face regional training sessions. VA indicated that target completion dates for implementing these recommendations range from September 2015 through September 2016. Regarding the recommendation that VA complete evaluations of Phase I and Phase II before national implementation of Phase II in all VAMCs, VA indicated that, by September 2016, it would complete its evaluations and determine what opportunities exist to modify policies and procedures, but did not explicitly state that the evaluations would be completed before national implementation. We continue to emphasize the importance of completing the evaluations before national implementation of Phase II in all VAMCs. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or draperd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. In addition to the contact named above, Janina Austin, Assistant Director; Jennie Apter; Kathryn Black; Jacquelyn Hamilton; Kelli Jones; Vikki L. Porter; and Karin Wallestad made key contributions to this report.

Summary: GAO and others have raised prior concerns about the adequacy and qualifications of VHA's nurse staffing. In part to address these concerns, VHA issued a directive in 2010 requiring all VAMCs to implement a standardized methodology for determining an adequate and qualified nurse workforce, which includes developing and executing nurse staffing plans. It also requires VAMCs to use the methodology on an ongoing basis to evaluate staffing plans. GAO was asked to provide information on nurse staffing at VAMCs. This report reviews the extent to which (1) VAMCs have implemented VHA's nurse staffing methodology, and (2) VHA oversees VAMCs' implementation and ongoing administration of the methodology. GAO reviewed documents and interviewed officials from VHA, seven VAMCs selected to ensure variation in factors such as geographic location, and regional offices for these VAMCs. GAO used federal internal control standards to evaluate VHA's oversight. GAO also interviewed representatives of veterans service organizations, nursing organizations, and unions. The seven Department of Veterans Affairs medical centers (VAMC) in GAO's review implemented the Veterans Health Administration's (VHA) nurse staffing methodology, experienced problems developing and executing the related nurse staffing plans, and some reported improvements in nurse staffing. Specifically, GAO found that each of the seven VAMCs had developed a facility-wide staffing plan—which outlines initiatives needed to ensure appropriate unit-level nurse staffing and skill mix—and taken steps to execute it. However, VAMCs experienced problems—such as lack of data resources and difficulties with training—in both the development and execution of their staffing plans. Some VAMC staff reported improvements in the adequacy and qualifications of their units' nursing staff when nurse staffing plan initiatives were executed. For example, at two VAMCs where the number of nurses was increased or where support services for nurses were put in place, such as a designated group of staff to assist in transporting patients to and from appointments off the unit, unit staff said the adequacy of the nursing staff had improved. However, some VAMC unit staff reported that unit nurse staffing continued to be inadequate and that nurse unit assignments and job duties were not always appropriate for their qualifications. VHA's oversight is limited for ensuring its nurse staffing methodology is implemented and administered appropriately. GAO found the following internal controls were limited in VHA's oversight process: Environmental assessment. VHA did not comprehensively assess each VAMC to ensure preparedness for implementing the methodology, including having the necessary technical support and resources, prior to the issuance of the directive requiring each VAMC to implement the methodology. Monitoring compliance. VHA does not have a plan for monitoring VAMCs to ensure compliance with the implementation and ongoing administration of the methodology. Evaluation. VHA has conducted limited evaluations of the methodology, and at least one of these evaluations has been significantly delayed. Timeliness of communication. VHA's protracted timeline for communicating methodology-related information may have hindered the ability of VAMCs to appropriately develop their staffing plans and to execute the initiatives contained in those plans. Organizational accountability. VHA did not define areas of responsibility or establish the appropriate line of reporting within VA's management structure for oversight of the implementation and ongoing administration of the methodology. Without these internal controls in place, VHA cannot ensure its methodology meets department goals, such as establishing a standardized methodology for determining an adequate and qualified nurse workforce at VAMCs, and ultimately, having nurse staffing that is adequate to meet veterans' growing and increasingly complex health care needs. GAO recommends VA: (1) assess VAMCs' ability to implement the methodology, (2) monitor VAMCs' ongoing compliance with the methodology, (3) complete timely evaluations, (4) improve the timeliness of communication with VAMCs, and (5) define areas of responsibility and reporting within VA's management structure. VA concurred with the recommendations.