VA IT Management: Organization Is Largely Centralized; Additional Actions Could Improve Human Capital Practices and Systems Development Processes
Published: 2016-08-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: VA relies extensively on IT to deliver services to millions of our nation's veterans. VA reported spending approximately $3.9 billion in 2015 and received appropriations of approximately $4.1 billion in 2016 to improve and maintain its IT resources. Even as the department has engaged in various attempts to improve its IT management capabilities, GAO has issued numerous reports that highlighted challenges in its efforts.
This study was to determine (1) how VA is organized to manage and perform key IT-related functions and the extent to which it has centralized the management of IT resources, (2) the extent to which VA has implemented effective IT human capital management, and (3) the extent to which VA has established key processes to effectively manage major system development and acquisition efforts. To conduct its study, GAO reviewed VA policies, procedures, and other documentation and compared the department's processes to best practices for human capital management and IT systems development and acquisition. GAO also interviewed VA officials.
section_title: What GAO Found
paragraphs: The Department of Veterans Affairs (VA) performs key information technology (IT)-related functions, such as leadership, strategic planning, systems development and acquisition, and systems operations and maintenance, largely through its centralized Office of Information & Technology (OI&T), led by the Chief Information Officer (CIO). VA's two IT governance boards are intended to play a role in other key functions, such as investment management. Nevertheless, the department faced challenges in effectively managing IT, including (1) preventing IT activities from occurring outside the control of OI&T, (2) maintaining collaboration between OI&T and business units, and (3) delivering efficient and cost-effective IT capabilities. In response to these and other challenges, the CIO initiated an effort in January 2016 to transform OI&T into a more veteran-focused organization that emphasized transparency, accountability, innovation, and teamwork. The transformation strategy calls for OI&T to stabilize and streamline core processes and platforms, mitigate weaknesses from information security and GAO assessments, and improve outcomes by institutionalizing a new set of IT management capabilities. The CIO intends to complete the transformation by the first quarter of 2017.
Key to an agency's success in effectively managing its IT systems is sustaining a workforce with the necessary knowledge, skills, and abilities to execute a range of management functions that support its mission and goals. VA took steps to implement effective IT human capital practices by documenting an IT human capital strategic plan and initiating an update based on changed priorities, analyzing workforce data, identifying skill gaps for the current year, and implementing an IT training program. However, OI&T had not consistently implemented all of these practices. Specifically, the office had not (1) tracked and reviewed historical and projected leadership retirements and (2) identified skills and competencies needed beyond the current year. Without annually tracking and reviewing data related to leadership retirements or identifying skills needed in future years, OI&T faces a risk of being unprepared to effectively respond to vacancies in key positions and not having the capabilities to deliver IT support that can contribute to improved services for veterans.
Key to successful development and acquisition of IT services is establishing documented processes that reflect best practices. Although there were gaps in some areas, VA's processes generally included best practices for project validation, project planning, requirements management, risk management, project monitoring and control, and process and product quality assurance. In addition, processes for developing and maintaining a project schedule had not fully addressed the majority of the associated best practices. Ensuring that these processes address all key practices will assist the department in effectively managing its IT system development and acquisitions.
section_title: What GAO Recommends
paragraphs: GAO is recommending that VA take two actions to assist the department in sustaining a workforce with the necessary knowledge, skills, and abilities to execute its mission and goals, as well as six actions to assist the department in developing comprehensive processes that reflect systems development best practices. VA generally agreed with GAO's conclusions and concurred with GAO's eight recommendations.

section_title: Letter
paragraphs: The Department of Veterans Affairs (VA) provides services and benefits,  such as medical care and compensation, education, and burial benefits,  to millions of veterans and their families each year. To accomplish this,  the department relies extensively on information technology (IT) systems.
However, we have previously reported on challenges that VA has  experienced in managing its IT projects and initiatives, including cost  overruns, schedule slippages, and performance problems. To address  these challenges, the department has, over many years, engaged in  initiatives aimed at realigning its IT program to provide greater authority  and accountability for its resources. In doing so, its goals were to, among  other things, centralize IT management under a department-level chief  information officer (CIO) and standardize budgets, operations, and the  development of systems using new management processes based on  industry best practices.
Recognizing the important role that IT plays in supporting VA’s mission,  you asked us to examine the department’s current IT organization.  Specifically, our objectives were to determine (1) how VA is organized to  manage and perform key IT-related functions and the extent to which it  has centralized the management of IT resources, (2) the extent to which  VA has implemented effective IT human capital management, and (3) the  extent to which VA has established key processes to effectively manage  major system development and acquisition efforts.
To address the first objective, we obtained and reviewed the department’s  documentation identifying and describing its IT organizational structure  and functions. These included the IT strategic plan, governance plan,  organization chart, and organizational descriptions. In addition, we  reviewed memoranda; testimony from VA officials, such as current and  previous secretaries and CIOs; and assessment reports that described  the department’s efforts to centralize the management of its IT resources.  We also interviewed responsible program officials regarding the  organizational structure and performance of key functions.
To address the second objective, we reviewed information describing the  department’s management of its IT human capital, such as human capital  strategic planning documentation, skills gap analyses, and human capital  performance measures. We assessed the department’s actions against  selected best practices that we and the Office of Personnel Management  (OPM) have identified, and against practices required by VA’s Office of  Human Resources and Administration. These included selected practices  for effective strategic human capital planning, workforce planning, and  strategic training. The practices are identified in our Key Principles for  Effective Strategic Workforce Planning; OPM’s final regulations to  implement certain provisions of the Chief Human Capital Officers Act of  2002; and VA’s workforce and succession planning directive. We also  interviewed officials responsible for the department’s IT human capital  management.
For the third objective, we reviewed policies, procedures, and supporting  documentation describing the department’s key processes for managing  major system development and acquisition efforts. We assessed the  processes against selected best practices that the Software Engineering  Institute (SEI) and we have identified. These included recognized  practices for project planning, requirements management, risk  management, project monitoring and control, project validation, quality  assurance, and project schedules. These practices are identified in SEI’s  Capability Maturity Model® Integration (CMMI®) for Development,  Version 1.3; SEI’s CMMI® for Acquisition, Version 1.3; and our Schedule  Assessment Guide. Further, we reviewed system development lifecycle  policies, procedures, and other supporting documentation, such as  templates for required project artifacts within the department’s ProPath  system and Project Management Accountability System (PMAS). A full  description of our objectives, scope, and methodology can be found in  appendix I.
We conducted this performance audit from February 2015 to August 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: VA’s mission is to promote the health, welfare, and dignity of all veterans  in recognition of their service to the nation by ensuring they receive  medical care, benefits, social support, and lasting memorials. It is the  second largest federal department and, in addition to its central office  located in Washington, D.C., has field offices throughout the United  States, as well as the U.S. territories and the Philippines.
The department’s three major components—the Veterans Benefits  Administration (VBA), the Veterans Health Administration (VHA), and the  National Cemetery Administration (NCA)—are primarily responsible for  carrying out its mission. More specifically, VBA provides a variety of  benefits to veterans and their families, including disability compensation,  educational opportunities, assistance with home ownership, and life  insurance. VHA provides health care services, including primary care and  specialized care, and it performs research and development to improve  veterans’ needs. Lastly, NCA provides burial and memorial benefits to  veterans and their families.
Collectively, the three components rely on approximately 340,000  employees to provide services and benefits. These employees work in  167 VA medical centers, approximately 1,200 community-based  outpatient clinics, 300 veterans centers, 56 regional offices, and 131  national and 90 state or tribal cemeteries situated throughout the nation.
subsections: section_title: VA Relies Extensively on Information Technology
paragraphs: VA operates and maintains an IT infrastructure that is intended to provide  the backbone necessary to meet the day-to-day operational needs of its  medical centers, veteran-facing systems, benefits delivery systems,  memorial services, and all other IT systems supporting the department’s  mission. The infrastructure is to provide for data storage, transmission,  and communications requirements necessary to ensure delivery of  reliable, available, and responsive support to all VA staff offices and  administration customers, as well as veterans.
In this regard, the department operates approximately 240 information  systems, manages 314,000 desktop computers and 30,000 laptops, and  administers nearly 460,000 network user accounts for employees and  contractors to facilitate providing benefits and health care to veterans.  These systems are used for the determination of benefits, benefits claims  processing, patient admission to hospitals and clinics, patient care  through telehealth, and access to health records, among other services.
For example, VBA relies on its disability benefits claims processing  system—the Veterans Benefits Management System—to collect and  store information such as military service records, medical examinations,  and treatment records from VA, the Department of Defense, and private  medical service providers. Information technology is widely used and  critically important to supporting the department in delivering health care  to veterans. VHA’s systems provide capabilities to establish and maintain  electronic health records that health care providers and other clinical staff  use to view patient information in inpatient, outpatient, and long-term care  settings. Specifically, the Veterans Health Information Systems and  Technology Architecture, known as VistA, consists of many computer  applications and modules that collect, among other things, information  about a veteran’s demographics, allergies, procedures, immunizations,  and medical diagnoses.
In 2014, VA issued its 6-year strategic plan, which emphasizes the  agency’s goal of increasing veterans’ access to benefits and services,  eliminating the disability claims backlog, and ending veteran  homelessness. According to the plan, the department intends to improve  access to benefits and services through the use of improved technology  to provide veterans with access to more effective care management. The  plan also calls for VA to eliminate the disability claims backlog by fully  implementing an electronic claims process that is intended to reduce  processing time and increase accuracy. Further, the department has an  initiative under way that provides services, such as health care, housing  assistance, and job training, to end veteran homelessness. To this end,  VA is working with other agencies, such as the Department of Health and  Human Services, to implement more coordinated data entry systems to  streamline and facilitate access to appropriate housing and services.
VA reported spending approximately $3.9 billion to improve and maintain  its IT resources in fiscal year 2015. Specifically, the department reported  spending approximately $548 million on new systems development  efforts, approximately $2.3 billion on maintaining existing systems, and  approximately $1 billion on payroll and administration.
For fiscal year 2016, the department received appropriations of  approximately $4.1 billion for IT. The department had requested  approximately $505 million for new systems development efforts,  approximately $2.5 billion for maintaining existing systems, and  approximately $1.1 billion for payroll and administration. In addition, in its  2016 budget submission the department requested appropriations for the  following purposes, among others:  improving veteran access to benefits and services ($378.1 million);  reducing the time it takes to process disability claims and increase  accuracy ($295.1 million); expanding information security ($180.3 million);  implementing the Veterans Access, Choice, and Accountability Act of  2014 (the Veterans Choice Act) ($440.6 million); maintaining the IT infrastructure ($1.828 billion); and  improving VistA ($182.6 million).
Further, for fiscal year 2017, the department’s budget request included  nearly $4.3 billion for IT. The department requested approximately $471  million for new systems development efforts, approximately $2.5 billion for  maintaining existing systems, and approximately $1.3 billion for payroll  and administration. In addition, in its 2017 budget submission, the  department requested appropriations to make improvements in a number  of areas, including the following:  veterans’ access to health care, to include enhancing health care- related systems, standardizing immunization data, and expanding  telehealth services ($186.7 million);  veterans’ access to benefits by modernizing systems supporting  benefits delivery, such as the Veterans Benefits Management System  and the Veterans Services Network ($236.3 million);  veterans’ experiences with VA by focusing on integrated service  delivery and streamlined identification processes ($171.3 million);
VA employees’ experiences by enhancing internal IT systems ($13  information security, including implementing strong authentication,  ensuring repeatable processes and procedures, adopting modern  technology, and enhancing the detection of cyber vulnerabilities and  protection from cyber threats ($370.1 million).
section_title: VA Has Faced Long- Standing Challenges in IT Management
paragraphs: Even as the department has engaged in various attempts to improve its  IT management capabilities, we have issued numerous reports that  highlighted challenges facing its efforts. In February 2014, we reported  that about 2 years after taking actions toward developing a single  interoperable electronic health record system with the Department of  Defense (DOD), VA and DOD announced that they would instead work  toward modernizing their separate existing health information systems  and work to ensure those systems could share information. However,  VA proceeded with this plan without developing a cost and schedule  analysis to support the assertion that pursuing a separate modernized  system while enabling interoperability with DOD’s system would be less  expensive and could be achieved faster than developing a single system,  which is contrary to best practice. We recommended that VA work with  DOD to develop a cost and schedule estimate for their modernization  approach, as well as their joint effort, and compare the estimates to  determine which approach would cost less and take less time. VA  concurred with our recommendation, but had not completed actions to  address it.
In addition, we reported in May 2010 that after spending an estimated  $127 million over 9 years on its outpatient scheduling system project, VA  had not implemented any of the planned system’s capabilities and was  essentially starting over by beginning a new initiative to build or purchase  another scheduling system. We also noted that VA had not developed a  project plan or schedule for the new initiative, stating that it intended to do  so after determining whether to build or purchase the new application. We  recommended that the department take six actions to improve key  systems development and acquisition processes essential to the second  outpatient scheduling system effort. The department generally concurred  with our recommendations, but as of May 2016, had not completed  actions to implement four of the six recommendations.
section_title: Key IT-Related Functions Provide a Foundation for IT Management
paragraphs: Effectively managing IT needs depends on federal departments and  agencies, including VA, having key functions in place. Toward this end,  we have identified and reported on a set of essential and complementary  functions that serve as a sound foundation for IT management. These  include the following:
Leadership: Effective leadership, such as that of a CIO, can drive  change, provide oversight, and ensure accountability for results.  Congress has also recognized the importance of having a strong  agency CIO. For example, as part of the Clinger-Cohen Act, Congress  required executive branch agencies to establish the position of  agency CIO. The act also gave these officials responsibility and  accountability for IT investments, including IT acquisitions, monitoring  the performance of IT programs, and advising the agency head on  whether to continue, modify, or terminate such programs. More  recently, in December 2014, Congress passed federal information  technology acquisition reform legislation (commonly referred to as  FITARA), which strengthened the role that agency CIOs are to play in  managing IT. For instance, the law requires the head of covered  agencies to ensure that the CIO has a significant role in the decision  process for IT budgeting, as well as the management, governance,  and oversight processes related to IT.
Strategic planning: Strategic planning defines what an organization  seeks to accomplish and identifies the strategies it will use to achieve  desired results. A defined strategic planning process allows an  agency to clearly articulate its strategic direction and to establish  linkages among planning elements such as goals, objectives, and  strategies. A well-defined IT strategic planning process helps ensure  that an agency’s IT goals are aligned with its strategic goals. Also,  as part of its strategic planning efforts, an organization should develop  an enterprise architecture, which is an important tool to help guide the  organization toward achieving the goals and objectives in its IT  strategic plan. In addition, the organization should implement human  capital management practices to sustain a workforce with the skills  necessary to execute its strategic plan.
Systems development and acquisition: Agencies should follow  disciplined processes for developing or acquiring IT systems. These  include defining the requirements that address the needs of the  system users, managing project risk to identify potential problems  before they occur, reliably estimating cost to help managers evaluate  affordability and performance against a project’s plans, and  developing an integrated and reliable master schedule that defines  when and how long work will occur and how each activity is related to  the others, among others. Best practices in these areas have been  identified by organizations such as SEI and GAO.
Quality assurance: Effective quality assurance supports the delivery  of high-quality products by providing staff and management with  appropriate visibility into, and feedback on, processes and associated  work products throughout the life of a systems development or  acquisition project. Quality assurance can also help ensure that  planned processes are implemented as intended.
Systems operations and maintenance: Given the size and  magnitude of the investments that executive branch agencies make in  this area (nearly 60 percent of VA’s reported IT spending in 2015), it is  important that operations and maintenance be managed effectively to  ensure that existing investments (1) continue to meet agency needs,  (2) deliver value, and (3) do not duplicate or overlap with other  investments. To accomplish this, agencies are required by the Office  of Management and Budget to perform annual operational analyses of  these investments, which are intended to serve as periodic  examinations of the investments’ performance against, among other  things, established costs, schedules, and performance goals.
Service management: Agencies should develop and implement a  process for ensuring that IT services, such as server management  and desktop support, are aligned with and actively support the  business needs of the organization. The Information Technology  Infrastructure Library identifies key practices for successful service  management. These include developing a service catalog that  identifies all IT services delivered by the service provider, as well as  establishing service-level agreements between the IT service provider  and its customer on the expected service-level targets.
IT investment management: IT projects can significantly improve an  organization’s performance, but they can also become costly, risky,  and unproductive. Agencies can maximize the value of these  investments and minimize the risks of acquisitions by having an  effective and efficient IT investment management and governance  process, as described in GAO’s guide to effective IT investment  management. Emphasizing the importance of investment  management, the Clinger-Cohen Act requires executive branch  agencies to establish a process for selecting, managing, and  evaluating IT investments in order to maximize the value and assess  and manage the risks of the acquisitions.
Information security and privacy: Effective security for federal IT  systems and data is essential to prevent data tampering, disruptions  in critical operations, fraud, and inappropriate disclosure of sensitive  information, including personal information entrusted to the  government by members of the American public. Recognizing the  importance of information security and privacy, Congress enacted the  Federal Information Security Modernization Act of 2014 (FISMA  2014), which requires executive branch agencies to develop,  document, and implement an agency-wide information security  program.  Additionally, in order to help agencies develop such a  program, the National Institute of Standards and Technology has  developed guidance for information security and privacy.
section_title: VA Manages and Performs Key IT- Related Functions through a Largely Centralized Organization, Which Is Taking Steps to Address Challenges
paragraphs: Since 2007, VA has been operating a centralized IT organization in which  most key functions intended for effective management of IT are  performed by the department’s Office of Information & Technology (OI&T)  and led by the department-level CIO. Two department-level IT  governance boards also have responsibility to assist in performing certain  functions. However, the department has faced challenges to effectively  managing IT using a centralized approach. These include ensuring that  (1) all projects are managed and controlled by OI&T, (2) effective  communication occurs between OI&T and the VA business units, and  (3) new IT capabilities are delivered efficiently and cost-effectively. To  address these and other challenges, the CIO recently announced a  transformation initiative aimed at improving OI&T’s accountability for, and  focus on, achieving a more transparent and veteran-centric organization.
subsections: section_title: VA Has Largely Centralized Its Management of IT-Related Functions within OI&T
paragraphs: OI&T has responsibility for managing the majority of VA’s IT-related  functions, under a largely centralized structure that the department  attempted for nearly two decades to establish. The department’s efforts  toward achieving a centralized management structure were strengthened  when the Military Quality of Life and Veterans Affairs Appropriations Act,  2006 established a new appropriation account for the department’s IT  systems. In addition, in order to obligate or expend amounts for IT  systems development, modernization, and enhancement, the  Consolidated Appropriations Act of 2016 required that the Secretary or  CIO submit to the Committees on Appropriations of both Houses of  Congress a certification of the amounts to be obligated and expended for  each development project.
Subsequent to the 2006 appropriations act, the Secretary of Veterans  Affairs assigned control over the IT appropriation to a department-level  CIO. In addition, the Secretary approved a centralized organization  structure for IT-related functions in February 2007. By April 2007, all of  the department’s personnel that had been involved in IT operations,  maintenance, and development, with the exception of those in the Office  of Inspector General, were permanently assigned to OI&T. (See appendix  II for a summary of key events in the history of VA’s IT centralization  efforts.)
The CIO serves as the head of OI&T and, accordingly, is responsible for  providing effective leadership over the department’s IT activities. The CIO  reports directly to the Office of the Secretary of Veterans Affairs through  the Deputy Secretary and advises the Secretary regarding the execution  of the IT appropriation. In addition, the CIO is expected to serve as the  principle advisor to top management officials, such as the Under  Secretaries of each of the three administrations, on matters relating to  information and technology management in the department. This official  also is tasked with reviewing and approving IT investments, as well as  overseeing the performance of IT programs and evaluating them to  determine whether to continue, modify, or terminate a program or project.
Under the CIO’s leadership, seven organizational units within OI&T had  responsibility for performing and managing the other specific IT-related  functions through March 2016.
Architecture, Strategy, and Design: This office was responsible for  the department’s IT strategic planning. Specifically, it was tasked with  providing a framework of strategies and processes to ensure IT  programs and projects are designed and executed to satisfy current  and future department needs.
Further, the office was tasked with performing other functions, such as  the development and maintenance of an integrated technical,  business, systems, and data architecture; development of systems  design, engineering, and integration standards; and the examination  of existing IT requirements and solutions for efficiency and potential  redundancy.
Product Development: Systems development was managed by this  office. Among other things, the office was responsible for developing  and testing software products, including new products and upgrades  to existing legacy systems, based on defined requirements from its  customers (which are the three administrations and other staff  offices).
In addition to systems development, the office had responsibility for  monitoring the progress of all IT projects and reporting that progress  to the department’s CIO and other senior leadership. This office was  also tasked with developing and maintaining policies and guidance for  implementing, and ensuring that responsible staff are trained in using,  the department’s project management and accountability system,  referred to as PMAS. (This management tool is discussed in greater  detail later in the report.)
IT Resource Management: Acquisition processes and strategies  were facilitated by the IT Resource Management office. Toward this  end, the office was tasked with providing acquisition program  management oversight and disseminating acquisition policy and  procedures, among other things.
In addition, the IT Budget and Finance office within IT Resource  Management was tasked with planning, executing, and overseeing  the department’s technology budget, as well as ensuring that the  appropriation funds are used appropriately. According to a director  within the IT Budget and Finance office, this office was responsible for  performing these functions for the fiscal year 2015 and 2016 IT  budgets. The director also stated that the CIO, through this office,  oversaw 100 percent of the IT appropriation. Further, in fiscal year  2016 the IT Budget and Finance office was charged with creating  certification letters signed by the CIO and submitted to the  Committees on Appropriations detailing the systems development,  modernization, and enhancement projects VA had planned for the IT  appropriation. To date, the CIO has certified projects for fiscal year  2016 in October 2015, February 2016, and April 2016.
Office of Quality, Performance, and Oversight: This office was  tasked with managing quality assurance activities. It was intended to  act as an independent party to determine if OI&T’s governance  processes are adequate and functioning as anticipated. The office  was also responsible for facilitating the reporting and understanding of  performance measures and metrics related to IT program activities  and strategic objectives. Further, it was charged with providing  independent systems integration testing and other services to help  ensure the integrity of the department’s systems and compliance with  applicable guidance, such that provided through PMAS.
In addition, the office was responsible for providing organizational  development and training for IT, to include coverage of information  security, privacy, and rules of behavior. It also was responsible for  establishing IT human capital policies and conducting workforce  planning.
Office of Service Delivery and Engineering: All operations and  maintenance activities associated with the department’s IT  environment, including those in medical centers and regional offices,  were directed by the Office of Service Delivery and Engineering. This  office also was tasked with overseeing and managing VA’s data  centers, network, and telecommunications; monitoring production for  all information systems; delivering operations services (including  deployment, maintenance, monitoring, and support) to all of the  department’s geographic locations; and engineering and designing all  of the IT system platforms and infrastructure.
Office of Information Security: The department’s information  security and privacy infrastructure was managed within this office. In  this regard, the office was responsible for, among other things,  ensuring the confidentiality, integrity, and availability of the  department’s information and information systems, as well as  cybersecurity, risk management, records management, incident  response, critical infrastructure protection, and business continuity.  The office also was charged with developing, implementing, and  overseeing the policies, procedures, training, communication, and  operations related to improving how the department and its partners  safeguard the personally identifiable information of veterans and VA  employees.
Office of Customer Advocacy: This office was in charge of service  management. In this regard, it was responsible for working with VHA,  VBA, NCA, and the department’s staff offices to ensure that OI&T  provides the services they need. The office also had responsibility for  collecting and analyzing customer satisfaction metrics, and for  responding to IT support requests at the department, business office,  and employee levels.
Figure 1 depicts the seven organizational units within OI&T that had  responsibility for performing and managing IT-related functions through  March 2016.
OI&T reported that its organizational units performed key IT-related  functions with the support of nearly 7,300 federal employees and  approximately 7,800 contractor staff. Nearly 600 of the federal employees  were located in Washington, D.C., at the VA Headquarters, while the vast  majority of the employees were assigned to other facilities, such as  medical centers and regional offices.
Table 1 identifies each OI&T organizational unit and the number of  employees and approximate number of contractors that supported VA’s  IT program.
In addition to these organizational resources, OI&T developed PMAS to  serve as a mechanism for overseeing the performance of IT programs  and ensuring accountability for results. PMAS was intended to provide  information to facilitate oversight through a dashboard that captures  project data, such as descriptions of the projects, baseline milestone  dates, actual milestone dates, contract information, and project status  information. The CIO’s responsibilities with regard to PMAS include  authorizing new projects and project increments in the system; approving  the funding needed for the projects; monitoring a project’s status via  reporting, review, and assessment; and addressing any significant issues  with the projects.
Further, according to the CIO and OI&T documented procedures,  TechStat reviews are required for every project that fails to deliver on its  committed delivery date. TechStat reviews are the mechanism that the  CIO, along with other senior leaders, used to determine whether a project  should continue, be modified, or be terminated.
section_title: Governance Boards Are to Assist the CIO with IT- Related Functions
paragraphs: In VA’s organization structure, the IT investment governance boards are  intended to play a role in strategic planning and IT investment  management, as they are responsible for providing the CIO with  recommendations on the IT investments they feel would best meet the  strategic and business objectives of the department. These two boards  are intended to provide executive oversight for IT initiative planning and  management.
IT Planning, Programming, Budgeting, and Execution Board: The  board is tasked with recommending (1) the prioritization of all IT  funding requests, including what should and should not be funded  during the year; (2) execution of the IT appropriation; and  (3) decisions as to whether to fund IT-related projects outside of the IT  appropriation. The IT Chief Financial Officer, who also serves as the  Deputy Assistant Secretary for the IT Resource Management office  within OI&T, serves as the board’s chairperson. The board includes  voting members from VA’s three administrations, as well as non- voting members from the OI&T organizational units, among others. In  March 2016, the CIO stated in congressional testimony that the roles  of this board will be assumed by a newly established Enterprise  Program Management Office (EPMO).
IT Leadership Board: This board is tasked with reviewing and  validating portfolio recommendations from the Planning,  Programming, Budgeting, and Execution Board, as well as making  decisions on any issues that cannot be resolved by that board.  According to the department’s FITARA implementation plan, the IT  Leadership Board has been tasked with approving the IT budget and  then submitting it to the VA Executive Board—the department’s  highest-level decision-making board chaired by the Secretary—for  approval. The CIO serves as the IT Leadership Board’s chairperson  and its members include the Under Secretaries for VA’s three  administrations and all VA staff offices.
section_title: VA Is Taking Action to Address Challenges to Effectively Managing IT
paragraphs: Although VA centralized its key functions in order to maintain better  control over resources, OI&T has faced challenges to fully implementing  and managing IT under a centralized organizational structure. In  particular, independent assessments of the department’s efforts in 2013  and 2015 showed that the office has had difficulty in preventing IT  activities from occurring outside the control of OI&T. It has also been  challenged in effectively collaborating with the department’s various  business units and in efficiently and cost-effectively delivering new IT  capabilities.
IT activities have occurred outside the control of OI&T. In  September 2013, the VA Inspector General reported that the  department’s Office of Acquisition, Logistics, and Construction had  used $13 million in supply funds, rather than approved IT funds, to  develop a suite of electronic contracting applications to support the  office’s procurement process. Further, the report noted that the  Office of Acquisition, Logistics, and Construction had not used the  department’s IT project management system, PMAS, to develop the  contracting system, although doing so was mandated by the  Secretary in June 2009 for all IT development projects department- wide.
According to the report, the office followed its own project  management process that was similar to PMAS. However, in doing  so, the system development effort was not subjected to OI&T’s risk  management, monitoring, oversight and control, and reporting  processes within PMAS. According to the report, officials in the Office  of Acquisition, Logistics, and Construction incorrectly believed that VA  allowed for a PMAS exception because they had used supply funds,  and not IT funds, for the development. The Inspector General  recommended that the Office of Acquisition, Logistics, and  Construction implement controls to ensure the systems development  project and all future IT development efforts fall within the control and  oversight of the PMAS process. The office concurred with the  recommendation.
More recently, in March 2015, the Inspector General reported that  VHA had improperly obligated $92.5 million and spent $73.8 million in  medical support and compliance funds, rather than IT funds, to  develop its health care claims processing system through the VHA  Financial Services Center. According to the report, the former VHA  Deputy Chief Business Officer stated that funds and development  services provided through the Financial Services Center were used in  hopes of achieving a faster delivery of the claims processing system.  The Inspector General reported that, by doing so, VHA had avoided  competing with other VA projects for IT systems appropriations. Thus,  the Inspector General recommended that the Under Secretary for  Health obtain the appropriate funding to support the development of  the claims processing system. The interim Under Secretary for Health  concurred with the recommendation and agreed to establish oversight  mechanisms and issue guidance to ensure that VHA uses the  appropriate funds.
Collaboration between OI&T and business units was not always  effective. In September 2015, challenges with collaboration between  OI&T and VA business units were highlighted by an independent  assessment of VA’s health care delivery systems and management  processes, including health IT processes. Several clinicians  interviewed for that assessment stated that collaboration between  systems developers and clinicians seemed to have disappeared after  the centralization, resulting in uncoordinated execution of health IT  strategies and limited development of new and improved capabilities  for existing IT systems. The assessment noted that although the goals  of OI&T and VHA did not conflict at the strategic level, the  organizations often did not agree on priorities for executing the  strategic plans. The assessment team recommended that VA and  VHA transform IT strategy, planning, and execution by, among other  things, establishing service-level agreements and refining the  planning and budgeting process to ensure that business needs are  effectively identified, prioritized, funded, and used to drive health IT  investments. The team recommended that VA also develop a  governance policy to ensure the strategic plans are executed well and  in a timely manner.
Similarly, staff within the Office of Finance informed us of their  dissatisfaction because OI&T controls the budget and the priorities for  systems development efforts and IT change management. These staff  stated that OI&T’s priorities do not always align with the needs of the  administrations or other staff offices, thus potentially delaying the  delivery of needed services.
PMAS limited VA’s efforts to efficiently and cost-effectively  deliver new IT capabilities. The September 2015 independent  assessment also stated that overly demanding processes for system  development, as defined by OI&T’s PMAS, impeded cost-effective  delivery of new health IT capabilities and limited VA’s ability to  measure the value of its investments. The assessment report further  stated that the PMAS process was schedule driven and risk averse,  leading many project managers to limit the amount of functionality in  each release, thereby increasing the total time for any useful  capability to be released. In addition, assessment interviewees  indicated that documentation required by PMAS consumed a  significant percentage of development time. Accordingly, the  assessment team recommended that VA establish product-focused  teams to ensure delivery of needed capabilities to users and that the  department’s system development process be modified from focusing  on documentation and schedule to focusing more on functionality  delivered.
subsections: section_title: CIO Transformation Strategy Aims to Address Challenges
paragraphs: Recognizing challenges in IT management, the CIO initiated an effort in  January 2016 to transform the focus and functions of OI&T. In doing so,  the CIO stressed the Secretary’s goal of achieving a more veteran- focused organization that places greater emphasis on transparency,  accountability, innovation, and teamwork. The CIO’s transformation  strategy calls for OI&T to focus on stabilizing and streamlining core  processes and platforms, mitigating weaknesses highlighted in  information security and GAO assessments, and improving outcomes by  institutionalizing a new set of IT management capabilities.
According to the CIO, one of the most significant challenges the  department faces is securing its assets, networks, systems, and data. To  address this challenge, in September 2015, as part of the CIO’s effort to  transform OI&T, VA documented a long-term cybersecurity strategy. This  strategy identified five goals for successfully building a comprehensive  cybersecurity capability: (1) protecting veterans’ information and VA data  by ensuring privacy concerns are addressed and ensuring secure  technology and data systems; (2) providing secure and resilient  information systems technology, business applications, publicly  accessible platforms, and shared data networks; (3) protecting VA  infrastructure and assets, including the boundary environments that  provide potential access and entry into VA; (4) enabling effective  operations by improving governance and organizational alignment at  enterprise, operational, and tactical levels (points of service interactions);  and (5) recruiting and retaining a talented cybersecurity workforce  covering multiple disciplines within cybersecurity.
As another part of this transformation, the CIO began transitioning the  oversight and accountability of IT projects from PMAS to a new project  management process called the Veteran-focused Integration Process  (VIP) in January 2016, in an effort to better streamline systems  development and speed up the delivery of new IT capabilities. According  to the VIP guide, all systems development projects that touch the VA  network or use money from the IT appropriation will be required to follow  VIP. The CIO intends for VIP to facilitate an expanded use of the Agile  system development methodology resulting in, for example, the delivery  of system functionality on a more frequent, 3-month delivery cycle,  whereas PMAS delivered functionality in 6-month cycles. In addition, VIP  is intended to streamline IT project management by reducing the number  of artifacts (i.e., documentation) that must be developed (from 58 to 8)  and the number of critical decision points (from 5 to 2). According to the  CIO’s transformation strategy, under the new VIP process, OI&T will also  be required to consider security and architecture standards early in  project planning phases, whereas PMAS required this later in the  development process. According to the CIO, the department intends to  have all IT projects managed under VIP by September 30, 2016.
In addition to implementing VIP, the CIO also intends to establish five  new functions within OI&T as part of the transformation:
Enterprise Program Management Office. This office began initial  operations in April 2016 and is intended to serve as OI&T’s portfolio  management and project tracking organization. According to the  strategy, the goals of the new organization are to align IT portfolios  with agency strategic objectives; enhance visibility and governance;  analyze and report on portfolio performance metrics; ensure the  overall health of the IT portfolio; and optimize resources for projects,  people, and timelines. EPMO includes six functional areas: (1) Intake  and Analysis of Alternatives, responsible for working with the VA  administrations and other staff offices to develop requirements to  meet the needs of veterans, provide analysis of alternative  approaches to meeting those requirements, and integrate information  security; (2) IT Portfolios, responsible for consolidating programs and  projects under five IT portfolios (Health, Benefits, Cemeteries,  Corporate, and Enterprise services); (3) Project Special Forces,  responsible for mitigating issues that put projects at risk of failure;  (4) Lean Systems Engineering, responsible for metrics gathering and  analysis, development of process tools, human resources, and  training; (5) Transition Release and Support, responsible for  managing OI&T’s integrated calendar supporting VIP; and  (6) Application Management, responsible for IT implementation  efforts, including testing, design, and data management.
The EPMO is expected to replace the Product Development office  and the Planning, Programming, Budgeting, and Execution board. As  of July 2016, VA had not provided information about whether and how  the transformation will affect the IT Leadership Board.
Account Management. The account management function, led by  three account managers, is to be responsible for managing the IT  needs of OI&T’s business partners—the VA’s major components.  Account managers are to interface directly with the VA administrations  and staff offices to understand their needs, help identify and define  the solutions to meet those needs, and represent their customers’  interests by reporting directly to the CIO. In this regard, account  managers are to submit their customers’ IT requirements to the  EPMO, ensure that their business needs are understood by OI&T,  and ensure that business solutions are designed to meet their  customers’ specifications. This function is also tasked with advocating  for their customers in the budget process. OI&T intends this new  function to address the challenge of effectively collaborating with  business units. As of May 2016, the three account managers were in  place.
Quality and Compliance. This function is to be responsible for  establishing effective policy governance and standards and ensuring  adherence to the policies and standards. In addition, the quality and  compliance function is expected to be charged with identifying,  monitoring, and measuring risks across the OI&T organization.
Data Management Organization. The organization is being  established to improve both service delivery and the veteran  experience by engaging with data stewards to ensure the accuracy  and security of the information collected by VA. The organization is  intended to institute a data governance strategy; engage with  business owners to ensure accuracy and security of collected data;  analyze data sources to form an enterprise data architecture; and  establish metrics for data efficiency, access, and value. OI&T also  intends for the organization to identify trends in the data collected on  each veteran that could improve their health care by providing  predictive care and anticipating needs.
Strategic Sourcing. This function is expected to be responsible for  establishing an approach to fulfilling the agency’s requirements with  vendors that provide solutions to those requirements, managing  vendor selection, tracking vendor performance and contract  deliverables, and sharing insights on new technologies and  capabilities to improve the workforce knowledge base.
Figure 2 depicts the OI&T organization as of April 2016, which includes  previously existing components and functions, as well as the new  functions described above. OI&T was still in the process of fully defining  the roles and responsibilities of the organizational units as of July 2016.
In addition to these new functions, the strategy calls for a transformation  of the services that OI&T provides in field offices, to include requiring  service-level agreements for all VA organizations and using those  agreements to define the support that is needed, migrating data and  applications to the cloud, and developing a strategy for consolidating data  centers. According to the CIO, the transformation strategy is expected to  be completed by the first quarter of fiscal year 2017, although the vast  majority of the plan, including establishing the five new functions, is to be  executed by the end of fiscal year 2016.
section_title: VA Generally Implemented IT Human Capital Management Best Practices but Has Not Projected Leadership Retirements or Fully Identified Skill Gaps
paragraphs: Key to an agency’s success in effectively managing its IT systems is  sustaining a workforce with the necessary knowledge, skills, and abilities  to execute a range of management functions that support the agency’s  mission and goals. Achieving such a workforce depends on effective  human capital management, which includes developing a human capital  strategic plan; analyzing the workforce; analyzing the gaps between  current skills and future needs, and developing strategies for filling the  gaps; and training and developing the workforce. Taking such steps is  consistent with activities outlined in human capital management practices  that we and the Office of Personnel Management have developed.
OI&T took steps toward implementing effective IT human capital practices  by documenting an IT human capital strategic plan and initiating an  update based on changed priorities, regularly analyzing workforce data,  identifying skill gaps for the current fiscal year, and implementing an IT  training program. However, OI&T did not implement other human capital  practices that are essential for analyzing the workforce and identifying IT  skill gaps. Specifically, the office had not (1) tracked and reviewed  historical and projected leadership retirements, and (2) identified the gaps  in future skill areas.
subsections: section_title: OI&T Developed a Human Capital Strategic Plan
paragraphs: Strategic human capital planning that is linked to an agency’s strategic  goals can be used as a tool to identify the workforce needed for the future  and to develop strategies for shaping the workforce. Since 2001, we have  identified strategic human capital management as a government-wide  high-risk area.
OPM regulations require agencies to develop a human capital strategic  plan that identifies goals and objectives that are consistent with the  agency’s strategic plans and annual performance goals. These goals and  objectives are to address each of the five systems for comprehensive  human capital management that are outlined in OPM’s Human Capital  Assessment and Accountability Framework (HCAAF). In addition,  OPM’s regulations require agencies to include in their human capital  strategic plans performance measures and milestones that assess the  agency’s progress in meeting the identified goals and objectives.
In October 2012, the VA Inspector General reported that OI&T had not  developed a human capital strategic plan and recommended that the  office do so. In response to the recommendation, OI&T developed a 6- year human capital strategic plan in September 2013. This plan outlined  four human capital strategic goals: maximize employee talent through recruitment, outreach, hiring and  sustain a productive, diverse workforce and achieve results by valuing  and recognizing performance in an environment in which all  employees are encouraged to contribute; ensure OI&T supports a culture of leadership and continuous learning; ensure the human capital strategic plan is aligned with other VA  strategic plans and integrated into workforce planning.
In addition to outlining the human capital strategic goals, this plan also  identified objectives and strategies for achieving them. For example, the  objectives included improving the processes for hiring new employees;  building a diverse OI&T workforce; eliminating competency gaps in key  leadership positions; and developing an office-wide integrated workforce  analysis capability. Strategies that are linked to the goals and objectives  include implementing a plan to streamline and improve employee  orientation, increasing the percentage of minorities and veterans in the  OI&T workforce, identifying and defining the competencies that OI&T  leaders must develop to ensure staff in leadership positions have the right  skills needed to meet organizational goals, and implementing a workforce  planning process. See appendix III for a summary of the goals,  objectives, and strategies outlined in OI&T’s strategic human capital plan.
The human capital strategic plan also was aligned with specific strategies  that were identified in VA’s 6-year Strategic Plan, as well as the strategic  goals within the department’s Office of Human Resources and  Administration Strategic Plan for fiscal year 2014-2020. For example,  the department’s strategic plan includes strategies for the development of  leadership, and OI&T’s plan also includes the same. Specifically, as  stated earlier, one of the goals within OI&T’s plan is to ensure the office  supports a culture of leadership and continuous learning. Strategies for  this IT human capital strategic goal include identifying and defining the  competencies that OI&T leaders must develop and retain.
The plan also was aligned with a Human Resources and Administration  strategic goal—to advocate for veteran employment within the  department. In addition, the plan identified strategies to recruit, hire, and  promote qualified veterans.
Further, OI&T’s human capital strategic goals are linked to each of the  systems for comprehensive human capital management outlined in  OPM’s framework. For example, one of the goals is to maximize  employee talent through recruitment, outreach, hiring, and retention,  which links to the talent management system in the framework. In  addition, the plan identifies performance measures that the office is to use  to monitor its progress in achieving each of the human capital goals.  These performance measures include, among others, improved  leadership abilities, and improvements in the number of hires and  percentages for minorities, employees with disabilities, and veterans.
As of March 2016, OI&T was in the process of updating the human capital  strategic plan as part of the CIO’s transformation strategy for OI&T.  According to the office’s human capital management director, the revised  plan is expected to be in place by the end of December 2016. By  developing a human capital strategic plan and ensuring that the plan is  updated to reflect the changing mission requirements of the office, OI&T  should be better positioned to provide the necessary strategic direction  for meeting its workforce needs and goals.
section_title: OI&T Regularly Analyzed Workforce Data but Was Not Tracking Leadership Retirements
paragraphs: Performing workforce analysis is a key component in strategic human  capital planning because it assists agencies in developing strategies for  acquiring, training, and retaining staff. According to OPM, workforce  analysis allows agencies to identify trends impacting their workforce. The  analysis also provides a basis for developing actions to address  workforce trends that may impact an agency’s future mission capabilities.  OPM also stresses the importance of ensuring the continuity of leadership  within an agency, and suggests that agencies perform ongoing workforce  analysis to identify current and future workforce and leadership needs.  Among other things, for relevant agency mission requirements, OPM  requires a workforce analysis that includes a forecast of future leadership  requirements and changes due to retirements. Similarly, VA’s workforce  planning policy requires each staff office, including OI&T, to annually  analyze historical workforce data and develop workforce projections that,  at a minimum, include employee counts, retirements and other losses,  new hires, workforce diversity, and leadership retirements. The  workforce analysis also is to include a determination of gaps in the  workforce and strategies to close those gaps.
OI&T conducted a workforce analysis in May 2013 that identified for the  office the number of employees, retirements and projected eligible  retirements through fiscal year 2018, employee turnover and hiring rates,  and workforce diversity percentages. Further, OI&T has analyzed most of  the required workforce information on a monthly basis. Specifically, the  analyses we reviewed for October 2015 to January 2016 included  workforce data and projections related to employee counts, losses, new  hires, and workforce diversity. According to the OI&T Director of Human  Capital Management, the office used these data to examine trends,  increase awareness of staffing related issues or opportunities, and  identify workforce gaps, among other things.
Further, the department’s Integrated Human Resources Management  Council was tasked with analyzing workforce data, such as retirement  rates, on a quarterly basis as part of an overall VA effort to review  performance metrics related to human resource strategic goals. The  Integrated Human Resources Management Council’s review results  included workforce gaps and strategies to close them. For example, in  fiscal year 2015 VA had not met its target for hiring veterans. One of the  strategies the council identified for increasing the number of veterans  hired was to collaborate with stakeholders to identify and implement  innovative and targeted recruitment opportunities.
Nevertheless, even as it has taken these actions, OI&T was not analyzing  historical leadership retirement data; nor was it developing leadership  retirement projections as required by the department’s workforce and  succession policy. Leadership positions include senior executives as  well as other staff at the General Schedule levels 13 through 15.  According to the OI&T Director of Human Capital Management, the office  collects data on all employee losses, including retirements. Further the  OI&T human capital strategic plan included projected eligible retirements.  However, historic data on leadership retirements were not analyzed and a  forecast of future leadership retirements was not developed, as called for  by VA’s workforce planning policy.
The OI&T Director of Human Capital Management stated that, although  the office has the capability to analyze leadership retirements, there is no  specific reason why it has not done so. Without tracking and forecasting  leadership retirements, OI&T faces a risk of being unprepared to identify  and effectively respond to vacancies in key leadership positions, which in  turn can contribute to ineffective IT management.
section_title: OI&T Identified Skill Gaps but Has Not Determined Its Needs beyond the Current Year
paragraphs: Determining core skills (e.g., problem solving), leadership skills (e.g.,  developing others), and technical skills (e.g., configuration management)  is essential for an organization such as OI&T to successfully achieve its  missions and goals. This is especially important as changes in national  security, technology, budget constraints, and other factors change the  environment within which organizations operate. The identified skills  should be linked to the agency’s mission and long-term goals in order to  identify the workforce needed for the future and develop strategies for  shaping this workforce. Agencies should then identify the gaps in current  and future skills needed to achieve results, and develop strategies for  filling the gaps.
OI&T had conducted annual gap analyses to identify its skill needs, and it  had developed strategies for filling the gaps. Specifically, the IT  Workforce Development group within the Office of Quality, Performance,  and Oversight began conducting the analyses in June 2013, and the  analyses were conducted annually thereafter. On an annual basis, the  analyses identified each skill area where a gap existed, along with the  percentage of staff that was determined to be below the targeted  proficiency level for a particular role and career level (e.g., entry,  intermediate, and senior).
In addition, the analyses identified the top 10 technical skill gaps within  OI&T for each year and compared the gaps from the previous year to  those in the current year. This comparison showed either an increase,  decrease, or no change in the percentage of staff below the targeted  proficiency level for a particular skill area. For example, in March 2016, IT  Workforce Development reported that the gap in staff proficiency in 9 of  the top 10 technical skill areas within OI&T had decreased from the  previous year and that the gap in staff proficiency in 1 skill area had  remained the same. Table 2 shows the comparison of the skills gaps for  the 2015 and 2016 analyses.
After the initial analysis was conducted in June 2013, the office identified  training that was either available or being developed to address each skill  gap. For example, in its March 2016 analysis report, the IT Workforce  Development group noted that 48 new courses were being developed  based on gaps identified in the March 2015 analysis. This training  covered the areas of emerging technologies, project management,  communications, interpersonal skills, analytical reasoning, and conflict  management. As of July 2015, OI&T had provided approximately 470 on- the-job training opportunities. According to the IT Workforce Development  group, on-the-job training is used to increase options for staff to  demonstrate advanced-level proficiency, including in many of the skill gap  areas identified. For example, the group developed an opportunity for  senior staff to guide junior staff in the area of project management. The  group also provided staff the opportunity to learn from subject matter  experts in the area of infrastructure design. As of April 2016, employees  had taken advantage of approximately 160 of these opportunities.
The IT Workforce Development group also recommended in its 2016 gap  analysis that several other actions be taken by OI&T to address the  identified skill gaps. For example, the group recommended that OI&T  continue to develop learning plans that focus on advance levels of  proficiency; develop training on VIP, including training on Agile program  management; and prioritize training for leadership-related competencies,  among others.
Nevertheless, while OI&T conducted annual skill gap analyses and  developed training courses and recommended other actions for  addressing skill gaps, the office has not identified the gaps in skill areas  that could be needed in future years. Specifically, the analyses focused  on gaps in the time periods when they were conducted; however, the  analyses did not identify the skills or the gaps in the skills that may have  been needed in future years in order to implement the goals and  objectives identified in the department’s long-term IT strategic plan or for  planned IT initiatives.
According to a program manager in the IT Workforce Development group,  the annual skill gap analyses were based on current staffing levels and on  what each organization within OI&T wanted to achieve through the  current fiscal year, but not on what skills would be needed for future  years. However, by only focusing on the current year and not including  future years in its skill gaps analysis, OI&T may not have been aware of  gaps in skills that would be needed to successfully accomplish other  goals, such as the completion of a multi-year IT development project, or  longer-term goals, such as the department’s future IT operating  environment as described in the office’s 5-year IT strategic plan. By  including the IT skills needed for future-years in its gap analyses, OI&T  could have increased assurance that its staff has the capabilities to  deliver long-term IT support that contributes to improved services for  veterans.
section_title: OI&T Took Steps to Implement a Training Program
paragraphs: Training programs are an integral part of a learning environment that can  enhance an agency’s ability to attract and retain employees with the core,  leadership, and technical skills needed to achieve results. The essential  aim of these programs is to assist the agency in achieving its mission and  goals by improving individual and, ultimately, organizational performance.  Training programs involve establishing learning priorities; identifying  training initiatives; ensuring delivery of learning opportunities; and  evaluating the program and demonstrating how training efforts contribute  to improved performance and results.
OI&T took several steps to implement a training program. These included  establishing an advisory board, developing competency models and  individual development plans, and providing training to its employees.  Specifically, the office established a Training Advisory Board which was  tasked with identifying and validating training needs in OI&T. The board,  which was established in November 2014, is headed by the OI&T Chief  Learning Officer and has senior management representatives from each  organizational unit. According to its charter, the advisory board discusses  and determines the priority of the training needs of each of the  organizational units. In addition, the charter states that the board is  intended to ensure that available training is aligned with VA’s mission,  strategic planning, and OI&T priorities. The board’s most recent meetings  were held in October 2015 and April 2016. During the meetings, the  board discussed the training needs for OI&T’s organizations, the  development of training courses, and upcoming learning opportunities.
The IT Workforce Development group used competency models and  individual development plans to identify training initiatives that could  address the learning needs of OI&T’s employees. According to an official  in the IT Workforce Development group, employees were aligned with  competency models based on their job responsibilities. These  competency models identified training needs for professional  development. Additionally, the IT Workforce Development official stated  that OI&T used individual development plans to reinforce the required  knowledge, skills, and abilities necessary for employees to progress into  new positions at higher skill levels. The individual development plan is  essentially a to-do list that is generated based on identified gaps in the  employee’s skills. The plan identifies the employee’s competency model,  training needs, and the due dates for the learning opportunities identified  to meet their training needs.
Further, OI&T has taken steps to ensure the delivery of learning  opportunities. Specifically, in fiscal year 2015, the office provided over  700 training courses. For example, it provided training on information  security awareness, which over 15,000 employees and contractors  completed, and on risk assessment, which was completed by 284  employees and contractors. The courses also covered a wide range of  other IT-related subjects, such as project management, enterprise  architecture, and cybersecurity. The training courses were delivered  through several methods, such as webinars, on-the-job training,  classrooms, and virtual classrooms. The IT Workforce Development  group used event announcements and the delivery of course schedules  to each organizational unit to help ensure employees were made aware  of the provided opportunities.
Lastly, OI&T took steps to evaluate its training program. Specifically, in  April 2015, the office performed an assessment that identified  weaknesses in its training program, such as the need to supplement  competency modeling efforts with learning opportunities that focus on IT  strategic business priorities for overall organizational development. The  assessment also identified the need to provide learning opportunities that  address emerging trends and technologies at all leadership levels.
Further, the assessment included strategies to improve employee,  leadership, and organizational development. For example, one strategy to  improve employee development was to survey employees to identify  challenges in job responsibilities and the least understood emerging  technologies, and then to use that information to provide theme-based  development opportunities to address the identified needs. Additionally,  OI&T gauged the satisfaction of the training attendees by providing  course evaluations, which assessed the relevance of the training to job  responsibilities and the effectiveness of the instructor. For example, in  February 2016, 90 percent of course attendees indicated that they were  satisfied with the courses; 83 percent indicated that the content of the  courses were relevant to their jobs; and 95 percent indicated that the  instructor was effective in conveying practical knowledge about the  subject matter.
By taking these steps to implement a training program, OI&T is better  positioned to ensure that it has the ability to train and develop an IT  workforce to effectively support the department’s mission and goals.
section_title: VA Documented Processes to Manage IT System Development and Acquisitions, but Gaps Existed
paragraphs: Instituting disciplined, repeatable practices for IT development and  acquisition is key to ensuring that investments in IT cost-effectively deliver  the capabilities needed to support an organization’s mission. SEI and  GAO have identified best practices that are essential to the development  and acquisition of products and services for IT projects. These include  practices related to project planning, requirements management, risk  management, project monitoring and control, product validation, and  process and product quality assurance, as well as developing and  maintaining a reliable, high-quality project schedule. According to SEI,  when an organization’s processes are documented, it allows the  organization to more consistently apply these best practices. Further,  documented processes are important for sustaining institutionalized best  practices regardless of future leadership changes. Of 123 best practices  that we selected to evaluate, VA’s documented processes reflected 97  practices, partially reflected 12 practices, and did not reflect 14 practices.
subsections: section_title: Most Project Planning Best Practices Were Addressed in OI&T’s Documented Processes
paragraphs: Project planning involves establishing and maintaining estimates of  project parameters, developing a project plan, and obtaining commitment  to the plan from those who are responsible for implementing and  supporting it. For example, establishing project estimates involves  developing a work breakdown structure that details project tasks,  responsibilities, and schedules; specifying estimates of work products and  task attributes for the project; and estimating the project’s effort and costs  for work products and tasks based on estimation rationale.
In addition, developing a project plan includes identifying major  milestones, schedule assumptions, constraints, and task dependencies;  and creating and maintaining a project plan that ties together the budget,  schedule, milestones, and stakeholder identification and interaction.
Further, obtaining commitment to the plan includes identifying  agreements regarding interfaces between project elements and other  projects; documenting commitments with an appropriate level of  signatories; and making adjustments to the project plan to reconcile  variances in estimated and available resources.
OI&T’s project planning processes reflected 13 of 15 selected best  practices. Of the 15 practices, the processes we examined included  criteria that addressed 4 of 5 practices for establishing project estimates.  These included developing a work breakdown structure that specifies  project tasks, responsibilities, and schedules; identifying products to be  acquired externally; providing estimates of work products and task  attributes; and planning project lifecycle phases.
One practice—to estimate the project’s effort and costs for work products  and tasks based on estimation rationale—was partially reflected in the  documented processes. In this regard, VA’s project scheduling tool  contained processes that called for project managers to estimate project  effort. However, the processes did not address how the managers were  to calculate the estimated effort to be expended to complete a project.
Further, the processes included criteria related to all 7 selected practices  for developing a project plan. These included identifying major  milestones, defining and quantifying resources needed for the project,  and creating and maintaining a project plan that ties together the budget,  schedule, milestones, and stakeholder identification and interaction.
Finally, the processes reflected 2 of 3 selected practices for obtaining  commitment to the plan. In particular, they included identifying  commitments regarding interfaces between project elements and other  projects, and for documenting commitments with an appropriate level of  signatories. However, the processes did not address making adjustments  to the project plan to reconcile variances in estimated and available  resources.
Until OI&T incorporates all the recommended project planning best  practices in its process documentation, the office increases the risk that  its project managers could inconsistently calculate estimates of the effort  needed across projects and fail to adjust project plans based on resource  availability, thus potentially affecting the timely completion of projects.  Table 3 summarizes the extent to which these best practices were  documented in OI&T’s project planning processes.
section_title: OI&T’s Documented Processes Addressed Almost All Requirements Management Best Practices
paragraphs: Best practices recommend that project teams manage requirements of  the project’s products and components, and ensure alignment between  those requirements and the project plans and work products. Managing  requirements can be accomplished by understanding and obtaining  commitment to requirements, and ensuring that project work and  associated requirements align with each other. Examples of these best  practices include analyzing requirements to ensure that established  criteria for managing requirements are met; documenting all requirements  and requirements changes; maintaining bidirectional requirements  traceability from a requirement to its derived (i.e., lower-level)  requirements and allocation to work products and back; and identifying  changes to be made to plans and work products based on modifications  to the requirements baseline.
OI&T had documented processes that reflected 10 of 11 selected best  practices for managing project requirements. For example, the processes  included assessing the impact of requirements on existing commitments,  negotiating and documenting changes to commitments, and maintaining a  requirements change history with the rationale for the changes. However,  the documentation did not address one of the practices. Specifically, it did  not include identifying changes that should be made to plans and work  products resulting from changes to the requirements baseline.
Until the office incorporates in its documented processes the best practice  for identifying changes to be made based on modifications to a project’s  requirements baseline, OI&T’s staff will not be best positioned to  implement the associated changes and maintain consistent processes  across IT projects. Table 4 summarizes the extent to which these best  practices were reflected in OI&T’s requirements management processes.
section_title: Majority of Risk Management Best Practices Were Documented in OI&T’s Processes
paragraphs: Risk management best practices call for the identification of potential  problems before they occur so that risk-handling activities can be planned  throughout the life of the project to mitigate adverse impacts on achieving  objectives. These practices involve preparing for risk management,  identifying and analyzing risks, and mitigating identified risks. For  example, preparing for risk management involves determining risk  sources and categories, and developing risk mitigation techniques. In  addition, identifying and analyzing risks includes determining those that  are associated with cost, schedule, and performance; and reviewing the  work breakdown structure and project plan to help ensure that all  aspects of the work effort have been considered. Further, mitigating risks  includes determining the levels and thresholds in which a risk becomes  unacceptable and triggers the execution of a risk mitigation plan or  contingency plan; establishing a schedule for each risk handling activity,  with a start date and anticipated completion date; determining the costs  and benefits of implementing the risk mitigation plan for each risk; and  collecting performance measures on risk handling activities.
OI&T established processes that reflected 26 of 28 selected best  practices for risk management. Of the 28 best practices, the office’s  processes addressed all 9 practices related to preparing for risk  management. For example, they defined criteria for evaluating and  quantifying risk likelihood and severity, established thresholds for risk  categories, and defined a time period for risk monitoring or reassessment.
In addition, the documented processes included criteria related to all 9  practices for identifying and analyzing risks. For example, the processes  reflected best practices for documenting the context, conditions, and  potential consequences of each risk; categorizing and grouping risks  according to defined risk categories; and prioritizing risks for mitigation.
With regard to mitigating risks, the documentation reflected 8 of 10  selected best practices. For example, it included identifying the person or  group responsible for addressing risks, developing contingency plans for  critical risks, and monitoring risk status. However, it did not address one  practice for determining the costs and benefits of implementing the risk  mitigation plan for each risk. Further, one practice—to collect  performance measures on risk handling activities—was partially reflected  in the processes. In this regard, VA’s processes called for the creation  and distribution of monthly performance reports, but they did not address  risk mitigation activities.
Although VA included 26 of 28 best practices related to risk management  in its processes, it is not positioned to fully ensure that the costs and  benefits of executing risk mitigation plans will be consistently determined  and that performance measures for risk mitigation efforts are collected.  Table 5 summarizes the extent to which these best practices were  documented in OI&T’s risk management processes.
section_title: OI&T’s Documented Processes Addressed Some but Not All Project Monitoring and Control Best Practices
paragraphs: The purpose of project monitoring and control is to provide an  understanding of the project’s progress so that appropriate corrective  actions can be taken when the project’s performance deviates  significantly from the plan. Toward this end, a project’s documented plan  is considered the basis for monitoring activities and corrective actions  taken when performance deviates significantly from the plan. Examples  of monitoring the project against the project plan include tracking  progress against the schedule, monitoring project costs and the effort  expended for project completion, documenting the results of data  management activity reviews, and identifying and documenting significant  issues and deviations from the project plan. In addition, managing  corrective actions to address identified issues involves determining and  documenting the appropriate actions needed to address the issues and  monitoring corrective actions for completion.
OI&T’s documented processes reflected 18 of 28 selected best practices  related to project monitoring and control. Of the 28 practices, the  processes reflected 12 of 22 selected practices for monitoring the project  against the project plan and all 6 selected practices related to managing  correction actions. For monitoring the project against the project plan, the  processes included tracking change requests and problem reports to  closure, documenting the results of milestone reviews, and reviewing the  project plan, project status, and risks at selected milestones. In addition,  for managing corrective actions, the processes included gathering issues  that require corrective action for analysis, analyzing the results of  corrective actions to determine effectiveness, and determining and  documenting lessons learned as a result of taking appropriate corrective  actions.
However, three of the selected practices were partially reflected in the  processes for monitoring the project against the project plan. Specifically,  documentation on monitoring project costs and the effort expended for  project completion included criteria for monitoring project costs; however,  it did not call for tracking expended effort. With regard to monitoring  resources provided and used, the processes call for VA to monitor  projects to ensure that staff and resources are provided, but they did not  clearly require monitoring for the utilization of those staff and resources.
Similarly, OI&T partially documented its processes for reviewing the  results of collecting and analyzing project performance measures, such  as customer service satisfaction metrics. In particular, the processes had  been developed to ensure the agreed-upon functionality was delivered to  customers. However, the customers’ level of satisfaction based on the  functionality requested and received for the project was not addressed.
In addition, the processes did not address seven of the selected  practices. These practices were associated with monitoring the  knowledge and skills of project staff, periodically reviewing data  management activities against the project plan, identifying and  documenting significant data management issues and their impacts,  documenting results of data management activity reviews, periodically  reviewing stakeholder involvement in projects, documenting the results of  stakeholder involvement status reviews, and tracking action items to  closure.
Until OI&T completely documents its processes for monitoring and  controlling projects, staff responsible for executing these steps may not  be fully aware of a project’s progress toward achieving the milestones  defined in the project plan. Table 6 summarizes the extent to which these  best practices were addressed in OI&T’s project monitoring and control  processes.
section_title: Validation Best Practices Were Reflected in OI&T’s Documented Processes
paragraphs: Best practices for product validation call for project teams to demonstrate  that a product or component fulfills its intended use when placed in its  intended environment. The methods employed to accomplish validation  can be applied to work products as well as to the product components.  Key practices, as identified by SEI, include preparing for validation by  identifying and selecting products, and validating the product or product  components. For example, preparing for validation includes identifying  the product, product component, and/or features to be validated  throughout the life of the project and identifying requirements for the  validation environment and passing them to the requirements  development process. Further, validating the product or its components  involves ensuring that activities (i.e., those that validate IT hardware and  services) are performed throughout the project lifecycle and that  deviations from validation procedures are documented.
Of the 17 selected best practices for product validation, OI&T’s  documented processes reflected all 10 practices related to preparing for  validation, and all 7 practices for carrying out the validation of the product  or its components. For example, the office’s processes addressed best  practices for identifying test equipment and tools, ensuring the validation  method was selected early in the development process, identifying and  documenting validation failure causes as needed, comparing actual  validation test results to expected results, and analyzing validation data  for defects.
By incorporating the selected validation best practices into its processes,  OI&T has increased assurance that staff responsible for performing  validation of the department’s IT products or their components will be  positioned to consistently apply key processes to ensure that products  perform as expected when placed in the intended environment. Table 7  summarizes the extent to which these best practices were addressed in  OI&T’s validation processes.
section_title: Process and Product Quality Assurance Best Practices Were Mostly Reflected in OI&T’s Documentation
paragraphs: Best practices for process and product quality assurance are intended to  support the delivery of high-quality products by providing project staff and  management with appropriate visibility and feedback on processes and  associated work products throughout the life of the project. Toward this  end, process and product quality assurance involves objectively  evaluating processes and work products, and providing objective insight  to staff and managers. Examples of objectively evaluating processes  and work products include establishing and maintaining clearly stated  criteria, based on business needs, for assessing processes and work  products; identifying lessons learned that could improve processes; and  documenting a description of the quality assurance reporting chain and  defining how it will ensure objectivity. In addition, examples of providing  objective insight include escalating noncompliance issues that cannot be  resolved in the project to the appropriate level of management; tracking  noncompliance issues to resolution; recording process and product  quality assurance activities in sufficient detail so that the status and  results are known; and periodically reviewing open noncompliance issues  and trends with management designated to receive and act on them.
OI&T had documented processes that addressed 12 of 14 selected best  practices related to process and product quality assurance. Specifically,  the processes reflected 4 of 5 practices for objectively evaluating  processes and work products. These included practices related to  establishing and maintaining clearly stated criteria, based on business  needs, for evaluating processes and work products; identifying each  noncompliance found during the evaluation; identifying lessons learned;  and evaluating selected work products at selected times.
One practice—to document a description of the quality assurance  reporting chain and define how it will ensure objectivity— was partially  addressed in the processes. In this regard, while a quality assurance  standard described how each review would ensure objectivity, the  description of the reporting chain was not included.
In addition, the office’s processes reflected 8 of 9 selected practices for  providing objective insight. These included documenting noncompliance  issues that cannot be resolved in the project, analyzing noncompliance  issues to determine if quality trends can be identified and addressed; and  ensuring that relevant stakeholders are aware of results of evaluations  and quality trends in a timely manner. However, they did not address the  periodic review of open noncompliance issues and trends with  appropriate management.
By not fully documenting best practices related to quality assurance in its  processes, OI&T has less assurance that staff will appropriately report  quality assurance issues and periodically review open issues and trends  so that the relevant manager can act on them. Table 8 summarizes the  extent to which these best practices were reflected in OI&T’s process and  product quality assurance processes.
section_title: Most Project Schedule Best Practices Were Partially Addressed or Not Addressed in OI&T’s Process Documentation
paragraphs: The success of a project depends, in part, on having an integrated and  reliable master schedule that defines when and how long work will occur,  and how each activity is related to the others. A project’s schedule  provides not only a road map for systematic project execution, but also  the means by which to gauge progress, identify and resolve potential  problems, and promote accountability at all levels of the program. VA’s  guide for IT project management deems project schedules as essential to  a project’s success and notes that failure to meet project schedule dates  may trigger additional project reviews by senior leaders, which could lead  to a project’s shutdown.
Further, our Schedule Assessment Guide defines 10 best practices  related to 4 characteristics that are important to developing high-quality,  reliable schedule estimates—comprehensive, controlled, well- constructed, and credible. Table 9 describes the characteristics of high- quality, reliable schedule estimates and their associated best practices  that guided our analysis.
Almost all of OI&T’s processes for developing project schedules either  partially addressed or did not address the 10 best practices. Specifically,  the documentation for developing comprehensive, controlled, well- constructed, credible project schedules reflected 1 best practice, partially  reflected 6 best practices, and did not reflect 3 best practices.
Without documented processes for developing schedules that reflect best  practices, OI&T is at increased risk that schedules created for its projects  will not be reliable and, therefore, will not be useful tools for measuring  program performance against approved project plans. Table 10  summarizes the results of our assessment of whether OI&T’s  documented processes reflected the 4 characteristics of a high-quality,  reliable schedule estimate, their associated best practices, and examples  of our rationale for the assessment.
section_title: VA Officials Asserted That Inclusion of Missing Best Practices in Documented Processes Was Not Necessary
paragraphs: In discussing OI&T process documentation, officials in the Product  Development and Service Delivery and Engineering offices, including an  Assistant Deputy CIO within Product Development, stated that the  department’s documented IT processes did not include all of the selected  best practices because they were considered basic, fundamental skills  that a properly trained project planner should possess. The officials  added that their inclusion would be overly burdensome and  counterproductive when working in an Agile environment.
However, the selected best practices do not apply to a particular  development approach; rather, they were designed to provide value  across different approaches, including Agile. In addition, documented  processes are key to the consistent implementation of best practices that  can withstand future leadership changes. Without incorporating these  best practices into its processes, VA is at risk of missing critical steps that  could help ensure that its processes are implemented as intended and  are consistently applied across IT projects.
section_title: Conclusions
paragraphs: While VA has largely centralized the performance of key IT-related  functions in OI&T, led by the CIO, it has faced challenges in fully  implementing and managing IT under this organizational structure. In  particular, the office has had difficulty in preventing IT activities from  occurring outside the control of OI&T and PMAS. It has also been  challenged in effectively collaborating with the department’s various  business units and in efficiently and cost-effectively delivering new IT  capabilities. The OI&T transformation effort recently initiated by the CIO,  and projected to be completed by the first quarter of 2017, is intended to  address these and other challenges.
OI&T has also taken steps to implement human capital management best  practices by developing and documenting and beginning to update a  human capital strategic plan that aligns with VA’s strategic plans, creating  action plans for implementing OI&T human capital goals and objectives,  reviewing its workforce quarterly, identifying skill gaps, and implementing  a training program. However, without annually tracking and reviewing  data related to leadership retirements or identifying skills needed in future  years, VA will be hindered in ensuring that it has the leadership and staff  with skills needed to support the successful delivery of IT capabilities.
Further, opportunities exist for VA to strengthen its processes for system  development and acquisition by including key system development best  practices. VA’s documented processes for managing IT system  development and acquisition generally reflected best practices in the key  areas we reviewed. However, there were gaps in most of these areas.  Ensuring that these processes address all key practices will assist the  department in effectively managing its IT system development and  acquisitions.
section_title: Recommendations for Executive Action
paragraphs: To assist VA in sustaining an IT workforce with the necessary knowledge,  skills, and abilities to execute its mission and goals, we recommend that  the Secretary of Veterans Affairs direct the Chief Information Officer to  track and review OI&T historical workforce data and projections  related to leadership retirements, and  identify IT skills needed beyond the current fiscal year to assist in  identifying future skills gaps.
To assist VA in establishing comprehensive and documented processes  that reflect system development and acquisition best practices, we  recommend that the Secretary of Veterans Affairs direct the Chief  Information Officer to revise OI&T’s documented processes related to project planning, to include (1) estimating the level of effort that will  need to be expended for work products and tasks, and (2) making  adjustments to the project plan to reconcile differences between  estimated and available resources;  requirements management, to include identifying changes to be made  to plans and work products as a result of requirements baseline  changes;  risk management, to include (1) determining costs and benefits of  implementing the risk mitigation plan for each risk and (2) collecting  performance measures on risk handling activities; project monitoring and control, to include the 10 best practices that  were missing from the guidance; process and product quality assurance, to include (1) documenting a  description of the quality assurance reporting chain and defining how  objectivity will be ensured, and (2) periodically reviewing open  noncompliance issues and trends with management that is  designated to receive and act on them; and project scheduling, to include the 9 best practices that were missing  from the guidance and revise the documented processes where the  guidance was contrary to best practices.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of our report (reprinted in appendix IV), VA  generally agreed with our conclusions and concurred with our eight  recommendations. Further, the department provided, and requested that  we include in our report, information describing OI&T’s new organizational  structure and its cyber security initiative. We have incorporated this  information in relevant sections of the report.
VA’s comments described steps that it had taken or planned to implement  our recommendations. For example, the department asserted that it had  taken steps that fully addressed our recommendation that it track and  review OI&T historical workforce data and projections related to  leadership retirements. In our follow up on the department’s  implementation of our recommendations, we will evaluate whether the  actions noted are fully responsive to this recommendation.
The department also discussed planned actions for addressing our  recommendation related to identifying future IT skills. Specifically, VA  stated that it plans to, among other actions, include in the next skills gap  analysis, long-term recommendations at the organizational level that  show the types of skills each organization needs to increase and which  proficiency level targets need the most emphasis.
Further, in response to our recommendations that it revise OI&T’s  documented processes related to project planning and requirements  management, VA stated that it plans to document changes to these  processes as the department transitions from PMAS to VIP. In addition,  with respect to the majority of the remaining deficiencies in VA’s  documented processes—related to risk management, project monitoring  and control, quality assurance, and project scheduling—the department  stated that our recommendations are to be addressed through the  implementation of VIP, Agile processes, and other systems development  process tools.
According to VA, its actions in response to our recommendations are  expected to be completed by the end of fiscal year 2017. If the  department ensures that these and other activities it identified are  appropriately documented and effectively implemented, then VA should  be better positioned to sustain an IT workforce with the necessary  knowledge, skills, and abilities to execute its mission and goals, and to  ensure critical systems development and acquisition processes are  implemented as intended and are consistently applied across IT projects.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of Veterans  Affairs and other interested parties. In addition, the report is available at  no charge on the GAO website at http://www.gao.gov.
Should you or your staffs have questions on matters discussed in this  report, please contact me at (202) 512-6304. I can also be reached by e- mail at melvinv@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.
GAO staff who made major contributions to this report are listed in  appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this study were to determine (1) how the Department of  Veterans Affairs (VA) is organized to manage and perform key  information technology (IT)-related functions and the extent to which it  has centralized the management of IT resources, (2) the extent to which  VA has implemented effective IT human capital management, and (3) the  extent to which VA has established key processes to effectively manage  major system development and acquisition efforts.
To address the first objective, we obtained and reviewed the department’s  documentation identifying and describing its IT organizational structure  and functions. These included the IT strategic plan, governance plan,  organizational descriptions contained within the VA’s Functional  Organization Manual, the IT organization chart, and VA’s project  management framework user guides. In addition, we reviewed  memoranda, testimony from VA officials, and assessment reports that  described the department’s efforts to centralize the management of its IT  resources and the challenges it faces in doing so. In addition, we  obtained and reviewed the Chief Information Officer’s (CIO) IT  transformation strategy intended to address the challenges in maintaining  a centralized program. We also interviewed responsible program officials  regarding the organizational structure and performance of key IT-related  functions.
To address the second objective, we reviewed information describing the  department’s management of its IT human capital, such as human capital  strategic planning documentation, skills gap analyses, and human capital  performance measures. We compared the department’s actions to  agency-level human capital management best practices that are also  applicable to major agency components such as the Office of Information  & Technology (OI&T). These best practices are those that we and the  Office of Personnel Management (OPM) have identified. We also  compared VA’s actions against those required by VA’s Office of Human  Resources and Administration. These included practices for effective  strategic human capital planning, workforce planning, and strategic  training. The practices are identified in our Key Principles for Effective  Strategic Workforce Planning;  OPM’s final regulations to implement  certain provisions of the Chief Human Capital Officers Act of 2002;  and  VA’s Workforce and Succession Planning directive.  We also interviewed  officials responsible for the department’s IT human capital management.
For the third objective, we reviewed policies, procedures, and supporting  documentation describing the department’s key processes for managing  major IT system development and acquisition efforts. We assessed the  processes against key best practices that the Software Engineering  Institute (SEI) and we have identified. The practices we selected are  fundamental to effective IT system development and acquisition. These  included recognized practices for project planning, requirements  management, risk management, project monitoring and control, project  validation, quality assurance, and project schedules. These practices are  identified in SEI’s Capability Maturity Model® Integration (CMMI®) for  Development, Version 1.3; SEI’s CMMI® for Acquisition, Version 1.3;  and our Schedule Assessment Guide. Further, we reviewed systems  development lifecycle policies, procedures, and other supporting  documentation such as templates for required project artifacts within the  department’s ProPath system and Project Management Accountability  System (PMAS). The scope of our review did not include assessment of  VA’s implementation of these processes.
Our methodology to determine the extent to which VA included systems  development and scheduling best practices developed by SEI and us in  its procedures and guidance included three levels of compliance: (1) the  program office provided evidence that satisfied the elements of the best  practice, (2) the program office provided evidence that satisfied some but  not all of the elements of the best practice, and (3) the program office  provided no evidence that satisfied the elements of the best practice.
We conducted this performance audit from February 2015 to August 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Timeline of VA’s Efforts to Organize the Management of IT Functions
paragraphs: The Department of Veterans Affairs (VA) has had a long history of  organizing its information technology (IT) functions. The information  contained in table 11 describes key events in the department’s efforts.
section_title: Appendix III: Summary of VA’s Office of Information & Technology Human Capital Goals, Objectives, and Strategies
paragraphs: The Department of Veterans Affairs (VA) Office of Information &  Technology (OI&T) developed a 6-year human capital strategic plan in  September 2013. The information contained in table 12 describes the  goals, objectives, and strategies outlined in OI&T’s strategic human  capital plan.
section_title: Appendix IV: Comments from the Department of Veterans Affairs
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Mark Bird (Assistant Director),  Nicole Jarvis (Analyst in Charge), Christopher Businsky, Juana  Collymore, Sharhonda Deloach, Jason Lee, Lee McCracken, and Christy  Tyson made key contributions to this report.