IT Workforce: Key Practices Help Ensure Strong Integrated Program Teams; Selected Departments Need to Assess Skill Gaps
Published: 2016-11-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In fiscal year 2017, the federal government is expected to spend more than $89 billion on IT. In many instances, agencies have not consistently applied best practices that are critical to successfully acquiring IT investments, such as ensuring program staff have the necessary knowledge and skills. In an effort to aid agencies in successfully delivering projects, the Office of Management and Budget has called for the development and use of IPTs for federal IT acquisitions to ensure that projects consist of the appropriate mix of individuals. GAO was asked to review IPTs for federal IT acquisitions and the federal government's IT workforce planning.
GAO's objectives were to (1) identify key characteristics of comprehensive IPTs responsible for managing major federal IT acquisitions, and (2) evaluate whether selected federal agencies are adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major IT acquisitions. To do so, GAO reviewed relevant literature; interviewed IPT experts; and evaluated IT workforce efforts at five departments: Commerce, Defense, Health and Human Services, Transportation, and the Treasury.
section_title: What GAO Found
paragraphs: Integrated program teams (IPT) are cross-functional or multidisciplinary groups of individuals that are organized and collectively responsible for delivering a product to an external or internal customer. GAO identified three characteristics that contribute to the creation and operation of a comprehensive IPT: (1) executive leadership through team support, empowerment, and oversight; (2) team composition; and (3) processes for team operations. GAO also identified 18 practices supporting these three characteristics (see figure). For example, executive leadership is effective when sufficient resources are provided and teams are empowered to act, team composition is more robust when the IPT has cross-functional and multidisciplinary skill sets, and team operations are streamlined when team guidelines are established and stakeholders are involved as active members. When implemented, these practices can increase the IPT's likelihood of success by having the right mix of expertise to recognize problems early and by having the requisite authority to do something about them.
While multiple factors contribute to a robust IPT, one aspect involves having a strong information technology (IT) workforce. To evaluate agencies' IT workforce planning efforts, GAO identified eight key workforce planning steps and activities based on relevant laws and guidance (see table).
Five federal departments had mixed progress in assessing their IT skill gaps. While all five departments had demonstrated important progress in either partially or fully implementing key IT workforce planning activities, each had shortfalls. For example, four departments had not demonstrated an established IT workforce planning process.
As shown in the figure, of the five departments, the Department of Defense had the most robust IT workforce planning process by fully or partially implementing all eight activities. However, the departments have not yet fully implemented all of the practices for various reasons. For example, policies were not comprehensive in requiring such activities or were not being applied to IT workforce planning at four departments, one department placed a greater emphasis on assessing its cybersecurity workforce, and two departments identified the need to perform more granular assessments of the workforce in order to identify skill gaps. Until the departments fully implement key workforce planning steps and activities, they risk not adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major acquisitions.
section_title: What GAO Recommends
paragraphs: GAO recommends that selected departments implement IT workforce planning practices to facilitate more rigorous analyses of gaps between current skills and future needs, and the development of strategies for filling the gaps. Four departments agreed and one, Defense, partially agreed with our recommendations.

section_title: Letter
paragraphs: In fiscal year 2017, the federal government is expected to spend more  than $89 billion on information technology (IT). Unfortunately, projects too  frequently fail or incur cost overruns and schedule slippages while  contributing little to mission-related outcomes. We have previously  reported that failed IT projects often suffered from a lack of disciplined  and effective management, such as project planning, requirements  definition, and program oversight and governance. In many instances,  agencies had not consistently applied best practices that are critical to  successfully acquiring IT investments, such as ensuring program staff  have the necessary knowledge and skills. As a result, we added  “Improving the Management of Information Technology Acquisitions and  Operations” to our high risk list in February 2015.
In an effort to aid agencies in successfully delivering IT projects, the  Office of Management and Budget (OMB) has called for the development  and use of integrated program teams (IPT) for federal IT acquisitions to  ensure that projects consist of the appropriate mix of individuals. OMB  defines an IPT as a cross-functional or multidisciplinary group of  individuals that are organized and collectively responsible for the specific  purpose of delivering a product to an external or internal customer. IPTs  include a program manager, other key experts, and stakeholders.
Part of having a sound IPT is ensuring an agency has a workforce that  contains the appropriate mix of skills and number of staff that possess  those skills. Law and guidance over the past 20 years has called for  federal agencies to assess IT knowledge and skill requirements, identify  gaps in meeting those requirements, and develop strategies and plans to  address deficiencies.
Accordingly, you asked us to review IPTs and the federal government’s IT  workforce planning. Specifically, our objectives were to (1) identify key  characteristics of a comprehensive IPT responsible for managing a major  federal IT acquisition, and (2) evaluate whether selected federal agencies  are adequately assessing and addressing gaps in knowledge and skills  that are critical to the success of major IT acquisitions.
To address our first objective, we reviewed practices applicable to IPTs  that were recommended by leading government and industry  organizations such as OMB, the Project Management Institute, Inc.,  MITRE Corporation, and GAO; reviewed practices recommended by  officials from two federal agency IPTs for major IT acquisitions, the offices  of the chief information officer (CIO) with responsibilities for those IPTs,  the Department of Defense (Defense) Office of the CIO, and two  agencies’ acquisition centers of excellence; performed a content analysis  on recommended IPT practices to identify key characteristics; and  validated the results of our analysis with the selected organizations and  internal experts.
To address our second objective, we reviewed relevant laws and  guidance to federal agencies and created an evaluation framework of IT  workforce planning practices that were consistent with strategic human  capital planning practices recommended by the Office of Personnel  Management (OPM), OMB, and GAO. We selected five departments— Commerce, Defense, Health and Human Services, Transportation, and  the Treasury—based on the following factors: (1) largest number of major  IT investments, (2) largest planned dollar total of major IT spending in  fiscal year 2016, and (3) largest planned percentage of total major IT  spending with development, modernization, and enhancements in fiscal  year 2016. We analyzed the extent to which departments were  implementing the set of practices from our IT workforce planning  framework and interviewed cognizant officials at the selected  departments.
We conducted this performance audit from July 2015 to November 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. See appendix I for a more  detailed discussion of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: IPTs are to bring together the different areas of expertise needed to  acquire a new product, such as engineering, manufacturing, purchasing,  and finance. The essence of the IPT approach is to concentrate this  expertise in a single team together with the authority to design, develop,  test, manufacture, and deliver a product. The hallmark of these teams is  their ability to efficiently make decisions that cross lines of expertise.
Over the last two decades, the federal government has taken steps to  improve IT acquisitions through the use of IPTs. Building on success from  private industry’s use of IPTs, Defense adopted them in an attempt to  improve its weapon system acquisitions. In 1996, the department  published a guide on integrated product and process development based  on its survey of policies and practices for over 80 government and  industry organizations. Defense’s intention was to use the teams in the  same manner as commercial firms—to integrate different functional  disciplines into a team responsible for all aspects of an acquisition.
Recognizing IPTs’ potential to improve the federal government’s  approach to managing IT investments, OMB has called for the  development and use of IPTs for federal IT acquisitions. In 2010, OMB  published a 25-point action plan that was intended to address many of the  most pressing, persistent challenges to the federal government’s  management of IT. One of the actions OMB cited was to require that  IPTs were in place before OMB approves program budgets. Additional  actions agencies are to take are to dedicate resources to the team  throughout the program lifecycle and hold team members accountable for  individual goals and overall program success.
Subsequently, OMB has incorporated revisions to its Capital  Programming Guide and related guidance, which require the use of IPTs  prior to OMB’s approval of an investment’s business case. Specifically,  OMB requires that each major IT investment establish an IPT that  includes, at a minimum, a fully dedicated program manager, a contracting  specialist (if applicable), an IT specialist, an IT security specialist, and a  business process owner or subject matter expert.
subsections: section_title: IT Workforce Planning Is Needed to Support Effective IPTs
paragraphs: Effective IT workforce planning is key to an agency’s success in  developing IPTs with the necessary knowledge, skills, and abilities to  execute a range of management functions that support the agency’s  mission and goals. Over the past 20 years, various laws were enacted  and guidance issued that call for agencies to perform workforce planning  activities that ensure the timely and effective acquisition of IT. These laws  and guidance focus on the importance of (1) setting the strategic direction  for workforce planning, (2) analyzing the workforce to identify skill gaps,  (3) developing strategies to address skill gaps, and (4) monitoring and  reporting on progress in addressing skill gaps. For example:
The Clinger-Cohen Act of 1996 requires agency CIOs to annually (1)  assess the requirements established for agency personnel regarding  knowledge and skill in information resource management and the  adequacy of such requirements for facilitating the achievement of  performance goals; (2) assess the extent to which the positions and  personnel at executive and management levels meet those  requirements; (3) develop strategies and specific plans for hiring,  training, and professional development to address any deficiencies;  and (4) report to the head of the agency on the progress made in  improving information resources management capability.
The E-Government Act of 2002 requires the Director of OPM, in  consultation with the Director of OMB, the Chief Information Officers  Council, and the Administrator of General Services to (1) analyze, on  an on-going basis, the personnel needs of the federal government  related to IT and information resource management; and (2) identify  where current IT and information resource management training does  not satisfy personnel needs. In addition, the law requires the Director  of OMB to ensure that agency heads collect and maintain  standardized information on their IT and information resources  management workforce.
In 2010, OMB issued its 25-point plan for IT reform and outlined  several action plans to build workforce capabilities, including  acquisition and program management. For example, OMB stated  that OPM will work with OMB to provide agencies with direct hiring  authority for program managers and directed OPM to create a  specialized career path. OMB also tasked agencies with identifying  program management competency gaps and reporting on those gaps.
Subsequent to the 25-point plan, in July 2011, OMB released  guidance for agencies to develop specialized IT acquisition cadres.  Among other things, this memorandum required agencies to analyze  current acquisition staffing challenges; determine if developing or  expanding the use of cadres would improve program results; and  outline a plan to pilot or expand cadres for an especially high-risk  area, if the agency determined that such an effort would improve  performance.
Further, in November 2011 OPM issued guidance for developing  career paths for IT program managers. OPM’s career path guide  was to build upon its IT Program Management Competency Model  released in July 2011 by serving as a roadmap for individuals  interested in pursuing a career in this area and providing employees  and their supervisors with a single-source reference to determine  appropriate training opportunities for career advancement.
In December 2014, Congress enacted legislation commonly referred  to as the Federal Information Technology Acquisition Reform Act  (FITARA). Among other things, the law aims to ensure timely  progress by federal agencies toward developing, strengthening, and  deploying IT acquisition cadres consisting of personnel with highly  specialized skills in IT acquisition, including program and project  managers. Agencies (other than Defense) are required to update their  acquisition human capital plans to address how they are meeting their  human capital requirements to support timely and effective  acquisitions.
To assist agencies in implementing the provisions of FITARA  and to  reinforce provisions of the Clinger-Cohen Act of 1996, OMB issued  guidance to agencies in June 2015. In doing so, OMB directed  agencies (other than Defense) to, among other things, (1) develop a  set of competency requirements for staff, including leadership  positions; and (2) develop and maintain a current workforce planning  process to ensure the agency can (a) anticipate and respond to  changing mission requirements, (b) maintain workforce skills in a  rapidly developing environment, and (c) recruit and retain the talent  needed to accomplish the mission. Each agency is to conduct an  annual self-assessment of its conformity with these requirements and  develop an implementation plan describing the changes it will make.
In October 2015, OMB required agencies to identify their top five  cybersecurity talent gaps by December 2015 as a one-time effort.  Specifically, agencies were to participate in an OPM cybersecurity  staffing exercise to identify the universe of their cyber talent,  understand challenges for retaining talent, and address gaps  accordingly.
The Federal Cybersecurity Workforce Assessment Act of 2015  required OPM, with support from the National Institute of Standards  and Technology, to establish a coding structure to be used in  identifying all federal civilian and non-civilian positions that require the  performance of IT, cybersecurity, or other cyber-related functions.  Agencies, in consultation with OPM, the National Institute of  Standards and Technology, and the Department of Homeland  Security, were then required to utilize this coding structure to annually  assess, among other things, the IT, cybersecurity, and other cyber- related work roles of critical need in the agency’s workforce. We have  additional planned work in this area.
OMB released its Federal Cybersecurity Workforce Strategy in July  2016. Among other things, the strategy cited the need for agencies  to examine specific IT, cybersecurity, and cyber-related work roles,  and identify personnel skills gaps, rather than merely examining the  number of vacancies by job series. The strategy identified several  actions that agencies could take to identify workforce needs, expand  the cybersecurity workforce through education and training, recruit  and hire highly skilled talent, and retain and develop highly skilled  talent.
Finally, in July 2016 OMB issued updated policy for the planning,  budgeting, governance, acquisition, and management of federal  information, personnel, equipment, funds, IT resources and  supporting infrastructure and services. Among other things, OMB’s  updated circular requires the agency’s chief human capital officer,  CIO, chief acquisition officer, and senior agency official for privacy to  develop a set of competency requirements for staff and develop and  maintain a current workforce planning process.
While the laws and guidance focus on IT workforce, there are also other  broader initiatives to improve federal human capital management. For  example, we and OPM developed human capital management models  that call for implementing workforce planning practices that can facilitate  the analysis of gaps between current skills and future needs and the  development of strategies for filling the gaps, as well as planning for  succession. In addition, our Standards for Internal Control in the Federal  Government stress that management should consider how best to retain  valuable employees, plan for their eventual succession, and ensure  continuity of needed skills and abilities.
section_title: GAO Has Previously Reported on Federal IPTs and IT Workforce Planning
paragraphs: We have previously reported on commercial and Defense teaming  practices and found that IPTs can develop and deliver superior products  within predicted time frames and budgets—often cutting calendar time in  half compared with earlier products developed without such teams. In  addition, we identified two elements that are essential to an IPT: the  knowledge and authority needed to recognize problems and make cross- cutting decisions expeditiously. We noted that knowledge is sufficient  when the team has the right mix of expertise to master the different facets  of product development and authority is present when the team is  responsible for making both day-to-day decisions and delivering the  product. We concluded that if a team lacks expertise, it will miss  opportunities to recognize potential problems early; without authority, it  can do little about them.
Regarding broader human capital planning efforts, we first designated  strategic human capital management across the government as a high- risk issue in 2001 because of the federal government’s long-standing lack  of a consistent approach to human capital management. In February  2011, we narrowed the focus of this high-risk issue to the need for  agencies to close mission-critical skill gaps. At that time, we noted that  agencies faced challenges effectively and efficiently meeting their  missions across a number of areas, including acquisition management.
With regard to IT workforce and human capital planning, we reported that  effectively addressing mission-critical skill gaps in IT requires a  multi-faceted response from OPM and agencies. Specifically, our high  risk update in February 2013 noted that OPM and agencies would need  to use a strategic approach that (1) involves top management,  employees, and other stakeholders; (2) identifies the critical skills and  competencies that will be needed to achieve current and future  programmatic results; (3) develops strategies that are tailored to address  skill gaps; (4) builds the internal capability needed to address  administrative, training, and other requirements important to support  workforce planning strategies; and (5) includes plans to monitor and  evaluate progress toward closing skill gaps and meeting other human  capital goals using a variety of appropriate metrics. We subsequently  reported in January 2015 that while the Chief Human Capital Officers  Council identified skill gaps in six government-wide occupations— including IT/cybersecurity and contract specialist/acquisition—it would be  important for lessons learned from these initial efforts to inform a new set  of skill gaps; key features of OPM’s efforts to predict emerging skill gaps  beyond those already identified were in the early planning stages; and  OPM and selected agencies could improve efforts to address skill gaps  by strengthening their use of quarterly data-driven reviews.
Further, we have reported that agencies across the federal government  have not always effectively planned for IT workforce challenges. For  example,
We recently determined that the Department of Veterans Affairs (VA)  had performed key steps such as documenting an IT human capital  strategic plan and regularly analyzing workforce data, but the  department had not tracked and reviewed historical and projected  leadership retirements and had not identified gaps in future skill  areas. We recommended that the department track and review  historical workforce data and projections related to leadership  retirements and identify IT skills needed beyond the current fiscal year  to assist in identifying  future skills gaps and it concurred with our  recommendations.
We identified that while the Federal Emergency Management Agency  had taken initial steps to assess the needs of its IT workforce, it had  not yet completed workforce planning efforts and lacked an  understanding of its regional IT workforce. For example, we noted  that while it had conducted a workforce assessment to identify skill  levels of employees in the agency’s Office of the CIO, it had not  completed recommended actions called for by this assessment. In  addition, its workforce planning efforts had not included an  assessment of the many IT staff located in the agency’s regions and  other offices. We concluded that the agency had less assurance that  its IT workforce will have the skills needed to successfully manage its  programs and recommended it establish time frames for completing  workforce planning efforts. The Department of Homeland Security  concurred with our recommendations.
In November 2015, we testified that, among other things, the U.S.  Census Bureau faced challenges in the area of workforce planning.  Specifically, we noted that while it had taken steps to develop an  enterprise-wide IT workforce planning process, as we recommended  in 2012, it had yet to fill key positions.
We have also identified human capital challenges at the Social  Security Administration (SSA). Specifically, we concluded that  SSA’s IT human capital program had identified skills and  competencies to support certain workforce needs, but lacked  adequate planning for the future. The agency had developed IT  human capital planning documents, such as an Information  Resources Management plan and skills inventory gap reports, which  identified near-term needs, such as skill sets for the following 2 years.  Nevertheless, SSA has not adequately planned for longer-term needs  because its human capital planning and analysis were not aligned  with long-term goals and objectives and the agency did not have a  current succession plan for its IT efforts. Accordingly, we  recommended that SSA identify long-term IT needs in its updated  human capital operating plan and the agency agreed.
Further, we determined that eight selected agencies had taken varied  steps to implement workforce planning practices for cybersecurity  personnel. For example, five of eight agencies, including the largest,  Defense, had established cybersecurity workforce plans or other  agency-wide activities addressing cybersecurity workforce planning.  However, all of the agencies faced challenges determining the size of  their cybersecurity workforce because of variations in how work was  defined and the lack of an occupational series specific to  cybersecurity. With respect to other workforce planning practices, all  agencies had defined roles and responsibilities for their cybersecurity  workforce, but these roles did not always align with guidelines issued  by the federal Chief Information Officers Council and National Institute  of Standards and Technology. We also noted that the robustness and  availability  of cybersecurity training and development programs varied  significantly among the agencies. We made recommendations aimed  at enhancing individual agency cybersecurity workforce planning  activities and to address government-wide cybersecurity workforce  challenges through better planning, coordination, and evaluation of  government-wide activities. The agencies agreed with the majority of  our recommendations.
Our review of the Food and Drug Administration concluded that,  among other things, the agency was not strategically managing IT  human capital—it had not determined its IT skill needs or analyzed  gaps between skills on hand and future needs. We recommended  that the agency complete key elements of IT human capital planning;  the Food and Drug Administration concurred with our  recommendation.
section_title: Three Characteristics Are Key to a Comprehensive Integrated Program Team; Shortfalls Have Contributed to Previous Acquisition Problems
paragraphs: Three key characteristics contribute to the creation and operation of  comprehensive IT acquisition IPTs: (1) effective executive leadership  through team support, empowerment, and oversight; (2) effective team  composition; and (3) effective processes for team operations. The 18 key  practices associated with these characteristics provide additional detail  regarding recommended practices for the successful implementation of  the characteristics.
subsections: section_title: Teams Should Have Support, Empowerment, and Oversight through Effective Leadership
paragraphs: The first characteristic of a strong IPT is that executive leadership,  external to the team, provides the team with support, empowerment, and  oversight. The seven key practices provide details of how executive  leadership can assist the IPT, as shown in figure 2.
There should be a common understanding between executive leadership,  the IPT program manager, and team members regarding the mission of  the IPT, its responsibilities, and the desired outcome of the program.  Individual  team members should have a shared definition of success, and  should be encouraged to prioritize team-wide goals over personal goals.  Agency officials highlighted the need to establish and maintain a shared  vision and goals. Specifically:
General Services Administration (GSA) officials told us that IPTs do  not set their own goals. The project is sponsored by management,  and the vision and goals should be communicated by leadership. The  IPT’s job is to develop a strategy to achieve the vision and goals. GSA  officials added that these teams should communicate the vision and  goals of the program during the solicitation phase. During the  performance phase, vision and goals should be aligned with service  level agreements.
Officials from the National Oceanic and Atmospheric Administration’s  (NOAA) Office of the CIO noted that the IPT should communicate its  vision and goals with its stakeholders for their review and approval  prior to the project commencement.
Officials with the Department of Commerce’s (Commerce) Office of  the CIO stated it is important to ensure the team is aware of the  shared vision and goals and that the team works with the sponsor,  stakeholders, and other management to ensure the vision is shared  and understood by all involved.
Representatives of NOAA’s Geostationary Operational Environmental  Satellite - R series program reported that team members need to be  able to perform tasks that are best for the IPT and the project,  regardless of whether those tasks were not originally assigned to  them. For example, members need to break out of the mentality of  being an “IT person” who does not need to assist with non-IT related  tasks. Regardless of what expertise the member holds, he or she  needs to work towards the mission of the project.
Representatives from the VA Veterans Benefits Management System  added that they work with partnering and parent organizations to  ensure decisions related to the functionality are aligned with agency  priorities.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology explained that an IPT should be  committed to a common purpose. Work products such as the project  charter and program management plan can be used to communicate  the IPT’s vision and goals.
subsections: section_title: Serve as an Advocate for the Team
paragraphs: Executive leadership should serve as an active promoter for the IPT and  its success. This focus can be demonstrated by leadership effectively and  enthusiastically communicating the vision and goals to outsiders,  collaborating with other programs upon which the team is dependent, and  managing the impact of challenges and opportunities from the external  environment.
For example, representatives from the VA Veterans Benefits  Management System stated that a major underlying contributor to the  progress of the program has been unwavering support from executive  leadership since the project’s inception. The IPT staff explained that  consistent oversight and executive involvement has had a positive and  direct impact on the project’s overall probability of success by removing  roadblocks, mitigating risks, and prioritizing funding.
section_title: Motivate Team Members for Success
paragraphs: Executive leadership should create an environment for the IPT that  facilitates success. This can include things such as team building  experiences, opportunities for career growth through challenges,  feedback and support, and recognition and rewards for good  performance. For example, key guidance from MITRE Corporation  recommends that periodic opportunities be provided for team members to  brief their respective executives on aspects of their work. This may  provide individual members with valuable recognition from high levels  within their own areas as well as feedback to the IPT from senior  management. The guidance also discusses the importance of  incentivizing individual team members by tying their performance  appraisals and rewards to team performance. Officials with two of the  agencies in our review also mentioned the value of motiving team  members:
Officials with Commerce’s Office of the CIO stated that a desirable  IPT characteristic includes teaming. Teaming requires a true peer  environment where team members are mutually accountable for  outcomes and consensus building is essential.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center noted that successful  IPTs have some level of social interaction outside of professional  duties to smooth out communication barriers and build rapport.
section_title: Empower Team to Act
paragraphs: The people assigned to an IPT should have the appropriate authority to  perform their assigned responsibilities. The boundaries of responsibility  and authority between the team and executive leadership should be  established early and documented. Executive leadership may honor  these agreements by allowing the IPT to select its members, coordinate  and implement decisions, take prudent risks, and exercise full day-to-day  responsibilities for delivering the product. An IPT should have the  authority to make decisions and not just serve as a mechanism for  communicating with stakeholders. Several agency officials also discussed  the benefits of empowering teams. Specifically:
Officials from the NOAA Office of the CIO stated that the extent of the  IPT’s authority should be set by the executive sponsors based on the  importance of the program. A risk-based approach should be used to  identify the threshold of decision authority.
According to officials with Commerce’s Office of the CIO, the IPT  should be able to make most decisions as long as it does not over-run  the budget, schedule, or scope. Empowerment is critical to making  and keeping the agreements essential to effective teams.
Representatives of NOAA’s Geostationary Operational Environmental  Satellite - R series program noted that executive leadership should  only help an IPT when appropriate, otherwise too strong of a hand  from leadership can result in micromanagement. These  representatives added that IPTs should have enough decision  authority to push forward the project. Further, they should not have to  constantly go up the management chain to make decisions, as this  would be a clear indicator that the team’s authority is non-existent.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center explained that the level  of decision making authority granted to an IPT depends on the level of  resources, experience, human capital, and organization. It is  important to hire quality resources, empower them to use critical  thinking and judgment, and support their efforts. Controls should be  implemented when these conditions are not present.
Officials with the VA Technology Acquisition Center and the Office of  Information and Technology told us that, in providing the best value to  the VA customer, the IPT should have the authority to decide on  requirements and the corresponding acquisition milestones and  process to accomplish effective lifecycle acquisition, contract, and  performance management.
According to officials from Defense’s Office of the Under Secretary of  Defense for Acquisition, Technology, and Logistics and the Office of  the CIO, a desirable characteristic of an IPT is to have empowered  team members.
section_title: Provide Sufficient Resources
paragraphs: Executive leaders outside of the IPT can help to ensure its success by  providing the resources necessary to do the assigned work. Resources  include funding to ensure the team has adequate staffing; physical  facilities; security; and IT tools such as environments, software, hardware,  and networks. In that regard, it is vital to have leadership support for the  IPT lead and team members’ roles and time commitments. Having this  support will help enable the people assigned to the IPT be able to devote  the necessary time and effort to make the program successful. Agency  officials highlighted the importance of providing the necessary resources.  Specifically:
Officials from NOAA’s Office of the CIO stated that each team  member should have the time to devote to the IPT without distraction.
Representatives of NOAA’s Geostationary Operational Environmental  Satellite - R series program explained that leadership should have the  appropriate understanding and awareness of what resources are  needed, such as a sufficient budget and skilled staff members. If  problems arise, senior executives need the ability to bring in additional  resources that help the program. IPTs should have access to a  resource pool of needed expertise or talent.
According to officials from the VA Office of Information and  Technology, it is critical that team members are committed and  provide the necessary time for the effort so that project plans are  fulfilled, consistent with legitimate constraints.
Officials with the VA Technology Acquisition Center noted that  members need to be committed and provide the necessary time to the  IPT.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center told us that it is critical  to have access to resources, such as budget and technology, to track  the program’s schedule and deliverables. In addition, leadership  support and flexibility are needed when staffing a team.
section_title: Provide Meaningful Oversight
paragraphs: Executive leadership outside of the IPT should periodically evaluate its  structure, operations, and results. Another important role of leadership  oversight is to ensure that the shared vision and goals are consistently  pursued throughout the program lifecycle. Conducting such evaluations  can result in the clarification of roles and responsibilities and constructive  feedback to team members, the discovery of unrecognized or unresolved  issues, and the establishment or modification of program goals for the  future. Members should be held accountable by leadership for their  individual performance as well as overall program success. Agency  officials highlighted the need to provide the IPT with meaningful oversight.  Specifically:
Officials with NOAA’s Office of the CIO stated that executive  managers from three offices (acquisitions, IT, and the customer  organization) work together to approve the IPT and provide oversight  to hold it accountable.
According to VA Veterans Benefits Management System officials,  IPTs at the VA were held accountable through the department’s  Project Management Accountability System, which required that IT  projects deliver customer-facing functionality every 6 months.  Through the project management process, milestones are to be  monitored regularly and communicated to appropriate stakeholders.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology stated that IPTs within the  department are subject to regular reviews with senior leadership  where metrics are used to monitor progress and make adjustments as  appropriate to ensure success. Representatives from the VA’s Office  of the CIO added that projects that miss their delivery date are to be  reviewed by a senior leader panel to identify root causes and lessons  learned.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center indicated that the IPT is  coordinated by a sector director, group manager, and acquisition  group manager based on client needs and resource availability.  These three leadership stakeholders are involved in the gate review  process and provide oversight. Projects are tracked using activity  based costing. The budget is set by the group manager, and each  employee tracks their time and cost against the budget. This activity  drives efficiency on the government side. Similarly, every project  employs a variation of project tracking using service level agreements,  deliverables, and quality assurance surveillance.
section_title: Resolve Critical Issues
paragraphs: IPT leadership, with sufficient autonomy, should be able to handle many  issues and conflicts on their own. However, when an issue arises that  threatens to derail the project and it cannot be handled within the IPT,  executive leadership must be available and involved enough to quickly  and effectively resolve the issue.
For example, officials from NOAA’s Geostationary Operational  Environmental Satellite - R series program explained that when an issue  cannot be resolved within the IPT, the program manager should raise the  issue as quickly as possible to a decision-making level where resolution  can be achieved. In addition, executive sponsors help projects by  negotiating and addressing concerns of stakeholders not directly  assigned to the team.
section_title: Team Composition Should Be Highly Effective
paragraphs: The second key characteristic of a comprehensive IPT for major IT  acquisitions is to establish a highly effective team. The five key practices  contribute to the overall success of this characteristic, as shown in   figure 3.
Based on the program’s requirements, agencies need to develop a  human resource management plan that identifies the skills needed to  complete the work. Agencies then need to determine the optimal size of  the core IPT team and availability  of other supplementary staff. In some  situations, subject matter experts may be assigned to work full time on  the team, or they may rotate in and out based on the need for their  expertise in a given phase. Agency officials with whom we spoke  mentioned the importance of determining the required skills, optimal size,  and availability. Specifically:
According to officials with Commerce’s Office of the CIO, the  assignment of core and supplementary staff and subject matter  experts depends on the work needed to be completed.
Officials from NOAA’s Office of the CIO also explained that the  number and composition of team members should be commensurate  with the complexity and the scope of the task.
Representatives from NOAA’s Geostationary Operational  Environmental Satellite - R series program told us that for larger  projects, staff should be changed at specific phases of the program,  such as when the program transitions from development to  operations.
Officials from Defense’s Office of the Under Secretary of Acquisition,  Technology, and Logistics and the Office of the CIO explained that the  program manager, stakeholders, and customers should be able to  assign any staff member who would be of value and assistance. Once  the effort is complete, the assigned staff member can be released  from the IPT based on the program manager’s assessment that the  resource is no longer needed. These officials added that in addition to  the program’s tasks, functional capabilities, and lifecycle phase, the  size of the IPT will vary based on the acquisition category of the  program.
Representatives from VA’s Veterans Benefits Management System  noted that supplementary staff, such as end users and subject matter  experts, can become a part of the core IPT during times that  necessitate a specific level of expertise. These individuals can return  to their regular duties once they have fulfilled their roles.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology stated that it is important to  ensure adequate representation from all major stakeholder  organizations, but also to safeguard that the IPT does not have too  many members.
According to representatives from the VA Technology Acquisition  Center and the Office of Information and Technology, different  members have different levels of engagement depending on the  scope of the IPT and the lifecycle phase of the program. Staff may be  added later or required to participate more frequently depending on  the needs of the program.
subsections: section_title: Identify Critical Subject Matter Roles and Responsibilities
paragraphs: Each program must determine the appropriate composition of its IPT.  Some roles and responsibilities are considered “core,” meaning that each  team must have these staff. For example, in order for OMB to approve  the IT investment, an IPT must at a minimum include: (1) a fully dedicated  IT program manager; (2) a contracting specialist (if applicable); (3) an IT  specialist; (4) an IT security specialist; and (5) a business process owner  or a subject matter expert. Other roles and responsibilities are  considered “supplementary,” meaning that they can be added as  necessary during certain phases of the lifecycle.
Our review of key guidance documentation and interviews with agency  officials indicates that an IPT for a major IT acquisition should include a  number of core disciplines. Although the descriptions of the disciplines  are more detailed than those required by OMB, they are still compatible  with OMB’s five categories. For example, the contracting, procurement,  and acquisition professionals could fit into OMB’s category of contracting  specialist. Similarly, the disciplines of software developers, test and  quality assurance managers, engineers, and architects could fit into  OMB’s category of IT specialist.
Officials from the eight organizations that we identified for our sample  were largely in agreement with these roles and responsibilities. For  example, they all told us that contracting, procurement, and acquisition  professionals were critical for an IPT. In addition, six organizations told us  that that security and privacy specialists and requirements managers  should be team members. At least four of the organizations also  mentioned customers and users, budget and finance managers, software  developers, test and quality assurance managers, engineers, risk  managers, and sponsors.
The core disciplines are shown in figure 4 by descending order in which  they were cited in literature and by selected organizations, and discussed  in more detail following the figure.
Contracting, procurement, and acquisition professionals
Contracting officers, who, according to OMB, evaluate technical  competency of contractors and establish acquisition strategies to  achieve the best value for taxpayer dollars. They and their support  staff analyze proposals, provide independent cost estimates, and  develop negotiation positions. They legally obligate the government  when they enter into contracts.
Contracting officer’s representatives, who manage ongoing contractor  performance to ensure the government’s interests are protected. They  are designated and authorized in writing by the contracting officer to  perform specific technical or administrative functions on contracts or  orders.
Purchasing agents and procurement specialists, who assist  contracting professionals with a wide range of duties, such as market  research and file preparation, tracking procurements, and preparing  awards for simple acquisitions.
Contract specialists, who, according to OMB, support the contracting  officers in accomplishing their many duties during the pre- and post- award acquisition phases. Contract specialists may also perform the  role of contracting officers for small dollar value procurements.
At the hub of an effective IPT is a strong program manager who stewards  the process from beginning to end while achieving the program’s cost,  schedule, and performance goals. The program manager could come  from either the technology or mission organization, so long as the person  possesses skills in both areas and operates under a strong governance  process. Further, he or she should ideally be highly proficient at technical,  business (both government and commercial business processes),  organizational, programmatic, and interpersonal levels.
Members of the organization who will approve, accept, and ultimately use  the deliverables or products of the project on a day-to-day basis should  be assigned to the IPT to ensure proper coordination; advise on  requirements including business policies, rules, and processes; and  validate the acceptability of the project’s results.
Personnel from budget and finance are necessary to secure the required  funding for the program. Their involvement during the acquisition process  can help in the development of the acquisition strategy and limit potential  problems related to financial and budgetary issues.
An IPT should include government employees who are experts in system  security. In addition, the sustained engagement of privacy specialists  helps ensure that personal data is properly managed.
Members of the team should have experience with traditional and modern  development techniques such as Agile software development concepts  and modular approaches.
The team should have a requirements manager who understands the  lifecycle of managing requirements starting with elicitation through the  requirements change management process to test and evaluation.
Test and quality assurance managers  The test manager brings a solid end-to-end view of the testing process,  including test management and the use of automated testing frameworks.  Quality assurance personnel ensure consistency and quality  documentation.
Engineers translate operational needs and requirements into a set of  system product and process solutions that satisfy customer needs.
Teams benefit from having IT architects who can develop a solution that  will interoperate with the agency’s internal and external systems.
IPTs should include risk managers who are knowledgeable in the  processes of risk management. This includes risk identification, analysis,  and response planning that may decrease the likelihood and impact of  negative events in the program.
Each IPT should include the sponsor, who is the executive-level person  or group who provides resources and support for the program and who is  ultimately responsible and accountable for enabling its success.
In addition to the core disciplines, other supplementary staff members  could be added to the IPT on an as-needed basis. These include  positions such as:
Administrative personnel. These personnel are responsible for  activities such as document management, communications, outreach,  and meeting facilitation and recording.
Organizational change managers. These managers execute  strategies and techniques required for effectively planning,  implementing, and evaluating change in the organization.
Configuration managers. These managers apply principles and  methods for planning and managing the implementation, update, and  integration of information systems components.
Earned value management specialists. These specialists can combine  scope, schedule, and resource measurements to help the program  team assess and measure project performance and progress.
Human resources representatives. Human resources representatives  are involved in (1) identifying and documenting project roles,  responsibilities, required skills, and reporting relationships; (2)  confirming human resource availability;  (3) improving competencies,  team member interaction, and team environments; and (4) tracking  team member performance, providing feedback, and resolving issues.
Governance managers. Governance managers are IT officials  involved with strategic capital planning and investment control  processes.
Schedulers. Schedulers plan, develop, manage, execute, and control  the program schedule in order to ensure the timely completion of the  project.
Designers. Designers have core knowledge of sustainable design.
Legal counsel. Counsel should have the expertise to assist with  acquisition-related and legal issues such as the award of contracts,  intellectual property rights, licenses, and other matters.
Operational support personnel. These personnel are responsible for  activities such as system operations, implementation/deployment  management, software maintenance, training, customer service, and  system performance. They can be from external organizations that  have a relationship with the program to provide specialized expertise  or fill a specified role, such as IT system installation, customization, or  training.
section_title: Establish a Cross-Functional, Multidisciplinary Team
paragraphs: Ideally,  the team will not only consist of members from a wide selection of  subject matter disciplines, but individual members will also have a variety  of technical skills and expertise. In addition, important soft skill sets may  also be considered. These include skills such as problem solving and  decision-making, interpersonal skills, ability to communicate and work  with customers, team building, and conflict resolution. One of the more  important soft skills is the ability to deal with adversity and ever changing  circumstances. Specifically for the program manager, important selection  criteria include: (1) lack of bias, (2) technical expertise, (3) project  management skills, (4) ability to manage external environment, (5) team  engagement skills, (6) decisiveness, (7) time management skills, (8)  ability to effectively elevate and delegate decisions, and (9) commitment  to the IPT’s work. Several agency officials discussed the value of both  technical and non-technical attributes. Specifically:
Officials with Commerce’s Office of the CIO explained that a desirable  characteristic of an IPT is integration, which is embodied by bringing  together different organizations that have a stake in the program,  each providing differing perspectives and skills which are vital to the  successful outcome. Each team should possess the knowledge to  collaboratively identify problems and propose solutions, minimizing  the amount of rework that has to be done.
According to officials from NOAA’s Office of the CIO, members should  be capable of providing input regarding multiple disciplines, not just  their area of primary responsibility. They added that on a  multidisciplinary team often the best team members are individuals  who have multiple types of expertise.
Representatives from NOAA’s Geostationary Operational  Environmental Satellite - R series program stated that IPTs should  have personnel with specialized skill sets to bridge any gaps in the  program. They should also have cross-functional skill sets that enable  effective communication with managers inside and outside of their  primary knowledge domain.
GSA Assisted Acquisition Services, Federal Systems Integration and  Management Center officials explained that they have created a  structure that integrates the acquisition workforce with the technical  resources under the same management. In addition, they have  developed a role called the acquisition project manager, which is a  hybrid position of technical expertise and acquisition knowledge that is  present for pre-award activities such as technical evaluations. The  representatives added that the integration of acquisition and technical  resources, sharing the same management, leads to more efficiency  and alignment with the mission.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology told us that it is desirable for IPT  members to have the necessary acquisition, business, technical and  programmatic knowledge, as well as the expertise to facilitate and  leverage the process to help define user needs, facilitate discussions,  make recommendations, resolve issues, and make decisions.
VA Veterans Benefits Management System officials explained that the  most critical characteristic of a comprehensive IPT is a joint team  consisting of IT resources and business resources, where the IT staff  understands the business needs of the users and the users have the  technical experience to assess alternative IT solutions and weigh the  tradeoffs of each. Officials told us that they attempted to ensure  frequent collaboration between business and IT team members during  key phases of projects—from planning to deployment. They stated  that this practice was essential to the delivery of functionality to end- users.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology stated that desirable non- technical skills of IPT members include problem solving and decision- making, team building, interpersonal skills, and conflict resolution.
Officials with Commerce’s Office of the CIO noted that adaptability is  a desirable characteristic of an IPT member.
Representatives from NOAA’s Geostationary Operational  Environmental Satellite - R series program said that IPT members  need to be flexible and open to change when needed to address  team-wide priorities.
section_title: Provide Necessary Training
paragraphs: The IPT needs to ensure that its members have received two types of  training: (1) training related to their specific roles and (2) training related  to their involvement in an IPT in general. The first type of training involves  measuring the skills and knowledge of the staff in relation to their roles.  Based on this assessment, the team should provide role-specific training  as necessary to support the execution of the program. Providing  cross-training to team members can reduce problems which arise during  member absences. For the second type of training, the team should  provide its members with IPT-specific training to create the knowledge  and culture necessary to ensure its success. This can include training on  items such as (1) the purpose of an IPT, (2) why an IPT is being used in  this case, (3) how an IPT needs to function, and (4) behavior and skills  required of IPT leaders and members. This training will require the use of  additional resources upfront, but it can result in greater team  effectiveness and efficiencies later. Officials from Commerce mentioned  the value in ensuring appropriate training. Specifically:
Officials from NOAA’s Office of the CIO told us that in addition to  having knowledge regarding the mission subject, team members need  the proper training in IPT framework and discipline.
Representatives from Commerce’s Office of the CIO explained that  training will assist with ensuring all of the team members are on board  with the new program, especially if they are being taken from their  normal daily duties.
section_title: Maintain a Consistent and Stable Team from Beginning to End
paragraphs: The core members of the team, including all leadership roles, should be  in place throughout the program lifecycle. IPTs should be in place from  the initial concept and development phases, through the delivery and  implementation of the last increment under the contract. The team lead  should be 100 percent dedicated to the program during this time frame.  However, there may be instances where other team member services are  not needed on a full-time basis. In those cases, team member support to  the IPT, when needed, should take priority over the team member’s other  duties. It is also ideal to have contractor staff who are stable and  consistent. Having continuity of government and contractor staff working  on a project throughout its lifecycle reinforces accountability. Agency  officials discussed the importance of maintaining stability in the team.  Specifically:
Representatives from the VA Veterans Benefits Management System  and NOAA’s Geostationary Operational Environmental Satellite - R  series program stated that core staff should be assigned to the IPT for  the duration of the program.
According to representatives from the VA Technology Acquisition  Center and the Office of Information and Technology, members  should be available for the duration of the IPT, adding that  commitment and regular and consistent participation are critical to the  success of the team.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center noted IPT members  should be assigned as soon as the requirements are known. Each  member should have a clear expectation for the level of involvement,  time frame for participation, and response at the conclusion of the  phase. They also stated that roles and responsibilities of team  members should be revisited at each phase of the acquisition.
Representatives from Commerce’s Office of the CIO noted that it is  desirable that backup representatives on the IPT be well apprised of  the team’s agenda, issues, and decisions.
section_title: Teams Should Operate Using Effective Processes
paragraphs: The third and final key characteristic is to operate the team using effective  processes. As shown in figure 5, six key practices provide additional  detail for the successful implementation of this characteristic.
The IPT should have a charter that documents its need, purpose, and  scope. The charter should further clarify and elaborate goals, outcomes,  and performance measures. In addition, it should be reviewed by, and  receive concurrence from, executive leadership. It should also be updated  as necessary, but at least at the start of each work phase. Agency  officials mentioned the value of documenting the team’s charter.  Specifically:
Officials with Commerce’s Office of the CIO stated that the charter  should be established early on in the planning process, and it should  include (1) the expectations of the program team, (2) the scope of the  IPT’s authority, (3) the metrics by which the success of the IPT will be  evaluated, (4) the identification of the customers, (5) the amount and  types of funds available to the IPT, and (6) the expertise the team  must have.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology told us that the charter should  document the roles and responsibilities of each team member.
According to officials with Defense’s Office of the Under Secretary of  Acquisition, Technology, and Logistics and the Office of the CIO, the  charter should document the customers, end users, and other  stakeholders who have a vested interest in the outcome of the IPT.
subsections: section_title: Establish Team Guidelines
paragraphs: For effective operations, it is important to create and document  IPT-specific decision-making processes that align with the organization’s  policies and procedures. These include rules and guidelines for things  such as: (1) team structuring, (2) formal lines of communication, (3)  establishment of authority, (4) resource allocation, (5) work review and  approval, and (6) reporting requirements. Several agency officials  discussed the importance of providing the IPT with rules and guidelines.  Specifically:
Officials from NOAA’s Office of the CIO explained that the team  should establish the mission criteria, schedule, and performance  measures before the program begins. The IPT should report on the  status at the expected intervals using monthly written status reports  and quarterly project reviews.
Representatives of NOAA’s Geostationary Operational Environmental  Satellite - R series program told us that team members should  understand their roles and responsibilities so they know what is  required of them and the value they bring to the team.
According to representatives from the VA Veterans Benefits  Management System, they document roles and responsibilities,  including an organizational chart and a responsibility assignment  matrix.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology told us that each member needs  to have his or her role defined based on expertise.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center stated that an IPT  should have defined roles and responsibilities, as well as consistent,  mature processes. To that end, they have established standard  operating procedures, training, and templates for organizations to use.  These include (1) a roles and responsibilities matrix template to  document the structure, roles and responsibilities, and operating  procedures; (2) a project management tool to document the  acquisition schedule and to provide a quick overview of dates for  major milestones; and (3) an acquisition start meeting template to  document the roles, responsibilities, existing information, acquisition  schedule, procurement integrity briefing, project administration, and  next steps.
Defense’s Office of the Under Secretary of Acquisition, Technology,  and Logistics and the Office of the CIO officials stated that each IPT’s  structure should be individualized based on the program’s task,  acquisition, type of capability, technology, operational mission, and  other various factors.
section_title: Facilitate Teamwork through the Use of Co-Location and Virtual Teams
paragraphs: Co-location places many or all of the most active team members in the  same physical location. The benefits of co-location can include easier  information sharing and communication, earlier identification of issues,  and more effective decision making. At a minimum, the IPT lead and the  contracting officer should be co-located. However, co-location is not  always practical. Another option is temporary co-location—bringing staff  members together on a limited basis during strategically important times.  For example, an especially important time is requirements definition,  which is an area where translation issues between business users and  developers can cause problems if requirements are not clearly  articulated, understood, and documented. As an alternative, effective  virtual teams can be created through the use of shared networks,  software, tools, databases, and teleconferencing, which allows team  members from different locations to work together regardless of the  physical location. According to recognized guidance, the benefits of  virtual teams include the addition of skilled resources, reduced travel and  re-location costs, and the proximity of team members to other critical  resources. Agency officials highlighted the need to make trade-offs to  facilitate teamwork. Specifically:
Officials with Commerce’s Office of the CIO explained that IPTs are  not periodic meetings; rather they are daily work routines where  members must work closely with their teammates and the program  manager to produce a viable product as defined by business  requirements. As such, co-location of all members is preferable as it  facilitates frequent and timely discussions, problem-solving, and  decision-making. NOAA demonstrated commitment to this approach  with its Geostationary Operational Environmental Satellite – R series  program. Although the program is overseen internally within  Commerce, it relies on acquisition experience and technical expertise  supplied by the National Aeronautics and Space Administration. In  order to leverage this knowledge, NOAA implemented an integrated  program management structure and located the program office at the  National Aeronautics and Space Administration’s Goddard Space  Flight Center. Officials with the Geostationary Operational  Environmental Satellite – R series program told us that bringing team  members to the Goddard Space Flight Center was advantageous  because the co-location ensured that engineering resources were  available, if needed.
Representatives from the VA Veterans Benefits Management System  told us that they use a mix of virtual teams and co-location as  appropriate. For example, temporary co-location often occurs during  requirements elicitation, system design, and user acceptance testing.
Officials from the Defense’s Office of the Under Secretary of  Acquisition, Technology, and Logistics and the Office of the CIO noted  that co-locating team members is desirable but not mandatory in  today’s virtual world. The use of virtual technology provides a means  for high success in meeting the goals and objectives of the IPT  without all members being co-located. For example, the use of virtual  technologies such as teleconferencing, portals, instant messaging,  and e-mails may help ensure requirements are clearly articulated and  adequate oversight of the contracted work is accomplished.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center told us that geography  matters and there is no substitute for face-to-face communication.  Ideally the program manager should work in the same location as the  customer at least 2 days a week to provide the best customer service  possible. However, officials added that they have examples of  successful IPTs where the team is not co-located. In those cases, the  use of virtual tools such as telephone, e-mail, online documents, video  chat, web messaging, and virtual meetings assisted in ensuring a  seamless acquisition process.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology stated that although they  consider co-location to be optimal, it is not always practical. As a  result, they often use on-line tools to facilitate effective virtual  meetings.
section_title: Involve Stakeholders as Active Team Members
paragraphs: Stakeholders are individuals or groups affected by the outcome of a  project, and may or may not be assigned to the core team. Either way,  stakeholder involvement should be viewed as a valuable asset to the  team, and stakeholder satisfaction should be seen as a key project  objective. Stakeholders should represent all relevant acquisition,  technical, support, and operational organizations. They are working team  members and should be included in activities such as planning, decision- making, commitments, communication, coordination, reviews, appraisals,  requirements definition, and resolution of problems and issues. Reviews  and exchanges are regularly conducted with stakeholders to ensure that  coordination issues receive appropriate attention and everyone involved  with the project is appropriately aware of status, plans, and activities. The  involvement of stakeholders is especially important in Agile development,  which emphasizes collaboration more than traditional approaches do. For  example, in Agile development, project status is primarily evaluated  based on demonstrations of working software provided to stakeholders  and customers. Agency officials with whom we spoke discussed the  value of actively involving  stakeholders. Specifically:
Officials with Commerce’s Office of the CIO stated that ensuring the  stakeholders were supportive from the beginning to the end of the  program was a critical success factor.
Representatives with NOAA’s Office of the CIO noted that the IPT  should engage with internal stakeholders through weekly or monthly  status reports, monthly or quarterly interim project reviews, and  milestone briefings.
Representatives from the VA Veterans Benefits Management System  told us that engaging with stakeholders requires ongoing collaboration  and open communication to ensure transparency and the optimization  of inputs and ideas. They also explained that the program solicited  users’ input on requirements for the development and enhancement  of the system. Field subject matter experts from across the country  participated in requirements, design, and testing sessions with system  developers.
Representatives from the VA Technology Acquisition Center stated  that IPTs should communicate with internal and external stakeholders  regarding the status and progress of the project.
According to officials in VA’s Office of Information and Technology,  the appropriate representation within the team ensures that all  stakeholders are aware of and committed to completing key project  deliverables on time. This approach enables and requires coordinated  teamwork from multiple organizations to be a key driver of the  project’s daily performance. In addition, they noted that the customer  community is involved in several ways throughout the project lifecycle.  For example, the business sponsor is a representative and advocate  for the customer community. The customer community is involved  throughout the lifecycle of the project, and makes the final  determination of acceptability in user testing to accept delivery of  capabilities.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center told us that IPTs should  engage with internal and external stakeholders as often as feasible.  They explained that it is important for the team to fully understand the  political landscape and be able to identify all critical stakeholders early  in the process. Subsequently, teams should engage with stakeholders  by keeping key players involved in the process to ensure they are  working toward meeting the mission and strategy goals.
Officials from Defense’s Office of the Under Secretary of Acquisition,  Technology, and Logistics and the Office of the CIO stated that teams  should engage with stakeholders on a regular basis and as frequently  as possible.
section_title: Emphasize the Importance of Regular and Effective Communication
paragraphs: Ideally,  the amount of communication and the manner in which it is  documented would be tailored to fit the needs of the team. For example,  teams may establish regularly scheduled meetings. Records from the  meetings can be maintained in a form that allows for ready access at a  later date. In addition, teams may establish methods to identify and  resolve conflicts that arise among stakeholders. According to recognized  guidance, IPTs need to be particularly sensitive to the internal dynamics  and style of interaction between members both during and outside of  meetings. It can be helpful to have ground rules about how to  respectfully share and acknowledge different points of view, ensure  balanced participation from all members, build trust and collegiality, deal  with conflict, and maintain morale and team spirit. The use of a facilitator  during meetings could be a valuable tool in effective communications.  Agency officials with whom we spoke mentioned the importance of  communication. Specifically:
Officials from Commerce’s Office of the CIO stated that  communications should be tailored to the technical level of the  audience. One effective method of communication is the use of a  program management tool to allow team members to access any  information at any time. Communications should be two-way; there  should be a way for the stakeholders to provide feedback to the IPT.
Officials with NOAA’s Office of the CIO told us that the frequency of  communication should be commensurate with the scope and  complexity of the milestone or urgency of the pace of tasks.
Representatives from the VA Veterans Benefits Management System  explained that they have found that frequent and structured meetings  are the most effective way of communicating progress and addressing  issues. The IPT participated in daily and weekly meetings to discuss  key topics, risks, and issues. In addition, functionality-specific  meetings occur on an as-needed basis to ensure continued  collaboration. Communication could be facilitated by an intranet site, a  suite of communication materials, downloadable toolkits, factsheets,  and explanatory videos designed to build awareness and provide  necessary information for customers and end-users. Officials reported  that information sharing and coordination with other VA organizations  ensures awareness and understanding of changes prior to  implementation. In addition, engaging stakeholders across multiple  vehicles on a recurring basis was integral to the responsive and  iterative approach of the team.
Representatives from the VA Technology Acquisition Center and the  Office of Information and Technology stated that they communicate  progress and issues by meeting regularly (e.g., weekly/monthly),  documenting results and issues in meeting minutes, conducting  formal milestone reviews, and escalating issues as necessary.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center told us that after an  initial acquisition kickoff meeting, the team works together to set up  standing weekly (or more frequent depending on the project)  meetings. During these meetings, they review schedules, plans, and  deliverables with each other and the client. The IPT should be  communicating often to ensure members are always working towards  the same goal. Once a week the program manager updates the  acquisition schedule so that leadership has the most up-to-date  information. Once a systemic process exists for schedule  communication, management can more easily identify outliers and  focus on addressing problem areas. During the various meetings,  team members must have the opportunity to voice dissent. Once  decisions are made, the team must reach buy-in and proceed.
According to officials from Defense’s Office of the Under Secretary of  Acquisition, Technology, and Logistics and the Office of the CIO, the  IPT should have regularly scheduled meetings, at least monthly, or  more frequently if software is being deployed more often. Meetings  are intended to bring together the program office, contractors,  technical expertise, stakeholders, customers, and others to view the  development, progress, and issues related to acquisition of the  capability. In addition, there should be opportunities to discuss  matters between meetings. There should be open discussions with no  secrets.
section_title: Ensure the Appropriate Use of Contractor Assistance
paragraphs: The IPT is often assigned the responsibility to oversee the performance  and acceptance of contractors’ deliverables or services. If the contractors  bear a large share of the risk for delivering the project’s results, they may  play a significant role on the project team. The goal is to ensure that  government personnel maintain control over key decisions and  expenditure of funds. Agency officials with whom we spoke highlighted  the importance of effectively overseeing contractors’ activities.  Specifically:
Officials from NOAA’s Office of the CIO stated that an IPT can benefit  from contractors serving as administrative staff and subject matter  experts. However, the number of government team members should  be equal to or greater than the number of contractors. Contractor staff  should not be making important business decisions, and government  staff should review and approve recommendations made by  contractors.
Officials with Commerce’s Office of the CIO told us that the ratio of  government staff to contractor staff should not matter as long as the  contractor is doing the scoped work, maintaining its metrics, being  held to specific performance standards, and overseen by the  contracting officer’s representative. However, they noted that IPTs  should try to keep a ratio of government to contractor staff that  ensures enough government participation so that decisions, changes,  and approvals, can be executed in an efficient manner, and the  government maintains accountability and traceability.
Officials with the VA Technology Acquisition Center and the Office of  Information and Technology stated that contractors can serve in  support functions such as document preparation and meeting  facilitation. However, any inherently government function that requires  decision-making should not be performed by contractors.  Representatives from the VA Technology Acquisition Center and the  Office of the CIO also agreed that that the majority of personnel on an  IPT should be government employees. In addition, officials noted that  any poor performance by contractors can be documented in  performance reviews and can lead to corrective actions, or if  necessary, contract termination. Finally, according to VA officials,  contractor personnel should sign non-disclosure agreements.
Representatives from the VA Veterans Benefits Management System  noted that contractors can complement the IPT’s effectiveness by  providing experiences or resources in areas such as systems  engineering, software development, testing, requirements analysis,  communication, change management, business architecture, and  program management. However, the team members need to follow  the roles, rules, and guidance set forth by the Federal Acquisition  Regulation. Specifically, the government staff are to perform and  complete inherently government tasks and contractors only perform  work based on the statement of work for which they have an approved  contract. In addition, VA officials noted that each contract is to have  a dedicated contracting officer representative responsible for ensuring  contract deliverables meet quality and schedule parameters  established by the government.
Representatives from GSA’s Assisted Acquisition Services, Federal  Systems Integration and Management Center explained that it is  important to document roles, responsibilities, and expectations of any  contractor team member. According to these officials, the contracting  officer, contract specialist, and program manager should be  government employees, not contractor employees. However, these  officials stated that functions such as assisting the program manager  with financial management, tracking schedules, preparing  deliverables, creating an operational framework, and serving as a  technical writer are appropriate uses of contractor resources.
section_title: Shortfalls in Building Strong IPTs Have Contributed to Significant Problems in Major IT Acquisitions
paragraphs: The presence—or absence—of these IPT characteristics can have an  impact on the success or failure of a program. We have previously  reported on weaknesses in practices related to each of three  characteristics (effective leadership, team composition, and processes)  that have adversely impacted major IT acquisitions.
Shortfalls in leadership. Our recent review of the Department of  Homeland Security’s Human Resources Information Technology  investment revealed a lack of oversight from the department’s executive  steering committee. For example, the executive steering committee met  only once from September 2013 through June 2015—in July 2014—and  was minimally involved with overseeing the investment’s 15 key areas  identified as needing improvement during that period. As a result of the  executive steering committee not meeting, important governance  activities were not completed, including review of the investment’s key  strategic planning document. We reported that this this lack of  involvement contributed to Department of Homeland Security being able  to deliver only 1 of 15 human resources capabilities which the department  said would be mostly implemented by June 2015. Until the executive  steering committee effectively carries out its oversight responsibility, the  department will be limited in its ability to improve human resource  investments which are needed to carry out its mission.
Shortfalls in team composition. Our previous work on efforts of VA to  share electronic health records with Defense highlights the importance of  having sufficient staff on the IPT. Specifically, in 2009 the two  departments set up an Interagency Program Office to oversee the  development of fully interoperable electronic health records. The office was given the responsibility for oversight and management, stakeholder  communication, and decision-making. Specific tasks included the  development of a plan, schedule, and performance measures to guide the  departments’ interoperability efforts. In February 2011, we reported that  the Interagency Program Office had not fulfilled some of its major  responsibilities. The office was subsequently re-chartered and given more  staff to complete its work. Specifically, officials with the Interagency  Program Office noted that it was authorized to eventually have a staff of  236 personnel, more than 7 times the number originally allotted.  However, the officials stated that as of January 2013, the office was  staffed at only 62 percent—146 personnel. Officials stated that hiring  additional staff remained one of its biggest challenges.
As another example, our ongoing work regarding the U.S. Census  Bureau’s plans to conduct the 2020 Census has demonstrated the need  to provide its staff training related to program roles and responsibilities.  Specifically, in 2013 the Census Bureau measured the skills and  knowledge of the staff by conducting a workforce competency  assessment. The assessment identified several mission-critical gaps. For  example, the agency found that competency gaps existed in cloud  computing, security integration and engineering, enterprise/mission  engineering lifecycle, requirements development, and Internet data  collection. It also found that enterprise-level competency gaps existed in  program and project management, budget and cost estimation, systems  development, data analytics, and shared services. These mission-critical  gaps would have challenged the Census Bureau’s ability to deliver critical  IT-related initiatives. In response, the agency identified actions to close  the competency gaps by December 2015, including several training- related actions. For example, it planned to implement centralized and  integrated professional training and development activities, create  environments conducive to on-the-job training from embedded personnel,  and leverage staff with related knowledge and experiences to provide  in-house workshops that share skills and experiences. We reported that  fully implementing these activities would be critical to ensuring that the  Census Bureau has the skills it needs to effectively develop and  implement systems vital to the upcoming Census.
Shortfalls in team processes. Our prior work on the Federal Aviation  Administration’s acquisitions program for air traffic control facilities  demonstrated the problems associated with processes that did not  facilitate sufficient involvement of stakeholders throughout the system’s  development. Specifically, the Federal Aviation Administration did not  include air traffic controllers in the design and testing of one of its integral  air traffic control programs. Because of this, issues that could have been  addressed early in the design phase were not discovered until  implementation, resulting in cost and schedule impacts. Partly due to not  having valuable stakeholder involvement, the program experienced a  cumulative 60 percent cost increase and schedule delays averaging 4  years. In response, the agency took steps to involve stakeholders by  signing a memorandum of understanding with the air traffic controllers’  union. The memorandum called for, among other things, a team of 16  controllers to be detailed from various locations to test and validate  software fixes with engineers at the Federal Aviation Administration  Technical Center.
As another example, our prior review of the United States Department of  Agriculture’s Farm Service Agency’s Modernize and Innovate the Delivery  of Agricultural Systems program showed that agency policies for effective  communication were not followed. For example, the program did not  keep records of which artifacts were reviewed during meetings to assess  system developmental progress and maturity and did not adequately  track decisions and corrective actions. This hindered executive level  governance over the program, which contributed, in part, to significant  challenges for the program. As a result of these challenges, further  development of the system was ultimately halted after the program had  spent about $423 million and delivered only about 20 percent of the  functionality which had been envisioned.
Implementing the key characteristics of IPTs for major IT acquisitions can  mitigate the risk of aforementioned problems from occurring. Also key to an agency’s success in developing IPTs is sustaining a workforce with the  necessary knowledge, skills, and abilities to execute a range of  management functions that support its mission and goals.
section_title: Selected Departments Had Mixed Progress in Assessing Their IT Skill Gaps
paragraphs: One key element cuts across the characteristics and practices needed to  build a strong IPT: IT workforce planning. For example, leadership needs  accurate information about the team’s staffing needs in order to provide  sufficient staffing resources. Agencies can help ensure IPTs have the  requisite skill sets, team size, and set of disciplines by developing  competency and staffing requirements, assessing gaps, and developing  strategies and plans to address those gaps through additional training or  other measures. Agencies can also utilize staffing strategies, such as  contractor assistance, if appropriate, to address IT skill gaps and ensure  effective team operations.
As discussed previously in this report, a number of federal laws as well as  guidance issued by OMB and OPM address IT workforce planning  activities for federal agencies. Several of the requirements and  recommended practices from those sources also generally align with key  principles and activities identified in our and OPM’s strategic workforce  planning models. Consequently, to support the evaluation of whether  selected federal agencies are adequately assessing and addressing gaps  in IT knowledge and skills, we established an evaluation framework  based on these laws and guidance, and vetted it with internal and  external stakeholders.
The framework contains four steps: (1) setting the strategic direction for  IT workforce planning, (2) analyzing the IT workforce to identify skill gaps,  (3) developing and implementing strategies to address IT skill gaps, and  (4) monitoring and reporting progress in addressing IT skill gaps. Each of  the four steps is supported by key activities. Table 1 summarizes the four  steps and eight key activities used in our evaluation of selected federal  agencies.
When effectively implemented, IT workforce planning activities can  facilitate the success of major IT acquisitions. Ensuring program staff  have the necessary knowledge and skills is a factor commonly identified  as critical to the success of major IT investments. If agencies are to  ensure that this critical success factor has been met, then IT skill gaps  need to be adequately assessed and addressed through a workforce  planning process.
Selected federal departments had mixed progress in assessing their IT  skill gaps. While they have demonstrated important progress in either  partially or fully implementing key IT workforce planning activities, each of  the five departments had shortfalls. The five departments have started  focusing on identifying cybersecurity staffing gaps, but more work  remains in assessing competency gaps and in broadening the focus to  include the entire IT community. Figure 6 illustrates the extent to which  selected departments have fully, partially, or not implemented key IT  workforce planning activities.
As shown in the figure, of the five departments, Defense had the most  robust implementation with all eight activities fully or partially  implemented.
subsections: section_title: Commerce
paragraphs: Commerce has not yet established a process to guide its IT workforce  planning activities. Of the eight activities to assist an agency with effective  workforce planning, Commerce has partially implemented three activities  and has not implemented five activities. Table 2 identifies the extent to  which Commerce has implemented key IT workforce planning steps and  activities.
Several circumstances have contributed to the department’s shortfalls in  IT workforce planning. For example, Commerce’s efforts to gather and  utilize information for IT workforce planning have been ad hoc. While  Commerce participated in a government-wide IT workforce assessment in  2011, it was unable to demonstrate how it used information from the  assessment for determining IT competency and staffing requirements,  identifying gaps, and developing strategies to address gaps. Further, the  department attempted to collect updated data in recent years on IT  workforce staffing levels, certifications, and skill gaps. However, it was  unable to demonstrate how they used this information for workforce  planning and noted that data collected in 2015 on skill gaps were not  used because the data were unreliable. In addition, Commerce officials  noted that while the department has begun to draft IT competency  requirements and has reviewed several sources, the department does not  yet have policies and procedures that incorporate key IT workforce  planning steps and activities.
Commerce officials acknowledged that shortfalls exist in the department’s  workforce planning activities and stated that they are working to develop  a more effective IT workforce assessment plan in fiscal year 2017. Until  Commerce fully implements key IT workforce planning activities,  management will have a limited ability to assess and address gaps in  knowledge and skills that are critical to the success of major acquisitions.  As a result, it will be more difficult to anticipate and respond to changing  staffing needs and control human capital risks when developing,  implementing, and operating critical IT systems.
section_title: Defense
paragraphs: Defense has established processes that have enabled it to perform key  analyses and assessments on its IT workforce. Specifically, of the eight  activities associated with workforce planning, the department has fully  implemented one and has partially implemented seven. Table 3 identifies  the extent to which Defense has implemented key IT workforce planning  steps and activities.
To its credit, Defense has established competency models, conducted  gap assessments, and is developing strategies for two of its three major  IT occupations and officials reported that the department is working on  revalidating competencies for its third major IT occupation by fiscal year  2016. According to department workforce data from March 2015, these  three occupations comprised approximately 93 percent of its IT  workforce. However, Defense has not yet established competency  models and conducted gap assessments for the remaining 15 IT  occupations because it has focused initially on developing competencies  and assessing gaps for its mission-critical IT occupations after developing  a tool in 2014 that it could use in performing such enterprise  assessments. Further, officials from Defense’s Office of the CIO have  recognized limitations with assessing IT skill gaps solely based on  existing occupational titles and position descriptions. As a result, the  department is in the process of implementing strategies and plans to  establish a set of work roles and a baseline set of knowledge, skills, and  abilities that apply to personnel spread across a number of functional  communities who perform significant cybersecurity, IT, and other  cyber-related roles. According to Defense’s Strategic Workforce Plan for  fiscal years 2014 through 2019, the department anticipates that this effort  will help in monitoring the overall health and capability of the workforce at  a granular level not previously attainable.
By not fully implementing key IT workforce planning activities, Defense  runs the risk that it cannot adequately assess and address gaps in  knowledge and skills that are critical to providing support for major  acquisitions. As a result, Defense may have difficulty in anticipating and  responding to evolving staffing needs while developing, implementing,  and operating critical IT systems.
The Department of Health and Human Services (HHS) has begun  building its capability to set the strategic direction for its IT workforce by  taking foundational steps towards establishing a workforce planning  process. Of the eight activities needed to perform workforce planning,  HHS partially implemented six and did not implement two. Table 4  identifies the extent to which HHS implemented key IT workforce planning  steps and activities.
Several explanations exist for the department’s shortfalls in IT workforce  planning.
HHS does not have comprehensive policies that require the CIO to  implement key IT workforce planning activities, such as conducting IT  skill gap assessments and developing strategies and plans to address  those gaps. Officials noted that they have such guidance under  development.
HHS has not recently conducted skill gap assessments of its IT or  cybersecurity workforce because it decided to focus first on  establishing competencies and career paths for its IT program  manager and cybersecurity positions. To this end, department officials  have acknowledged that more comprehensive IT workforce planning  needs to occur to address requirements of laws, such as the Federal  Cybersecurity Workforce Assessment Act of 2015, and have begun  taking steps to do so. In particular, HHS identified planned steps that,  when implemented, could help the department to code and assess  skill gaps for its IT and cybersecurity workforce at a more granular  level.
HHS officials expressed concerns regarding the variety and  complexity of requirements for IT workforce planning from federal  laws and other guidance. Officials noted that additional guidance from  OMB could help federal agencies avoid potentially  duplicative and  inefficient implementation of the various workforce planning steps and  activities.
To its credit, HHS has recently taken positive steps to implement key IT  workforce planning activities. In June 2015, the department initiated a  cybersecurity workforce development program and subsequently  established competencies and career paths for cybersecurity related  occupations. According to officials, HHS is currently working to merge the  ongoing cybersecurity workforce development efforts into a broader IT  workforce planning effort that will allow the department to conduct gap  assessments and develop strategies, such as retention and recruitment  plans, based on a more comprehensive workforce planning process.
Until HHS fully implements key IT workforce planning activities,  management will have a limited ability to assess and address gaps in  knowledge and skills that are critical to the success of major acquisitions.  As a result, it will be difficult for HHS to anticipate and respond to  changing staffing needs and control human capital risks when developing,  implementing, and operating critical IT systems.
section_title: Transportation
paragraphs: The Department of Transportation (Transportation) is building its  capability to assess IT skill gaps, but it currently does not have  foundational elements of a workforce planning process. Specifically, it has  partially implemented five activities and has not implemented three  activities. Table 5 identifies the extent to which Transportation  implemented key IT workforce planning steps and activities.
Transportation officials provided explanations for the department’s  shortfalls in IT workforce planning. For example, the department has only  recently established policies to guide its workforce planning activities. In  addition, the department did not consider hiring staff to ensure that  program managers were certified to be a priority.
To its credit, Transportation has plans to improve its IT workforce. For  example, the department is planning to (1) develop plans by the end of  fiscal year 2016 that could lead to the creation of an IT acquisition cadre  and use of direct hiring authority, (2) draft an approach for  cross-functional training of IT acquisition and program personnel, and (3)  develop a career path for program managers by fiscal year 2017. While  these are positive steps, these actions are not yet rooted in an IT  workforce planning process that lays the foundation for how the  department is to analyze its IT workforce skill gaps, develop and  implement strategies to address gaps, and monitor and evaluate progress  in addressing the gaps.
By not fully implementing key IT workforce planning activities,  Transportation has a limited ability to assess and address gaps in  knowledge and skills that are critical to the success of major acquisitions.  As a result, it will be difficult to anticipate and respond to changing staffing  needs and control human capital risks when developing, implementing,  and operating critical IT systems.
section_title: Treasury
paragraphs: The Department of the Treasury (Treasury) has taken steps to improve its  IT and program management workforce by partially implementing seven  activities; however, the department has not yet implemented one key  workforce planning activity. Table 6 identifies the extent to which Treasury  has implemented key IT workforce planning steps and activities.
According to Treasury’s March 2016 FITARA Common Baseline  Implementation Plan, the department is working to develop plans and  processes by which the CIO, chief human capital officer, and senior  procurement executive will develop a common set of competency  requirements for IT staff and a workforce planning process. Treasury has  a policy for utilizing a department-wide strategic workforce planning  model. However, the department has deferred the use of this model on  the IT workforce until the results of an Internal Revenue Service IT  workforce assessment pilot are available and have been reviewed by the  CIO and chief human capital officer. Treasury’s FITARA Common  Baseline Implementation  Plan noted that the timing of these steps is  difficult to estimate because each decision point depends on the outcome  of the preceding step and because staff and managers would need to  coordinate workforce planning  with other time demands. According to its  implementation plan, among other milestones, the chief human capital  officer is to provide a recommendation to the CIO for a process and tools  to conduct a competency-based workforce planning study of the Treasury  IT workforce in fiscal year 2017.
While Treasury’s plans are a good starting point, it is too soon to tell  whether these plans will address the shortcomings we identified. By not  fully implementing key IT workforce planning activities, Treasury has a  limited ability to assess and address gaps in knowledge and skills that are  critical to the success of major acquisitions. Until the department  addresses the shortcomings in its IT workforce planning activities, it will  be more difficult to anticipate and respond to changing staffing needs and  control human capital risks when developing, implementing, and  operating critical IT systems.
section_title: Conclusions
paragraphs: Agencies can improve the success of their IPTs for IT acquisitions  through effective executive leadership that emphasizes team support,  empowerment, and oversight; effective team composition; and effective  processes for team operations. When implemented, these key practices  can better position agencies to efficiently make decisions that cross lines  of expertise.
The right mix of expertise to recognize problems early and the requisite  authority to do something about them are contingent upon effective IT  workforce planning. While the five selected departments have  demonstrated progress by implementing key IT workforce planning  activities, each had shortfalls. To their credit, all five had at least partially  analyzed their IT workforce and implemented certain activities to enhance  workforce skills. In addition, several departments identified promising  activities that, when implemented, could bolster effectiveness in  assessing and addressing IT skill gaps. For example, Defense and HHS  have initiated efforts to assess the IT and cybersecurity workforce at a  more granular level and the Internal Revenue Service within Treasury is  piloting a workforce assessment tool that may help the department  assess its skill gaps. However, only Defense demonstrated that it had  implemented a workforce planning process. Further, none of the  departments identified IT competency gaps for their entire workforce, and  only three were performing some level of monitoring towards the closure  of identified skill gaps. Until departments fully implement the key activities  and steps, they will have a limited ability to assess and address gaps in  knowledge and skills that are critical to the success of major IT  acquisitions.
section_title: Recommendations for Executive Action
paragraphs: To facilitate the analysis of gaps between current skills and future needs,  the development of strategies for filling the gaps, and succession  planning, we are making a total of five recommendations to the  Secretaries of Commerce, Defense, Health and Human Services,  Transportation, and the Treasury.
section_title: Agency Comments and Our Evaluation
paragraphs: We received comments on a draft of this report from the five departments  to which we made recommendations—Commerce, Defense, Health and  Human Services, Transportation, and the Treasury. Of those five  departments, four agreed with our recommendations and one, Defense,  partially agreed. The following summarizes each department’s comments.
In written comments, Commerce agreed with our recommendation  and stated that it will work to develop an action plan to address the  key workforce planning steps and activities identified in this report.  The department’s comments are reprinted in appendix II.
In its written comments, Defense partially agreed with our  recommendation and discussed its progress on selected activities. In  commenting on the activity involving the development of strategies  and plans to address gaps in competencies and staffing (activity 5),  the department stated that it had provided additional information on its  related efforts in its Strategic Workforce Plan for fiscal years 2016  through 2021. However, the department did not attach a copy of the  workforce plan. Moreover, the workforce plan, as described, does not  cover staffing gaps or address Defense’s entire IT workforce.
In addition to its workforce planning strategies, the department  commented on its efforts to develop competencies, as part of the  activity to establish requirements for competencies and staffing  (activity 2). The department stated that it has established  competencies for the Information Technology Management and  Computer Science series and is scheduled to complete competencies  for the Telecommunications series, its third and final mission-critical IT  occupation, by the end of December 2016. The department further  stated that, in addition to its three mission-critical occupations, it has  developed competencies for its next largest IT occupation, Computer  Clerk and Assistant (series 0335). This is to result in a total of 95  percent of workforce competencies developed or to be developed by  the end of December 2016.
Nevertheless, while establishing competencies is an important step  toward robust IT workforce planning, it remains important for Defense  to also identify how it intends to complete the workforce planning  cycle for staff represented by each of the occupations for which it has  developed or plans to develop competencies. Assessing competency  gaps and developing  gap mitigation strategies are important next  steps.
Looking across all of the activities, the department further stated that,  in addition to the four occupations comprising 95 percent of the IT  workforce, half of the department’s remaining 14 IT occupations are  filled with less than 100 staff, and that several of the remaining  occupations are likely to experience attrition. The department added  that it does not believe it is necessary to develop competencies,  identify competency gaps, and develop gap closure strategies for the  remaining 5 percent of its IT workforce, as these individuals would  likely not have a role on integrated program teams or be in a position  critical to the success of major IT acquisitions.
We understand the importance of Defense prioritizing its efforts to  focus on the 95 percent of its workforce that are likely to have a role  on integrated program teams or be in a position critical to the success  of major IT acquisitions. However, identifying gaps and developing  strategies for the remaining 5 percent of its IT workforce will become  more relevant as the department begins its transition to focus on work  roles and knowledge, skills, and abilities rather than occupations.
Therefore, we maintain that our recommendation is warranted. The  department’s comments are reprinted in appendix III.
HHS also provided written comments, in which the department agreed  with our recommendation and identified several planned or ongoing  actions to address it. The department stated that it is growing its  capability for mature workforce analytics and planning, in part, through  the identification and definition of critical IT and cybersecurity role  categories and competency requirements. The department also noted  that it plans to utilize codes from OPM and the National Institute of  Standards and Technology in its efforts to assess skill gaps and  staffing needs. In addition, HHS provided new documentation of its  completion of an IT program manager career path. We modified our  assessment and corresponding recommendation for the relevant  activity (activity 6) to reflect these efforts. The department’s comments  are reprinted in appendix IV.
In written comments, Transportation agreed with our recommendation  and described planned actions to address it. Specifically, the  department stated that it is leveraging its Strategic Workforce  Planning Guide to navigate its efforts in assessing IT skill gaps and  enhancing workforce proficiency. The department also noted that  actions underway include finalizing plans for creating IT acquisition  cadres and using direct hiring authority, developing an approach for  cross-functional training of IT acquisition and program management  personnel, and collaborating across departmental offices on the  development of an IT program manager career path. Transportation’s  comments are reprinted in appendix V.
In its written comments, Treasury agreed with our recommendation  and identified planned and ongoing efforts to address it. Specifically,  the department noted that, while an agency-wide assessment of the  IT workforce has not yet been completed, several component  bureaus—including the Internal Revenue Service, which comprises  over 80 percent of the total Treasury IT workforce—have been  assessing their workforce. In addition, the department stated that its  FITARA Implementation Plan identifies intended steps for meeting the  requirements established by Congress and OMB, including IT  workforce management, and that it is on track to establish a more  robust workforce planning capability by the end of fiscal year 2017.  Treasury’s written comments are reprinted in appendix VI.
In addition to its written comments on the recommendations, the  department also provided emails containing its comments on selected  findings in the draft report, which we addressed as appropriate.  Specifically:
Treasury stated that it did not agree with our assessment that the  department has not implemented the key IT workforce planning  activity to establish and maintain a workforce planning process  (activity 1). However, the department’s efforts have not been  sufficient to justify a higher rating for this activity. As noted in the  report, while the department has a strategic workforce planning  model, it has deferred applying the model to the IT workforce  pending a review of a workforce assessment tool being piloted  within the Internal Revenue Service. We do not consider this  situation to reflect an established and maintained IT workforce  planning process because the pilot has not been completed and  reviewed by the CIO and chief human capital officer, which was  identified as a foundational step in its FITARA implementation  plan. Further, decisions are yet to be made regarding (1) which IT  workforce planning approach the department will utilize: the  process still being piloted at the Internal Revenue Service, the  department’s existing strategic workforce planning model that  does not specifically address how IT components are to be  assessed, or a hybrid approach; and (2) how and when its  selected approach will be applied to assess skill gaps for the  department’s IT workforce.
Treasury also stated that we did not give the department credit for  a cybersecurity skill gap assessment that it performed, which  relates to the key IT workforce planning activity to assess gaps in  competencies and staffing (activity 4). We disagree. We noted in  the report that Treasury had identified staffing gaps for IT  positions, including its mission-critical occupation and  cybersecurity specific positions. This was based in part on the  cybersecurity staffing gap assessment that the department  performed, which factored into our rating of “partially  implemented” for this activity.
The department did not agree with our assessment that it had not  implemented the key IT workforce planning activity to report to  agency leadership on progress in addressing competency and  staffing gaps (activity 8). After evaluating reports to senior  department leaders and OMB regarding its cybersecurity staffing  gaps and efforts to address the gaps through a pilot cybersecurity  recruitment and retention program—which the department  provided with its comments—we changed our rating from “not  implemented” to “partially implemented” and revised the  description for this activity.
In addition to the aforementioned departments, we sought comments  from other federal agencies, including GSA, OMB, OPM, and VA. We  received e-mails from GSA, OMB, and OPM stating that they had no  comments on the report. We also received an e-mail from VA with a  technical comment that we addressed as appropriate.
We are sending copies of this report to interested congressional  committees, the Secretary of Commerce, the Secretary of Defense, the  Secretary of Health and Human Services, the Secretary of  Transportation, the Secretary of the Treasury, the Director of the Office of  Personnel Management, and the Director of the Office of Management  and Budget, and other interested parties. In addition, this report will be  available on the GAO Web site at http://www.gao.gov.
If you or your staffs have any questions on the matters discussed in this  report, please contact me at (202) 512-9286 or at pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix VII.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) identify key characteristics of a comprehensive  integrated program team (IPT) responsible for managing a major federal  information technology (IT) acquisition, and (2) evaluate whether selected  federal agencies are adequately assessing and addressing gaps in  knowledge and skills that are critical to the success of major IT  acquisitions.
To identify key characteristics of comprehensive IPTs, we listed over  1,000 recommended practices. In doing so, we reviewed approximately  20 documents containing practices applicable to IPTs that were  recommended by government and industry organizations, including:
Department of Defense, Office of the Under Secretary of Defense  (Acquisition and Technology), DoD Integrated Product and Process  Development Handbook, August 1998;
American Council for Technology-Industry Advisory Council, Key  Success Factors for Major Programs that Leverage IT, May 2014;
The MITRE Corporation, Integrated Project Team (IPT) Start-up  Guide, October 2008;
Software Engineering Institute at Carnegie Mellon University, CMMI®  for Acquisition, Version 1.3, November 2010;
Office of Management and Budget (OMB), 25 Point Implementation  Plan to Reform Federal Information Technology Management,  December 2010;
OMB, Guidance for Specialized Information Technology Acquisition  Cadres, July 2011;
OMB, Contracting Guidance to Support Modular Development, June
Project Management Institute, Inc., A Guide to the Project  Management Body of Knowledge (PMBOK® Guide), Fifth Edition,  2013. PMBOK is a trademark of the Project Management Institute,  Inc.;
OMB, Circular No. A-11, Preparation, Submission, and Execution of  the Budget, Part 7 (Capital Programming Guide, Version 3.0), June  2015; and
OMB, U.S. Digital Services Playbook.
To corroborate and validate the recommended practices and to identify  examples of how the practices can be applied, we interviewed agency  officials from eight different organizations: two agency IPTs, four offices of  the chief information officer (CIO) with oversight responsibilities for IPTs,  and two agencies’ acquisition centers of excellence.
The two agency IPTs we interviewed were the Department of  Commerce’s Geostationary Operational Environmental Satellite - R  series and the Department of Veterans Affairs’ Veterans Benefits  Management System. We selected these two IPTs from a pool of 13  candidates based on the extent to which they met multiple selection  criteria and did not overlap with ongoing GAO work. The selection  criteria were: (a) suggested by OMB officials as being high-impact  programs that have had a challenging past performance and have  current increased oversight from OMB; (b) active engagement with  OMB’s U.S. Digital Service; (c) observed by OMB has having  relatively strong teaming practices; (d) observed through the course of  GAO reviews as having relatively strong teaming practices; and (e)  observed through the course of GAO reviews as having a history of  problems from which lessons learned could be derived.
The four offices of the CIO we interviewed were from the Departments  of Commerce, Defense, and Veterans Affairs. We selected  Commerce, National Oceanic and Atmospheric Administration (within  the Department of Commerce), and Veterans Affairs CIO officials  because their departments had oversight responsibilities for the two  IPTs we interviewed. We also selected CIO officials from the  Department of Defense due to the department’s history in adopting  IPT concepts.
The two agencies’ acquisition centers of excellence we interviewed  were the General Services Administration’s Assisted Acquisition  Services and the Department of Veterans Affairs’ Technology  Acquisition Center. We selected these two centers of excellence  based on OMB’s 2011 guidance for agencies for formulating  specialized IT acquisition cadres.
In our interviews, we asked these officials to describe desirable  characteristics of comprehensive IPTs, identify additional factors that  are critical to the success of IPTs, and provide examples. We added  their responses to our database of IPT recommended practices.
We conducted a content analysis on the recommended IPT practices  from our literature research and from interviews with agency officials. In  doing so, team members individually reviewed the practices and assigned  them to various categories and sub-categories. Then the team members  met to compare their categorization schemes, discuss the differences,  and reach agreement on the final overarching characteristics and specific  sub-practices. We validated the outcome of our content analysis by  sending the results to the agency IPTs, CIOs, and centers of excellence  which we had interviewed, as well as two internal experts. We then  obtained their comments and revised our final list of IPT characteristics  and practices as appropriate. Finally, we created a graphic based on the  total number of times literature and agency officials identified specific  roles and responsibilities. In doing so, we incorporated the roles and  responsibilities that were most frequently cited as core disciplines and  organized the graphic in descending order.
To evaluate whether selected federal agencies are adequately assessing  and addressing gaps in knowledge and skills that are critical to the  success of major IT acquisitions, we selected departments for our review,  created an evaluation framework and validated it, and assessed the  selected departments’ documents and activities against the evaluation  framework.
We selected the departments for our review based on a judgmental and  non-generalizable sample of agencies’ data from the Federal IT  Dashboard as of August 31, 2015. Specifically, we selected five  departments—Commerce, Defense, Health and Human Services,  Transportation, and the Treasury—based on the following factors: (1)  largest number of major IT investments, (2) largest planned dollar total of  major IT spending in fiscal year 2016, and (3) largest planned percentage  of total major IT spending with development, modernization, and  enhancements in fiscal year 2016.
To create the evaluation framework, we reviewed relevant laws and  guidance to identify steps and activities for federal agencies to conduct IT  workforce planning, including:
Clinger-Cohen Act of 1996;
E-Government Act of 2002;
Legislation commonly referred to as the Federal Information  Technology Acquisition Reform Act;
Federal Cybersecurity Workforce Assessment Act of 2015;
OMB’s 25 Point Implementation Plan to Reform Federal Information
OMB’s Guidance for Specialized Information Technology Cadres;
OMB’s Management and Oversight of Federal Information
OMB’s Cybersecurity Strategy and Implementation Plan for the
OMB’s Federal Cybersecurity Workforce Strategy;
OMB’s Circular A-130, Managing Information as a Strategic
The Office of Personnel Management’s (OPM) Workforce Planning
OPM’s workforce planning guidance on key elements and suggested
OPM’s IT Program Management Career Path Guide; and
GAO guidance on federal internal control standards as well as key  principles for effective strategic workforce planning.
We developed a framework consisting of four IT workforce planning steps  and eight key activities, as shown in table 7.
We validated the evaluation framework by soliciting comments from  internal experts as well as officials from OMB and OPM and updated it as  appropriate. We did not perform a legal analysis as to whether the  selected agencies were in actual compliance with the laws we used to  derive our evaluation framework.
We assessed the selected departments’ documents and activities against  the evaluation framework by requesting relevant IT workforce and human  capital policies, as well as documentation from each of its most recent IT  workforce competency and staffing gap assessments. We analyzed the  policies and documents to determine whether they were consistent with  the workforce planning steps and activities found in our framework. The  documents we analyzed included: competency gap assessments, staffing assessments, strategic workforce plans, documentation of monitoring progress in addressing IT workforce skill  IT workforce competencies and staffing requirements,  gaps and reporting to department officials, documentation regarding IT acquisition cadres, documentation on cross-functional training of acquisition and IT documentation regarding career paths for IT program managers, documentation of efforts to strengthen IT program management, and documentation of the use of direct hire authority.
We also interviewed cognizant officials at the five departments to discuss  their policies, documentation, and implementation of the workforce  planning steps and activities. Based on our assessment of the  documentation and discussion with department officials, we assessed the  extent to which the agencies implemented, partially implemented, or did  not implement the steps and activities. We considered an activity to be  fully implemented if a department addressed all of the underlying  practices for the activity; partially implemented if it addressed some but  not all of the underlying practices for the activity; and not implemented if it  did not address any of the underlying practices for the activity.
We conducted this performance audit from July 2015 to November 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: Comments from the Department of Health and Human Services
section_title: Appendix V: Comments from the Department of Transportation
section_title: Appendix VI: Comments from the Department of the Treasury
section_title: Appendix VII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, individuals making contributions  to this report included Colleen Phillips (Assistant Director), Josh Leiling  (Analyst-in-Charge), Chris Businsky, Raj Chitikila, Rebecca Eyler, Torrey  Hardee, Jamelyn Payan, Andrew Stavisky, Mark Stefan, and Brian  Vasquez.