IPTs are to bring together the different areas of expertise needed to acquire a new product, such as engineering, manufacturing, purchasing, and finance. The essence of the IPT approach is to concentrate this expertise in a single team together with the authority to design, develop, test, manufacture, and deliver a product. The hallmark of these teams is their ability to efficiently make decisions that cross lines of expertise. Over the last two decades, the federal government has taken steps to improve IT acquisitions through the use of IPTs. Building on success from private industry’s use of IPTs, Defense adopted them in an attempt to improve its weapon system acquisitions. In 1996, the department published a guide on integrated product and process development based on its survey of policies and practices for over 80 government and industry organizations. Defense’s intention was to use the teams in the same manner as commercial firms—to integrate different functional disciplines into a team responsible for all aspects of an acquisition. Recognizing IPTs’ potential to improve the federal government’s approach to managing IT investments, OMB has called for the development and use of IPTs for federal IT acquisitions. In 2010, OMB published a 25-point action plan that was intended to address many of the most pressing, persistent challenges to the federal government’s management of IT. One of the actions OMB cited was to require that IPTs were in place before OMB approves program budgets. Additional actions agencies are to take are to dedicate resources to the team throughout the program lifecycle and hold team members accountable for individual goals and overall program success. Subsequently, OMB has incorporated revisions to its Capital Programming Guide and related guidance, which require the use of IPTs prior to OMB’s approval of an investment’s business case. Specifically, OMB requires that each major IT investment establish an IPT that includes, at a minimum, a fully dedicated program manager, a contracting specialist (if applicable), an IT specialist, an IT security specialist, and a business process owner or subject matter expert. Effective IT workforce planning is key to an agency’s success in developing IPTs with the necessary knowledge, skills, and abilities to execute a range of management functions that support the agency’s mission and goals. Over the past 20 years, various laws were enacted and guidance issued that call for agencies to perform workforce planning activities that ensure the timely and effective acquisition of IT. These laws and guidance focus on the importance of (1) setting the strategic direction for workforce planning, (2) analyzing the workforce to identify skill gaps, (3) developing strategies to address skill gaps, and (4) monitoring and reporting on progress in addressing skill gaps. For example: The Clinger-Cohen Act of 1996 requires agency CIOs to annually (1) assess the requirements established for agency personnel regarding knowledge and skill in information resource management and the adequacy of such requirements for facilitating the achievement of performance goals; (2) assess the extent to which the positions and personnel at executive and management levels meet those requirements; (3) develop strategies and specific plans for hiring, training, and professional development to address any deficiencies; and (4) report to the head of the agency on the progress made in improving information resources management capability. The E-Government Act of 2002 requires the Director of OPM, in consultation with the Director of OMB, the Chief Information Officers Council, and the Administrator of General Services to (1) analyze, on an on-going basis, the personnel needs of the federal government related to IT and information resource management; and (2) identify where current IT and information resource management training does not satisfy personnel needs. In addition, the law requires the Director of OMB to ensure that agency heads collect and maintain standardized information on their IT and information resources management workforce. In 2010, OMB issued its 25-point plan for IT reform and outlined several action plans to build workforce capabilities, including acquisition and program management. For example, OMB stated that OPM will work with OMB to provide agencies with direct hiring authority for program managers and directed OPM to create a specialized career path. OMB also tasked agencies with identifying program management competency gaps and reporting on those gaps. Subsequent to the 25-point plan, in July 2011, OMB released guidance for agencies to develop specialized IT acquisition cadres. Among other things, this memorandum required agencies to analyze current acquisition staffing challenges; determine if developing or expanding the use of cadres would improve program results; and outline a plan to pilot or expand cadres for an especially high-risk area, if the agency determined that such an effort would improve performance. Further, in November 2011 OPM issued guidance for developing career paths for IT program managers. OPM’s career path guide was to build upon its IT Program Management Competency Model released in July 2011 by serving as a roadmap for individuals interested in pursuing a career in this area and providing employees and their supervisors with a single-source reference to determine appropriate training opportunities for career advancement. In December 2014, Congress enacted legislation commonly referred to as the Federal Information Technology Acquisition Reform Act (FITARA). Among other things, the law aims to ensure timely progress by federal agencies toward developing, strengthening, and deploying IT acquisition cadres consisting of personnel with highly specialized skills in IT acquisition, including program and project managers. Agencies (other than Defense) are required to update their acquisition human capital plans to address how they are meeting their human capital requirements to support timely and effective acquisitions. To assist agencies in implementing the provisions of FITARA and to reinforce provisions of the Clinger-Cohen Act of 1996, OMB issued guidance to agencies in June 2015. In doing so, OMB directed agencies (other than Defense) to, among other things, (1) develop a set of competency requirements for staff, including leadership positions; and (2) develop and maintain a current workforce planning process to ensure the agency can (a) anticipate and respond to changing mission requirements, (b) maintain workforce skills in a rapidly developing environment, and (c) recruit and retain the talent needed to accomplish the mission. Each agency is to conduct an annual self-assessment of its conformity with these requirements and develop an implementation plan describing the changes it will make. In October 2015, OMB required agencies to identify their top five cybersecurity talent gaps by December 2015 as a one-time effort. Specifically, agencies were to participate in an OPM cybersecurity staffing exercise to identify the universe of their cyber talent, understand challenges for retaining talent, and address gaps accordingly. The Federal Cybersecurity Workforce Assessment Act of 2015 required OPM, with support from the National Institute of Standards and Technology, to establish a coding structure to be used in identifying all federal civilian and non-civilian positions that require the performance of IT, cybersecurity, or other cyber-related functions. Agencies, in consultation with OPM, the National Institute of Standards and Technology, and the Department of Homeland Security, were then required to utilize this coding structure to annually assess, among other things, the IT, cybersecurity, and other cyber- related work roles of critical need in the agency’s workforce. We have additional planned work in this area. OMB released its Federal Cybersecurity Workforce Strategy in July 2016. Among other things, the strategy cited the need for agencies to examine specific IT, cybersecurity, and cyber-related work roles, and identify personnel skills gaps, rather than merely examining the number of vacancies by job series. The strategy identified several actions that agencies could take to identify workforce needs, expand the cybersecurity workforce through education and training, recruit and hire highly skilled talent, and retain and develop highly skilled talent. Finally, in July 2016 OMB issued updated policy for the planning, budgeting, governance, acquisition, and management of federal information, personnel, equipment, funds, IT resources and supporting infrastructure and services. Among other things, OMB’s updated circular requires the agency’s chief human capital officer, CIO, chief acquisition officer, and senior agency official for privacy to develop a set of competency requirements for staff and develop and maintain a current workforce planning process. While the laws and guidance focus on IT workforce, there are also other broader initiatives to improve federal human capital management. For example, we and OPM developed human capital management models that call for implementing workforce planning practices that can facilitate the analysis of gaps between current skills and future needs and the development of strategies for filling the gaps, as well as planning for succession. In addition, our Standards for Internal Control in the Federal Government stress that management should consider how best to retain valuable employees, plan for their eventual succession, and ensure continuity of needed skills and abilities. We have previously reported on commercial and Defense teaming practices and found that IPTs can develop and deliver superior products within predicted time frames and budgets—often cutting calendar time in half compared with earlier products developed without such teams. In addition, we identified two elements that are essential to an IPT: the knowledge and authority needed to recognize problems and make cross- cutting decisions expeditiously. We noted that knowledge is sufficient when the team has the right mix of expertise to master the different facets of product development and authority is present when the team is responsible for making both day-to-day decisions and delivering the product. We concluded that if a team lacks expertise, it will miss opportunities to recognize potential problems early; without authority, it can do little about them. Regarding broader human capital planning efforts, we first designated strategic human capital management across the government as a high- risk issue in 2001 because of the federal government’s long-standing lack of a consistent approach to human capital management. In February 2011, we narrowed the focus of this high-risk issue to the need for agencies to close mission-critical skill gaps. At that time, we noted that agencies faced challenges effectively and efficiently meeting their missions across a number of areas, including acquisition management. With regard to IT workforce and human capital planning, we reported that effectively addressing mission-critical skill gaps in IT requires a multi-faceted response from OPM and agencies. Specifically, our high risk update in February 2013 noted that OPM and agencies would need to use a strategic approach that (1) involves top management, employees, and other stakeholders; (2) identifies the critical skills and competencies that will be needed to achieve current and future programmatic results; (3) develops strategies that are tailored to address skill gaps; (4) builds the internal capability needed to address administrative, training, and other requirements important to support workforce planning strategies; and (5) includes plans to monitor and evaluate progress toward closing skill gaps and meeting other human capital goals using a variety of appropriate metrics. We subsequently reported in January 2015 that while the Chief Human Capital Officers Council identified skill gaps in six government-wide occupations— including IT/cybersecurity and contract specialist/acquisition—it would be important for lessons learned from these initial efforts to inform a new set of skill gaps; key features of OPM’s efforts to predict emerging skill gaps beyond those already identified were in the early planning stages; and OPM and selected agencies could improve efforts to address skill gaps by strengthening their use of quarterly data-driven reviews. Further, we have reported that agencies across the federal government have not always effectively planned for IT workforce challenges. For example, We recently determined that the Department of Veterans Affairs (VA) had performed key steps such as documenting an IT human capital strategic plan and regularly analyzing workforce data, but the department had not tracked and reviewed historical and projected leadership retirements and had not identified gaps in future skill areas. We recommended that the department track and review historical workforce data and projections related to leadership retirements and identify IT skills needed beyond the current fiscal year to assist in identifying future skills gaps and it concurred with our recommendations. We identified that while the Federal Emergency Management Agency had taken initial steps to assess the needs of its IT workforce, it had not yet completed workforce planning efforts and lacked an understanding of its regional IT workforce. For example, we noted that while it had conducted a workforce assessment to identify skill levels of employees in the agency’s Office of the CIO, it had not completed recommended actions called for by this assessment. In addition, its workforce planning efforts had not included an assessment of the many IT staff located in the agency’s regions and other offices. We concluded that the agency had less assurance that its IT workforce will have the skills needed to successfully manage its programs and recommended it establish time frames for completing workforce planning efforts. The Department of Homeland Security concurred with our recommendations. In November 2015, we testified that, among other things, the U.S. Census Bureau faced challenges in the area of workforce planning. Specifically, we noted that while it had taken steps to develop an enterprise-wide IT workforce planning process, as we recommended in 2012, it had yet to fill key positions. We have also identified human capital challenges at the Social Security Administration (SSA). Specifically, we concluded that SSA’s IT human capital program had identified skills and competencies to support certain workforce needs, but lacked adequate planning for the future. The agency had developed IT human capital planning documents, such as an Information Resources Management plan and skills inventory gap reports, which identified near-term needs, such as skill sets for the following 2 years. Nevertheless, SSA has not adequately planned for longer-term needs because its human capital planning and analysis were not aligned with long-term goals and objectives and the agency did not have a current succession plan for its IT efforts. Accordingly, we recommended that SSA identify long-term IT needs in its updated human capital operating plan and the agency agreed. Further, we determined that eight selected agencies had taken varied steps to implement workforce planning practices for cybersecurity personnel. For example, five of eight agencies, including the largest, Defense, had established cybersecurity workforce plans or other agency-wide activities addressing cybersecurity workforce planning. However, all of the agencies faced challenges determining the size of their cybersecurity workforce because of variations in how work was defined and the lack of an occupational series specific to cybersecurity. With respect to other workforce planning practices, all agencies had defined roles and responsibilities for their cybersecurity workforce, but these roles did not always align with guidelines issued by the federal Chief Information Officers Council and National Institute of Standards and Technology. We also noted that the robustness and availability of cybersecurity training and development programs varied significantly among the agencies. We made recommendations aimed at enhancing individual agency cybersecurity workforce planning activities and to address government-wide cybersecurity workforce challenges through better planning, coordination, and evaluation of government-wide activities. The agencies agreed with the majority of our recommendations. Our review of the Food and Drug Administration concluded that, among other things, the agency was not strategically managing IT human capital—it had not determined its IT skill needs or analyzed gaps between skills on hand and future needs. We recommended that the agency complete key elements of IT human capital planning; the Food and Drug Administration concurred with our recommendation. Three key characteristics contribute to the creation and operation of comprehensive IT acquisition IPTs: (1) effective executive leadership through team support, empowerment, and oversight; (2) effective team composition; and (3) effective processes for team operations. The 18 key practices associated with these characteristics provide additional detail regarding recommended practices for the successful implementation of the characteristics. The first characteristic of a strong IPT is that executive leadership, external to the team, provides the team with support, empowerment, and oversight. The seven key practices provide details of how executive leadership can assist the IPT, as shown in figure 2. There should be a common understanding between executive leadership, the IPT program manager, and team members regarding the mission of the IPT, its responsibilities, and the desired outcome of the program. Individual team members should have a shared definition of success, and should be encouraged to prioritize team-wide goals over personal goals. Agency officials highlighted the need to establish and maintain a shared vision and goals. Specifically: General Services Administration (GSA) officials told us that IPTs do not set their own goals. The project is sponsored by management, and the vision and goals should be communicated by leadership. The IPT’s job is to develop a strategy to achieve the vision and goals. GSA officials added that these teams should communicate the vision and goals of the program during the solicitation phase. During the performance phase, vision and goals should be aligned with service level agreements. Officials from the National Oceanic and Atmospheric Administration’s (NOAA) Office of the CIO noted that the IPT should communicate its vision and goals with its stakeholders for their review and approval prior to the project commencement. Officials with the Department of Commerce’s (Commerce) Office of the CIO stated it is important to ensure the team is aware of the shared vision and goals and that the team works with the sponsor, stakeholders, and other management to ensure the vision is shared and understood by all involved. Representatives of NOAA’s Geostationary Operational Environmental Satellite - R series program reported that team members need to be able to perform tasks that are best for the IPT and the project, regardless of whether those tasks were not originally assigned to them. For example, members need to break out of the mentality of being an “IT person” who does not need to assist with non-IT related tasks. Regardless of what expertise the member holds, he or she needs to work towards the mission of the project. Representatives from the VA Veterans Benefits Management System added that they work with partnering and parent organizations to ensure decisions related to the functionality are aligned with agency priorities. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology explained that an IPT should be committed to a common purpose. Work products such as the project charter and program management plan can be used to communicate the IPT’s vision and goals. Executive leadership should serve as an active promoter for the IPT and its success. This focus can be demonstrated by leadership effectively and enthusiastically communicating the vision and goals to outsiders, collaborating with other programs upon which the team is dependent, and managing the impact of challenges and opportunities from the external environment. For example, representatives from the VA Veterans Benefits Management System stated that a major underlying contributor to the progress of the program has been unwavering support from executive leadership since the project’s inception. The IPT staff explained that consistent oversight and executive involvement has had a positive and direct impact on the project’s overall probability of success by removing roadblocks, mitigating risks, and prioritizing funding. Executive leadership should create an environment for the IPT that facilitates success. This can include things such as team building experiences, opportunities for career growth through challenges, feedback and support, and recognition and rewards for good performance. For example, key guidance from MITRE Corporation recommends that periodic opportunities be provided for team members to brief their respective executives on aspects of their work. This may provide individual members with valuable recognition from high levels within their own areas as well as feedback to the IPT from senior management. The guidance also discusses the importance of incentivizing individual team members by tying their performance appraisals and rewards to team performance. Officials with two of the agencies in our review also mentioned the value of motiving team members: Officials with Commerce’s Office of the CIO stated that a desirable IPT characteristic includes teaming. Teaming requires a true peer environment where team members are mutually accountable for outcomes and consensus building is essential. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center noted that successful IPTs have some level of social interaction outside of professional duties to smooth out communication barriers and build rapport. The people assigned to an IPT should have the appropriate authority to perform their assigned responsibilities. The boundaries of responsibility and authority between the team and executive leadership should be established early and documented. Executive leadership may honor these agreements by allowing the IPT to select its members, coordinate and implement decisions, take prudent risks, and exercise full day-to-day responsibilities for delivering the product. An IPT should have the authority to make decisions and not just serve as a mechanism for communicating with stakeholders. Several agency officials also discussed the benefits of empowering teams. Specifically: Officials from the NOAA Office of the CIO stated that the extent of the IPT’s authority should be set by the executive sponsors based on the importance of the program. A risk-based approach should be used to identify the threshold of decision authority. According to officials with Commerce’s Office of the CIO, the IPT should be able to make most decisions as long as it does not over-run the budget, schedule, or scope. Empowerment is critical to making and keeping the agreements essential to effective teams. Representatives of NOAA’s Geostationary Operational Environmental Satellite - R series program noted that executive leadership should only help an IPT when appropriate, otherwise too strong of a hand from leadership can result in micromanagement. These representatives added that IPTs should have enough decision authority to push forward the project. Further, they should not have to constantly go up the management chain to make decisions, as this would be a clear indicator that the team’s authority is non-existent. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center explained that the level of decision making authority granted to an IPT depends on the level of resources, experience, human capital, and organization. It is important to hire quality resources, empower them to use critical thinking and judgment, and support their efforts. Controls should be implemented when these conditions are not present. Officials with the VA Technology Acquisition Center and the Office of Information and Technology told us that, in providing the best value to the VA customer, the IPT should have the authority to decide on requirements and the corresponding acquisition milestones and process to accomplish effective lifecycle acquisition, contract, and performance management. According to officials from Defense’s Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics and the Office of the CIO, a desirable characteristic of an IPT is to have empowered team members. Executive leaders outside of the IPT can help to ensure its success by providing the resources necessary to do the assigned work. Resources include funding to ensure the team has adequate staffing; physical facilities; security; and IT tools such as environments, software, hardware, and networks. In that regard, it is vital to have leadership support for the IPT lead and team members’ roles and time commitments. Having this support will help enable the people assigned to the IPT be able to devote the necessary time and effort to make the program successful. Agency officials highlighted the importance of providing the necessary resources. Specifically: Officials from NOAA’s Office of the CIO stated that each team member should have the time to devote to the IPT without distraction. Representatives of NOAA’s Geostationary Operational Environmental Satellite - R series program explained that leadership should have the appropriate understanding and awareness of what resources are needed, such as a sufficient budget and skilled staff members. If problems arise, senior executives need the ability to bring in additional resources that help the program. IPTs should have access to a resource pool of needed expertise or talent. According to officials from the VA Office of Information and Technology, it is critical that team members are committed and provide the necessary time for the effort so that project plans are fulfilled, consistent with legitimate constraints. Officials with the VA Technology Acquisition Center noted that members need to be committed and provide the necessary time to the IPT. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center told us that it is critical to have access to resources, such as budget and technology, to track the program’s schedule and deliverables. In addition, leadership support and flexibility are needed when staffing a team. Executive leadership outside of the IPT should periodically evaluate its structure, operations, and results. Another important role of leadership oversight is to ensure that the shared vision and goals are consistently pursued throughout the program lifecycle. Conducting such evaluations can result in the clarification of roles and responsibilities and constructive feedback to team members, the discovery of unrecognized or unresolved issues, and the establishment or modification of program goals for the future. Members should be held accountable by leadership for their individual performance as well as overall program success. Agency officials highlighted the need to provide the IPT with meaningful oversight. Specifically: Officials with NOAA’s Office of the CIO stated that executive managers from three offices (acquisitions, IT, and the customer organization) work together to approve the IPT and provide oversight to hold it accountable. According to VA Veterans Benefits Management System officials, IPTs at the VA were held accountable through the department’s Project Management Accountability System, which required that IT projects deliver customer-facing functionality every 6 months. Through the project management process, milestones are to be monitored regularly and communicated to appropriate stakeholders. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology stated that IPTs within the department are subject to regular reviews with senior leadership where metrics are used to monitor progress and make adjustments as appropriate to ensure success. Representatives from the VA’s Office of the CIO added that projects that miss their delivery date are to be reviewed by a senior leader panel to identify root causes and lessons learned. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center indicated that the IPT is coordinated by a sector director, group manager, and acquisition group manager based on client needs and resource availability. These three leadership stakeholders are involved in the gate review process and provide oversight. Projects are tracked using activity based costing. The budget is set by the group manager, and each employee tracks their time and cost against the budget. This activity drives efficiency on the government side. Similarly, every project employs a variation of project tracking using service level agreements, deliverables, and quality assurance surveillance. IPT leadership, with sufficient autonomy, should be able to handle many issues and conflicts on their own. However, when an issue arises that threatens to derail the project and it cannot be handled within the IPT, executive leadership must be available and involved enough to quickly and effectively resolve the issue. For example, officials from NOAA’s Geostationary Operational Environmental Satellite - R series program explained that when an issue cannot be resolved within the IPT, the program manager should raise the issue as quickly as possible to a decision-making level where resolution can be achieved. In addition, executive sponsors help projects by negotiating and addressing concerns of stakeholders not directly assigned to the team. The second key characteristic of a comprehensive IPT for major IT acquisitions is to establish a highly effective team. The five key practices contribute to the overall success of this characteristic, as shown in figure 3. Based on the program’s requirements, agencies need to develop a human resource management plan that identifies the skills needed to complete the work. Agencies then need to determine the optimal size of the core IPT team and availability of other supplementary staff. In some situations, subject matter experts may be assigned to work full time on the team, or they may rotate in and out based on the need for their expertise in a given phase. Agency officials with whom we spoke mentioned the importance of determining the required skills, optimal size, and availability. Specifically: According to officials with Commerce’s Office of the CIO, the assignment of core and supplementary staff and subject matter experts depends on the work needed to be completed. Officials from NOAA’s Office of the CIO also explained that the number and composition of team members should be commensurate with the complexity and the scope of the task. Representatives from NOAA’s Geostationary Operational Environmental Satellite - R series program told us that for larger projects, staff should be changed at specific phases of the program, such as when the program transitions from development to operations. Officials from Defense’s Office of the Under Secretary of Acquisition, Technology, and Logistics and the Office of the CIO explained that the program manager, stakeholders, and customers should be able to assign any staff member who would be of value and assistance. Once the effort is complete, the assigned staff member can be released from the IPT based on the program manager’s assessment that the resource is no longer needed. These officials added that in addition to the program’s tasks, functional capabilities, and lifecycle phase, the size of the IPT will vary based on the acquisition category of the program. Representatives from VA’s Veterans Benefits Management System noted that supplementary staff, such as end users and subject matter experts, can become a part of the core IPT during times that necessitate a specific level of expertise. These individuals can return to their regular duties once they have fulfilled their roles. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology stated that it is important to ensure adequate representation from all major stakeholder organizations, but also to safeguard that the IPT does not have too many members. According to representatives from the VA Technology Acquisition Center and the Office of Information and Technology, different members have different levels of engagement depending on the scope of the IPT and the lifecycle phase of the program. Staff may be added later or required to participate more frequently depending on the needs of the program. Each program must determine the appropriate composition of its IPT. Some roles and responsibilities are considered “core,” meaning that each team must have these staff. For example, in order for OMB to approve the IT investment, an IPT must at a minimum include: (1) a fully dedicated IT program manager; (2) a contracting specialist (if applicable); (3) an IT specialist; (4) an IT security specialist; and (5) a business process owner or a subject matter expert. Other roles and responsibilities are considered “supplementary,” meaning that they can be added as necessary during certain phases of the lifecycle. Our review of key guidance documentation and interviews with agency officials indicates that an IPT for a major IT acquisition should include a number of core disciplines. Although the descriptions of the disciplines are more detailed than those required by OMB, they are still compatible with OMB’s five categories. For example, the contracting, procurement, and acquisition professionals could fit into OMB’s category of contracting specialist. Similarly, the disciplines of software developers, test and quality assurance managers, engineers, and architects could fit into OMB’s category of IT specialist. Officials from the eight organizations that we identified for our sample were largely in agreement with these roles and responsibilities. For example, they all told us that contracting, procurement, and acquisition professionals were critical for an IPT. In addition, six organizations told us that that security and privacy specialists and requirements managers should be team members. At least four of the organizations also mentioned customers and users, budget and finance managers, software developers, test and quality assurance managers, engineers, risk managers, and sponsors. The core disciplines are shown in figure 4 by descending order in which they were cited in literature and by selected organizations, and discussed in more detail following the figure. Contracting, procurement, and acquisition professionals Contracting officers, who, according to OMB, evaluate technical competency of contractors and establish acquisition strategies to achieve the best value for taxpayer dollars. They and their support staff analyze proposals, provide independent cost estimates, and develop negotiation positions. They legally obligate the government when they enter into contracts. Contracting officer’s representatives, who manage ongoing contractor performance to ensure the government’s interests are protected. They are designated and authorized in writing by the contracting officer to perform specific technical or administrative functions on contracts or orders. Purchasing agents and procurement specialists, who assist contracting professionals with a wide range of duties, such as market research and file preparation, tracking procurements, and preparing awards for simple acquisitions. Contract specialists, who, according to OMB, support the contracting officers in accomplishing their many duties during the pre- and post- award acquisition phases. Contract specialists may also perform the role of contracting officers for small dollar value procurements. At the hub of an effective IPT is a strong program manager who stewards the process from beginning to end while achieving the program’s cost, schedule, and performance goals. The program manager could come from either the technology or mission organization, so long as the person possesses skills in both areas and operates under a strong governance process. Further, he or she should ideally be highly proficient at technical, business (both government and commercial business processes), organizational, programmatic, and interpersonal levels. Members of the organization who will approve, accept, and ultimately use the deliverables or products of the project on a day-to-day basis should be assigned to the IPT to ensure proper coordination; advise on requirements including business policies, rules, and processes; and validate the acceptability of the project’s results. Personnel from budget and finance are necessary to secure the required funding for the program. Their involvement during the acquisition process can help in the development of the acquisition strategy and limit potential problems related to financial and budgetary issues. An IPT should include government employees who are experts in system security. In addition, the sustained engagement of privacy specialists helps ensure that personal data is properly managed. Members of the team should have experience with traditional and modern development techniques such as Agile software development concepts and modular approaches. The team should have a requirements manager who understands the lifecycle of managing requirements starting with elicitation through the requirements change management process to test and evaluation. Test and quality assurance managers The test manager brings a solid end-to-end view of the testing process, including test management and the use of automated testing frameworks. Quality assurance personnel ensure consistency and quality documentation. Engineers translate operational needs and requirements into a set of system product and process solutions that satisfy customer needs. Teams benefit from having IT architects who can develop a solution that will interoperate with the agency’s internal and external systems. IPTs should include risk managers who are knowledgeable in the processes of risk management. This includes risk identification, analysis, and response planning that may decrease the likelihood and impact of negative events in the program. Each IPT should include the sponsor, who is the executive-level person or group who provides resources and support for the program and who is ultimately responsible and accountable for enabling its success. In addition to the core disciplines, other supplementary staff members could be added to the IPT on an as-needed basis. These include positions such as: Administrative personnel. These personnel are responsible for activities such as document management, communications, outreach, and meeting facilitation and recording. Organizational change managers. These managers execute strategies and techniques required for effectively planning, implementing, and evaluating change in the organization. Configuration managers. These managers apply principles and methods for planning and managing the implementation, update, and integration of information systems components. Earned value management specialists. These specialists can combine scope, schedule, and resource measurements to help the program team assess and measure project performance and progress. Human resources representatives. Human resources representatives are involved in (1) identifying and documenting project roles, responsibilities, required skills, and reporting relationships; (2) confirming human resource availability; (3) improving competencies, team member interaction, and team environments; and (4) tracking team member performance, providing feedback, and resolving issues. Governance managers. Governance managers are IT officials involved with strategic capital planning and investment control processes. Schedulers. Schedulers plan, develop, manage, execute, and control the program schedule in order to ensure the timely completion of the project. Designers. Designers have core knowledge of sustainable design. Legal counsel. Counsel should have the expertise to assist with acquisition-related and legal issues such as the award of contracts, intellectual property rights, licenses, and other matters. Operational support personnel. These personnel are responsible for activities such as system operations, implementation/deployment management, software maintenance, training, customer service, and system performance. They can be from external organizations that have a relationship with the program to provide specialized expertise or fill a specified role, such as IT system installation, customization, or training. Ideally, the team will not only consist of members from a wide selection of subject matter disciplines, but individual members will also have a variety of technical skills and expertise. In addition, important soft skill sets may also be considered. These include skills such as problem solving and decision-making, interpersonal skills, ability to communicate and work with customers, team building, and conflict resolution. One of the more important soft skills is the ability to deal with adversity and ever changing circumstances. Specifically for the program manager, important selection criteria include: (1) lack of bias, (2) technical expertise, (3) project management skills, (4) ability to manage external environment, (5) team engagement skills, (6) decisiveness, (7) time management skills, (8) ability to effectively elevate and delegate decisions, and (9) commitment to the IPT’s work. Several agency officials discussed the value of both technical and non-technical attributes. Specifically: Officials with Commerce’s Office of the CIO explained that a desirable characteristic of an IPT is integration, which is embodied by bringing together different organizations that have a stake in the program, each providing differing perspectives and skills which are vital to the successful outcome. Each team should possess the knowledge to collaboratively identify problems and propose solutions, minimizing the amount of rework that has to be done. According to officials from NOAA’s Office of the CIO, members should be capable of providing input regarding multiple disciplines, not just their area of primary responsibility. They added that on a multidisciplinary team often the best team members are individuals who have multiple types of expertise. Representatives from NOAA’s Geostationary Operational Environmental Satellite - R series program stated that IPTs should have personnel with specialized skill sets to bridge any gaps in the program. They should also have cross-functional skill sets that enable effective communication with managers inside and outside of their primary knowledge domain. GSA Assisted Acquisition Services, Federal Systems Integration and Management Center officials explained that they have created a structure that integrates the acquisition workforce with the technical resources under the same management. In addition, they have developed a role called the acquisition project manager, which is a hybrid position of technical expertise and acquisition knowledge that is present for pre-award activities such as technical evaluations. The representatives added that the integration of acquisition and technical resources, sharing the same management, leads to more efficiency and alignment with the mission. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology told us that it is desirable for IPT members to have the necessary acquisition, business, technical and programmatic knowledge, as well as the expertise to facilitate and leverage the process to help define user needs, facilitate discussions, make recommendations, resolve issues, and make decisions. VA Veterans Benefits Management System officials explained that the most critical characteristic of a comprehensive IPT is a joint team consisting of IT resources and business resources, where the IT staff understands the business needs of the users and the users have the technical experience to assess alternative IT solutions and weigh the tradeoffs of each. Officials told us that they attempted to ensure frequent collaboration between business and IT team members during key phases of projects—from planning to deployment. They stated that this practice was essential to the delivery of functionality to end- users. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology stated that desirable non- technical skills of IPT members include problem solving and decision- making, team building, interpersonal skills, and conflict resolution. Officials with Commerce’s Office of the CIO noted that adaptability is a desirable characteristic of an IPT member. Representatives from NOAA’s Geostationary Operational Environmental Satellite - R series program said that IPT members need to be flexible and open to change when needed to address team-wide priorities. The IPT needs to ensure that its members have received two types of training: (1) training related to their specific roles and (2) training related to their involvement in an IPT in general. The first type of training involves measuring the skills and knowledge of the staff in relation to their roles. Based on this assessment, the team should provide role-specific training as necessary to support the execution of the program. Providing cross-training to team members can reduce problems which arise during member absences. For the second type of training, the team should provide its members with IPT-specific training to create the knowledge and culture necessary to ensure its success. This can include training on items such as (1) the purpose of an IPT, (2) why an IPT is being used in this case, (3) how an IPT needs to function, and (4) behavior and skills required of IPT leaders and members. This training will require the use of additional resources upfront, but it can result in greater team effectiveness and efficiencies later. Officials from Commerce mentioned the value in ensuring appropriate training. Specifically: Officials from NOAA’s Office of the CIO told us that in addition to having knowledge regarding the mission subject, team members need the proper training in IPT framework and discipline. Representatives from Commerce’s Office of the CIO explained that training will assist with ensuring all of the team members are on board with the new program, especially if they are being taken from their normal daily duties. The core members of the team, including all leadership roles, should be in place throughout the program lifecycle. IPTs should be in place from the initial concept and development phases, through the delivery and implementation of the last increment under the contract. The team lead should be 100 percent dedicated to the program during this time frame. However, there may be instances where other team member services are not needed on a full-time basis. In those cases, team member support to the IPT, when needed, should take priority over the team member’s other duties. It is also ideal to have contractor staff who are stable and consistent. Having continuity of government and contractor staff working on a project throughout its lifecycle reinforces accountability. Agency officials discussed the importance of maintaining stability in the team. Specifically: Representatives from the VA Veterans Benefits Management System and NOAA’s Geostationary Operational Environmental Satellite - R series program stated that core staff should be assigned to the IPT for the duration of the program. According to representatives from the VA Technology Acquisition Center and the Office of Information and Technology, members should be available for the duration of the IPT, adding that commitment and regular and consistent participation are critical to the success of the team. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center noted IPT members should be assigned as soon as the requirements are known. Each member should have a clear expectation for the level of involvement, time frame for participation, and response at the conclusion of the phase. They also stated that roles and responsibilities of team members should be revisited at each phase of the acquisition. Representatives from Commerce’s Office of the CIO noted that it is desirable that backup representatives on the IPT be well apprised of the team’s agenda, issues, and decisions. The third and final key characteristic is to operate the team using effective processes. As shown in figure 5, six key practices provide additional detail for the successful implementation of this characteristic. The IPT should have a charter that documents its need, purpose, and scope. The charter should further clarify and elaborate goals, outcomes, and performance measures. In addition, it should be reviewed by, and receive concurrence from, executive leadership. It should also be updated as necessary, but at least at the start of each work phase. Agency officials mentioned the value of documenting the team’s charter. Specifically: Officials with Commerce’s Office of the CIO stated that the charter should be established early on in the planning process, and it should include (1) the expectations of the program team, (2) the scope of the IPT’s authority, (3) the metrics by which the success of the IPT will be evaluated, (4) the identification of the customers, (5) the amount and types of funds available to the IPT, and (6) the expertise the team must have. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology told us that the charter should document the roles and responsibilities of each team member. According to officials with Defense’s Office of the Under Secretary of Acquisition, Technology, and Logistics and the Office of the CIO, the charter should document the customers, end users, and other stakeholders who have a vested interest in the outcome of the IPT. For effective operations, it is important to create and document IPT-specific decision-making processes that align with the organization’s policies and procedures. These include rules and guidelines for things such as: (1) team structuring, (2) formal lines of communication, (3) establishment of authority, (4) resource allocation, (5) work review and approval, and (6) reporting requirements. Several agency officials discussed the importance of providing the IPT with rules and guidelines. Specifically: Officials from NOAA’s Office of the CIO explained that the team should establish the mission criteria, schedule, and performance measures before the program begins. The IPT should report on the status at the expected intervals using monthly written status reports and quarterly project reviews. Representatives of NOAA’s Geostationary Operational Environmental Satellite - R series program told us that team members should understand their roles and responsibilities so they know what is required of them and the value they bring to the team. According to representatives from the VA Veterans Benefits Management System, they document roles and responsibilities, including an organizational chart and a responsibility assignment matrix. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology told us that each member needs to have his or her role defined based on expertise. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center stated that an IPT should have defined roles and responsibilities, as well as consistent, mature processes. To that end, they have established standard operating procedures, training, and templates for organizations to use. These include (1) a roles and responsibilities matrix template to document the structure, roles and responsibilities, and operating procedures; (2) a project management tool to document the acquisition schedule and to provide a quick overview of dates for major milestones; and (3) an acquisition start meeting template to document the roles, responsibilities, existing information, acquisition schedule, procurement integrity briefing, project administration, and next steps. Defense’s Office of the Under Secretary of Acquisition, Technology, and Logistics and the Office of the CIO officials stated that each IPT’s structure should be individualized based on the program’s task, acquisition, type of capability, technology, operational mission, and other various factors. Co-location places many or all of the most active team members in the same physical location. The benefits of co-location can include easier information sharing and communication, earlier identification of issues, and more effective decision making. At a minimum, the IPT lead and the contracting officer should be co-located. However, co-location is not always practical. Another option is temporary co-location—bringing staff members together on a limited basis during strategically important times. For example, an especially important time is requirements definition, which is an area where translation issues between business users and developers can cause problems if requirements are not clearly articulated, understood, and documented. As an alternative, effective virtual teams can be created through the use of shared networks, software, tools, databases, and teleconferencing, which allows team members from different locations to work together regardless of the physical location. According to recognized guidance, the benefits of virtual teams include the addition of skilled resources, reduced travel and re-location costs, and the proximity of team members to other critical resources. Agency officials highlighted the need to make trade-offs to facilitate teamwork. Specifically: Officials with Commerce’s Office of the CIO explained that IPTs are not periodic meetings; rather they are daily work routines where members must work closely with their teammates and the program manager to produce a viable product as defined by business requirements. As such, co-location of all members is preferable as it facilitates frequent and timely discussions, problem-solving, and decision-making. NOAA demonstrated commitment to this approach with its Geostationary Operational Environmental Satellite – R series program. Although the program is overseen internally within Commerce, it relies on acquisition experience and technical expertise supplied by the National Aeronautics and Space Administration. In order to leverage this knowledge, NOAA implemented an integrated program management structure and located the program office at the National Aeronautics and Space Administration’s Goddard Space Flight Center. Officials with the Geostationary Operational Environmental Satellite – R series program told us that bringing team members to the Goddard Space Flight Center was advantageous because the co-location ensured that engineering resources were available, if needed. Representatives from the VA Veterans Benefits Management System told us that they use a mix of virtual teams and co-location as appropriate. For example, temporary co-location often occurs during requirements elicitation, system design, and user acceptance testing. Officials from the Defense’s Office of the Under Secretary of Acquisition, Technology, and Logistics and the Office of the CIO noted that co-locating team members is desirable but not mandatory in today’s virtual world. The use of virtual technology provides a means for high success in meeting the goals and objectives of the IPT without all members being co-located. For example, the use of virtual technologies such as teleconferencing, portals, instant messaging, and e-mails may help ensure requirements are clearly articulated and adequate oversight of the contracted work is accomplished. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center told us that geography matters and there is no substitute for face-to-face communication. Ideally the program manager should work in the same location as the customer at least 2 days a week to provide the best customer service possible. However, officials added that they have examples of successful IPTs where the team is not co-located. In those cases, the use of virtual tools such as telephone, e-mail, online documents, video chat, web messaging, and virtual meetings assisted in ensuring a seamless acquisition process. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology stated that although they consider co-location to be optimal, it is not always practical. As a result, they often use on-line tools to facilitate effective virtual meetings. Stakeholders are individuals or groups affected by the outcome of a project, and may or may not be assigned to the core team. Either way, stakeholder involvement should be viewed as a valuable asset to the team, and stakeholder satisfaction should be seen as a key project objective. Stakeholders should represent all relevant acquisition, technical, support, and operational organizations. They are working team members and should be included in activities such as planning, decision- making, commitments, communication, coordination, reviews, appraisals, requirements definition, and resolution of problems and issues. Reviews and exchanges are regularly conducted with stakeholders to ensure that coordination issues receive appropriate attention and everyone involved with the project is appropriately aware of status, plans, and activities. The involvement of stakeholders is especially important in Agile development, which emphasizes collaboration more than traditional approaches do. For example, in Agile development, project status is primarily evaluated based on demonstrations of working software provided to stakeholders and customers. Agency officials with whom we spoke discussed the value of actively involving stakeholders. Specifically: Officials with Commerce’s Office of the CIO stated that ensuring the stakeholders were supportive from the beginning to the end of the program was a critical success factor. Representatives with NOAA’s Office of the CIO noted that the IPT should engage with internal stakeholders through weekly or monthly status reports, monthly or quarterly interim project reviews, and milestone briefings. Representatives from the VA Veterans Benefits Management System told us that engaging with stakeholders requires ongoing collaboration and open communication to ensure transparency and the optimization of inputs and ideas. They also explained that the program solicited users’ input on requirements for the development and enhancement of the system. Field subject matter experts from across the country participated in requirements, design, and testing sessions with system developers. Representatives from the VA Technology Acquisition Center stated that IPTs should communicate with internal and external stakeholders regarding the status and progress of the project. According to officials in VA’s Office of Information and Technology, the appropriate representation within the team ensures that all stakeholders are aware of and committed to completing key project deliverables on time. This approach enables and requires coordinated teamwork from multiple organizations to be a key driver of the project’s daily performance. In addition, they noted that the customer community is involved in several ways throughout the project lifecycle. For example, the business sponsor is a representative and advocate for the customer community. The customer community is involved throughout the lifecycle of the project, and makes the final determination of acceptability in user testing to accept delivery of capabilities. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center told us that IPTs should engage with internal and external stakeholders as often as feasible. They explained that it is important for the team to fully understand the political landscape and be able to identify all critical stakeholders early in the process. Subsequently, teams should engage with stakeholders by keeping key players involved in the process to ensure they are working toward meeting the mission and strategy goals. Officials from Defense’s Office of the Under Secretary of Acquisition, Technology, and Logistics and the Office of the CIO stated that teams should engage with stakeholders on a regular basis and as frequently as possible. Ideally, the amount of communication and the manner in which it is documented would be tailored to fit the needs of the team. For example, teams may establish regularly scheduled meetings. Records from the meetings can be maintained in a form that allows for ready access at a later date. In addition, teams may establish methods to identify and resolve conflicts that arise among stakeholders. According to recognized guidance, IPTs need to be particularly sensitive to the internal dynamics and style of interaction between members both during and outside of meetings. It can be helpful to have ground rules about how to respectfully share and acknowledge different points of view, ensure balanced participation from all members, build trust and collegiality, deal with conflict, and maintain morale and team spirit. The use of a facilitator during meetings could be a valuable tool in effective communications. Agency officials with whom we spoke mentioned the importance of communication. Specifically: Officials from Commerce’s Office of the CIO stated that communications should be tailored to the technical level of the audience. One effective method of communication is the use of a program management tool to allow team members to access any information at any time. Communications should be two-way; there should be a way for the stakeholders to provide feedback to the IPT. Officials with NOAA’s Office of the CIO told us that the frequency of communication should be commensurate with the scope and complexity of the milestone or urgency of the pace of tasks. Representatives from the VA Veterans Benefits Management System explained that they have found that frequent and structured meetings are the most effective way of communicating progress and addressing issues. The IPT participated in daily and weekly meetings to discuss key topics, risks, and issues. In addition, functionality-specific meetings occur on an as-needed basis to ensure continued collaboration. Communication could be facilitated by an intranet site, a suite of communication materials, downloadable toolkits, factsheets, and explanatory videos designed to build awareness and provide necessary information for customers and end-users. Officials reported that information sharing and coordination with other VA organizations ensures awareness and understanding of changes prior to implementation. In addition, engaging stakeholders across multiple vehicles on a recurring basis was integral to the responsive and iterative approach of the team. Representatives from the VA Technology Acquisition Center and the Office of Information and Technology stated that they communicate progress and issues by meeting regularly (e.g., weekly/monthly), documenting results and issues in meeting minutes, conducting formal milestone reviews, and escalating issues as necessary. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center told us that after an initial acquisition kickoff meeting, the team works together to set up standing weekly (or more frequent depending on the project) meetings. During these meetings, they review schedules, plans, and deliverables with each other and the client. The IPT should be communicating often to ensure members are always working towards the same goal. Once a week the program manager updates the acquisition schedule so that leadership has the most up-to-date information. Once a systemic process exists for schedule communication, management can more easily identify outliers and focus on addressing problem areas. During the various meetings, team members must have the opportunity to voice dissent. Once decisions are made, the team must reach buy-in and proceed. According to officials from Defense’s Office of the Under Secretary of Acquisition, Technology, and Logistics and the Office of the CIO, the IPT should have regularly scheduled meetings, at least monthly, or more frequently if software is being deployed more often. Meetings are intended to bring together the program office, contractors, technical expertise, stakeholders, customers, and others to view the development, progress, and issues related to acquisition of the capability. In addition, there should be opportunities to discuss matters between meetings. There should be open discussions with no secrets. The IPT is often assigned the responsibility to oversee the performance and acceptance of contractors’ deliverables or services. If the contractors bear a large share of the risk for delivering the project’s results, they may play a significant role on the project team. The goal is to ensure that government personnel maintain control over key decisions and expenditure of funds. Agency officials with whom we spoke highlighted the importance of effectively overseeing contractors’ activities. Specifically: Officials from NOAA’s Office of the CIO stated that an IPT can benefit from contractors serving as administrative staff and subject matter experts. However, the number of government team members should be equal to or greater than the number of contractors. Contractor staff should not be making important business decisions, and government staff should review and approve recommendations made by contractors. Officials with Commerce’s Office of the CIO told us that the ratio of government staff to contractor staff should not matter as long as the contractor is doing the scoped work, maintaining its metrics, being held to specific performance standards, and overseen by the contracting officer’s representative. However, they noted that IPTs should try to keep a ratio of government to contractor staff that ensures enough government participation so that decisions, changes, and approvals, can be executed in an efficient manner, and the government maintains accountability and traceability. Officials with the VA Technology Acquisition Center and the Office of Information and Technology stated that contractors can serve in support functions such as document preparation and meeting facilitation. However, any inherently government function that requires decision-making should not be performed by contractors. Representatives from the VA Technology Acquisition Center and the Office of the CIO also agreed that that the majority of personnel on an IPT should be government employees. In addition, officials noted that any poor performance by contractors can be documented in performance reviews and can lead to corrective actions, or if necessary, contract termination. Finally, according to VA officials, contractor personnel should sign non-disclosure agreements. Representatives from the VA Veterans Benefits Management System noted that contractors can complement the IPT’s effectiveness by providing experiences or resources in areas such as systems engineering, software development, testing, requirements analysis, communication, change management, business architecture, and program management. However, the team members need to follow the roles, rules, and guidance set forth by the Federal Acquisition Regulation. Specifically, the government staff are to perform and complete inherently government tasks and contractors only perform work based on the statement of work for which they have an approved contract. In addition, VA officials noted that each contract is to have a dedicated contracting officer representative responsible for ensuring contract deliverables meet quality and schedule parameters established by the government. Representatives from GSA’s Assisted Acquisition Services, Federal Systems Integration and Management Center explained that it is important to document roles, responsibilities, and expectations of any contractor team member. According to these officials, the contracting officer, contract specialist, and program manager should be government employees, not contractor employees. However, these officials stated that functions such as assisting the program manager with financial management, tracking schedules, preparing deliverables, creating an operational framework, and serving as a technical writer are appropriate uses of contractor resources. The presence—or absence—of these IPT characteristics can have an impact on the success or failure of a program. We have previously reported on weaknesses in practices related to each of three characteristics (effective leadership, team composition, and processes) that have adversely impacted major IT acquisitions. Shortfalls in leadership. Our recent review of the Department of Homeland Security’s Human Resources Information Technology investment revealed a lack of oversight from the department’s executive steering committee. For example, the executive steering committee met only once from September 2013 through June 2015—in July 2014—and was minimally involved with overseeing the investment’s 15 key areas identified as needing improvement during that period. As a result of the executive steering committee not meeting, important governance activities were not completed, including review of the investment’s key strategic planning document. We reported that this this lack of involvement contributed to Department of Homeland Security being able to deliver only 1 of 15 human resources capabilities which the department said would be mostly implemented by June 2015. Until the executive steering committee effectively carries out its oversight responsibility, the department will be limited in its ability to improve human resource investments which are needed to carry out its mission. Shortfalls in team composition. Our previous work on efforts of VA to share electronic health records with Defense highlights the importance of having sufficient staff on the IPT. Specifically, in 2009 the two departments set up an Interagency Program Office to oversee the development of fully interoperable electronic health records. The office was given the responsibility for oversight and management, stakeholder communication, and decision-making. Specific tasks included the development of a plan, schedule, and performance measures to guide the departments’ interoperability efforts. In February 2011, we reported that the Interagency Program Office had not fulfilled some of its major responsibilities. The office was subsequently re-chartered and given more staff to complete its work. Specifically, officials with the Interagency Program Office noted that it was authorized to eventually have a staff of 236 personnel, more than 7 times the number originally allotted. However, the officials stated that as of January 2013, the office was staffed at only 62 percent—146 personnel. Officials stated that hiring additional staff remained one of its biggest challenges. As another example, our ongoing work regarding the U.S. Census Bureau’s plans to conduct the 2020 Census has demonstrated the need to provide its staff training related to program roles and responsibilities. Specifically, in 2013 the Census Bureau measured the skills and knowledge of the staff by conducting a workforce competency assessment. The assessment identified several mission-critical gaps. For example, the agency found that competency gaps existed in cloud computing, security integration and engineering, enterprise/mission engineering lifecycle, requirements development, and Internet data collection. It also found that enterprise-level competency gaps existed in program and project management, budget and cost estimation, systems development, data analytics, and shared services. These mission-critical gaps would have challenged the Census Bureau’s ability to deliver critical IT-related initiatives. In response, the agency identified actions to close the competency gaps by December 2015, including several training- related actions. For example, it planned to implement centralized and integrated professional training and development activities, create environments conducive to on-the-job training from embedded personnel, and leverage staff with related knowledge and experiences to provide in-house workshops that share skills and experiences. We reported that fully implementing these activities would be critical to ensuring that the Census Bureau has the skills it needs to effectively develop and implement systems vital to the upcoming Census. Shortfalls in team processes. Our prior work on the Federal Aviation Administration’s acquisitions program for air traffic control facilities demonstrated the problems associated with processes that did not facilitate sufficient involvement of stakeholders throughout the system’s development. Specifically, the Federal Aviation Administration did not include air traffic controllers in the design and testing of one of its integral air traffic control programs. Because of this, issues that could have been addressed early in the design phase were not discovered until implementation, resulting in cost and schedule impacts. Partly due to not having valuable stakeholder involvement, the program experienced a cumulative 60 percent cost increase and schedule delays averaging 4 years. In response, the agency took steps to involve stakeholders by signing a memorandum of understanding with the air traffic controllers’ union. The memorandum called for, among other things, a team of 16 controllers to be detailed from various locations to test and validate software fixes with engineers at the Federal Aviation Administration Technical Center. As another example, our prior review of the United States Department of Agriculture’s Farm Service Agency’s Modernize and Innovate the Delivery of Agricultural Systems program showed that agency policies for effective communication were not followed. For example, the program did not keep records of which artifacts were reviewed during meetings to assess system developmental progress and maturity and did not adequately track decisions and corrective actions. This hindered executive level governance over the program, which contributed, in part, to significant challenges for the program. As a result of these challenges, further development of the system was ultimately halted after the program had spent about $423 million and delivered only about 20 percent of the functionality which had been envisioned. Implementing the key characteristics of IPTs for major IT acquisitions can mitigate the risk of aforementioned problems from occurring. Also key to an agency’s success in developing IPTs is sustaining a workforce with the necessary knowledge, skills, and abilities to execute a range of management functions that support its mission and goals. One key element cuts across the characteristics and practices needed to build a strong IPT: IT workforce planning. For example, leadership needs accurate information about the team’s staffing needs in order to provide sufficient staffing resources. Agencies can help ensure IPTs have the requisite skill sets, team size, and set of disciplines by developing competency and staffing requirements, assessing gaps, and developing strategies and plans to address those gaps through additional training or other measures. Agencies can also utilize staffing strategies, such as contractor assistance, if appropriate, to address IT skill gaps and ensure effective team operations. As discussed previously in this report, a number of federal laws as well as guidance issued by OMB and OPM address IT workforce planning activities for federal agencies. Several of the requirements and recommended practices from those sources also generally align with key principles and activities identified in our and OPM’s strategic workforce planning models. Consequently, to support the evaluation of whether selected federal agencies are adequately assessing and addressing gaps in IT knowledge and skills, we established an evaluation framework based on these laws and guidance, and vetted it with internal and external stakeholders. The framework contains four steps: (1) setting the strategic direction for IT workforce planning, (2) analyzing the IT workforce to identify skill gaps, (3) developing and implementing strategies to address IT skill gaps, and (4) monitoring and reporting progress in addressing IT skill gaps. Each of the four steps is supported by key activities. Table 1 summarizes the four steps and eight key activities used in our evaluation of selected federal agencies. When effectively implemented, IT workforce planning activities can facilitate the success of major IT acquisitions. Ensuring program staff have the necessary knowledge and skills is a factor commonly identified as critical to the success of major IT investments. If agencies are to ensure that this critical success factor has been met, then IT skill gaps need to be adequately assessed and addressed through a workforce planning process. Selected federal departments had mixed progress in assessing their IT skill gaps. While they have demonstrated important progress in either partially or fully implementing key IT workforce planning activities, each of the five departments had shortfalls. The five departments have started focusing on identifying cybersecurity staffing gaps, but more work remains in assessing competency gaps and in broadening the focus to include the entire IT community. Figure 6 illustrates the extent to which selected departments have fully, partially, or not implemented key IT workforce planning activities. As shown in the figure, of the five departments, Defense had the most robust implementation with all eight activities fully or partially implemented. Commerce has not yet established a process to guide its IT workforce planning activities. Of the eight activities to assist an agency with effective workforce planning, Commerce has partially implemented three activities and has not implemented five activities. Table 2 identifies the extent to which Commerce has implemented key IT workforce planning steps and activities. Several circumstances have contributed to the department’s shortfalls in IT workforce planning. For example, Commerce’s efforts to gather and utilize information for IT workforce planning have been ad hoc. While Commerce participated in a government-wide IT workforce assessment in 2011, it was unable to demonstrate how it used information from the assessment for determining IT competency and staffing requirements, identifying gaps, and developing strategies to address gaps. Further, the department attempted to collect updated data in recent years on IT workforce staffing levels, certifications, and skill gaps. However, it was unable to demonstrate how they used this information for workforce planning and noted that data collected in 2015 on skill gaps were not used because the data were unreliable. In addition, Commerce officials noted that while the department has begun to draft IT competency requirements and has reviewed several sources, the department does not yet have policies and procedures that incorporate key IT workforce planning steps and activities. Commerce officials acknowledged that shortfalls exist in the department’s workforce planning activities and stated that they are working to develop a more effective IT workforce assessment plan in fiscal year 2017. Until Commerce fully implements key IT workforce planning activities, management will have a limited ability to assess and address gaps in knowledge and skills that are critical to the success of major acquisitions. As a result, it will be more difficult to anticipate and respond to changing staffing needs and control human capital risks when developing, implementing, and operating critical IT systems. Defense has established processes that have enabled it to perform key analyses and assessments on its IT workforce. Specifically, of the eight activities associated with workforce planning, the department has fully implemented one and has partially implemented seven. Table 3 identifies the extent to which Defense has implemented key IT workforce planning steps and activities. To its credit, Defense has established competency models, conducted gap assessments, and is developing strategies for two of its three major IT occupations and officials reported that the department is working on revalidating competencies for its third major IT occupation by fiscal year 2016. According to department workforce data from March 2015, these three occupations comprised approximately 93 percent of its IT workforce. However, Defense has not yet established competency models and conducted gap assessments for the remaining 15 IT occupations because it has focused initially on developing competencies and assessing gaps for its mission-critical IT occupations after developing a tool in 2014 that it could use in performing such enterprise assessments. Further, officials from Defense’s Office of the CIO have recognized limitations with assessing IT skill gaps solely based on existing occupational titles and position descriptions. As a result, the department is in the process of implementing strategies and plans to establish a set of work roles and a baseline set of knowledge, skills, and abilities that apply to personnel spread across a number of functional communities who perform significant cybersecurity, IT, and other cyber-related roles. According to Defense’s Strategic Workforce Plan for fiscal years 2014 through 2019, the department anticipates that this effort will help in monitoring the overall health and capability of the workforce at a granular level not previously attainable. By not fully implementing key IT workforce planning activities, Defense runs the risk that it cannot adequately assess and address gaps in knowledge and skills that are critical to providing support for major acquisitions. As a result, Defense may have difficulty in anticipating and responding to evolving staffing needs while developing, implementing, and operating critical IT systems. The Department of Health and Human Services (HHS) has begun building its capability to set the strategic direction for its IT workforce by taking foundational steps towards establishing a workforce planning process. Of the eight activities needed to perform workforce planning, HHS partially implemented six and did not implement two. Table 4 identifies the extent to which HHS implemented key IT workforce planning steps and activities. Several explanations exist for the department’s shortfalls in IT workforce planning. HHS does not have comprehensive policies that require the CIO to implement key IT workforce planning activities, such as conducting IT skill gap assessments and developing strategies and plans to address those gaps. Officials noted that they have such guidance under development. HHS has not recently conducted skill gap assessments of its IT or cybersecurity workforce because it decided to focus first on establishing competencies and career paths for its IT program manager and cybersecurity positions. To this end, department officials have acknowledged that more comprehensive IT workforce planning needs to occur to address requirements of laws, such as the Federal Cybersecurity Workforce Assessment Act of 2015, and have begun taking steps to do so. In particular, HHS identified planned steps that, when implemented, could help the department to code and assess skill gaps for its IT and cybersecurity workforce at a more granular level. HHS officials expressed concerns regarding the variety and complexity of requirements for IT workforce planning from federal laws and other guidance. Officials noted that additional guidance from OMB could help federal agencies avoid potentially duplicative and inefficient implementation of the various workforce planning steps and activities. To its credit, HHS has recently taken positive steps to implement key IT workforce planning activities. In June 2015, the department initiated a cybersecurity workforce development program and subsequently established competencies and career paths for cybersecurity related occupations. According to officials, HHS is currently working to merge the ongoing cybersecurity workforce development efforts into a broader IT workforce planning effort that will allow the department to conduct gap assessments and develop strategies, such as retention and recruitment plans, based on a more comprehensive workforce planning process. Until HHS fully implements key IT workforce planning activities, management will have a limited ability to assess and address gaps in knowledge and skills that are critical to the success of major acquisitions. As a result, it will be difficult for HHS to anticipate and respond to changing staffing needs and control human capital risks when developing, implementing, and operating critical IT systems. The Department of Transportation (Transportation) is building its capability to assess IT skill gaps, but it currently does not have foundational elements of a workforce planning process. Specifically, it has partially implemented five activities and has not implemented three activities. Table 5 identifies the extent to which Transportation implemented key IT workforce planning steps and activities. Transportation officials provided explanations for the department’s shortfalls in IT workforce planning. For example, the department has only recently established policies to guide its workforce planning activities. In addition, the department did not consider hiring staff to ensure that program managers were certified to be a priority. To its credit, Transportation has plans to improve its IT workforce. For example, the department is planning to (1) develop plans by the end of fiscal year 2016 that could lead to the creation of an IT acquisition cadre and use of direct hiring authority, (2) draft an approach for cross-functional training of IT acquisition and program personnel, and (3) develop a career path for program managers by fiscal year 2017. While these are positive steps, these actions are not yet rooted in an IT workforce planning process that lays the foundation for how the department is to analyze its IT workforce skill gaps, develop and implement strategies to address gaps, and monitor and evaluate progress in addressing the gaps. By not fully implementing key IT workforce planning activities, Transportation has a limited ability to assess and address gaps in knowledge and skills that are critical to the success of major acquisitions. As a result, it will be difficult to anticipate and respond to changing staffing needs and control human capital risks when developing, implementing, and operating critical IT systems. The Department of the Treasury (Treasury) has taken steps to improve its IT and program management workforce by partially implementing seven activities; however, the department has not yet implemented one key workforce planning activity. Table 6 identifies the extent to which Treasury has implemented key IT workforce planning steps and activities. According to Treasury’s March 2016 FITARA Common Baseline Implementation Plan, the department is working to develop plans and processes by which the CIO, chief human capital officer, and senior procurement executive will develop a common set of competency requirements for IT staff and a workforce planning process. Treasury has a policy for utilizing a department-wide strategic workforce planning model. However, the department has deferred the use of this model on the IT workforce until the results of an Internal Revenue Service IT workforce assessment pilot are available and have been reviewed by the CIO and chief human capital officer. Treasury’s FITARA Common Baseline Implementation Plan noted that the timing of these steps is difficult to estimate because each decision point depends on the outcome of the preceding step and because staff and managers would need to coordinate workforce planning with other time demands. According to its implementation plan, among other milestones, the chief human capital officer is to provide a recommendation to the CIO for a process and tools to conduct a competency-based workforce planning study of the Treasury IT workforce in fiscal year 2017. While Treasury’s plans are a good starting point, it is too soon to tell whether these plans will address the shortcomings we identified. By not fully implementing key IT workforce planning activities, Treasury has a limited ability to assess and address gaps in knowledge and skills that are critical to the success of major acquisitions. Until the department addresses the shortcomings in its IT workforce planning activities, it will be more difficult to anticipate and respond to changing staffing needs and control human capital risks when developing, implementing, and operating critical IT systems. Agencies can improve the success of their IPTs for IT acquisitions through effective executive leadership that emphasizes team support, empowerment, and oversight; effective team composition; and effective processes for team operations. When implemented, these key practices can better position agencies to efficiently make decisions that cross lines of expertise. The right mix of expertise to recognize problems early and the requisite authority to do something about them are contingent upon effective IT workforce planning. While the five selected departments have demonstrated progress by implementing key IT workforce planning activities, each had shortfalls. To their credit, all five had at least partially analyzed their IT workforce and implemented certain activities to enhance workforce skills. In addition, several departments identified promising activities that, when implemented, could bolster effectiveness in assessing and addressing IT skill gaps. For example, Defense and HHS have initiated efforts to assess the IT and cybersecurity workforce at a more granular level and the Internal Revenue Service within Treasury is piloting a workforce assessment tool that may help the department assess its skill gaps. However, only Defense demonstrated that it had implemented a workforce planning process. Further, none of the departments identified IT competency gaps for their entire workforce, and only three were performing some level of monitoring towards the closure of identified skill gaps. Until departments fully implement the key activities and steps, they will have a limited ability to assess and address gaps in knowledge and skills that are critical to the success of major IT acquisitions. To facilitate the analysis of gaps between current skills and future needs, the development of strategies for filling the gaps, and succession planning, we are making a total of five recommendations to the Secretaries of Commerce, Defense, Health and Human Services, Transportation, and the Treasury. We received comments on a draft of this report from the five departments to which we made recommendations—Commerce, Defense, Health and Human Services, Transportation, and the Treasury. Of those five departments, four agreed with our recommendations and one, Defense, partially agreed. The following summarizes each department’s comments. In written comments, Commerce agreed with our recommendation and stated that it will work to develop an action plan to address the key workforce planning steps and activities identified in this report. The department’s comments are reprinted in appendix II. In its written comments, Defense partially agreed with our recommendation and discussed its progress on selected activities. In commenting on the activity involving the development of strategies and plans to address gaps in competencies and staffing (activity 5), the department stated that it had provided additional information on its related efforts in its Strategic Workforce Plan for fiscal years 2016 through 2021. However, the department did not attach a copy of the workforce plan. Moreover, the workforce plan, as described, does not cover staffing gaps or address Defense’s entire IT workforce. In addition to its workforce planning strategies, the department commented on its efforts to develop competencies, as part of the activity to establish requirements for competencies and staffing (activity 2). The department stated that it has established competencies for the Information Technology Management and Computer Science series and is scheduled to complete competencies for the Telecommunications series, its third and final mission-critical IT occupation, by the end of December 2016. The department further stated that, in addition to its three mission-critical occupations, it has developed competencies for its next largest IT occupation, Computer Clerk and Assistant (series 0335). This is to result in a total of 95 percent of workforce competencies developed or to be developed by the end of December 2016. Nevertheless, while establishing competencies is an important step toward robust IT workforce planning, it remains important for Defense to also identify how it intends to complete the workforce planning cycle for staff represented by each of the occupations for which it has developed or plans to develop competencies. Assessing competency gaps and developing gap mitigation strategies are important next steps. Looking across all of the activities, the department further stated that, in addition to the four occupations comprising 95 percent of the IT workforce, half of the department’s remaining 14 IT occupations are filled with less than 100 staff, and that several of the remaining occupations are likely to experience attrition. The department added that it does not believe it is necessary to develop competencies, identify competency gaps, and develop gap closure strategies for the remaining 5 percent of its IT workforce, as these individuals would likely not have a role on integrated program teams or be in a position critical to the success of major IT acquisitions. We understand the importance of Defense prioritizing its efforts to focus on the 95 percent of its workforce that are likely to have a role on integrated program teams or be in a position critical to the success of major IT acquisitions. However, identifying gaps and developing strategies for the remaining 5 percent of its IT workforce will become more relevant as the department begins its transition to focus on work roles and knowledge, skills, and abilities rather than occupations. Therefore, we maintain that our recommendation is warranted. The department’s comments are reprinted in appendix III. HHS also provided written comments, in which the department agreed with our recommendation and identified several planned or ongoing actions to address it. The department stated that it is growing its capability for mature workforce analytics and planning, in part, through the identification and definition of critical IT and cybersecurity role categories and competency requirements. The department also noted that it plans to utilize codes from OPM and the National Institute of Standards and Technology in its efforts to assess skill gaps and staffing needs. In addition, HHS provided new documentation of its completion of an IT program manager career path. We modified our assessment and corresponding recommendation for the relevant activity (activity 6) to reflect these efforts. The department’s comments are reprinted in appendix IV. In written comments, Transportation agreed with our recommendation and described planned actions to address it. Specifically, the department stated that it is leveraging its Strategic Workforce Planning Guide to navigate its efforts in assessing IT skill gaps and enhancing workforce proficiency. The department also noted that actions underway include finalizing plans for creating IT acquisition cadres and using direct hiring authority, developing an approach for cross-functional training of IT acquisition and program management personnel, and collaborating across departmental offices on the development of an IT program manager career path. Transportation’s comments are reprinted in appendix V. In its written comments, Treasury agreed with our recommendation and identified planned and ongoing efforts to address it. Specifically, the department noted that, while an agency-wide assessment of the IT workforce has not yet been completed, several component bureaus—including the Internal Revenue Service, which comprises over 80 percent of the total Treasury IT workforce—have been assessing their workforce. In addition, the department stated that its FITARA Implementation Plan identifies intended steps for meeting the requirements established by Congress and OMB, including IT workforce management, and that it is on track to establish a more robust workforce planning capability by the end of fiscal year 2017. Treasury’s written comments are reprinted in appendix VI. In addition to its written comments on the recommendations, the department also provided emails containing its comments on selected findings in the draft report, which we addressed as appropriate. Specifically: Treasury stated that it did not agree with our assessment that the department has not implemented the key IT workforce planning activity to establish and maintain a workforce planning process (activity 1). However, the department’s efforts have not been sufficient to justify a higher rating for this activity. As noted in the report, while the department has a strategic workforce planning model, it has deferred applying the model to the IT workforce pending a review of a workforce assessment tool being piloted within the Internal Revenue Service. We do not consider this situation to reflect an established and maintained IT workforce planning process because the pilot has not been completed and reviewed by the CIO and chief human capital officer, which was identified as a foundational step in its FITARA implementation plan. Further, decisions are yet to be made regarding (1) which IT workforce planning approach the department will utilize: the process still being piloted at the Internal Revenue Service, the department’s existing strategic workforce planning model that does not specifically address how IT components are to be assessed, or a hybrid approach; and (2) how and when its selected approach will be applied to assess skill gaps for the department’s IT workforce. Treasury also stated that we did not give the department credit for a cybersecurity skill gap assessment that it performed, which relates to the key IT workforce planning activity to assess gaps in competencies and staffing (activity 4). We disagree. We noted in the report that Treasury had identified staffing gaps for IT positions, including its mission-critical occupation and cybersecurity specific positions. This was based in part on the cybersecurity staffing gap assessment that the department performed, which factored into our rating of “partially implemented” for this activity. The department did not agree with our assessment that it had not implemented the key IT workforce planning activity to report to agency leadership on progress in addressing competency and staffing gaps (activity 8). After evaluating reports to senior department leaders and OMB regarding its cybersecurity staffing gaps and efforts to address the gaps through a pilot cybersecurity recruitment and retention program—which the department provided with its comments—we changed our rating from “not implemented” to “partially implemented” and revised the description for this activity. In addition to the aforementioned departments, we sought comments from other federal agencies, including GSA, OMB, OPM, and VA. We received e-mails from GSA, OMB, and OPM stating that they had no comments on the report. We also received an e-mail from VA with a technical comment that we addressed as appropriate. We are sending copies of this report to interested congressional committees, the Secretary of Commerce, the Secretary of Defense, the Secretary of Health and Human Services, the Secretary of Transportation, the Secretary of the Treasury, the Director of the Office of Personnel Management, and the Director of the Office of Management and Budget, and other interested parties. In addition, this report will be available on the GAO Web site at http://www.gao.gov. If you or your staffs have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VII. Our objectives were to (1) identify key characteristics of a comprehensive integrated program team (IPT) responsible for managing a major federal information technology (IT) acquisition, and (2) evaluate whether selected federal agencies are adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major IT acquisitions. To identify key characteristics of comprehensive IPTs, we listed over 1,000 recommended practices. In doing so, we reviewed approximately 20 documents containing practices applicable to IPTs that were recommended by government and industry organizations, including: Department of Defense, Office of the Under Secretary of Defense (Acquisition and Technology), DoD Integrated Product and Process Development Handbook, August 1998; American Council for Technology-Industry Advisory Council, Key Success Factors for Major Programs that Leverage IT, May 2014; The MITRE Corporation, Integrated Project Team (IPT) Start-up Guide, October 2008; Software Engineering Institute at Carnegie Mellon University, CMMI® for Acquisition, Version 1.3, November 2010; Office of Management and Budget (OMB), 25 Point Implementation Plan to Reform Federal Information Technology Management, December 2010; OMB, Guidance for Specialized Information Technology Acquisition Cadres, July 2011; OMB, Contracting Guidance to Support Modular Development, June Project Management Institute, Inc., A Guide to the Project Management Body of Knowledge (PMBOK® Guide), Fifth Edition, 2013. PMBOK is a trademark of the Project Management Institute, Inc.; OMB, Circular No. A-11, Preparation, Submission, and Execution of the Budget, Part 7 (Capital Programming Guide, Version 3.0), June 2015; and OMB, U.S. Digital Services Playbook. To corroborate and validate the recommended practices and to identify examples of how the practices can be applied, we interviewed agency officials from eight different organizations: two agency IPTs, four offices of the chief information officer (CIO) with oversight responsibilities for IPTs, and two agencies’ acquisition centers of excellence. The two agency IPTs we interviewed were the Department of Commerce’s Geostationary Operational Environmental Satellite - R series and the Department of Veterans Affairs’ Veterans Benefits Management System. We selected these two IPTs from a pool of 13 candidates based on the extent to which they met multiple selection criteria and did not overlap with ongoing GAO work. The selection criteria were: (a) suggested by OMB officials as being high-impact programs that have had a challenging past performance and have current increased oversight from OMB; (b) active engagement with OMB’s U.S. Digital Service; (c) observed by OMB has having relatively strong teaming practices; (d) observed through the course of GAO reviews as having relatively strong teaming practices; and (e) observed through the course of GAO reviews as having a history of problems from which lessons learned could be derived. The four offices of the CIO we interviewed were from the Departments of Commerce, Defense, and Veterans Affairs. We selected Commerce, National Oceanic and Atmospheric Administration (within the Department of Commerce), and Veterans Affairs CIO officials because their departments had oversight responsibilities for the two IPTs we interviewed. We also selected CIO officials from the Department of Defense due to the department’s history in adopting IPT concepts. The two agencies’ acquisition centers of excellence we interviewed were the General Services Administration’s Assisted Acquisition Services and the Department of Veterans Affairs’ Technology Acquisition Center. We selected these two centers of excellence based on OMB’s 2011 guidance for agencies for formulating specialized IT acquisition cadres. In our interviews, we asked these officials to describe desirable characteristics of comprehensive IPTs, identify additional factors that are critical to the success of IPTs, and provide examples. We added their responses to our database of IPT recommended practices. We conducted a content analysis on the recommended IPT practices from our literature research and from interviews with agency officials. In doing so, team members individually reviewed the practices and assigned them to various categories and sub-categories. Then the team members met to compare their categorization schemes, discuss the differences, and reach agreement on the final overarching characteristics and specific sub-practices. We validated the outcome of our content analysis by sending the results to the agency IPTs, CIOs, and centers of excellence which we had interviewed, as well as two internal experts. We then obtained their comments and revised our final list of IPT characteristics and practices as appropriate. Finally, we created a graphic based on the total number of times literature and agency officials identified specific roles and responsibilities. In doing so, we incorporated the roles and responsibilities that were most frequently cited as core disciplines and organized the graphic in descending order. To evaluate whether selected federal agencies are adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major IT acquisitions, we selected departments for our review, created an evaluation framework and validated it, and assessed the selected departments’ documents and activities against the evaluation framework. We selected the departments for our review based on a judgmental and non-generalizable sample of agencies’ data from the Federal IT Dashboard as of August 31, 2015. Specifically, we selected five departments—Commerce, Defense, Health and Human Services, Transportation, and the Treasury—based on the following factors: (1) largest number of major IT investments, (2) largest planned dollar total of major IT spending in fiscal year 2016, and (3) largest planned percentage of total major IT spending with development, modernization, and enhancements in fiscal year 2016. To create the evaluation framework, we reviewed relevant laws and guidance to identify steps and activities for federal agencies to conduct IT workforce planning, including: Clinger-Cohen Act of 1996; E-Government Act of 2002; Legislation commonly referred to as the Federal Information Technology Acquisition Reform Act; Federal Cybersecurity Workforce Assessment Act of 2015; OMB’s 25 Point Implementation Plan to Reform Federal Information OMB’s Guidance for Specialized Information Technology Cadres; OMB’s Management and Oversight of Federal Information OMB’s Cybersecurity Strategy and Implementation Plan for the OMB’s Federal Cybersecurity Workforce Strategy; OMB’s Circular A-130, Managing Information as a Strategic The Office of Personnel Management’s (OPM) Workforce Planning OPM’s workforce planning guidance on key elements and suggested OPM’s IT Program Management Career Path Guide; and GAO guidance on federal internal control standards as well as key principles for effective strategic workforce planning. We developed a framework consisting of four IT workforce planning steps and eight key activities, as shown in table 7. We validated the evaluation framework by soliciting comments from internal experts as well as officials from OMB and OPM and updated it as appropriate. We did not perform a legal analysis as to whether the selected agencies were in actual compliance with the laws we used to derive our evaluation framework. We assessed the selected departments’ documents and activities against the evaluation framework by requesting relevant IT workforce and human capital policies, as well as documentation from each of its most recent IT workforce competency and staffing gap assessments. We analyzed the policies and documents to determine whether they were consistent with the workforce planning steps and activities found in our framework. The documents we analyzed included: competency gap assessments, staffing assessments, strategic workforce plans, documentation of monitoring progress in addressing IT workforce skill IT workforce competencies and staffing requirements, gaps and reporting to department officials, documentation regarding IT acquisition cadres, documentation on cross-functional training of acquisition and IT documentation regarding career paths for IT program managers, documentation of efforts to strengthen IT program management, and documentation of the use of direct hire authority. We also interviewed cognizant officials at the five departments to discuss their policies, documentation, and implementation of the workforce planning steps and activities. Based on our assessment of the documentation and discussion with department officials, we assessed the extent to which the agencies implemented, partially implemented, or did not implement the steps and activities. We considered an activity to be fully implemented if a department addressed all of the underlying practices for the activity; partially implemented if it addressed some but not all of the underlying practices for the activity; and not implemented if it did not address any of the underlying practices for the activity. We conducted this performance audit from July 2015 to November 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, individuals making contributions to this report included Colleen Phillips (Assistant Director), Josh Leiling (Analyst-in-Charge), Chris Businsky, Raj Chitikila, Rebecca Eyler, Torrey Hardee, Jamelyn Payan, Andrew Stavisky, Mark Stefan, and Brian Vasquez.

Summary: In fiscal year 2017, the federal government is expected to spend more than $89 billion on IT. In many instances, agencies have not consistently applied best practices that are critical to successfully acquiring IT investments, such as ensuring program staff have the necessary knowledge and skills. In an effort to aid agencies in successfully delivering projects, the Office of Management and Budget has called for the development and use of IPTs for federal IT acquisitions to ensure that projects consist of the appropriate mix of individuals. GAO was asked to review IPTs for federal IT acquisitions and the federal government's IT workforce planning. GAO's objectives were to (1) identify key characteristics of comprehensive IPTs responsible for managing major federal IT acquisitions, and (2) evaluate whether selected federal agencies are adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major IT acquisitions. To do so, GAO reviewed relevant literature; interviewed IPT experts; and evaluated IT workforce efforts at five departments: Commerce, Defense, Health and Human Services, Transportation, and the Treasury. Integrated program teams (IPT) are cross-functional or multidisciplinary groups of individuals that are organized and collectively responsible for delivering a product to an external or internal customer. GAO identified three characteristics that contribute to the creation and operation of a comprehensive IPT: (1) executive leadership through team support, empowerment, and oversight; (2) team composition; and (3) processes for team operations. GAO also identified 18 practices supporting these three characteristics (see figure). For example, executive leadership is effective when sufficient resources are provided and teams are empowered to act, team composition is more robust when the IPT has cross-functional and multidisciplinary skill sets, and team operations are streamlined when team guidelines are established and stakeholders are involved as active members. When implemented, these practices can increase the IPT's likelihood of success by having the right mix of expertise to recognize problems early and by having the requisite authority to do something about them. While multiple factors contribute to a robust IPT, one aspect involves having a strong information technology (IT) workforce. To evaluate agencies' IT workforce planning efforts, GAO identified eight key workforce planning steps and activities based on relevant laws and guidance (see table). Five federal departments had mixed progress in assessing their IT skill gaps. While all five departments had demonstrated important progress in either partially or fully implementing key IT workforce planning activities, each had shortfalls. For example, four departments had not demonstrated an established IT workforce planning process. As shown in the figure, of the five departments, the Department of Defense had the most robust IT workforce planning process by fully or partially implementing all eight activities. However, the departments have not yet fully implemented all of the practices for various reasons. For example, policies were not comprehensive in requiring such activities or were not being applied to IT workforce planning at four departments, one department placed a greater emphasis on assessing its cybersecurity workforce, and two departments identified the need to perform more granular assessments of the workforce in order to identify skill gaps. Until the departments fully implement key workforce planning steps and activities, they risk not adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major acquisitions. GAO recommends that selected departments implement IT workforce planning practices to facilitate more rigorous analyses of gaps between current skills and future needs, and the development of strategies for filling the gaps. Four departments agreed and one, Defense, partially agreed with our recommendations.