Managing For Results: Agencies Have Elevated Performance Management Leadership Roles, but Additional Training Is Needed
Released: 2013-04-16T00:00:00

Letter

Background

GPRAMA made a number of changes to agency performance management roles, and provided the officials in these roles with specific duties. Among other things, the requirements for these roles reflected Congress’s intention to increase accountability of senior agency leadership for performance and results. Although these roles existed at some agencies prior to GPRAMA, it established them in law, added responsibilities, and elevated some of them. Later OMB guidance established additional performance management roles related to implementation of GPRAMA. The primary roles with responsibilities under GPRAMA and in OMB guidance are:

Agency head: GPRAMA gave each agency’s head broad responsibility for performance management. Among other things, the agency head is responsible for identifying agency priority goals and, along with the COO discussed in the next paragraph, conducting quarterly priority goal progress reviews.

Chief operating officer: The COO role existed at agencies prior to GPRAMA’s enactment, with responsibilities such as improving agency management and performance outlined in two presidential memoranda. GPRAMA maintained these previously established responsibilities, and added others to bring them in line with other GPRAMA requirements. It also required that the deputy agency head or equivalent serve as COO.

Performance improvement officer: The PIO role was created by a 2007 executive order. GPRAMA established the role in law and elevated it, specifying that it be given to a “senior executive” at each agency and that the PIO report directly to the agency’s COO. The various duties of the PIO include advising the agency head and COO on goal-setting and measurement and reviewing progress toward agency priority goals.

Deputy performance improvement officer: The deputy PIO role was not included in GPRAMA, but later OMB guidance directed agencies with a PIO who is a political appointee or other official with a limited- term appointment to also appoint a career senior executive as deputy PIO.

Goal leader: GPRAMA directs agencies to identify a goal leader who is responsible for each priority goal and make this information available to OMB to be published online. Leaders for agency priority goals are identified, with their photos, on the performance.gov website. A similar position existed at some agencies prior to GPRAMA, as earlier OMB guidance had encouraged agencies to identify officials who were responsible for High Priority Performance Goals (these goals have since been renamed as agency priority goals).

Deputy goal leader: The deputy goal leader role was not included in GPRAMA, but later OMB guidance directed agencies to identify a deputy goal leader to support the goal leader. In cases where the goal leader was a political appointee, OMB encouraged agencies to assign a career senior executive as the deputy.

GPRAMA also assigned responsibilities to OPM related to agency performance management. By January 2012, the agency was to identify skills and competencies needed by government personnel for setting goals, evaluating programs, and analyzing and using performance information for improving government efficiency and effectiveness. GPRAMA also directed OPM, by January 2013, to incorporate these competencies into relevant position classifications and to work with each agency to incorporate the skills and competencies into employee training.

OMB has a leadership and coordinating role in agency implementation of GPRAMA. OMB issued guidance on implementation through memoranda and in Circular A-11. Under GPRAMA, OMB is to ensure the operation of a public website that includes information on cross-agency priority goals and agency priority goals, among other performance-related information. This information is included on the performance.gov website. OMB’s Deputy Director for Management, or his or her designee, is directed to chair the PIC and preside at its meetings, determine meeting agendas, direct its work, and establish and direct its subgroups.

GPRAMA also included specific requirements for the PIC. The PIC was initially created by a 2007 executive order, but GPRAMA established it in law and included additional specific responsibilities. In addition to directing OMB’s Deputy Director for Management to chair the PIC, GPRAMA specified that council membership include the PIOs from the 24 CFO Act agencies, as well as any other PIOs and individuals identified by OMB’s Deputy Director for Management. The PIC’s duties are detailed in GPRAMA and later OMB guidance and include facilitating the exchange of useful practices and developing tips and tools to strengthen agency performance management. According to the PIC’s Executive Director, the PIC and the Executive Director are supported by two federal employees and four contractors. The PIC also typically has two to four detailees from other federal agencies. The PIC is administratively located within the General Service Administration’s Office of Executive Councils. According to a PIC staff member, the Office of Executive Councils provides infrastructure, analytical support, and project management capacity to the PIC and other interagency management councils, such as the Chief Financial Officers Council.

Agencies Have Elevated Performance Management Roles, but Performance Staff Competency Assessment and Training Are Needed

Designation of Senior- Level Officials to Key Performance Management Roles Has Helped to Elevate Accountability for Agency Performance

The 24 CFO Act agencies have all assigned senior-level officials to the key performance management roles—chief operating officer, performance improvement officer, and goal leader—required under GPRAMA, according to OMB and the results of our PIO survey. Figures 1 and 2 illustrate the performance management leadership teams at HHS and NSF, respectively.

Chief operating officer. GPRAMA’s requirement that each agency’s deputy head, or equivalent, take on the role of COO helped to ensure high-level involvement in performance management. GPRAMA required the COO to be involved in activities such as quarterly performance reviews. As we discuss later, most (21) PIOs we surveyed reported that their agencies’ COOs were involved in quarterly performance reviews to a large extent. The COOs at both HHS and NSF told us they were involved in selecting their agencies’ priority goals and they chaired the quarterly performance review meetings. HHS’s COO said that he considered it his role to make sure that everyone in the agency paid attention to performance and knew that he considered it important. NSF’s COO said that she saw performance management as a primary concern and integrally connected to all aspects of her work at NSF.

Performance improvement officer. Although the PIO role existed prior to GPRAMA, PIOs were not required to report directly to the COO until GPRAMA was enacted. GPRAMA elevated the role by putting this requirement in place. According to our PIO survey, at 8 agencies the PIO who was already in place took on the additional responsibilities required by GPRAMA. At the other 16 agencies, the current PIO began in the role after the implementation of GPRAMA.

All PIOs we surveyed reported that they have other agency roles in addition to being the PIO. Most PIOs (21) reported that these roles gave them access and authority that has been helpful in the PIO role, and most (20) reported that these roles gave them knowledge and experience that has been helpful in the PIO role. These roles included planning, administration, management, and budget and finance. The PIOs at HHS and NSF were both also chief financial officers (CFO), and told us that their other agency roles helped them in their PIO roles. HHS’s PIO said that her joint role allowed her to align budget development with performance management—she made sure that the budget was built so that dollars were spent on efforts that would be able to perform well. NSF’s PIO told us that the two roles worked well together because much of the data used to evaluate performance at the agency was maintained by the agency’s budget division, which also reports to her in her CFO capacity.

NSF’s COO noted that the PIO role at NSF was designated for reasons relating more to the individual, including the PIO’s past experience with performance management, than to her role as CFO. The COO said that NSF’s next PIO may not necessarily be the person in the CFO role. Although it was common for the PIOs we surveyed to also have the CFO role, PIOs whose other roles were in planning, administration, and management reported that those roles were helpful as well.

Nearly all PIOs (23) reported that their level of authority and access to agency leadership helped them perform their duties. Additionally, almost half of surveyed PIOs (11) reported directly to the agency head in their non-PIO role; they had even higher level authority and access to leadership than the PIO role alone would have provided. OMB staff added that because they were senior level officials, PIOs had the authority and ability to assemble the right people to implement performance management at their agencies.

Our survey results did not indicate great differences between political appointees and career civil servants in carrying out PIO duties. OMB staff told us that the requirement that agencies with political appointee PIOs have career civil servant deputy PIOs was in place to address the higher likelihood of turnover among political appointees. However, our PIO survey results did not indicate great differences in either time spent on PIO duties or turnover so far. Additionally, nearly all the agencies (22), not just those with political appointee PIOs, have chosen to designate deputy PIOs.

Goal leader. Agencies assigned senior-level officials with expertise in the goal to take on the role of priority goal leader. This was true at HHS and NSF, and was also true across agencies, according to OMB staff.

Officials at NSF and HHS said that priority goals had senior-level leaders who helped to bring attention and resources to the goal. For example, the U.S. Assistant Secretary for Health was one of two goal leaders for HHS’s priority goal on reducing tobacco consumption. He previously worked on tobacco cessation at the state level, was a public health professor, and published journal articles on tobacco control and health promotion. The goal’s other leader told us that because the Assistant Secretary was widely known within the public health community, his involvement brought respect and attention to the goal.

As discussed previously, the PIO and goal leader positions were filled by senior-level agency officials. Most of these officials had other responsibilities, such as serving as a CFO. HHS and NSF officials told us that while the PIO and goal leader roles were performed by senior-level officials at the highest levels of the agency, they relied on deputies who generally managed the day-to-day aspects of performance management.

Deputy performance improvement officer. Nearly all (22) of the CFO Act agencies have assigned officials to the Deputy PIO role, according to our PIO survey. Both HHS and NSF have assigned staff to the deputy PIO role. Officials at HHS and NSF also told us that PIOs tended to provide high-level vision and oversight and be a voice for performance management at their agencies, while deputy PIOs handled the day-to-day management of performance management for the agency. According to officials at HHS and NSF, responsibilities of deputy PIOs at these agencies included, among other things, coordinating with priority goal leaders, preparing for agency quarterly performance reviews, and attending PIC meetings, as appropriate. PIOs reported that most deputies devoted half or more of their time to performance-related duties.

Deputy PIOs also had other roles and titles at agencies, similar to the PIOs. Based on our analysis of their other titles, deputy PIOs most commonly had other roles also related to performance, while others filled roles in areas such as budget and finance and administration and management.

Deputy goal leader. HHS and NSF both assigned officials to deputy goal leader roles to support most of their goal leaders. As with the PIO/deputy PIO division, the goal leaders at our case study agencies were senior-level officials. According to agency officials, goal leaders provided high-level vision and oversight and lent their influence to ensure that the goal was prioritized, while their deputy goal leaders managed the goal on a day-to-day basis. For example, deputy goal leaders at these agencies were responsible for monitoring staff carrying out the goal and preparing reports on the goal for quarterly performance reviews. Two goal leaders we spoke with—one at HHS and one at NSF—had two deputy goal leaders. Moreover, at NSF, one goal leader we spoke with was supported by a deputy from a different operating division. The goal leader told us that this structure provided a cross-agency perspective and facilitated coordination.

PIOs reported that they and other key performance management officials at their agencies were involved in central aspects of performance management. We asked PIOs whether agency heads, COOs, PIOs, deputy PIOs, and goal leaders had large, moderate, small, or no involvement in four primary tasks that summarize the performance management responsibilities required by GPRAMA: performance measurement and analysis; communicating agency progress toward goals; and agency quarterly performance reviews. strategic and performance planning and goal setting; As shown in figure 3, the PIOs we surveyed reported that most performance management officials had large involvement in these key tasks.

Officials at HHS and NSF emphasized the importance of commitment to performance management at all levels, which was in part reflected in officials’ involvement in these key aspects. PIOs who reported large involvement for themselves generally reported larger involvement for other officials, suggesting that agencies with a strong commitment to performance management were following this philosophy.

OPM Has Identified Competencies for Performance Management Staff, but Competency Assessment and Additional Training Would Be Beneficial

GPRAMA directed OPM to take certain actions to support agency hiring and training of performance management staff. As noted earlier, OPM, in consultation with the PIC, was charged with three responsibilities under GPRAMA: (1) identify key skills and competencies needed by performance management staff; (2) incorporate these skills and competencies into relevant position classifications; and (3) work with agencies to incorporate these key skills into agency training. OPM has completed its work on its first two responsibilities, and is working to support agency training.

OPM identified 15 core competencies for performance management staff, in accordance with GPRAMA, and published them in a January 2012 memorandum from the OPM Director. (See app. III for a list of the competencies with definitions.) OPM also identified competences needed by PIOs and goal leaders. A manager of classification and assessment policy at OPM told us that OPM’s work to identify these competencies included a review of GPRAMA and related information. OPM also worked with a PIC working group focused on capability building to review the competencies it identified.

Figure 4 shows PIOs’ responses to our survey question about the extent to which their staff had these competencies.

PIOs generally reported that their staff had the competencies identified by OPM to a large extent, although they reported that the competencies below were not as widespread as others:

Performance measurement: the knowledge of the principles and methods for evaluating program or organizational performance using financial and nonfinancial measures.

Information management: the ability to identify a need for information, know how to gather information and organize and maintain information or information management systems.

Organizational performance analysis: the knowledge of the methods and tools used to analyze program, organizational, and mission performance.

Planning and evaluating: the ability to organize work, set priorities, and determine resource requirements. This includes determining short- or long-term goals and strategies to achieve them, coordinating with other organizations or parts of the organization to accomplish goals, and monitoring progress and evaluating outcomes.

Some individual agencies found competency gaps in similar areas. HHS’s Deputy PIO told us that further improvement of HHS’s performance staff’s analytical skills would help the agency to more effectively implement GPRAMA. Also, as we reported in February 2013, Small Business Administration officials identified a skills gap among some of their staff in working with data.

OPM officials had planned to develop a competency assessment tool that could be used to determine needs at each individual agency. Developing the tool was identified as a critical and manageable “next step” at a January 2012 meeting focused on incorporating key performance management competencies into agency training. Meeting participants included OMB, OPM, and agency members of the Chief Learning Officers (CLO) Council, which facilitates collaboration among CLOs. OPM officials told us that they took action to follow up on other “next steps” related to training identified at the meeting, which are discussed in the following section. However, an OPM official relayed to us that at this time, OPM does not plan to conduct a formal competency assessment using a competency assessment tool. A group manager in OPM’s Training and Executive Development division told us that the agency is focused on identifying critical skills gaps across the federal government. She said that some of the government-wide skills that OPM plans to focus on are related to skills needed for performance management, such as data analysis.

OPM identified relevant position classifications that are related to the competencies for performance management staff, and worked with the PIC Capability Building working group to develop related guidance and tools for agencies. A manager of Classification and Assessment Policy at OPM told us that the competencies best fit into an existing classification series for management and program analysts. In addition, OPM worked with the PIC’s Capability Building working group to develop position descriptions for performance management staff. In December 2012, the Capability Building working group released to agencies a draft performance analyst position design, recruitment, and selection toolkit. The draft toolkit included position description templates for performance analysts, job opportunity announcement templates, and recruiting resources, among other information. NSF officials told us that they found the Capability Building working group’s performance analyst position description helpful and used it to develop the agency’s deputy PIO position, which has been recently filled. The toolkit may also be of use to other agencies planning to hire new performance management staff. About half (11) of PIOs reported that their agencies planned to hire new staff, in addition to training existing staff, in order to address competency gaps.

According to OPM, the 15 core competencies for performance management staff are moderately to highly trainable, and OPM has taken steps to work with agencies to incorporate the competencies into training programs for relevant staff. Most (18) PIOs reported that their agencies planned to train staff in order to strengthen their performance management competencies. OPM’s Director stated in a January 2012 memorandum that the agency would work with CLOs to incorporate GPRAMA competencies into agency training programs. OPM worked with the CLO Council and the PIC Capability Building working group to develop a website—the Training and Development Policy Wiki—that lists some training resources for performance management staff. OPM also sponsored two webcasts focused on sharing agency experiences using performance management tools. A manager in OPM’s Training and Executive Development division told us that OPM was developing an interactive, online course focused on writing measurable performance goals that align with organizational goals, which she expected would be completed by July 2013. According to the official, both the webcasts and the work on the online course were the result of “next steps” identified at the January 2012 meeting between OPM, OMB, and members of the CLO Council.

OPM Human Resources specialists said they were also working to help the PIC to develop a website with more extensive resources, including information on training as well as performance management career path information. According to OPM officials and the PIC’s Executive Director, the PIC will develop the content for this website and it will be modeled on OPM’s Human Resources University website, which provides human resources career path information and links to related training. A Workforce Development Manager at OPM said the agency will support the PIC on the technical aspects of the site based on its experience developing the Human Resources University site. According to OMB staff and OPM, the performance management website is scheduled to launch by the end of 2013.

In addition to OPM’s actions on performance management training, individual agencies have taken action to develop their own performance management training to address competency gaps. For example, as we described in our recent report on quarterly performance reviews, SBA developed courses focused on skills such as spreadsheet development and analysis, presentation delivery, and other analytic and presentation skills.

According to our survey of PIOs, less than half (9) of the PIOs we surveyed rated the level of access to and availability of performance management training at their agencies as helpful, as shown in figure 5.

OPM’s efforts so far to work with agencies to incorporate performance improvement skills and competencies into agency training have been relatively broad-based and have not been informed by specific assessments of agency training needs. As described earlier, the agency has not followed through on its plan to measure agency staff competency levels in key areas required for performance management. Our survey results suggest that certain areas need to be improved, but without a more comprehensive assessment, it will be difficult for OPM to target its efforts—both to identify training that addresses agency needs, and to make training available through its performance website, which is under development, or through other means.

PIOs and Selected Agencies Found the PIC Useful, but Additional Planning and Performance Assessment Are Needed

PIOs Reported that the Most Helpful Aspects of the PIC Were Sharing of Best Practices and Development of Tips and Tools

PIOs we surveyed generally found the PIC’s work to be helpful to their agencies. We asked the PIOs to rate the helpfulness of selected functions that GPRAMA and OMB guidance direct the PIC to perform. As shown in figure 6, PIOs generally rated the PIC’s work in these areas as helpful. The PIC’s work promoting communication and developing tools incorporated several practices that our past work has identified as necessary for building collaborative working relationships. These include establishing means to operate across agency boundaries and identifying and addressing needs by leveraging resources.

Most (17) PIOs we surveyed reported that they have been able to apply successful practices and other information and tools shared by the PIC. PIOs surveyed reported some examples of information shared by the PIC that they applied at their agencies, including information on performance management positions, goal-setting, and quarterly performance reviews.

PIOs we surveyed and agency officials we interviewed reported that the PIC has been particularly helpful in facilitating the exchange of successful practices among agencies. An OMB staff member told us that facilitating this type of exchange was the PIC’s greatest strength, and that doing so also helped the group identify best practices. As shown in figure 6, just over half of PIOs reported that the PIC was very helpful in this area. Senior agency officials we interviewed and PIOs we surveyed provided examples of ways in which the PIC facilitated information exchange. For example, a PIO we surveyed reported that his agency’s approach to quarterly performance reviews was informed by examples shared by other agencies in the PIC’s Internal Agency Reviews working group.

An OMB staff member told us that in addition to helping agencies, the PIC’s facilitation of information exchange has benefitted OMB. One of the tasks GPRAMA charged the PIC with was submitting recommendations to streamline and improve performance management policies and requirements to OMB. PIOs we surveyed generally reported that this function was helpful to their agencies. An OMB staff member told us that the PIC helped OMB staff determine best practices and use that information to inform policy. For example, they said that in response to feedback from PIC members, OMB added information on “other indicators” to its 2012 Circular A-11 guidance.

Another area in which PIOs reported that the PIC was particularly helpful was in developing and providing tips, tools, training, and other capacity- building mechanisms. Half of PIOs reported that the PIC was very helpful in this area. PIOs we surveyed and officials we interviewed reported various ways in which their agencies have used PIC information. For example, five of the PIOs we surveyed reported that their agencies used PIC information on goal setting.

PIOs Reported Strong Agency Participation in the PIC and in Its Working Groups

The PIC holds two types of meetings—a “principals only” meeting open to PIOs only, and a broader meeting open to PIOs as well as other agency staff—both of which are well attended, according to PIOs we surveyed. OMB staff told us that these two types of meetings were generally held on alternating months. As shown in figure 7, most of the PIOs we surveyed told us that they regularly attended the “principals only” PIC meetings. OMB staff told us that in order to encourage senior-level attendance, PIOs were not permitted to send substitutes in their place. Most PIOs also reported that their deputies or other staff members regularly attended the broader PIC meetings. OMB staff estimated that two to three representatives from each agency typically attended these meetings. In our previous work, we identified regular participation in activities such as meetings as an important feature of effective collaboration.

Agency participation in PIC working groups was also strong, and PIOs and other agency officials reported using information and products shared through working groups. The PIC established five working groups, three of which were actively meeting at the time of our review, to focus on specific topics (see table 1). According to the PIC’s Executive Director, PIC working groups focused on issues related to implementation of GPRAMA and related guidance and provided a forum for staff from different agencies who were working on similar issues to connect with each other. He told us that the PIC identified the working group topics based on informal input from PIC members, though the council might in the future solicit more formal input through a survey. Agencies could also participate in separate OMB working groups that focused on informing policy and guidance. For example, at the time of our review, an OMB working group was focusing on informing guidance for strategic planning required under GPRAMA.

In addition to the three working groups described in table 1, there were two PIC working groups—the Goal Setting and the Benefits Processing working groups—that no longer regularly met, according to OMB staff. OMB and PIC staff said that these groups could restart again in the future, if agency needs arise. The Goal Setting working group focused on helping agencies set priority goals for fiscal year 2013, and produced a draft guide to goal setting. OMB staff said that the group may start meeting again to focus on strategic goals and objectives or on the next round of priority goal setting. In addition, the Benefits Processing working group, which focused on promoting consistency in agencies’ benefits processing, was no longer regularly meeting because it had completed its tasks.

Most (18) PIOs we surveyed reported that they or other staff members from their agencies participated in at least one working group, with some agencies participating in multiple groups and one agency participating in all five. These 18 agencies participated in an average of three working groups each, according to PIOs. PIOs reported that they generally did not participate personally in working groups. At HHS, the Deputy PIO said that both he and members of his team participate in working groups. PIOs we surveyed and agency officials we interviewed reported using working group products or information. For example, a PIO reported on the survey that she used the Goal Setting group’s guide on developing priority goals.

The PIC Coordinated with Small Agencies and Interagency Management Councils

According to representatives of small agencies, PIC and OMB staff effectively coordinated with them and were receptive to their feedback. GPRAMA directed the PIC to coordinate with nonmember agencies, which include most small agencies. According to a representative of the Small Agency Council (SAC), a management association of small agencies, PIC meetings generally addressed issues affecting the CFO Act agencies. She said that she understood the PIC’s focus, as larger agencies generally have more consistency in their implementation of GPRAMA requirements. However, she further stated that smaller agencies may require more assistance due to their more diverse missions and fewer resources. Although the broader PIC meetings were open to all agencies, the SAC representative told us that few small agencies found that the PIC met their needs. Instead, PIC and OMB staff communicated with small agencies through the SAC’s Performance Improvement Committee, which was established in March 2011. This committee is similar to the PIC in its attention to implementation of GPRAMA, but meeting objectives focus on issues facing small agencies. According to SAC representatives, the committee functions as a way for small agencies to give voice to their concerns, as well as a forum for OMB and the PIC to focus on the needs of small agencies, which may have unique issues in implementing GPRAMA and other requirements. SAC representatives told us that they have been satisfied with the support provided by OMB and the PIC in both of these areas. SAC Performance Improvement Committee meeting agendas from recent months included presentations from PIC and OMB staff. For example, a recent meeting included presentations from OMB staff on updated guidance contained in OMB’s Circular A-11 and on small agencies’ use of the performance.gov website.

In addition to coordinating with non-member agencies, GPRAMA directed the PIC to coordinate with other interagency management councils. While two PIOs we surveyed reported that this coordination was very helpful, most (16) rated this function as moderately helpful.

The PIC’s Functions Are Established in GPRAMA and Guidance, but It Has Not Routinely Assessed its Performance

GPRAMA and OMB guidance specify the PIC’s functions and roles, but the PIC has not regularly assessed how well it has been fulfilling these roles. GPRAMA directed the PIC to perform several functions, such as helping agencies share practices that have led to performance improvements. According to the PIC’s Executive Director, the PIC conducted a survey of PIOs prior to the enactment of GPRAMA. It also surveyed attendees of a January 16, 2013, PIC meeting, one of the broader meetings that is open to PIOs as well as other agency staff. The survey covered topics such as participants’ expectations for the meeting and assessments of the usefulness of the agenda items covered. However, the PIC has not done this on a regular basis, or gathered member feedback about its overall performance. As we have previously reported, practices that help to sustain collaboration include having federal agencies create the means to monitor and evaluate their collaborative efforts to enable them to identify areas for improvement. Although our survey indicated that PIOs generally found the PIC’s work helpful in selected areas, without more comprehensive and regular assessment of member opinions, it will be difficult for the PIC to ensure that, going forward, it is meeting its members’ current and emerging needs. The PIC’s Executive Director, who started in this position in November 2012, told us that he was considering conducting a survey of PIC members as well as administering evaluation forms at the end of every meeting. Regularly soliciting feedback allows organizations to monitor member input on an ongoing basis. For example, the SAC Performance Improvement Committee administers an evaluation form at each of its meetings. These forms allow members to rate the usefulness of the meeting and their satisfaction with particular aspects of it and to suggest topics for upcoming meetings.

Without formal and regular member feedback, the PIC is missing opportunities to tap a resource for identifying topics for future working groups and PIC meetings. PIC staff told us that identification of working group topics and meeting agenda items was generally based on informal input. Our review of PIC meeting agendas from February 2009 through September 2012 showed that since the enactment of GPRAMA in January 2011, both the “principals only” and broader PIC meetings focused on issues related to GPRAMA implementation. Going forward, PIC members’ needs will naturally evolve as GPRAMA implementation deepens within agencies and new questions and issues arise. The PIC’s Executive Director told us that the topic areas covered by the PIC in the future will most likely include new issues related to GPRAMA implementation, and an increased emphasis on cross-agency connections.

Additionally, the PIC has not updated its strategic plan since GPRAMA was enacted in January 2011. OMB staff provided us with a copy of the PIC’s Strategic Action Plan, which was implemented in January 2009 and covered fiscal years 2009 through 2013. This plan included four strategic goals, along with objectives and implementing strategies for each. OMB staff also provided us with information from a 2010 update of the strategic plan that focused on two new PIC goals. As we have previously reported, practices that help ensure effective collaboration include the use of strategic plans as tools to drive collaboration and establish goals and strategies for achieving results. Our prior work also identified several leading practices in federal strategic planning, among them that organizations involve stakeholders in strategic planning. Although the PIC has a strategic plan in place, the PIC last updated the plan prior to the enactment of GPRAMA. An up-to-date strategic plan that incorporates the input of its members and reflects the changes in federal performance management required by GPRAMA could help the PIC be reasonably assured that it has established a framework to effectively guide and assess its work. The PIC’s Executive Director told us that he intended to work with the PIC to update the strategic plan, which will be informed by PIC member feedback.

Conclusions

Senior agency officials’ commitment to and accountability for improving performance are important factors in determining the success of performance and management improvement initiatives. Through our PIO survey, we found that officials with responsibilities under GPRAMA were greatly involved in central, key aspects of performance management. These officials were supported by performance management staff, and PIOs reported that they were generally satisfied with their staff skills. However, our survey results showed that PIOs believed that certain competencies could be strengthened. OPM planned to directly assess performance management competency gaps at agencies, but has not yet done so. An assessment directly focused on performance management competencies could provide information on any gaps and inform agencies’ efforts to address them. OPM could also use this information to ensure that its work with agencies to incorporate competencies into training for agency staff, as required by GPRAMA, is effective. In particular, this information could inform OPM’s coordination of the sharing of training resources among agencies, both through its Training and Development Policy Wiki website and the website planned for performance management professionals. Through these websites, OPM could target resources to areas in which it has identified competency gaps. Such sharing of agency training resources offers the opportunity to maximize efficiency, and OPM is well positioned to play a coordinating role in this area through its expertise and relationships with the PIC and CLO Council. OPM has worked with these councils related to training under GPRAMA in the past. Additionally, the councils include representatives from across government with expertise in performance management and training, so they provide OPM with the ability to efficiently obtain input and share resources on performance management training.

The PIC plays a significant role in agency implementation of GPRAMA, and our survey results indicate that PIOs generally found the PIC helpful to their agencies. Both GPRAMA and related OMB guidance described the PIC’s functions, but the PIC has not regularly collected member feedback on its own performance, and has not updated its strategic plan since GPRAMA was enacted in January 2011. Functions such as creating working groups and developing meeting agendas have been based on informal feedback. Regularly collecting formal feedback from members, such as through a survey, would help the PIC identify areas in which it could ensure it maintains its usefulness, as well as new areas on which member agencies would like for it to focus its meetings and working groups. Obtaining such feedback would allow the PIC to monitor its performance and identify issues as they arise. This will be particularly important as agencies become more accustomed to GPRAMA processes and their needs change. In addition, the PIC lacks an up-to-date strategic plan. Its most recent update was in 2010, so it does not reflect any changes in goals or priorities that may have resulted from GPRAMA. An up-to-date plan could provide the PIC with a basis for directing and evaluating its performance in implementing GPRAMA. A strategic plan that incorporates input from PIC members could also serve as a tool for encouraging collaboration and reinforcing accountability.

Recommendations for Executive Action

To improve performance management staff capacity to support performance management in federal agencies, we recommend that the Director of OPM, in coordination with the PIC and the CLO Council, work with agencies to take the following three actions: Identify competency areas needing improvement within agencies. Identify agency training that focuses on needed performance management competencies.

Share information about available agency training on competency areas needing improvement.

To ensure that the PIC has a clear plan for accomplishing its goals and evaluating its progress, we recommend that the Director of OMB work with the PIC to take the following two actions:

Conduct formal feedback on the performance of the PIC from member agencies, on an ongoing basis.

Update its strategic plan and review the PIC’s goals, measures, and strategies for achieving performance, and revise them if appropriate.

Agency Comments and Our Evaluation

We provided a draft of this report to the Acting Director of OMB, Director of OPM, Secretary of HHS, and Director of NSF for review and comment. OMB staff agreed with our recommendation that it work with the PIC to conduct regular feedback on the PIC’s performance, and update the PIC’s strategic plan and review the PIC’s goals, measures, and strategies for achieving performance. The staff also provided technical comments, which we incorporated as appropriate. OPM agreed with our recommendation that it identify competency areas needing improvement in agencies, and use this information to identify and share information about training that focuses on needed performance management competencies. OPM explained that it will work with agencies, and in particular with PIOs, to assess the competencies of the performance management workforce. OPM also stated that it will support the use of the PIC’s performance learning website to facilitate the identification and sharing of training related to competencies in need of improvement. OPM’s written comments are reprinted in appendix V.

HHS did not have comments. NSF provided technical comments, which we incorporated.

We are sending copies of this report to the Acting Directors of OMB, OPM, and NSF, and the Secretary of HHS, as well as the appropriate congressional committees and other interested parties. In addition, this report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-6806 or mihmj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Agencies Subject to the Chief Financial Officers Act

Appendix II: Objectives, Scope, and Methodology

The Government Performance and Results Act Modernization Act of 2010 (GPRAMA) requires GAO to review the act’s implementation, and this report is part of a series of reviews planned around the requirement. The objectives of this report are to (1) examine the status of federal agency implementation of the performance management leadership roles under GPRAMA; and (2) evaluate the role of the PIC in facilitating the exchange of best practices and improving program management and performance.

To achieve our objectives, we focused our review on the 24 agencies covered by the Chief Financial Officers Act of 1990 (CFO Act). Several provisions of GPRAMA apply specifically to these agencies, including that the Performance Improvement Council (PIC) include them as members. We focused our examination of performance management leadership on the management roles that have specific responsibilities under GPRAMA and related OMB guidance, with the exception of agency head. These are the chief operating officer (COO), performance improvement officer (PIO), deputy PIO, priority goal leader, and deputy goal leader. In looking at the PIC, we evaluated its role in facilitating the exchange of best practices and improving agency program management and performance. GPRAMA and related OMB guidance charge the PIC with performing several additional functions, such as supporting OMB in implementing requirements related to federal government priority goals, also referred to as cross-agency priority goals. We did not include these other functions in our review.

To address both objectives, we conducted a survey of PIOs at the 24 CFO Act agencies. Through our survey, we collected information regarding PIOs’ and other key officials’ characteristics, their involvement in performance management under GPRAMA, and PIO and agency participation in the PIC. Appendix IV presents the survey questions we asked, and summarizes the responses we received. We received responses from all 24 PIOs (a 100 percent response rate). Selected results from our survey were also reported in another GAO report that focused on quarterly performance reviews under GPRAMA. We administered the web-based survey from October 18, 2012 to December 14, 2012. Respondents were sent an e-mail invitation to complete the survey on a GAO web server using a unique username and password. During the data collection period, we sent reminder e-mails and made phone calls to nonresponding agencies. Because this was not a sample survey, it has no sampling errors. The practical difficulties of conducting any survey may also introduce nonsampling errors, such as difficulties interpreting a particular question, which can introduce unwanted variability into the survey results. We took steps to minimize nonsampling errors by pretesting the questionnaire in person with PIOs and deputy PIOs at three different agencies. We conducted these pretests to make sure that the questions were clear and unbiased, the data and information were readily obtainable, and that the questionnaire did not place an undue burden on respondents. Additionally, a senior methodologist within our office independently reviewed a draft of the questionnaire prior to its administration. We made appropriate revisions to the content and format of the questionnaire after the pretests and independent review. All data analysis programs used to generate survey results were independently verified for accuracy. Additionally, in reviewing the answers from agencies, we confirmed that PIOs had correctly bypassed inapplicable questions (skip patterns). Based on our findings, we determined that the survey data were sufficiently reliable for the purposes of this report.

In addition, in order to understand GPRAMA implementation in more detail and put survey results in context for both objectives, we conducted in-depth studies of two agencies’ implementation of performance management leadership roles under GPRAMA and participation in the PIC—the Department of Health and Human Services (HHS) and the National Science Foundation (NSF). We selected these two agencies because they have differing characteristics that may affect implementation, such as agency size and the career status of the official in the PIO role. HHS is a relatively large agency when ranked according to annual budget and number of staff, while NSF is a relatively small agency. Additionally, HHS’s PIO is a political appointee, while NSF’s PIO is a career civil servant. In making our selection, we excluded agencies with certain characteristics, including those with a PIO that was relatively new to the role at the time of our survey and agencies that had been the subject of recent case studies on performance management by us or other organizations. We conducted interviews with both selected agencies’ COOs, PIOs, and deputy PIOs.

In addition, in order to understand the priority goal leader role and its contributions to performance management, we selected three of HHS’s six priority goals and two of NSF’s three priority goals and interviewed the responsible goal leaders. We selected these goals on the basis of several characteristics that may affect their management. These include: (1) number of priority goal leaders—we selected some goals with one leader and some with multiple leaders; (2) number of agency components involved in the goal—we selected goals with varying numbers of components and other stakeholders involved; (3) type of goal—we selected some process goals and some outcome goals; and (4) relationship to cross-agency priority goals—we included one goal in our set that relates to a cross-agency priority goal. The three goals we selected for HHS were: (1) improve the quality of early childhood education: (2) improve patient safety; and (3) reduce cigarette smoking. The two goals we selected for NSF were: (1) develop a diverse and highly qualified science and technology workforce; and (2) increase opportunities for research and education through public access to high- value digital products of NSF-funded research. For priority goals in which two leaders were assigned, we interviewed one of the responsible leaders. In several cases, deputy/ lieutenant goal leaders also attended the interviews.

We also addressed our first objective by reviewing GPRAMA and OMB guidance related to the key management roles. We reviewed information provided to us by OMB on the officials in the roles, along with information on them that is publically available through OMB’s performance.gov website and agency websites. We also interviewed OMB staff and officials at OPM about their work under GPRAMA, including their work with agencies in implementing GPRAMA. To understand agencies’ perspectives and experiences in implementing the performance management leadership roles under GPRAMA, we analyzed relevant results from our survey of PIOs and included related questions in our interviews with officials at HHS and NSF. We also obtained relevant documentation, such as meeting agendas, from these two agencies.

To address our second objective, we reviewed GPRAMA and related OMB guidance on the PIC. We analyzed PIC meeting agendas from February 2009 through September 2012, and we observed part of the September 12, 2012, PIC meeting. We also reviewed documents related to the PIC and its working groups and website, and interviewed OMB and PIC staff. We also interviewed OPM officials about their work with the PIC. To understand agencies’ participation in and use of the PIC, we analyzed relevant results from our survey of PIOs and included related questions in our interviews with officials at HHS and NSF. We also interviewed the PIC’s Executive Director, and officials from the Small Agency Council and the chair of its Performance Improvement Committee, which interacts with the PIC.

We conducted our work from May 2012 to April 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Core Competencies for Performance Management Staff

In a January 2012 memorandum, OPM’s Director identified 15 competencies that are essential for performance management staff to have in order to perform their roles. The memorandum included the following definitions of each competency.

Accountability - Holds self and others accountable for measurable high- quality, timely, and cost-effective results. Determines objectives, sets priorities, and delegates work. Accepts responsibility for mistakes. Complies with established control systems and rules.

Attention to Detail - Is thorough when performing work and conscientious about attending to detail.

Customer Service - Works with clients and customers (that is, any individuals who use or receive the services or products that your work unit produces, including the general public, individuals who work in the agency, other agencies, or organizations outside the Government) to assess their needs, provide information or assistance, resolve their problems, or satisfy their expectations; knows about available products and services; is committed to providing quality products and services.

Influencing/Negotiating - Persuades others; builds consensus through give and take; gains cooperation from others to obtain information and accomplish goals.

Information Management - Identifies a need for and knows where or how to gather information; organizes and maintains information or information management systems.

Oral Communication - Expresses information (for example, ideas or facts) to individuals or groups effectively, taking into account the audience and nature of the information (for example, technical, sensitive, controversial); makes clear and convincing oral presentations; listens to others, attends to nonverbal cues, and responds appropriately.

Organizational Awareness - Knows the organization’s mission and functions, and how its social, political, and technological systems work and operates effectively within them; this includes the programs, policies, procedures, rules, and regulations of the organization.

Organizational Performance Analysis - Knowledge of the methods, techniques, and tools used to analyze program, organizational, and mission performance; includes methods that deliver key performance information (for example, comparative, trend, diagnostic, root cause, predictive) used to inform decisions, actions, communications, and accountability systems.

Partnering - Develops networks and builds alliances; collaborates across boundaries to build strategic relationships and achieve common goals.

Performance Measurement - Knowledge of the principles and methods for evaluating program or organizational performance using financial and nonfinancial measures, including identification of evaluation factors (for example, workload, personnel requirements), metrics, and outcomes.

Planning and Evaluating - Organizes work, sets priorities, and determines resource requirements; determines short- or long-term goals and strategies to achieve them; coordinates with other organizations or parts of the organization to accomplish goals; monitors progress and evaluates outcomes.

Problem Solving - Identifies and analyzes problems; weighs relevance and accuracy of information; generates and evaluates alternative solutions; makes recommendations.

Reasoning - Identifies rules, principles, or relationships that explain facts, data, or other information; analyzes information and makes correct inferences or draws accurate conclusions.

Technical Competence - Uses knowledge that is acquired through formal training or extensive on-the-job experience to perform one’s job; works with, understands, and evaluates technical information related to the job; advises others on technical issues.

Written Communication - Writes in a clear, concise, organized, and convincing manner for the intended audience.

Appendix IV: Results from Survey of Performance Improvement Officers

Performance Improvement Officer (You and Your Role)

Please note: Please provide what you personally believe is the most correct answer to each question, even if it is different from the opinions of others at your agency. 1. Besides Performance Improvement Officer (PIO), what other title(s), if any, do you have (e.g. CFO)?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 2. When did you start serving in the position(s) you identified in Question 1? 3. Which of the following best describes your hiring status? 4. Who do you report to in the role(s) you identified in Question 1? (check all that apply)

Number of respondents 24 4a. If other, please specify: Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 5. When did you start serving as PIO at your agency? 6. Who do you report to in your role as PIO? (check all that apply) 1. Chief Operating Officer (i.e. Deputy Secretary or equivalent) 6a. If other, please specify: Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 7. How many PIOs has your agency had (including yourself) since GPRAMA was enacted in January 2011? 8. On average, how much time per month do you spend performing duties related to your role as PIO?

If other factors specified in question 10, what was each additional factor?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents.

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 11. To what extent, if at all, are PIO responsibilities considered in your annual performance expectations and appraisals?

Not applicable - no annual performance expectations or appraisals 1 12. What suggestions, if any, do you have for improving the effectiveness of the PIO role?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents.

Deputy Performance Improvement Officer(s)

13. Does your agency have a Deputy Performance Improvement Officer(s)?

Yes - we have 2 Deputy Performance Improvement Officers 2 14. When was the Deputy Performance Improvement Officer position created at your agency? 15. How many Deputy PIOs has your agency had (including the current person in that position) since GPRAMA was enacted in January 2011?

Number of respondents 22 16. What other title(s), if any, does each Deputy Performance Improvement Officer have?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 17. Who does each Deputy Performance Improvement Officer report to in the role(s) identified in question 16?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 18. On average, how much time per month does each Deputy PIO spend performing duties related to his/her role as PIO?

Number of DPIOs Listed by Agencies 24 19. When did the Deputy Performance Improvement Officer start in his/her position?

Number of DPIOs Listed by Agencies 24 20. To what extent, if at all, are Deputy PIO responsibilities specifically considered in his/her annual performance expectations and appraisals?

Not applicable - no annual performance expectations or appraisals 21. What suggestions, if any, do you have for improving the effectiveness of the Deputy PIO role?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents.

See Appendix IV for explanations of competencies.

22k. Planning and Evaluating

Number of respondents 24 23. What are your plans, if any, to strengthen staff competencies?

Not applicable - competencies are not sufficiently available, but no action is planned 1 23a. If other, please specify: Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents.

GPRAMA provides senior agency officials with specific duties and responsibilities related to performance management and achievement of performance goals. The key agency officials identified in the Act are Agency Head, Chief Operating Officer (COO) who is the Deputy Secretary or equivalent position; Performance Improvement Officer (PIO); and Goal Leader. Later guidance from the Office of Management and Budget (OMB) directs agencies that have a political appointee serving as the PIO to appoint a career senior executive to serve as Deputy PIO. The following questions address these roles and their responsibilities at your agency. 24. How much involvement, if any, does each of the following officials have in strategic and performance planning and goal setting?

No opinion Not applicable 2 0 No opinion Not applicable 0 0 25. How much involvement, if any, does each of the following officials have in performance measurement and analysis?

No opinion Not applicable 2 0 No opinion Not applicable 0 0 26. How much involvement, if any, does each of the following officials have in communicating agency progress toward goals, both internally and externally?

Quarterly Performance Reviews

As of June 2011, GPRAMA required agencies to review progress toward its priority goals on at least a quarterly basis. The quarterly performance reviews are to involve key leadership and other relevant parties and should, at minimum, focus on the agency's priority goals. They are to include reviewing progress and trends, coordinating within and outside the agency, assessing agencies’, activities', and policies’' contributions to goals, categorizing goals by risk, and identifying strategies for improvement.

When we refer to "quarterly performance reviews" in the following questions, we refer to all aspects of the regularly-scheduled reviews required under GPRAMA, including preparation, review, and follow-up. Some agencies refer to these reviews as "stat" reviews. Additionally, although we refer to them as "quarterly performance reviews," agencies may conduct these reviews on a regularly occurring basis more frequently than quarterly. 27. Does your agency conduct GPRAMA-required quarterly performance reviews? 28. Did your agency conduct quarterly performance reviews (or similar reviews) before the GPRAMA requirement took effect in June 2011? 29. When did your agency begin conducting quarterly reviews?

Two PIOs did not respond to this question. 30. How does your agency conduct its quarterly performance reviews? 31. How often does your agency conduct performance reviews (although GPRAMA requires quarterly reviews, some agencies have established other review cycles to meet their management needs)?

Number of respondents 24 31a. How often?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 32. How much involvement, if any, does each of the following officials have in your agency's quarterly performance reviews?

No opinion Not applicable 4 0 33. For each of the following officials, has their involvement in agency performance management increased, remained about the same, or decreased as a result of your agency's quarterly performance reviews?

Performance Improvement Council

GPRAMA establishes in law the Performance Improvement Council (PIC). The PIC is an interagency council made up of agency PIOs that is charged with assisting OMB with topics related to GPRAMA and facilitating the exchange of useful practices among agencies. 36. How often do you attend the every-other-month Performance Improvement Council meetings that are for PIOs only (not deputies or staff)?

Rarely or never attend meetings 2 37. How often do you attend the every-other-month Performance Improvement Council meetings that are open to PIOs, Deputy PIOs, and staff?

Rarely or never attend meetings 12 38. How often does a Deputy PIO or another representative(s) from your agency attend the every-other-month Performance Improvement Council meetings that are open to PIOs, Deputy PIOs, and staff?

Rarely or never attend meetings 1 39. Which of the following working groups of the Performance Improvement Council do/did you actively participate in personally as the PIO? (check all that apply)

Number of respondents 24 3. Internal Agency Reviews working group 6. I do not personally participate in any working groups 39a. If other working group, please specify: Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 40. In which of the following working groups of the Performance Improvement Council do/did other representatives from your agency actively participate? (check all that apply)

If other aspect(s) specified in question 41, what was each additional aspect?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents.

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 42. To what extent are you able to apply successful practices and other information and tools shared by the PIC to your agency's performance management? 42a. Please provide an example or examples of case(s) in which you have applied successful practices and other information and tools shared by the PIC in your agency.

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 43. What suggestions do you have for improving the effectiveness of the PIC, if any?

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents. 44. Please provide any comments that would expand upon your responses to any of the questions in the survey.

Data for this question is intentionally not reported because it is difficult to summarize and/or could identify respondents.

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgements

In addition to the contact named above, Sarah Veale, Assistant Director, and Kathleen Padulchick, Analyst-in-Charge, supervised the development of this report. Virginia Chanley, Lois Hanshaw, Linda Kohn, Jill Lacey, Albert Sim, and Meredith Trauner made significant contributions to all aspects of this report.