Securities and Exchange Commission: Actions Needed to Address Limited Progress in Resolving Long-Standing Personnel Management Challenges
Published: 2016-12-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Dodd-Frank Wall Street Reform and Consumer Protection Act contains a provision for GAO to report triennially on SEC's personnel management. GAO's first report in 2013 (GAO-13-621) identified a number of challenges, such as SEC's lack of a mechanism to monitor supervisors' use of its performance management system, and included seven recommendations.
This report examines (1) employee views on SEC's organizational culture since 2013 and (2) SEC's current personnel management practices. GAO surveyed all SEC employees (staff in its six key divisions and offices, staff in all other offices and divisions, and all senior officers, with response rates of 69, 60, and 70 percent, respectively); evaluated SEC policies and procedures against relevant criteria; and analyzed information on SEC's practices.
section_title: What GAO Found
paragraphs: Employee views on the Securities and Exchange Commission's (SEC) organizational culture have generally improved since 2013. Employees GAO surveyed cited improved levels of morale and trust within the agency compared to 2013 and noted that SEC was less hierarchical and risk-averse. However, GAO's survey indicated that SEC still operates in a compartmentalized way and that there is little communication and collaboration between divisions.
SEC made limited progress on improving personnel management. SEC has addressed two of seven recommendations from GAO's 2013 report, but it faces added challenges in cross-divisional collaboration and hiring and promotion.
Mechanisms to monitor supervisors' use of performance management system . Recently, SEC began to monitor how supervisors (1) provide feedback to staff, (2) recognize and reward staff, and (3) address poor performance. SEC's efforts address the related 2013 recommendation.
Accountability system . SEC implemented a system to monitor its human capital programs and inform its human capital goals consistent with Office of Personnel Management (OPM) guidance. SEC's efforts address the related 2013 recommendation.
Workforce and succession planning . SEC has developed a workforce and succession plan in response to two of GAO's recommendations, but the plan does not include some elements of OPM guidance, such as a skills gap analysis for all SEC staff. As a result, SEC continues to lack assurance that all staff have the necessary skills.
Performance management. Although GAO found in 2013 that SEC's performance management system was generally consistent with relevant criteria, SEC redesigned it in 2014 without first examining its effectiveness—a recommendation GAO made in 2013. SEC officials stated they do not plan any future reviews because they are piloting a new system. As a result, SEC lacks assurance that the new system will perform better than the current one.
Communication and collaboration . SEC has made little progress to address GAO's two recommendations related to improving cross-divisional collaboration. While SEC has recognized some staff for collaborating, it has yet to set expectations for all staff to collaborate across divisions as needed or implement relevant best practices to break down existing silos. As a result, SEC staff still report that divisions operate in isolation. Other than the SEC Chair's Office, which has competing demands on its time, no official has authority to affect the daily operations of the entire agency. Other organizations rely on their Chief Operating Officer (COO) to make such changes, but because SEC's COO lacks such authority, the agency will likely continue to face challenges.
In addition, GAO found that because SEC has not identified skills gaps among its hiring specialists, its training of these staff is limited. As a result, SEC lacks assurance that its hiring specialists have the necessary skills to hire and promote the most qualified applicants, in accordance with key principles of an effective control system.
section_title: What GAO Recommends
paragraphs: SEC should (1) provide authority to the COO or other official to enhance cross-divisional collaboration and (2) develop and implement training for hiring specialists that is informed by a skills gap analysis. GAO also reiterates the need to address the remaining five prior unaddressed recommendations on workforce planning, performance management, and intra-agency collaboration. SEC agreed with the second recommendation but disagreed with the first one. In particular, SEC disagreed that enhancing the role of the COO would be the optimal means to achieve further enhancements. GAO maintains that this recommendation will help improve cross-divisional communication and collaboration, as discussed in the report.

section_title: Letter
paragraphs: The Securities and Exchange Commission’s (SEC) mission is to protect  investors; maintain fair, orderly, and efficient securities markets; and  facilitate capital formation. To carry out its mission, SEC requires public  companies to disclose meaningful financial and other information to the  public, examines firms it regulates, and investigates potential securities  law violations. SEC oversees more than 25,000 market participants,  including nearly 12,000 investment advisers, approximately 10,500  mutual funds and exchange-traded funds, and nearly 4,500 broker- dealers. Under the Dodd‐Frank Wall Street Reform and Consumer  Protection Act (the Dodd-Frank Act), the agency’s jurisdiction was  expanded to include certain participants in the derivatives markets,  private fund advisers, and municipal advisers, among other changes.  The large number of entities SEC oversees, combined with the increased  complexity of these entities, underscores the importance of SEC’s  regulatory oversight of the securities markets.
Effectively carrying out its regulatory responsibilities requires that SEC  attract and retain a high-quality workforce. However, because SEC faced  a high-profile enforcement failure during the 2007–2009 financial crisis,  we and others have previously reported on the personnel management  challenges SEC has faced in building and retaining such a workforce.  These challenges have included a lack of emphasis on personnel  management and weaknesses in linking pay and performance. We most  recently reported on SEC’s personnel management challenges in 2013.  In our July 2013 report, we made seven recommendations to improve  SEC’s personnel management, including developing comprehensive  workforce plans, implementing mechanisms to monitor how supervisors  use the performance management system, conducting periodic  validations of the performance management system, exploring  collaboration practices of leading organizations, and regularly assessing  these efforts.
SEC is not unique in facing personnel management challenges. For  example, in February 2011, we found a number of cross-cutting skill gaps  across the federal government and recommended that actions be taken  by the Office of Personnel Management (OPM), federal agencies, and the  Chief Human Capital Officers (CHCO) Council to address critical skills  gaps that cut across several agencies. In response, OPM and agencies  launched several initiatives. For example, in September 2011, OPM and  the CHCO Council established an interagency working group to identify  and mitigate selected skills gaps in mission-critical occupations. Most  recently, in January 2015, we recommended that OPM and the CHCO  Council incorporate lessons learned from their initial efforts to close skill  gaps to strengthen future approaches. Strategic human capital  management has remained on our list of high-risk areas since 2001.
Section 962 of the Dodd-Frank Act includes a provision for us to report  triennially on SEC’s personnel management, including the competence of  professional staff; the effectiveness of supervisors; and issues related to  employee performance assessments, promotion, and intra-agency  communication. This report examines (1) employees’ views of SEC’s  organizational culture and personnel management, and the extent to  which these views have changed since our 2013 report and (2) the extent  to which selected SEC personnel management practices have been  implemented consistent with relevant standards.
To examine employees’ views of SEC’s organizational culture and the  extent to which they have changed since 2013, we implemented three  web-based surveys, based largely on our 2013 surveys, of all 4,236  nonsupervisory and supervisory staff and 148 senior officers. The three  surveys were of (1) the mission-critical office and divisions, (2) all other  offices and divisions, and (3) all senior officers, and their response rates  were approximately 69 percent, 60 percent, and 70 percent respectively.  Our three surveys consisted of both multiple-choice and open-ended  questions and the results from the multiple choice questions are detailed  in a separate GAO report (GAO-17-95SP). We do not make any attempt  to extrapolate the findings to the eligible staff who chose not to complete  our surveys.
To determine the extent to which selected SEC personnel management  practices have been implemented consistent with relevant standards, we  examined a variety of policies and identified the ones that were designed  consistent with relevant criteria and tested their implementation. First,  we assessed SEC policies, procedures, and other documents related to  personnel management against applicable federal regulations, OPM’s  Human Capital Assessment and Accountability Framework (HCAAF), key  human capital practices, our prior work on personnel management and  workforce issues, and federal standards for internal control. Second,  based on our assessment of SEC documents (e.g., plans, policies,  standard operating procedures, and annual reports), interviews with  officials from SEC’s mission-critical office and divisions and the Office of  Human Resources, and discussions with our internal human capital  officials, we identified four practices that were designed consistent with  relevant OPM criteria and best practices: hiring and promotions,  recognizing and rewarding performance, providing feedback, and  addressing unacceptable performance. Finally, we tested these four  practices to determine whether SEC was implementing them consistent  with its policies and procedures. This testing consisted of performing file  reviews and data analysis. We also conducted interviews with relevant  SEC staff in the Office of Human Resources and the mission-critical office  and divisions. As part of this effort, we reviewed actions SEC had taken to  address the seven recommendations from our 2013 report related to four  personnel management areas: (1) workforce planning, (2) performance  management, (3) communication and collaboration, and (4) human  capital accountability.
For all objectives, we conducted semi-structured interviews with a  nonprobability sample of SEC supervisors selected based on whether  they worked in the mission-critical office and divisions, their position and  grade level, geographic location, and their willingness to participate. We  also interviewed former SEC employees, officials from the SEC Office of  Inspector General (IG), and self-selected representatives and members of  the SEC employees’ union. We selected a nonprobability sample of  former employees to interview that reflected the diversity of former  employees in terms of pay grade, occupational category, and tenure,  among other factors. The views of these officials are not generalizable.
We assessed the reliability of all of the data we used during this review  and determined they were sufficiently reliable for our purposes, which  include describing trends and views on personnel management practices  at SEC. To assess the reliability of OPM’s Federal Employee Viewpoint  Survey (FEVS) data, we examined descriptive statistics and data  distribution and we reviewed missing data. We also reviewed FEVS  technical documentation, as well as the statistical code OPM uses to  generate the index and variance estimates, and we interviewed officials  responsible for collecting, processing, and analyzing the data. We used  SEC data derived from the Department of the Interior’s Federal  Personnel/Payroll System to construct our sample frames for the three  surveys, test the implementation of various personnel management  practices, and develop summary tables in our appendixes. To determine  the reliability of these data, we interviewed SEC staff responsible for the  data to determine how data were collected, what controls existed over the  data, and any limitations on the data. In addition, where possible, we  compared data elements to the original source documents to corroborate  the accuracy of the data where available. Appendix II provides more  information on our scope and methodology.
We conducted this performance audit from July 2015 to December 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: SEC consists of a five-member Commission that oversees the agency’s  operations and provides final approval over staff interpretation of federal  securities laws, proposals for new or amended rules to govern securities  markets, and enforcement activities. The Commission, which is headed  by the SEC Chair, oversees 5 divisions, 23 offices, and 11 regional  offices. Figure 1 illustrates SEC’s organizational structure.
SEC’s divisions and offices are organized by functional responsibility.  Table 1 summarizes the roles and responsibilities of the one office and  five divisions that primarily implement SEC’s mission: the Office of  Compliance Inspections and Examinations and the Divisions of  Corporation Finance, Enforcement, Investment Management, Economic  and Risk Analysis, and Trading and Markets. The mission-critical office  and divisions are supported by other offices, such as the Office of  Financial Management, the Office of Information Technology, and the  Office of Human Resources. The Office of Information Technology  supports SEC and its employees in all aspects of information technology  (IT) and has overall management responsibility for SEC’s IT program,  including application development, infrastructure operations and  engineering, user support, IT program management, capital planning,  security, and enterprise architecture. SEC’s Office of Human Resources  provides overall responsibility for the strategic management of SEC’s  personnel management and assesses compliance with federal  regulations for areas such as recruitment, retention, leadership and staff  development, and performance management. However, certain divisions  have internal human resource coordinators that liaise between the Office  of Human Resources and their respective division heads. The Office of  Information Technology and the Office of Human Resources report to  SEC’s Office of the Chief Operating Officer (COO), which in turn reports  to the Office of the Chair.
To carry out its mission, SEC employs staff with a range of skills and  backgrounds, including attorneys, accountants, and economists. As of  February 2016, SEC employed 4,674 staff. Of these, approximately 40  percent were attorneys, 21 percent were accountants or financial  analysts, and 6 percent were examiners. The remaining 33 percent were  other professional, technical, administrative, and clerical staff. From fiscal  years 2013 through 2015, SEC hired 1,310 employees. To help SEC  attract and retain qualified employees, Congress enacted the Investor  and Capital Markets Fee Relief Act (Pay Parity Act) in 2002, which  allowed SEC to implement a new compensation system with unique pay  scales comparable to those of other federal financial regulators. SEC  staff are represented by the National Treasury Employees Union (which  we refer to in this report as the SEC employees’ union).
subsections: section_title: Personnel Management Challenges at SEC
paragraphs: Effectively carrying out its regulatory responsibilities requires that SEC  attract and retain a high-quality workforce. However, we and others have  previously reported on the personnel management challenges SEC has  faced in building and retaining such a workforce. These personnel  management challenges included challenges related to establishing a  constructive organizational culture and developing effective personnel  management practices.
For example, a 2011 SEC Inspector General (IG) report found that the  level of communication between the Office of Compliance Inspections  and Examinations (OCIE) and the Division of Enforcement after a  referral—that is, the extent to which noteworthy information from an  examination was passed on to the Division of Enforcement for further  investigation or action—was not always consistent in the regional offices,  which the IG noted can hinder SEC’s ability to achieve its mission. In  addition, a Boston Consulting Group report also noted in 2011 that SEC’s  culture impaired communication and collaboration between divisions.  According to the report, each division’s internal structure was tailored to  division-specific needs, and SEC historically placed limited emphasis on  using formalized mechanisms for cross-divisional collaboration.
More recently, in July 2013, we found that SEC’s organizational culture  hindered its ability to effectively fulfill its mission and identified a number  of personnel management deficiencies. We also noted that  organizations with more constructive cultures generally perform better  and are more effective. Within constructive cultures, employees exhibit a  stronger commitment to mission focus, accountability, coordination, and  adaptability. We found a number of deficiencies in four areas related to  SEC’s personnel management and made seven recommendations to  help SEC address personnel management challenges:
Workforce planning: We found that SEC had not developed a  comprehensive workforce plan. In addition, we found that SEC had  not developed a comprehensive management succession plan to fill  agency supervisory positions. As a result, we recommended that the  Chairman of SEC direct the COO and Office of Human Resources to  (1) prioritize efforts to expeditiously develop a comprehensive  workforce plan, including a succession plan, and establish time  frames for implementation and mechanisms to help ensure that the  plans are regularly updated; and (2) incorporate OPM guidance as  they develop the workforce and succession plans by developing a  formal action plan to identify and close competency gaps and fill  supervisory positions and institute a fair and transparent process for  identifying high-potential leaders from within the agency.
Performance management: We found that while SEC had  performance standards related to supervisors’ use of the performance  management system, we did not identify specific mechanisms to  monitor supervisors’ use of the system. In addition, we found no  evidence that SEC had validated the system with its staff to help  ensure its credibility. As a result, we recommended that the  Chairman of SEC direct the COO and Office of Human Resources to  (1) create mechanisms to monitor how supervisors use the  performance management system to recognize and reward  performance, provide meaningful feedback to staff, and effectively  address unacceptable performance, for example, by requiring ongoing  feedback discussions with higher-level supervisors; and (2) conduct  periodic validations (with staff input) of the performance management  system and make changes, as appropriate, based on these  validations.
Communication and collaboration: We found that although SEC  had taken steps to improve intra-agency communication and  collaboration, barriers still existed. In addition, we found that staff  continued to identify barriers to effective communication and  collaboration among the divisions, within the divisions, and between  staff and management, contrary to collaborative best practices. As a  result, we recommended that the Chairman of SEC direct the COO to  (1) identify and implement incentives for all staff to support an  environment of open communication and collaboration, such as  setting formal expectations for its supervisors to foster such an  environment, and recognizing and awarding exceptional teamwork  efforts; and (2) explore communication and collaboration best  practices and implement those that could benefit SEC.
Human capital accountability: We found that SEC had not  developed an accountability system to monitor and evaluate its  personnel management programs and systems. As a result we  recommended that the Chairman of SEC direct the COO and Office of  Human Resources to prioritize and expedite efforts to develop and  implement a system to monitor and evaluate personnel management  activities, policies, and programs, including establishing and  documenting the steps necessary to ensure completion of the system.
SEC agreed with our recommendations and acknowledged that  improvements could be made in its personnel management. We discuss  the progress SEC has made toward addressing these recommendations  throughout this report. In addition, appendix I summarizes the status of  our 2013 recommendations, as of December 2016.
section_title: OPM’s Human Capital Assessment and Accountability Framework
paragraphs: OPM advocates the use of HCAAF, which is a set of tools and strategies  available to federal agencies that assist officials in achieving results in  personnel management programs. HCAAF is designed to guide the  assessment of agency efforts, while allowing enough flexibility for  agencies to tailor these efforts to their missions, plans, and budgets. The  framework uses five standards for success, lists key questions to  consider, and suggests performance indicators for measuring progress  and results. The five standards for success are as follows:
Strategic alignment: Agency strategies for human capital  management are aligned with mission, goals, and organizational  objectives and are integrated into its strategic plan and performance  budget.
Leadership and knowledge management: Agency leaders and  managers effectively manage people, ensure continuity of leadership,  sustain a learning environment that drives continuous improvement in  performance, and provide a means to share critical knowledge across  the organization.
Results-oriented performance culture: The agency has a diverse,  results-oriented, high-performing workforce and a performance  management system that effectively differentiates between high and  low levels of performance and links individual/team/unit performance  to organizational goals and desired results.
Talent management: The agency has closed gaps or deficiencies in  skills, knowledge, and competencies for mission-critical occupations  and made meaningful progress toward closing such gaps or  deficiencies in all occupations used in the agency.
Accountability: A data-driven, results-oriented planning and  accountability system guides the agency’s decisions on human capital  management.
OPM has provided some updates to the HCAAF model to federal  agencies and recently revised its personnel management regulations (the  basis for HCAAF).  According to OPM, the revised regulations, which  are scheduled to go into effect April 11, 2017, should reinforce existing  content and streamline the systems to make the framework more  practical to use. The new framework, called the Human Capital  Framework, will replace HCAAF and reduce the number of systems from  five to four (strategic planning and alignment, talent management,  performance culture, and evaluation).
section_title: Employee Views of Organizational Culture and Personnel Management Generally Improved, but Views on Cross- Divisional Collaboration Remain Unfavorable
paragraphs: Based on responses to our surveys of all SEC employees, we determined  that views of the agency’s organizational culture have generally improved  since 2013. Organizational culture is the underlying assumptions,  beliefs, values, attitudes, and expectations shared by an organization’s  members that affect their behavior and the behavior of the organization  as a whole. In July 2013, we reported that SEC’s organizational culture  was not constructive and could hinder its ability to effectively fulfill its  mission. We previously found that organizations with more constructive  cultures generally perform better and are more effective at fulfilling their  mission; within constructive cultures, employees also exhibit a stronger  commitment to mission focus, accountability, coordination, and  adaptability. Although we determined that employee views of SEC’s  organizational culture have generally improved, employee perceptions  about management’s efforts to improve cross-divisional collaboration  remain low and have not changed since 2013.
subsections: section_title: Employees’ Views of Organization Culture Generally Improved from 2013 to 2016
paragraphs: While some staff continue to raise concerns, generally employees’ views  related to morale, trust, hierarchy, and risk aversion have improved since  2013.
subsections: section_title: Morale
paragraphs: Our survey and other evidence indicate that both nonsupervisors’ and  supervisors’ views of morale have improved since 2013 (see fig. 2).  Based on our 2016 survey results, about 43 percent of nonsupervisors  responded positively (strongly or somewhat agree) when asked whether  employee morale is generally high most of the time, compared with about  30 percent in 2013. Also, about 51 percent of supervisors responded  positively to this question in 2016, up from about 39 percent in 2013.
Similarly, in response to a related question about morale, around 17  percent of nonsupervisors who completed our 2016 survey said that  senior officers in their division or office worked (to a great extent) to make  improvements in workforce morale, up from about 10 percent in 2013.  Also in 2016, about 28 percent of supervisors responded to our survey  that senior officers in their division or office worked to a great extent to  make improvements in workforce morale, about a 10 percentage point  increase from 18 percent in 2013.
While our survey results suggest that morale has improved, many SEC  employees we spoke with cited concerns related to favoritism and a lack  of workplace diversity and promotion opportunities that resulted in low  morale among some employees. Additionally, SEC employees from the  mission-critical office and divisions provided 369 written responses to our  survey questions that addressed challenges related to morale at SEC.  For example, one employee described a work environment that promoted  staff based on favoritism and an unwillingness by senior officers to make  the necessary changes (including addressing low performing staff) to  improve employee morale. (We discuss our assessment of SEC’s policies  and practices related to promotions and addressing unacceptable  performance later in this report.)
However, our positive survey findings are generally consistent with the  2016 OPM Federal Employee Viewpoint Survey (FEVS). OPM estimated  that SEC’s overall Global Satisfaction Index score—which measures  employee satisfaction with job, pay, and their organization and is  calculated based on FEVS results—increased from about 59 percent in  2012 to 77 percent in 2016. Although this index score may not directly  correlate to employee perceptions of morale, it is an important indicator of  employee views about whether the agency sufficiently values its staff.
section_title: Atmosphere of Trust
paragraphs: Our survey indicates that nonsupervisors’ views about an atmosphere of  trust at SEC have improved, increasing from approximately 45 percent in  2013 to 55 percent in 2016. However, SEC employees from the mission- critical office and divisions provided 112 written responses to our survey,  raising concerns about SEC’s atmosphere of distrust. In addition, 1  former employee described SEC’s promotion process as lacking  transparency and favoring certain employees. This perceived lack of  transparency and favoritism can erode trust between staff and  management because it raises questions about the fairness of SEC’s  promotion process. The views of supervisors on this issue improved  slightly between 2013 and 2016, and positive responses from senior  officers increased from about 81 percent in 2013 to 83 percent in 2016,  as illustrated in figure 3.
Similar to our findings, OPM recently found a slight increase in employee  trust at SEC. OPM estimated that the engagement index for supervisors  calculated based on FEVS results increased from an estimated 72  percent in 2012 to 73 percent in 2016. SEC scores are similar to those  of the National Credit Union Administration (NCUA), which had an  estimated score of 79 percent in both 2012 and 2015 according to OPM  and a score identical to SEC’s in 2016 (73 percent). According to OPM,  the government-wide average score was 65 percent in 2016.
section_title: Hierarchy and Risk Aversion
paragraphs: Our 2016 survey and other evidence indicate that both nonsupervisors’  and supervisors’ views about a hierarchical culture have improved. We  reported in 2013 that some SEC staff described the agency’s culture as  “hierarchical” (that is, decisions are made from the top with little if any  solicitation of input from staff). In 2013, about 38 percent of  nonsupervisory staff who responded to our survey strongly or somewhat  agreed that they had a voice in decisions that affected them; in contrast,  about 50 percent of nonsupervisory staff strongly or somewhat agreed  with this statement in 2016 (see fig. 4). Supervisors’ positive responses to  this statement also increased slightly—about 64 percent strongly or  somewhat agreed in 2013, compared to about 67 percent in 2016. Similar  to our survey results, when OPM surveyed SEC employees about  whether they have a feeling of personal empowerment with respect to  work processes, OPM estimated that 33 percent of staff had a positive  attitude (agreed and strongly agreed) in 2012, which increased to 51  percent of staff in 2016. Despite significantly more positive survey  results from nonsupervisory staff, senior officers held less positive views  than they did in 2013; about 100 percent strongly or somewhat agreed in  2013, compared to about 84 percent in 2016.
Regarding excessive risk aversion—that is, the condition in which an  agency’s ability to function effectively is hindered by the fear of taking on  risk—our survey and other evidence indicate that nonsupervisors’ and  supervisors’ views have improved significantly since 2013. The  percentage of survey respondents who agreed that fear of public  scandals had made SEC overly cautious and risk averse fell from about  55 percent in 2013 to about 46 percent in 2016 for nonsupervisory staff,  and from about 58 percent in 2013 to about 49 percent in 2016 for  supervisory staff (see fig. 5). While our survey results show  improvements, SEC employees from the mission-critical office and  divisions provided 125 written survey comments related to concerns  about risk-averse leaders. A few staff who provided written comments  stated that some supervisors and peers fear bad publicity and are still  risk-averse, which results in a refusal to admit wrongs or a dislike of being  questioned by subordinates. One employee noted that the fear of the  appearance of impropriety limits SEC’s ability to bring in industry experts.  In addition, one former nonsupervisory employee described a work  environment that did not encourage change or innovation. This employee  stated that she would have been reprimanded for presenting new ideas.
Furthermore, the percentage of staff responding to our survey who  agreed that the fear of being wrong had made some senior officers  reluctant to take a stand on important issues fell from about 47 percent in  2013 to about 38 percent in 2016 for nonsupervisory staff; from about 44  percent in 2013 to about 39 percent in 2016 for supervisors; and from  about 23 percent to about 19 percent for senior officers (see fig. 6).
Similarly, OPM estimated that employees’ views of SEC leaders improved  from 2012 to 2016. OPM created an engagement index based on the  FEVS results that measures employees’ views about the integrity of their  leaders, including their perceptions of their leaders’ behavior related to  communication and workforce motivation. OPM estimated that the  engagement index score for SEC in this area increased from 49 percent  in 2012 to 63 percent in 2016. While this index captures elements of  leadership behaviors beyond top-down decision making and risk  aversion, it reflects employees’ perceptions of how well senior leaders  communicate the goals and priorities of the organization, among other  things, and, as such, captures staff attitudes toward the perceived levels  of hierarchy. When compared with the average government-wide score in  2016 of 53 percent, SEC’s estimated score of 63 percent is 10  percentage points higher.
section_title: Employees’ Survey Responses Indicate That SEC Continues to Operate in a Compartmentalized Way
paragraphs: Our 2016 survey results indicate that SEC continues to operate in a  compartmentalized manner. In 2013, we reported that such an  environment can hinder SEC’s ability to effectively carry out its mission by  limiting communication and collaboration among the divisions. For  example, consistent with our 2013 findings, with the exception of the  Division of Enforcement, at least one-third of nonsupervisory staff  responding to our 2016 survey never contacted staff in other divisions or  offices in headquarters in the past 12 months for work-related issues (see  fig. 7).
According to a number of staff who provided written comments to our  2016 survey, SEC is comprised of “silos”—that is, work is  compartmentalized in each division or office, and little communication or  collaboration occurs between the divisions. Several current and one  former SEC employees we spoke with expressed similar views. For  example, some employees cited a culture that was not supportive of  cross-divisional communication. Of the 187 employees we interviewed,  78 considered issues around siloed communication, which includes  communication and collaboration between and within units and divisions,  as an area where SEC needs to improve. Further, one former employee  stated that communication between offices was only encouraged at the  most senior levels in the agency. This employee also said that although  SEC’s Commission required the breaking up of silos after the collapse of  Bernard L. Madoff Investment Securities, LLC, this requirement was  never implemented at the staff level. Additionally, we received 597  written responses to our survey questions (of a total of 1,947 responses)  citing various issues and challenges related to communication and  collaboration at SEC. Finally, supervisors we interviewed said that it is  sometimes difficult to know who to contact if you need to collaborate with  a particular individual or a group with whom you do not normally work.  (We discuss our assessment of SEC’s efforts to improve communication  and collaboration later in this report.)
OPM found improvement at SEC related to cross-divisional  communication and collaboration. In OPM’s 2016 FEVS survey, an  estimated 60 percent of SEC employees responded positively (agree and  strongly agree) when asked if managers promote communication among  different work units (for example, about projects, goals, needed  resources), compared to an estimated 47 percent in 2012. In addition,  one former employee who had been a senior officer at SEC described a  substantial improvement in communication after the 2007–2009 financial  crisis.
section_title: SEC Has Taken Some Steps to Improve Personnel Management, but Progress Has Been Limited Overall
paragraphs: SEC has developed mechanisms to monitor supervisors’ use of its  performance management system and developed and implemented a  system to monitor and evaluate personnel management activities,  consistent with our 2013 recommendations in these areas, but progress  to improve personnel management in other areas has been limited. Since  our prior review, SEC has developed mechanisms to monitor supervisors’  use of the performance management system to provide performance  feedback, reward strong performance, and address unacceptable  performance. In addition, SEC has implemented an accountability system  to evaluate personnel management activities, policies, and programs.
However, SEC’s actions to address personnel management practices in  the areas of workforce and succession planning, performance  management, and cross-divisional communication and collaboration have  not been sufficient to address our 2013 recommendations. Further, we  found that SEC lacks controls over some aspects of its hiring and  promotions.
subsections: section_title: SEC Has Created Mechanisms to Monitor Supervisors’ Use of the Performance Management System
paragraphs: We found that SEC has implemented mechanisms to monitor how  supervisors use the performance management system. According to  OPM guidance, an effective performance management system provides  mechanisms to monitor how supervisors use that system and provide  feedback to staff. OPM does not provide specific requirements on the  structure of these mechanisms, allowing agency discretion. In our July  2013 report, we recommended that SEC create mechanisms to monitor  how supervisors use the performance management system to provide  meaningful feedback to staff, recognize and reward performance, and  effectively address unacceptable performance. Based on our review,  SEC’s efforts to implement mechanisms to monitor supervisors’ use of  the system in these areas are sufficient to address our 2013  recommendation.
subsections: section_title: Providing Meaningful Performance Feedback
paragraphs: SEC has taken steps to monitor the performance feedback supervisors  provide to employees. While we did not independently assess the quality  of feedback provided to employees, we examined SEC’s process for  monitoring feedback as well as our survey results that relate to the  performance feedback employees receive. Consistent with the OPM  guidance, each year SEC monitors whether supervisors are providing the  required feedback by reviewing a random sample of 5 percent of  performance work plans each fiscal year; these work plans contain  documentation that the supervisor provided the interim and final  performance feedback to the employee. SEC’s review of the random  sample of the performance work plans involves assessing the  documentation to determine whether employees and supervisors have  completed the formal performance appraisal process and whether the  supervisor provided feedback to the employee. We found that while  SEC’s random selection of 5 percent of performance work plans produces  results which are not generalizable, the methodology is sufficient to  gauge general compliance with SEC policies.
SEC’s review of performance work plans for fiscal year 2014, the most  recent review available, found that 96 percent of the sampled employees  had discussed performance expectations with their supervisors, 92  percent had a midyear performance review, and 98 percent had received  end of year feedback. While the fiscal year 2014 review did not make any  new recommendations for improvement, SEC’s fiscal year 2013 review of  performance work plans, the first time SEC had conducted these reviews,  made recommendations that included providing additional online  resources to supervisors and uploading copies of performance work plans  to an electronic OPM database. In response to the 2013 review, SEC  provided supervisors with more online resources about the performance  management process in SEC’s shared database and continued to upload  performance work plans into staffs’ electronic official personnel folders. In  addition, SEC plans to continue these reviews annually, according to SEC  officials.
Overall, survey responses by nonsupervisors related to feedback  improved modestly since 2013 (see fig. 8). The percentage of  nonsupervisory staff who agreed that their current direct supervisor  provided useful and constructive feedback increased from about 65  percent to about 70 percent from 2013 to 2016. A few employees who  provided written responses to our survey noted that supervisors in their  workgroup provided constructive feedback. In addition, SEC’s level of  positive responses on performance feedback was similar to that of other  agencies in OPM’s 2016 FEVS. Specifically, when employees were  asked whether they agreed with the statement “Discussions with my  supervisor about my performance are worthwhile,” OPM estimated that  66 percent of SEC respondents agreed or strongly agreed with the  statement, compared to 63 percent of respondents government-wide.
While employees’ views related to the quality of feedback provided have  generally improved since 2013, some supervisors and staff we met with  identified some areas of concern that are common across the  government. For example, we interviewed two groups of supervisors and  both groups told us that the quality of feedback an employee receives can  be inconsistent and is often dependent on their particular supervisor. In  addition, supervisors in one group stated that they are sometimes  reluctant to provide negative feedback to staff for fear of retaliation by the  SEC employees’ union. Finally, in our conversations with SEC employees  and in comments on our survey, some employees told us that feedback  was not consistently substantive and timely.
section_title: Rewarding Strong Performance
paragraphs: In response to our July 2013 recommendation, SEC has also  implemented mechanisms to monitor how supervisors recognize and  reward performance. For example, during the course of our review, the  accountability group in the Office of Human Resources issued a report on  its review of SEC’s award program. The purpose of the review was to  help ensure that SEC’s Employee Recognition Program and  performance-based cash awards were in compliance with applicable  federal laws, rules, and regulations and SEC policies and procedures,  and to determine how awards were distributed across the agency.  According to the report, SEC’s accountability group took a number of  steps to monitor how awards were being distributed to SEC employees.  First, they reviewed the criteria and justification for the incentive awards.  Second, they analyzed demographic data to determine the distribution of  incentive awards throughout SEC. Third, they interviewed key staff  responsible for the SEC awards program to help ensure they understood  how to administer the program in compliance with relevant SEC policies  and procedures and federal laws and regulations. We found that SEC’s  actions to review the program are consistent with OPM guidance on  monitoring supervisors’ use of the performance management system.
Overall, SEC’s accountability group found that the awards program had  improved over time and that information about the program was well- communicated and highly visible to staff, automated, and sufficiently  funded. However, the group recommended that supervisors ensure that  nominating staff document the justification for all awards that are not  based on a rating of record, as required by regulation. SEC officials  responsible for the awards program agreed with the recommendation  and, according to the accountability group’s report, are taking actions to  review submitted awards to ensure all program requirements are met,  including the requirement for supervisors to ensure that each award  recommendation is justified. According to the accountability group’s  planning document, the group plans to evaluate the awards program  every 3 years through 2027. In addition, SEC officials told us that they  plan to make additional program improvements based on a 2014 SEC  Office of Equal Employment Opportunity initial review of SEC’s awards  data. In response to this initial review, SEC is conducting an analysis to  determine if SEC’s policies, practices, or procedures are creating any  barriers in recognition and awards. SEC expects to complete this analysis  in March 2017.
Similar to SEC findings, we also found that SEC’s award program is  designed and implemented consistent with OPM’s HCAAF, which notes,  among other things, that an agency’s award system should have clear  criteria and include a variety of types of awards. We performed our own  independent analysis of SEC’s awards program by reviewing a  nongeneralizable sample of 71 award packages and found that SEC is  implementing its awards program consistent with its policies and  procedures. We reviewed these packages to determine whether awards  packages are consistent with awards criteria, which includes whether the  awards have written justifications and required signatures by the staff  submitting award recommendations and staff reviewing the awards, and  whether the awards are accurately reflected in the employees’ personnel  records. We found that all award packages we reviewed had a written  justification describing what the employee or group of employees had  accomplished to receive consideration for the award and had the requisite  signatures from the division or office, as well as from the Office of Human  Resources, indicating that the relevant officials had reviewed the awards  package and approved it. Finally, we found that for all award packages  we reviewed, the approved cash award or time-off award was accurately  reflected in all the employees’ personnel records.
section_title: Addressing Unacceptable Performance
paragraphs: In response to our 2013 recommendation, SEC has implemented  mechanisms to monitor supervisor practices to address unacceptable  performance. Consistent with OPM guidance and federal regulations,  SEC supervisors are required to gather relevant information, such as  examples of work products that do not meet performance standards and  any relevant e-mails discussing the individual’s performance. They must  also document the unacceptable performance prior to putting a  permanent employee on a performance improvement plan or terminating  employment for a probationary employee (generally employees who have  been on the job for less than 1 year). According to SEC officials, the  Office of General Counsel, which is now responsible for coordinating  SEC’s practices related to addressing unacceptable performance, tracks  employees who receive an annual performance rating of “unacceptable”  (which would generally precipitate a performance improvement plan). It  also follows up with the employee’s supervisors to ensure the supervisors  are taking the required steps to address the performance issue. As  documented in the performance improvement plans, the Office of General  Counsel is to ensure that supervisors (1) describe the unacceptable  performance, (2) describe what actions the employee needs to take to  address the unacceptable performance, (3) specify the amount of time  the employee will have to improve his or her performance, and (4)  describe the consequences if the employee’s performance fails to  improve.
Based on our review of performance improvement plans for permanent  employees and actions taken against probationary employees, we found  that SEC has implemented its practices related to addressing  unacceptable performance consistent with its policies and procedures.  Specifically, we reviewed all 16 performance improvement plans SEC  issued in fiscal years 2013 through 2015 and found that these plans  contained all the required information. We also reviewed all files related  to terminations of employees during probationary periods for fiscal years  2013 through 2015 (20 in all) and found that they all contained the  required information under the regulations. Specifically, these files  described the unacceptable performance and the effective date of the  termination, which in all cases was within the 1-year probationary  period.
section_title: SEC Has Developed a Human Capital Accountability System to Help Assess Its Personnel Management Programs
paragraphs: In response to our 2013 recommendation, SEC has designed and  implemented a human capital accountability system (that is, a system  designed to facilitate regular assessments of an agency’s personnel  management programs). In our July 2013 report, we found that SEC  had not implemented a way to monitor and evaluate its personnel  management. As a result, we recommended that SEC prioritize and  expedite efforts to develop and implement a system to monitor and  evaluate personnel management activities, policies, and programs,  including establishing and documenting the steps necessary to ensure  completion of the system. Since that time, SEC has taken steps to  address the recommendation.
In 2015, SEC designed a human capital accountability system, including  an underlying plan and standard operating procedures. According to  OPM’s HCAAF, a human capital accountability system should evaluate  results and provide consistent means for an agency to monitor and  analyze its performance on all aspects of its human capital management  policies, programs, and systems. OPM guidance also states that the  accountability system should contribute to an agency’s performance by  identifying and monitoring necessary improvements. An accountability  system should also provide for annual assessments of an agency’s  progress and results related to human capital management. SEC’s  accountability system requires that staff in the Office of Human  Resources review programs, recommend corrective actions and track the  status of those actions, and provide an annual assessment of the  progress. Steps SEC has taken to implement the system include the  following:
SEC evaluated its Student Loan Repayment Program in October 2015  and found weaknesses in internal controls—for example, controls  related to documentation of the decision to accept or reject an  application for the program—and made recommendations for  strengthening these controls.
Since September 2014, SEC has also conducted quarterly reviews of  personnel actions and recruitment case files and identified  weaknesses such as incorrect offer letters and missing evidence of  rankings of job candidates, which we discuss in more detail later.  SEC’s quarterly reviews also identified some positive findings,  including that SEC’s job opportunity announcements had few  significant errors.
In April 2016, SEC provided its first annual human capital  accountability report to the Chief Human Capital Officer. This report  summarizes the actions SEC took to review its human capital  programs in fiscal year 2015 and lists the remaining steps necessary  to fully implement the system.
In addition, the results of SEC’s human capital accountability system have  informed the agency’s human capital goals and spending priorities.  According to HCAAF, the results of the human capital accountability  system should inform an agency’s human capital goals and objectives as  well as its spending priorities. For example, SEC relied on the results of  its review of the Student Loan Repayment Program to set goals related to  attracting and retaining talent. Specifically, SEC found that the program  lacked a process to document why some employees’ applications were  denied and therefore was unable to ensure that qualified and talented  employees benefited from the program. According to SEC officials, in  response to the results of the review, SEC broadened its goal of attracting  and retaining talented employees by incorporating goals related to  improvements to the management and oversight of the program.  Likewise, officials said that SEC used the results of its quarterly reviews  of its recruitment case files, which found improvements in aspects of the  recruitment and hiring process, to set more challenging goals to hire  larger numbers of staff in a more efficient manner. Based on the results of  these reviews, SEC requested additional Office of Human Resources  staff in its 2017 budget justification request, according to SEC officials.  These linkages are important for providing assurance that SEC’s human  capital accountability system is contributing to its human capital goals and  priorities. As a result of these findings, we have concluded that SEC  addressed our 2013 recommendation to develop and implement a system  to monitor and evaluate personnel management activities, policies, and  programs.
In addition, although SEC completed only about half of its planned  reviews of human capital programs for fiscal years 2015 and 2016, it is  taking steps to address this issue. SEC staff told us that because this was  their first human capital accountability system, they had not developed  specific criteria for selecting programs to review, other than those they  were required to review by regulation. After discussions we held with  relevant SEC officials throughout the course of our review, in January  2016 SEC established criteria for programs to be reviewed through fiscal  year 2027. SEC staff in the Office of Human Resources now assign a  priority level to each program, function, or activity they plan to review  based on its regulatory review requirement, the necessary  implementation costs, and the number of employees affected. The higher  the priority, the more often the program, function, or activity is to be  evaluated. SEC staff also told us that they are in the process of updating  the standard operating procedures for the accountability system and that  these procedures would be finalized by early calendar year 2017. In  addition, when SEC staff planned the fiscal years 2015 and 2016 reviews,  they said that they did not anticipate the resources required to complete  them. As such, they now plan in a way that takes into account available  resources, which will limit the number of reviews in the future. By applying  these newly established priorities and planning procedures, SEC should  be in a better position to complete key program reviews that are an  essential component of its human capital accountability system.
section_title: SEC Continues to Face a Number of Personnel Management Challenges
paragraphs: While SEC has taken some actions to address our 2013  recommendations on workforce and succession planning, performance  management, and cross-divisional communication and collaboration, we  found that these actions were insufficient to address our 2013  recommendations. In addition, we found that while SEC’s hiring and  promotion policies and procedures are generally consistent with OPM and  other relevant criteria, SEC lacks adherence to controls over some  aspects of hiring and promotions.
subsections: section_title: Workforce and Succession Plans
paragraphs: Consistent with our 2013 recommendation that SEC prioritize efforts to  create a workforce and succession plan consistent with OPM guidance,  SEC has recently developed plans, but they do not include some key  components of strategic workforce and succession planning identified by  OPM and our previous work. In our July 2013 report, we found that SEC  had not yet developed a comprehensive workforce and succession plan.  We recommended that the COO and the Office of Human Resources (1)  develop a comprehensive workforce and succession plan and (2)  incorporate relevant OPM guidance as they develop this plan. SEC has  not yet fully addressed these recommendations.
In July 2016, SEC finalized its workforce plan for fiscal years 2016  through 2018, which included some elements of OPM guidance and best  practices we have previously identified. For example, OPM guidance  states that effective workforce planning aligns workforce requirements  with agency strategic plans. Furthermore, key principles for effective  workforce planning we have identified call for agencies to include plans to  monitor and evaluate the agency’s progress toward meeting its human  capital goals. SEC’s workforce plan is aligned with its strategic plans,  references the goals outlined in those plans, and includes performance  measures to monitor and evaluate SEC’s progress toward its goals. In  addition, key principles for effective workforce planning we have identified  also call for agencies to involve top management, staff, and other  stakeholders in the workforce planning process. SEC’s workforce plan  involves relevant stakeholders, including division and office leadership,  SEC University (SEC’s lead office for training), and focus groups of SEC  employees.
However, the workforce plan does not meet all the key principles for  effective workforce planning:
Skills gap analysis. SEC’s workforce plan lacks a comprehensive  skills gap analysis. OPM has stated, and our past work has shown,  that an agency should identify the critical skills its workforce needs,  develop a comprehensive assessment of the gaps in those skills, and  develop strategies to address those gaps. In 2015, SEC entered into  a contract with OPM to conduct a skills gap analysis of mission-critical  occupations, but the contract did not include an analysis of all  occupations at SEC because the agency chose to prioritize select  occupations in the mission-critical office and divisions and the Office  of Information Technology. As a result, the skills gap analysis did not  include an assessment of the competency of 33 percent of SEC’s  workforce, including mission-support staff, such as staff in the Office  of Human Resources, and supervisors. Without assessing the skills of  these key positions, SEC does not have assurance that its personnel  across the agency, including those responsible for carrying out critical  personnel management functions, have the skills necessary to fulfill  SEC’s mission.
Workforce structure. SEC’s workforce plan does not inform decision  making about the structure of the workforce. OPM guidance states  that an agency’s workforce plan should inform decision making about  how best to structure the organization and deploy the workforce.  However, the plan does not identify the optimal number of attorneys  (key staff responsible for carrying out SEC’s mission) SEC should  employ or the percentage of the workforce that should be located in  the regional offices. It also lacked information on the type of skills  needed by, for example, attorneys.
Links to budget. SEC’s workforce plan is not clearly linked to its  budget formulation, which we and OPM have previously identified as  a best practice. For example, the workforce plan does not identify  the personnel costs of the current workforce, nor does it identify the  number of employees SEC intends to hire and their associated cost.  When linked to the budgeting process, workforce planning provides  information that agencies need to help ensure that their annual budget  requests include adequate funds to implement their human capital  strategies.
In addition, the component of SEC’s workforce plan that addresses  succession planning lacks information on workforce attrition and lacks a  process for identifying future leaders. OPM guidance states that agencies  should have a leadership succession planning and management system  that is based on accurate data on the current workforce and accurate  projections of attrition at all leadership levels. OPM guidance also states  that agencies should develop a fair and accurate process to identify a  diverse pool of high-potential leaders. SEC’s succession plan describes  the various levels of leadership at SEC and what is required for  successful performance at each level. It also includes the leadership  competencies for all leadership positions and senior officers and the  courses and services available to develop those competencies. However,  it does not include data on SEC’s current workforce and attrition  projections for SEC leaders, which are important for determining current  and future workforce needs. In addition, the succession plan does not  identify a fair and accurate process for identifying and selecting leaders,  which may prevent the process from being transparent to employees.  Developing a clearer process for selecting leaders could help to address  employee concerns related to the promotion process. For example, only  15 percent of the nonsupervisory staff who responded to our 2016 survey  agreed that the criteria for promoting staff are clearly defined, a modest  improvement from our 2013 survey but still a relatively small percentage  (see fig. 9).
Since our 2013 report, SEC has provided us with various documents and  plans to demonstrate their response to our recommendations. However,  as we previously discussed, SEC’s recently developed workforce plan  lacks a comprehensive skills gap analysis plan, does not inform decision  making about the structure of the workforce, and is not clearly linked to its  budget formulation. As a result, SEC has not fully addressed the  recommendations from our July 2013 report related to workforce  planning, and we maintain that these 2013 recommendations are still  valid.
section_title: Performance Management System
paragraphs: In 2014, SEC decided to redesign its performance management system  without formally assessing it, which is inconsistent with our previous  recommendation that SEC periodically validate the system in order to  enhance its credibility. In our July 2013 report, we found that the design of  SEC’s performance management system reflected many elements of  OPM guidance, but SEC staff expressed concerns about implementation  of the system. Consistent with best practices, we recommended that  SEC’s COO and Office of Human Resources conduct periodic validations  (with staff input) of the performance management system and make  changes, as appropriate, based on these validations. At the time of this  review, SEC had not conducted periodic validations of its performance  management system as we recommended—nor are any planned,  according to SEC staff—and therefore the recommendation is still  unaddressed.
While SEC’s policies state that the Office of Human Resources is to  perform an assessment of the system on an annual basis, Office of  Human Resources officials told us that SEC has not conducted a formal  assessment of the performance management system because the  agency is in the process of developing a new system. Office of Human  Resources officials stated that they decided to develop a new  performance management system in 2014 due to continued criticism of  the current system by the SEC employees’ union.
In developing its new performance management system, SEC did not  follow best practices that we and OPM have identified. For example,  OPM’s HCAAF states that agencies should base their human capital  management decisions (including those related to changes to the  performance management system) on the results of data and planning.  Additionally, key practices for effective performance management we  have identified call for agencies to involve employees and other  stakeholders when they design and periodically evaluate their  performance management systems. However, since our 2013 report,  SEC has not reviewed the effectiveness of its existing system and has  had limited stakeholder involvement in the development of the new  performance management system. SEC management did not assess the  existing system to understand if the issues raised by employees were  related to the system’s design or its implementation. As a result, SEC  lacked information on if and what changes needed to be made and how  best to make them. Instead, SEC developed a new performance  management system with some limited consultations with the union in  2015 and conducted a pilot of the new system with non-bargaining staff in  May 2016.
We maintain that our prior recommendation should be implemented and  that SEC should conduct periodic validations of any performance  management system it has in place by, for example, obtaining staff input  and general agreement on the competencies, rating procedures, and  other aspects of the system. Only then should SEC make changes, as  appropriate, based on these validations. Without evaluating its  performance management system to identify problems and potential  solutions, SEC may not have assurance that the new system will perform  better than the current system. Furthermore, SEC’s planned changes to  its performance management system will require additional resources that  could be targeted toward its other personnel management challenges.
section_title: Cross-Divisional Collaboration and Communication
paragraphs: SEC has not addressed our previous recommendations targeted at  improving collaboration and communication across SEC. While SEC has  created some incentives to support communication and collaboration  across divisions, barriers related to cross-divisional communication and  collaboration still remain. In our July 2013 report, we found that SEC  faced barriers to communication and collaboration, especially among the  various divisions and offices. We recommended that the SEC COO (1)  identify and implement incentives for all staff to support an environment of  open communication and collaboration and (2) explore communication  and collaboration best practices and implement those that could benefit  SEC. SEC has not yet addressed these recommendations.
Since our 2013 report, SEC has demonstrated some improvement in  communication and collaboration within divisions and offices. For  example, in group interviews, supervisors from five of the six largest  divisions and offices at SEC agreed that there is sufficient communication  and collaboration within their division. Furthermore, our 2016 survey  results showed some improvements related to communication and  collaboration. For example, 44 percent of nonsupervisory staff agreed  that information is adequately shared across groups in their division or  office, compared to 34 percent in our 2013 survey (see fig. 10).
SEC has also implemented some incentives and procedures for staff to  communicate and collaborate. For example, SEC’s annual agency-wide  awards program includes awards that recognize outstanding teams,  including cross-divisional teams. SEC has also implemented tools and  procedures to facilitate collaboration. For example, SEC developed a  tracking system that facilitates collaboration on interdivisional  memorandums, and the Division of Economic and Risk Analysis  developed an electronic system that allows other divisions to request data  it collects. In addition, the Division of Enforcement created formal liaisons  that other divisions and offices can contact. Managers in four of the five  largest divisions and offices told us that these liaisons help to facilitate  cross-divisional communication and collaboration with the Division of  Enforcement.
However, incentives for staff to support an environment of open  communication and collaboration are not present for all staff across SEC.  According to OPM guidance, supervisors and managers should foster an  environment of communication and collaboration. SEC has added  performance expectations for 53 percent of supervisors to encourage  communication and collaboration, including intra-agency communication  and collaboration. For example, the Office of Compliance Inspections and  Examinations sets expectations for its Assistant Directors (SK-17 level) to  “promote and maintain an environment of cooperation and create a high  level of team cohesion by empowering all staff” and “work with other  Program areas and Offices, especially by pro-actively sharing relevant  information.” In addition, the Division of Corporation Finance sets  expectations for its accountants to “engage in appropriate internal and  external communications to resolve issues” and to “provide relevant  technical information and work-related knowledge, skills, and lessons  learned within and/or beyond the work unit.” However, we found that  these expectations were not present for 47 percent of all supervisors  across divisions and occupations. As a result, SEC has not fully  addressed our 2013 recommendation to identify and implement  incentives for all staff to support an environment of open communication  and collaboration.
In addition, SEC has not demonstrated the use of best practices to  improve communication and collaboration within and across SEC  divisions and offices. We have previously identified best practices related  to collaboration, including supervisors fostering an environment of open  communication, promoting frequent communication among collaborating  divisions, and establishing compatible policies and procedures to operate  across agency boundaries. When we asked officials from the COO’s  office whether they had researched best practices for improving  communication and collaboration across SEC, they provided two  examples. First, SEC officials told us that they reached out to officials at  the Federal Deposit Insurance Corporation (FDIC) to discuss how FDIC  had obtained high survey scores related to communication and  collaboration. This outreach resulted in the creation of SEC’s “All  Invested” initiative, which SEC described as an initiative to encourage  collaboration and community to help the agency achieve its mission and  make SEC the best place in government to work. Second, SEC officials  mentioned a best practice in which they launched a “values campaign” to  promote important values, including teamwork, as a part of the “All  Invested” initiative. However, many of the supervisors and staff we spoke  with told us that the “All Invested” initiative was more of a marketing  campaign than a substantive change. SEC has also established a  number of working groups to improve communication and collaboration,  but these working groups are often focused on specific topics and do not  provide a means for divisions and offices to collaborate on the full range  of their day-to-day work activities. As a result of SEC’s limited action, we  maintain that SEC has not taken sufficient steps to fully address our 2013  recommendation to explore and implement best practices to improve  communication and collaboration within and across SEC divisions and  offices.
Of the seven recommendations that we made in 2013, SEC has made the  least progress on the recommendations related to enhancing intra-agency  communication and collaboration. One reason for this may be that, other  than the Office of the Chair, there is no senior-level office or official that  has authority over the daily operations of all SEC divisions and offices  (see previous fig. 1). For example, the COO is responsible for approving  budget requests, staffing levels, and reorganization requests for the SEC  as a whole. However, each mission-critical office and division has its  own director that is responsible for policies and programs that affect the  operations of each individual office and division. For example, the  Director of Enforcement and his staff facilitate communications with other  divisions and offices to conduct investigations and coordinate on policy or  legislative briefings. According to the March 2011 Boston Consulting  Group report, the function of the SEC COO has historically focused on  the annual congressional appropriation cycle, internal budgeting process,  and administrative duties. Based on a number of interviews with  relevant staff in the Office of Human Resources, we found that this  structure and the limited authority of the COO may help to explain in part  the inability of the COO to explore and implement best practices that  could affect the daily operations throughout SEC. For example, these  staff told us that the divisions and offices play the key roles in exploring  and implementing best practices that could affect daily operations  throughout SEC, not the COO.
Key principles we have identified for organizational transformation call for  agencies to create a position such as a COO or Chief Management  Officer with authority over all operations of the agency; such a position is  one approach to help agencies address long-standing management  challenges. For example, there needs to be a single point within the  agency with the responsibility and authority to ensure successful  implementation of functional management and transformational change  efforts. Further, it is not practical to expect an official like the Chair’s Chief  of Staff to undertake this vital responsibility due to competing demands on  their time in helping to execute the Chair’s policy and program agendas.
The lack of a central position or office with authority over the daily  operations of all SEC divisions and offices makes it difficult to lead SEC- wide changes to address long-standing management challenges related  to communication and collaboration. Because of the COO’s limited  authority and the absence of another SEC official, other than the SEC  Chair, with the authority over the divisions and offices to take action to  facilitate efforts to improve cross-divisional communication and  collaboration, progress in this area will likely continue to be limited. An  environment of limited intra-agency communication may continue to  increase the risk of inefficiencies and less-than-optimal decision making,  which may affect SEC’s ability to achieve its mission, as was the case  with SEC’s actions related to the Bernard Madoff ponzi scheme and other  enforcement failures.
section_title: Hiring and Promotions
paragraphs: SEC’s hiring and promotion policies and procedures are generally  consistent with OPM and other relevant criteria, but SEC lacks assurance  that staff, particularly hiring specialists, know how to implement the  policies and procedures correctly. OPM’s HCAAF specifies, among other  things, that agencies’ hiring processes should help ensure that positions  are developed and validated by appropriate staff, that position  descriptions are established, and that appropriate assessment tools (e.g.,  processes for comparing application packages to qualifications and  conducting panel interviews) are developed prior to initiating a hiring  request. In addition, agencies should follow merit system principles and  must observe prohibited personnel practices to ensure a fair process and  may have to follow veterans’ preference requirements. Key principles  for hiring that we have identified call for agencies to use vacancy  announcements that are clear, user friendly, and comprehensive.  Finally, federal internal control standards note that agencies should have  procedures to determine whether a particular candidate fits the  organizational needs and has the competence for the proposed role.
Consistent with these criteria, SEC’s hiring and promotion policies and  procedures require hiring specialists from the Office of Human Resources  and hiring managers from the divisions and offices to follow specific steps  and document these steps in recruitment case files. These steps include  the following: documenting consultations between SEC hiring specialists and  division hiring managers over the position to be filled by a hiring or  promotion action;  including descriptions of the position, job analysis, and the vacancy  announcement in the case file for the position; documenting the review of applications to help ensure they meet  issuing a certificate of eligibles, which lists all the applicants who are  determined to be best qualified for the position posted; providing evidence of whether hiring managers reviewed each  certificate of eligibles within established time frames and made a  selection from the list of certified eligibles; and obtaining signed offer letters and supporting documentation, including  starting salary.
In addition, in 2015, SEC finalized standard operating procedures related  to the review and maintenance of case files. These procedures require  hiring specialists to complete checklists at various stages in the hiring and  promotion process to help ensure that documents are uploaded  appropriately. Supervisors are responsible for reviewing and approving  the checklists before moving to the next stage in the process. In addition,  they are also responsible for conducting periodic compliance reviews to  ensure adherence to these procedures.
However, although these policies and procedures meet relevant criteria,  we found a large number of deficiencies when we tested the policies’  implementation. We reviewed a random sample of cases files from fiscal  years 2013 through 2015 to determine if SEC was following its hiring and  promotion policies and procedures. Based on our analysis of this  sample, we estimate that for 94 percent of the case files for fiscal years  2013 through 2015, SEC staff did not consistently follow at least one  policy or procedure for hiring and promotion actions, including the  following examples:
Documentation missing: Based on our analysis of the sample, we  estimate that 16 percent of case files during these fiscal years had no  evidence that applicants were reviewed to ensure they met the  minimum job requirements. Further, we estimate that 16 percent of  the case files had no certificate of eligibles, which makes it difficult to  determine how officials selected the best qualified applicants. We  also found deficiencies once candidates were selected for the  position. For example, for 23 job offers made, we found no documents  that showed how the initial salary was determined.
Supervisory approvals missing: Based on our analysis of the  sample, we estimate that 18 percent of case files for fiscal years 2013  through 2015 contained documents describing the consultation  between the hiring specialist and the hiring manager about the  position that were not completed or signed. As a result, determining  if the hiring specialist and hiring manager reviewed and developed the  documentation that is meant to support and defend the hiring decision  is difficult. In addition, we estimate that 16 percent of case files had  certificates of eligibles that were not signed by the responsible  officials. The selecting officials are responsible for returning the  certificate with their selection indicated, and their signature serves to  assure the Office of Human Resources that they have provided their  approval to extend an offer of employment, according to SEC staff.
Time frames not observed: Based on our analysis of the sample, we  estimate that 29 percent of case files for fiscal years 2013 through  2015 had certificates of eligibles that were not returned on time, nor  was there documentation on why they were not returned on time.  This is particularly important because some SEC employees told us  that SEC cannot always hire the most qualified people due to slow  processing times. In addition, for 20 job offers, the offer letter was  sent before the initial salary determination was made, which is against  SEC policy.
SEC has also conducted reviews of its case files and identified  deficiencies similar to those we found during our review. As discussed  previously, as part of SEC’s implementation of its human capital  accountability system, SEC has implemented an internal quality control  process to help ensure that case files are accurate and complete, but this  process occurs after hiring and promotion decisions are made. The  purpose of this quality control process is to identify common deficiencies  in case files in order to improve the hiring and promotion process. SEC  has conducted four quarterly reviews of case files since September 2014  and continues to identify weaknesses similar to those we found, such as  missing evidence of ranking of job candidates and missing documentation  to support the initial salary of the candidate. SEC categorized the  deficiencies into five levels of severity and found that the frequency of the  minor (e.g., missing checklists) and moderate (e.g., missing descriptions  of the position) deficiencies has slowly decreased over time, with minor  deficiencies decreasing from 48 percent of case files reviewed during its  September 2015 quarterly review to 41 percent in the most recent March  2016 review. Likewise, according to SEC, the moderate deficiencies  decreased from 16 percent to 9 percent of case files reviewed over this  same time period. However, the frequency of significant (e.g., missing  reviews of whether applicants meet minimum qualifications) and major  (e.g., missing audit of certificate of eligibles) deficiencies had slowly  increased over time. Significant deficiencies have increased from 24  percent of case files reviewed during the September 2015 quarterly  review to 33 percent in the most recent March 2016 review. Likewise, the  major deficiencies increased from 12 percent to 17 percent of case files  reviewed over the same time period. SEC had no critical deficiencies  (e.g., violation of veterans’ preference) over this period.
OPM, the National Credit Union Administration (NCUA), and an outside  consultant also reviewed SEC’s hiring and promotion practices and  identified similar deficiencies in a sample of case files they reviewed,  including the following examples.
OPM found that although SEC had various options for staff to  document their rationale for deeming an applicant who did not meet  minimum qualifications as “best qualified,” in some cases, staff did not  provide sufficient documentation. As a result, OPM had difficulty  reconstructing some minimum qualifications to assess whether a  candidate met the minimum qualifications for the job posting. However  OPM did find that the qualifications were accurate.
NCUA found that applicants were not consistently notified of the  status of their application at key stages. OPM has noted that it is a  good practice to keep applicants notified of their status.
An outside consultant that reviewed SEC’s internal promotion actions  from fiscal years 2011 through 2014 found, for example, that SEC’s  lack of full adherence to uniform personnel practices and guidelines  appeared to result in a loss of promotional opportunities and unequal  treatment in the selection stage for certain groups. Specifically, the  consultant found that disparities existed in the way human resources  specialists processed promotion actions and that they failed to apply  processes and procedures established by OPM to promote fair and  equal opportunities for all employees. These practices included  restricting vacancy announcements to specific offices within SEC and  early closing of vacancy announcements. As a result of these  practices, the consultant reported that well-qualified applicants who  perform similar functions in another area of SEC may not be selected  or applicants may not have sufficient time to apply.
SEC has taken a number of actions to address deficiencies identified by  OPM, NCUA, and the consultant. For example, in December 2015 during  our review, SEC began to mandate that all case file documents be  uploaded to an electronic case file. Previously, SEC case files were  maintained in a paper format. According to SEC staff, the electronic case  files allow for easier access and monitoring than paper files, allow for  controls over what documents are stored in the case files, and avoid  problems with documents being misplaced or lost. To address issues that  NCUA found, SEC now notifies applicants at key stages of the application  process. SEC’s actions in response to recommendations in the  consultant’s report include providing general training to all staff on how  SEC promotes staff, publicizing the promotion process to all staff, and  providing resources to staff interested in promotions, including guidance  regarding writing resumes and preparing for interviews.
SEC has recently taken some steps to improve its hiring and promotion  practices, which may help to address the types of errors that we found in  our review of files from fiscal years 2013 through 2015. As discussed  previously, we found that hiring specialists failed to include the required  documentation for 94 percent of case files we reviewed. The presence of  errors in a large percentage of case files indicates that Office of Human  Resource supervisors, who are responsible for overseeing the work of the  hiring specialists, did not identify and resolve these issues as required by  SEC policy. According to federal internal control standards, managers  should perform ongoing monitoring as part of the normal course of  operations. Ongoing monitoring includes regular supervisory activities  and reconciliations, and may include automated tools, which can increase  objectivity and efficiency. SEC officials told us that the electronic case  files they began to create in December 2015, during the course of our  review, should allow them to more easily monitor and audit whether all  documentation is complete and properly uploaded at every stage of the  hiring and promotion process, and these actions are consistent with  federal internal control standards. Few completed electronic case files  were available during our review, and therefore it is too early to evaluate  the effectiveness of the new process. However, SEC’s steps may help to  ensure that staff adhere to policies and procedures and may help to  address the types of errors that we found in our file review.
We also found issues related to the training of SEC’s hiring specialists,  which may be another factor contributing to the high rate of errors in the  case files we reviewed. When we spoke to SEC hiring managers, they  expressed some concern over the competence of hiring specialists in the  Office of Human Resources. Hiring specialists also told us that they  received only limited training. Based on our review of training offered by  SEC University, we did not find any specific training on hiring and  promotions targeted at hiring specialists. Further, SEC’s 2015 training  plan for the Office of Human Resources has a course on adjudicating  veterans preference, but no courses are specifically targeted at how to  implement each stage of the hiring and promotion process. A key  principle of an effective control environment states that managers should  demonstrate a commitment to recruit, develop, and retain competent  individuals. For example, managers should establish expectations of  competence for key roles to help the entity achieve its objectives. This  competence is gained largely from professional experience, training, and  certifications. In addition, as previously discussed, we found that SEC has  not assessed whether some of its mission-support staff, including key  hiring specialists, have the necessary skills to conduct their work. OPM  has stated that agencies should conduct a learning needs analysis to  identify skills gaps across their entire workforce, and we found that SEC  has yet to fully address our 2013 recommendation to conduct such an  analysis. Rather, SEC only conducted a skills gap analysis of staff in the  mission-critical office and divisions and in the Office of Information  Technology. Without providing necessary training that is informed by a  comprehensive skills gap analysis, SEC may lack assurance that hiring  specialists have the skills required to conduct their work effectively, and  that it is hiring and promoting the most qualified applicants.
section_title: Conclusions
paragraphs: A high-performing workforce is critical to SEC effectively carrying out its  regulatory responsibilities in increasingly complex markets. While SEC  has taken steps to address our 2013 recommendations, its progress has  been limited, and five of the seven recommendations are not fully  addressed. We maintain that these five recommendations—in the areas  of workforce and succession planning, the performance management  system, and communication and collaboration—should be addressed in  order for SEC to fulfill its mission effectively.
One of the most protracted personnel management challenges at SEC  remains communication and collaboration, and SEC’s limited progress in  addressing our 2013 recommendations in this area points to a lack of  leadership in breaking down silos that prevent divisions and offices from  working more efficiently and effectively with each other. Apart from the  office of the Chair, which has broader responsibilities both within and  outside the agency, heads of each division and office are responsible for  their daily operations and are not accountable to any senior-level official,  such as the Chief Operating Officer. Our prior work has found that having  a senior-level position within the agency that has the responsibility and  authority to ensure that changes are implemented can help address  protracted personnel management challenges such as communication  and collaboration.
Finally, our review found that SEC’s training related to hiring and  promotion practices may be inadequate. According to federal internal  control standards, managers should demonstrate a commitment to  recruit, develop, and retain competent individuals, which depends in part  on adequate training. Because SEC has not conducted a skills gap  analysis across its entire workforce as we previously recommended in  2013, including its hiring personnel, it lacks the information needed to  develop an effective training program. Training for hiring specialists that is  informed by a comprehensive skills gap analysis should better enable  SEC management to hire and promote the most qualified applicants.
section_title: Recommendations for Executive Action
paragraphs: To help SEC address identified personnel management challenges, the  Chair should take the following two actions:
Enhance or expand the responsibilities and authority of the COO or  other official or office so they can help ensure that improvements to  communication and collaboration across SEC are made. For instance,  if the duties of the COO were expanded, the COO could establish  liaisons in each mission-critical office and division for SEC employees  to contact or develop procedures to help facilitate communication and  collaboration among the mission-critical office and divisions.
Develop and implement training for hiring specialists that is informed  by a skills gap analysis.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided SEC a draft of this report for its review and comment. SEC  provided written comments that are reprinted in appendix IX. In its written  comments, SEC agreed with the majority of our findings and one of our  two recommendations, but it disagreed with the other.  In its letter, SEC  stated that it has made a number of improvements in its personnel  management since our 2013 report. The letter also highlighted that the  released rankings of the Best Places to Work in the Federal Government  show that SEC’s workforce is among the most engaged in the  Government, ranking now 6 out of 27 mid-sized agencies; SEC has  climbed nine places in the rankings since our 2013 report. According to  the letter, this improvement, among other indicators, illustrates the  agency's success in building, sustaining, and growing an organization that  fosters and values innovation, communication, collaboration, and  transparency. In its letter, SEC also acknowledged that further  improvements could be made, and it noted that our report contained  useful information to strengthen personnel management at the  Commission.
Related to personnel management, SEC acknowledged our second  recommendation in this report to develop and implement training for its  hiring specialists. Specifically, the letter stated that SEC University in  coordination with the Talent Acquisition Group in SEC’s Office of Human  Resources will prioritize its competency assessment of its human  resource specialists (which includes hiring specialists) and develop  training plans to address any skill gaps identified.  SEC also agreed that it  still needs to conduct periodic validations of its performance management  system. According to the letter, SEC worked with OPM recently to  validate the new system that it is piloting and that the Commission will  continue to work with OPM to conduct additional surveys of supervisors  and employees regarding the efficiency and effectiveness of SEC's  performance management program. SEC also stated in its letter that  improvements can be made to its workforce and succession planning. In  its letter, SEC stated that it had already begun the planning process to  conduct a competency skills gap analysis on the non-mission critical  workforce in fiscal year 2017, and will develop appropriate action plans to  address the skill gaps that are identified. In addition SEC stated that the  Office of Human Resources is in the process of enhancing its current  succession planning program, and it will work with all SEC division  directors and office heads to institute additional fair and transparent  processes for identifying high-potential employees and communicating to  them opportunities for leadership development.
SEC disagreed with our characterization of the current state of its intra- agency communication and collaboration.  In its letter, SEC stated that it  believes that significantly more progress has been made by the  Commission to resolve the recommendations from our 2013 report  (addressing interdivisional communication and collaboration) than our  report recognizes.  SEC also stated that interactions at both the staff and  senior leadership levels are continuous, and that it has instituted both  formal and informal mechanisms for additional coordination where it is  required, which have proven to be successful.  SEC also stated that  cross-divisional interaction may not be necessary, or even appropriate,  for some non-supervisory staff, and it noted concern with our reliance on  anecdotal accounts from one former employee.  We acknowledged the  improvement SEC has made, for example, by noting that the percentage  of non-supervisory and supervisory staff responding that information is  adequately shared across groups in their division or office increased from  2013 to 2016.  However, we found substantial evidence that siloed  communication remains a challenge at SEC. For instance, 78 of the 187  employees we interviewed (over 40 percent) cited issues around siloed  communication as an area where SEC needs to improve.  Additionally, of  the 1,947 written responses we received to our survey questions, 597 of  them cited various challenges related to communication and  collaboration. We provided examples from several current and one former  employee to illustrate the siloed communication at SEC.  Further, we  recognize that not all staff at SEC may need to coordinate and collaborate  for work-related issues. However, staff in mission-critical offices and  divisions should be enabled to collaborate and communicate with staff in  other offices and divisions. As acknowledged in our report, the Division of  Enforcement created formal liaisons that other divisions and offices can  contact, and these liaisons help to facilitate cross-divisional  communication and collaboration within the division. Based on our survey  results, staff in the Division of Enforcement more frequently interacted  with staff from other mission-critical offices and divisions.  As SEC  acknowledged in its response, the Division of Economic and Risk  Analysis is similar to the Division of Enforcement in that staff should be  routinely communicating and collaborating with them. However, unlike the  Division of Enforcement, the Division of Economic and Risk Analysis  lacks a mechanism to easily facilitate cross-divisional communication and  collaboration. Our survey results show that interaction between Division  of Economic and Risk Analysis staff and staff from other mission-critical  offices and divisions is limited.
SEC also disagreed with our recommendation related to enhancing the  responsibilities and authority of the COO or other official or office to help  ensure that improvements to communication and collaboration across  SEC are made.  In its letter, SEC stated that given the current legal and  management structure of SEC, as well as the requirements of its mission,  SEC does not believe that a position of that description would improve the  ability of SEC to discharge its obligations to protect investors. SEC also  stated that the agency’s structure and the authority of the Chief of Staff  and the Deputy Chiefs of Staff enables them, in close consultation with  the Chair, to effectively pursue changes to enhance coordination and  collaboration throughout SEC.  We are not suggesting that an additional  layer of management is needed to help improve the ability of SEC to  discharge its obligations to protect investors. Rather, we are  recommending that the authority of the COO or some other official be  enhanced in order to ensure that each mission-critical office and division  establish a mechanism or develop procedures to facilitate communication  and collaboration.  SEC provided no evidence to illustrate why relevant  best practices that GAO has identified can work in other federal agencies  that have varied structures cannot benefit the Commission. The best  practices we have identified call for institutionalized accountability for  addressing management issues and leading transformational change  because the management weaknesses in some agencies are deeply  entrenched and long-standing, and it can take at least 5 to 7 years of  sustained attention and continuity to fully implement transformations and  change management initiatives.  The typical tenure of an SEC Chair is  shorter than the time needed to affect such change. Since 2001, SEC has  had 6 Chairs and none of them have had a tenure that lasted 4 years.    SEC also noted that our conclusions would have been better informed  with the full perspectives of the agency’s Chief of Staff and Deputy Chiefs  of Staff.  We met with the Chief of Staff and Deputy Chief of Staff during  our review and they discussed efforts by SEC to address cross-divisional  communication and collaboration changes.  While we acknowledge  efforts SEC has made to improve collaboration and communication  across the agency, the evidence we present indicates that SEC should do  more to identify a single point of contact with the responsibility and  authority to ensure the successful implementation of a communication  and collaboration process.   As a result, we maintain that the  recommendation would help ensure such change.
We are sending copies of this report to the appropriate congressional  committees, the Chair of the Securities and Exchange Commission, and  other interested parties. In addition, the report is available at no charge on  the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at 202-512-8678 or clementsm@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix X.
section_title: Appendix I: Status of GAO’s 2013 Personnel Management Recommendations to the Securities and Exchange Commission
paragraphs: Table 2 provides the status of recommendations we made to the  Securities and Exchange Commission (SEC) in 2013. As the table  shows, five of the seven recommendations remain open, as of December  2016.
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: This report examines (1) employees’ views of the Securities and  Exchange Commission’s (SEC) organizational culture and personnel  management, and the extent to which these views have changed since  our 2013 report and (2) the extent to which selected SEC personnel  management practices have been implemented consistent with relevant  standards.
subsections: section_title: Analysis of Employees’ Views of SEC’s Organizational Culture and Personnel Management
paragraphs: To examine employees’ views of SEC’s organizational culture and the  extent to which they have changed since 2013, we conducted surveys of  all SEC staff, a content analysis of open-ended responses to our survey,  individual interviews with SEC staff, and structured group interviews with  SEC supervisors:  Surveys: To examine employees’ views of SEC’s organizational culture  and the extent to which these views have changed since 2013, we  implemented three web-based surveys of all 4,236 nonsupervisory and  supervisory staff and 148 senior officers. The three surveys were  administered to the following number of staff during the following time  periods:  1.  the survey to the mission-critical office and divisions was administered  to 2,627 staff from October 2015 to March 2016;  2.  the survey to all other offices and divisions was administered to 1,609  staff from May 2016 to September 2016; and  3.  the survey to all 148 senior officers was administered from April 2016  to July 2016.
The surveys were administered to the different groups at various  timeframes to, for example, allow for the maximum response rate given  the competing demands of SEC staff at different times of the year. We  chose to survey all staff at SEC instead of a sample to obtain information  from the largest feasible number of SEC employees. We analyzed the  results of our 2016 survey of all supervisory and nonsupervisory staff and  senior officers and also compared the results from 2016 surveys to the  mission-critical office and divisions and the senior officers to the results  from the 2013 surveys. In addition, we reviewed the Office of Personnel  Management’s (OPM) 2016 Federal Employee Viewpoint Survey (FEVS)  results to obtain additional perspectives from SEC staff on the agency’s  personnel-management-related issues.
Each GAO survey to SEC staff included questions on (1) personnel  management issues related to recruitment, training, staff development,  and resources; (2) communication between and within divisions and  offices; (3) leadership and management; (4) performance management  and promotions; and (5) organizational culture and climate. The separate  survey of all SEC senior officers (those at the SO-1, SO-2, and SO-3 pay  grades) covered the same topic areas but omitted questions not relevant  for senior officers and included additional questions specifically relevant  for senior officers. The survey to all other offices and divisions also  covered the same topic areas, but had some questions omitted, such as  the question related to the number of times they had interacted with the  mission-critical office and divisions over the past year. Our survey  included both multiple-choice and open-ended questions.
For our survey of the mission-critical office and divisions, 1,819  nonsupervisors and supervisors responded to our survey for a response  rate of 69 percent; for our survey of all other offices and divisions, 969  nonsupervisors and supervisors responded for a response rate of 60  percent. A total of 104 of the 148 senior officers responded to our survey  of all senior officers for a response rate of 70 percent. For all surveys,  except the one for senior officers, we carried out a statistical nonresponse  bias analysis using available administrative data and determined that we  could not assume that the nonrespondents were missing at random. For  this reason, the results of the staff survey are presented as tabulations  from a census survey. 31 percent of mission-critical employees and 40  percent of all other employees who chose not to complete our survey.
To minimize certain types of errors, commonly referred to as nonsampling  errors, and enhance data quality, we employed recognized survey design  practices in the development of the questionnaires and the collection,  processing, and analysis of the survey data. To develop our survey  questions, we drew on information from one-on-one interviews, focus  group sessions held during our 2013 review, and prior GAO SEC  personnel management surveys. For the surveys of the mission-critical  office and divisions and senior officers, we took steps to ensure our  survey questions from 2013 were still relevant and to determine if new  issues warranted new questions. To do this, we sent draft survey  questions to SEC officials in the mission-critical office and divisions and  senior officers who volunteered to review our draft questions to obtain  their feedback on the survey questions. For the survey of the other offices  and divisions within SEC, we pretested the questionnaire with SEC  employees to validate the survey questionnaire and to ensure that we  were not omitting relevant questions from the survey. We met with six  SEC staff for the pretest of the survey. During survey development, we  reviewed the survey to ensure the ordering of survey sections was  appropriate and that questions in each section were clearly stated and  easy to comprehend. A GAO survey expert reviewed and provided  feedback on our survey instrument. To reduce nonresponse, another  source of nonsampling error, we undertook an intensive follow-up effort  that included multiple e-mail reminders to encourage SEC employees to  complete the questionnaire and a series of phone call reminders to  nonrespondents to encourage participation and to troubleshoot any  potential logistical issues with access to the questionnaire. We minimized  processing errors by having a second independent data analyst conduct  an accuracy check of the computer programs used for data analysis. We  also had respondents complete questionnaires online to eliminate errors  associated with manual data entry. On the basis of our application of  these practices and follow-up procedures, we determined that the data  were of sufficient quality for our purposes.
Content analysis: To analyze the information we obtained from the  open-ended survey responses, we conducted a content analysis on the  1,947 responses to the six open-ended survey questions from the survey  of the mission-critical office and divisions using a text analytics tool. Two  analysts developed coding categories based on requirements in section  962 of the Dodd-Frank Wall Street Reform and Consumer Protection Act  (Dodd-Frank Act), our researchable objectives, and information collected  during individual interviews, as well as findings from our July 2013  report. Coding categories were as follows: (1) communication and  collaboration, (2) hierarchy, (3) risk aversion, (4) atmosphere of trust, (5)  morale, (6) performance management, (7) training and hiring, and (8)  awards.
The team provided a “lexicon” of key words and phrases associated with  each of the eight coding categories to the text analytics subject matter  expert. For example, the key words and phrases associated with  communication and collaboration included “communication,”  “transparency,” and “working together.” For each of the six open-ended  questions, the subject matter expert developed a computer program using  the lexicon and provided the team with the categorized open-ended  responses. The goal of this analysis was to determine the number of  respondents who mentioned at least one challenge in the respective  coding category. After obtaining the categorized open-ended responses,  two GAO analysts collaboratively reviewed the output and revised or  “calibrated” the lexicon based on each result. This review involved  verifying the coding of a judgmental sample of responses. The subject  matter expert then reran the program with the updated lexicon. This  iterative process allowed the subject matter expert to refine the program  to isolate comments focused on challenges associated with the coding  categories. While human correction and evaluation of the content can  help improve the quality of the machine generated coding, a certain,  undetermined error remains. To minimize error and improve accuracy, the  calibration process continued for three iterations.
Comments that were flagged by the text analytics tool as capturing a  challenge within a particular coding category were coded with a “1.” Once  coding was completed and the discrepancies were resolved, an analyst  tallied the total number of responses with a “1” in each of the coding  categories. This number indicated the proportion of respondents who  expressed concern in their open-ended responses for each category. This  process was repeated for each of the eight challenge categories. We do  not make any attempt to extrapolate the findings to the eligible staff who  chose not to complete our surveys.
Individual interviews: We interviewed or obtained written responses  from 185 employees (144 nonsupervisory staff, 17 supervisory staff, and  24 staff who chose not to disclose their supervisory status)—in person for  those at SEC headquarters and by telephone or e-mail for those in  headquarters and regional offices—during 2 weeks in September 2015  and 1 week in February 2016 to obtain their views on personnel  management challenges at the agency. We coordinated with SEC to send  a broadcast message over its internal system to all staff to invite them to  meet with us either in person or to call a GAO toll-free number or use a  GAO e-mail account to provide their views on SEC’s organizational  culture. At headquarters, we established office hours during which  employees could speak with GAO analysts. During the same period, we  set up a GAO toll-free phone number and e-mail account to communicate  with employees in the regional offices or headquarters who could not  attend the office hours. We asked certain key questions of every person  we interviewed and interjected additional questions as appropriate. We  also explained that our review was initiated due to a provision in section  962 of the Dodd-Frank Act, and provided a description of this section  when asked by SEC employees. We then asked them about (1) what  personnel management practices were working well, (2) what challenges  existed in personnel management, (3) what initiatives, if any, SEC had  taken to address these challenges, and (4) whether these individuals had  any recommendations to address such challenges. For those staff who  were not familiar with what areas encompassed personnel management,  we presented them with a list of areas for them to think about. Employees  were encouraged to talk openly and freely. To maintain the confidentiality  of individual responses, we did not record individual names in our  transcripts. Instead, we collected and analyzed the information by division  and rank only, and we aggregated our findings so that no individual  comments could be identified.
We conducted a separate analysis to summarize key themes that  emerged from these individual interviews. GAO analysts independently  reviewed notes taken from these interviews and made a judgment about  appropriate codes that described the themes. The analysts compared  their decisions and reconciled any disagreements, resulting in the  following set of coding categories: (1) staff competence; (2) appropriate  levels of supervisory and non-supervisory staff; (3) effectiveness of  supervisors; (4) promoting staff criteria; (5) siloed communication issues;  and (6) other cross-cutting challenges. Once the coding structure was  finalized, one GAO analyst separately reviewed and coded each  response and tabulated the frequency of statements expressing certain  themes, while a second analyst verified the information to ensure the  tabulation was correct and that the analyst concurred with the results.  Where there were discrepancies, a third analyst was asked to review the  statements and to make a final determination about how a specific  statement would be coded.
We also interviewed former SEC employees, officials from the SEC Office  of Inspector General (IG), and self-selected representatives and members  of the SEC employees’ union. We selected a nonprobability sample of  former employees to interview that reflected the diversity of former  employees in terms of pay grade, occupational category, and tenure,  among other factors. The results of these interviews with SEC  employees, former SEC employees, and members and representatives of  the SEC employees’ union are not generalizable, but provide views and  experiences.
Structured group interviews: We also conducted structured group  interviews with supervisors from the mission-critical office and divisions at  SEC, including supervisors in regional offices. The purpose of these  structured group interviews was to obtain their views on the following  personnel management practices related to: hiring, promotions,  supervision, strong and unacceptable performance, training, and  communication and collaboration. Our universe of supervisors consisted  of staff at the SK-15 or SK-17 level, with the exception of staff at the SK- 15 level in the Division of Enforcement, who are not considered  management. We obtained the e-mail addresses of all supervisors in  these divisions and sent them an invitation to participate in our structured  group interviews. We held eight meetings with the supervisors who  agreed to participate in our structured group interviews. We held a  meeting with each of the mission-critical office and divisions on  communication and collaboration. At two of these meetings, we had both  SK-15s and SK-17s, including staff from regional offices. In addition, we  had two meetings that covered the noncommunication topics. One of the  meetings consisted of SK-15s and the other consisted of SK-17s from  each of the mission-critical office and divisions. These meetings also  included regional office staff. The results of these meetings are not  generalizable, but provide views on selected personnel management  practices.
section_title: Analysis of SEC Personnel Management Practices
paragraphs: To determine the extent to which selected SEC personnel management  practices have been implemented consistent with relevant standards, we  first assessed whether these practices were designed consistent with  OPM’s Human Capital Assessment and Accountability Framework  (HCAAF), key human capital practices we have identified in prior work,  and federal internal control standards. As part of this effort, we reviewed  actions SEC had taken to address the seven recommendations from our  2013 report related to four personnel management areas: (1) workforce  and succession planning, (2) performance management, including  performance feedback, (3) communication and collaboration, and (4)  human capital accountability. In addition to these four personnel  management areas, we also assessed the design and implementation of  four personnel management practices based on information we obtained  from SEC staff. Specifically, they identified these as potential areas of  concern (or areas of improvement) during individual interviews with us as  well as in responses to our surveys—actions to recognize and reward  performance, actions to address unacceptable performance, hiring and  promotions, and training.
Workforce and succession planning: To determine whether SEC’s  workforce and succession planning practices were designed consistent  with relevant criteria, we obtained a copy of SEC’s 2016 strategic  workforce plan. We reviewed this strategic plan and compared it to  OPM’s HCAAF and our prior work on strategic workforce planning. For  example, we reviewed the skills gap analysis from the plan and compared  it to OPM’s HCAAF related to identifying critical skills for an agency’s  workforce. As part of this effort, we worked with our Director of Workforce  Planning to review SEC’s strategic workforce plan and determine whether  the plan was sufficient to address our 2013 recommendations. In  addition, we met with SEC’s Workforce Policy Group, Office of Human  Resources, and senior leaders from the divisions to obtain information on  what actions they had taken to address our recommendations related to  workforce and succession planning.
Performance feedback: To determine what steps SEC had taken to  address our 2013 recommendation, in conjunction with staff from our  Human Capital Office, we examined SEC’s process for monitoring  feedback as well as our survey results that relate to the performance  feedback employees receive. We obtained information on SEC’s process  for monitoring feedback by reviewing documentation and interviewing  staff from the Office of Human Resources. As part of this effort, we  obtained a description of their audit of a sample of performance work  plans.
Performance management: In our prior review (GAO-13-621), we  determined that the design of SEC’s performance management system  reflected many elements of OPM guidance, but SEC staff expressed  concerns about implementation of the system. However, since our prior  review, SEC decided to make changes to its performance management  system. Therefore, in this review, we set out to determine the extent to  which SEC’s changes to its performance management system were  consistent with relevant standards. To do this, we met with SEC staff from  the Office of Human Resources and union officials to inquire about what  actions SEC had taken to address our 2013 recommendations. Upon  learning that SEC decided to redesign its performance management  system in 2014, we assessed the actions SEC took to redesign its system  and compared these actions to OPM’s HCAAF, our previous work on  performance management systems, and federal internal control  standards. We also worked with staff from our Human Capital Office to  assess the actions SEC had taken to redesign its system against the  relevant criteria.
Communication and collaboration: To determine whether SEC’s  communication and collaboration practices were designed consistent with  relevant criteria, we assessed SEC’s actions to address our 2013  recommendations against OPM’s HCAAF, our previous work, and federal  internal control standards. Specifically, we reviewed SEC’s award  program to determine if the program provided incentives to support an  environment of open communication and collaboration, including  determining whether SEC provided awards for teamwork. In addition, we  reviewed 79 performance expectations, known as performance work  plans; all 28 supervisory performance work plans (including 5 plans under  the pilot performance management system), and 51 nonsupervisory performance work plans for the mission-critical office and divisions. For  the nonsupervisors, we selected the performance work plans for the  “mission-critical” staff we identified in our 2013 review—accountants,  attorneys, economists, examiners, and financial analysts at the SK-12  through SK-16 levels. The results of our review of performance work  plans are not generalizable, but provide us with information on how  communication and collaboration are addressed as part of an employee’s  performance expectations. To determine whether the performance work  plans contained expectations that addressed our 2013 recommendation,  one GAO analyst read through each performance work plan and  determined whether it contained all four of the following categories that  were derived from the recommendation:(1) communication, (2)  collaboration, (3) intra-agency communication and collaboration, and (4)  fostering an environment of open communication and collaboration—for  example, encouraging acceptance, exchanging of information, and  sharing of diverse points of view regardless of individual differences. A  second GAO analyst independently conducted a similar analysis of each  performance work plan. A third GAO analyst reviewed the results of both  sets of reviews to reconcile any differences. In cases where the reviews  differed, the third GAO analyst reviewed each analyst’s assessment and  made a judgment on which one was correct. In addition, we met with SEC  staff in the Office of Human Resources and senior officials from the  divisions to discuss what actions they had taken to address our 2013  recommendations.
Human capital accountability: To determine whether SEC’s human  capital accountability system was designed consistent with relevant  criteria, we obtained and reviewed SEC documentation on its recently  developed human capital accountability system, including policies  establishing the system, standard operating procedures, and evidence of  the system’s implementation, such as a review of SEC’s student loan  repayment program. We compared SEC’s documentation on its human  capital accountability system to OPM’s HCAAF and federal internal  control standards to determine whether this system addresses the  recommendation from our 2013 review. For example, we reviewed  SEC’s schedule of human capital program reviews, the reports generated  from reviews of specific human capital programs, and actions taken by  SEC to address any identified weaknesses in internal controls and  compared them to OPM’s HCAAF related to human capital accountability.  We also met with staff from SEC’s Human Capital Strategy Group in the  Office of Human Resources to determine what criteria they used to  determine which human capital programs to review.
Actions to recognize and reward performance: To determine whether  SEC’s policies on awards have been designed consistent with relevant  criteria, we compared these policies to OPM’s HCAAF and our prior work  on linking individual performance to organizational success. Specifically,  we reviewed the following policies and documents related to awards:
SEC’s administrative regulations that govern its recognition programs
SEC’s collective bargaining agreement; and
SEC’s guidance for cash and time-off awards.
We worked with staff from our Human Capital Office to assess these  policies against OPM’s HCAAF and our prior work, and determined that  they were consistent with relevant criteria. For example, we reviewed  SEC’s processes for monitoring how supervisors recognize and reward  performance and compared them to OPM’s HCAAF related to awarding  staff.
We next determined whether SEC’s practices related to awards were  being implemented consistent with the agency’s policies. To do this, we  obtained a list of all incentive awards (cash and time-off) for fiscal years  2013 through 2015. We randomly selected 96 award packages from fiscal  years 2013 through 2015. Of these 6 were from fiscal year 2012 and out  of the scope of our review, 19 were performance-based compensation  amounts, which were out of the scope of our review, and 1 was later  found by SEC to be out of our scope, thus 71 were analyzed. Because  the database we used to draw our sample contained data that was out of  the scope of our review (such as some packages from fiscal year 2012  and others that were not incentive awards), we did not attempt to  generalize the results of the sample. We were, however, able to use the  71 award packages to assess SEC’s implementation of its awards  program. Each package should have contained the following information: written justification for the award;  the dollar amount or hours off for the award; evidence of approval (i.e., signatures) by the recommending official,  reviewing official (both usually from the divisions or office that  nominated the person for the award), and the office of human  resources staff in the awards program area; evidence of the Chief Operating Officer’s approval for certain high  dollar amount or time-off awards; and a copy of the award recipient’s official personnel record (SF-50) with  the correct dollar amount or time-off hours noted.
We then had one GAO analyst review each of the 71 award packages  and complete a checklist noting whether the package contained the  information previously mentioned. A second GAO analyst then reviewed  each of the 71 packages and reviewed how the first analyst coded the  checklist. If the second analyst did not agree with the coding of the first  analyst, that information was noted and both analysts met to discuss any  disagreements. During this meeting, the two analysts were able to  discuss their disagreements and reach consensus on the proper coding  of the award packages. We also analyzed the award data for fiscal years  2013 through 2015, which we present in appendix IV. In addition, we met  with staff from SEC’s Office of Human Resources who are responsible for  the awards program to obtain a better understanding of SEC’s awards  program.
Actions to address unacceptable performance: To determine whether  SEC’s policies to address unacceptable performance have been  designed consistent with relevant criteria, we compared these policies to  OPM’s HCAAF and our prior work on linking individual performance to  organizational success. Specifically, we reviewed the following policies  and documents related to addressing unacceptable performance:
SEC’s overview of its employee misconduct and nonperformance  support program,
SEC’s collective bargaining agreement,  the performance management standard operating procedures for non- bargaining-unit employees, and
SEC’s senior officer performance management administrative  regulation.
We worked with staff from our Human Capital Office to assess these  policies against OPM’s HCAAF and our prior work, and determined that  they were consistent with relevant criteria. For example, we examined the  processes SEC has to monitor supervisors’ practices to address  unacceptable performance and compared them to OPM guidance and  federal regulations on addressing such performance.
We next determined whether SEC’s practices related to addressing  unacceptable performance were being implemented consistent with the  agency’s policies. To do this, we obtained all performance improvement  plans for fiscal years 2013 through 2015 (16 in all) and compared the  information in these plans against what SEC’s policies require. We first  had one GAO analyst review each performance improvement plan,  compare the information in these plans and supporting documents to  what SEC requires for these plans, and record the findings in a  spreadsheet. We then had a second GAO analyst review the work of the  first analyst to determine if the spreadsheet was coded correctly. The two  analysts conferred on any differences in the coding and were able to  reach consensus on the proper coding.
We also reviewed documentation associated with probationary  employees who are terminated due to performance issues. SEC’s policies  related to addressing unacceptable performance do not apply to  probationary employees. The actions SEC can take against these  employees are governed by federal regulations related to actions taken  against probationary employees for unsatisfactory performance or  conduct. Similar to our approach with the performance improvement  plans, we obtained all files related to probationary employees terminated  due to unacceptable performance for fiscal years 2013 through 2015 (20  in all). We again had one GAO analyst review each file and compare the  information in these files and any supporting documents to what OPM  regulations require, such as a description of the unacceptable  performance, and record the findings in a spreadsheet. We then had a  second GAO analyst review the work of the first analyst to determine if  the spreadsheet was coded correctly. The two analysts conferred on any  differences in the coding and were able to reach consensus on the proper  coding. We also reviewed our recent work on the federal government’s  actions to address unacceptable performance to compare actions SEC  had taken with actions taken across the federal government. In addition,  we met with the Office of General Counsel at SEC to obtain an  understanding of the agency’s policies to address unacceptable  performance.
Hiring and promotions: To determine whether SEC’s hiring and  promotion policies have been designed consistent with relevant criteria,  we compared these policies to OPM’s HCAAF, the President’s 2011  memorandum on improving federal recruitment and hiring, our prior work  on best practices in human capital management, and federal internal  control standards. Specifically, we reviewed the following policies and  documents related to hiring and promotions:
SEC’s hiring program overview, which describes the entire hiring  process, including specific responsibilities of staff in the Office of  Human Resources and the divisions; a description of the various hiring authorities available to SEC; a description of how SEC establishes initial pay for new hires;
SEC policies on promotions for non-bargaining-unit positions; and
SEC’s collective bargaining agreement.
We worked with staff from our Human Capital Office to assess these  policies against OPM’s HCAAF and our prior work, and determined that  they were consistent with relevant criteria. For example, we assessed  SEC’s processes for hiring and promotions and compared them to OPM’s  HCAAF related to hiring.
We next determined whether SEC’s hiring and promotion policies were  being implemented consistent with the agency’s policies. To do this, we  reviewed recruitment case files, which are the documentation that  supports a hiring or promotion announcement, to determine if SEC was  following its hiring and promotion policies. We randomly selected 102  recruitment case files for review. Of these, 3 were duplicates and 18  recruitment case files could not be analyzed because they had already  been audited by OPM or SEC. This left us with a final sample of 81  recruitment case files. We express our confidence in the precision of  estimates derived from the sample of recruitment case files as 95 percent  confidence intervals. This is the interval that would contain the actual  population values for 95 percent of the samples that we could have  drawn. We then developed a checklist that contained steps SEC should  take during various stages of the hiring and promotion process. We  provided this checklist to staff from our Human Capital Office for their  review and also ensured that our checklist was consistent with the  checklists that SEC had developed to assess its hiring and promotion  processes. We then assembled a group of GAO analysts to review the  case files and complete the checklists. In order to ensure consistent  completion of the checklist, each GAO analyst’s first few checklists were  independently reviewed by another analyst. After this stage, the two GAO  analysts compared the results of the checklists. Any discrepancies were  discussed among the two analysts as well as shared with the entire group  of GAO analysts. Once the entire group reached consensus on how to  address the discrepancies identified during this stage, all 81 case files  were reviewed against the checklist.
The next step in the process involved an independent second review of  the completed 81 case file checklists. All 81 case files were checked by a  GAO analyst who was not involved in the initial review. Any discrepancies  found during this stage were discussed among the analysts, and a  consensus was reached on how to address the discrepancy. Once this  process was completed, the information from the checklists was tallied to  identify any deficiencies in the hiring and promotion process.
To obtain the views of the key SEC staff involved in the hiring and  promotion process, we developed a set of structured interview questions  and conducted interviews with 18 of the 23 hiring specialists in the Office  of Human Resources and 16 hiring managers from the divisions. We  provided the set of structured interview questions to staff from our Human  Capital Office for review and revised the questions based on their  expertise. We attempted to interview all 23 hiring specialists, but 5 did not  respond to our request. We chose the 16 hiring managers to interview by  obtaining the list of names, titles, e-mail addresses, and phone numbers  of all hiring managers from the six mission-critical office and divisions. For  divisions that had more than two hiring managers, we randomly selected  two managers, except for the Division of Enforcement and the Office of  Compliance Inspections and Examinations. For this division and office,  we selected a judgmental sample of hiring managers, based on input  from SEC on what types of staff or regions would more likely be involved  in the hiring process. We used this approach for the Division of  Enforcement and the Office of Compliance Inspections and Examinations  because they have a large regional presence, and we interviewed 2  managers in headquarters and 2 in the regional offices for each. The  responses to the structured interview questions are not representative of  the views of all SEC staff involved in the hiring process but provide useful  information on the types of views and concerns held by these staff.
We also reviewed OPM’s 2014 review and the National Credit Union  Administration’s 2015 review of SEC’s hiring and promotion practices, as  well as an outside consultant group review of SEC’s internal promotion  actions from fiscal years 2011 through 2014. We assessed the methods  used in these reviews and determined that they were reliable for our  purposes. Finally, we reviewed SEC’s internal quality reviews of its hiring  and promotion practices that were implemented as part of its human  capital accountability system.
Training: To determine whether SEC’s policies on training for staff who  work in the mission-critical office and divisions were designed consistent  with relevant criteria, we compared these policies to OPM’s HCAAF and  our prior work on assessing strategic training and development efforts in  the federal government. Specifically, we reviewed SEC’s training  policy—SEC Administrative Regulation 6-28: Training and  Development—and the collective bargaining agreement. We also met  with our Chief Learning Officer to help us determine how to assess these  policies against OPM’s HCAAF and our prior work, and determined that  they were designed consistent with relevant criteria.
In order to assess the implementation of SEC’s training practices, we, in  conjunction with our Chief Learning Officer, determined that the best  measure of a training program is the views of the supervisors because  they are in the best position to determine if their staff have the necessary  skills to accomplish their work. As a result, we asked selected SEC  supervisors about the effectiveness of training during structured group  interviews of SEC supervisors. We selected these supervisors based on  whether they worked in the mission-critical office and divisions, pay grade  and occupation, geographic location, and willingness to meet with us. We  also analyzed survey results on training collected from our 2016 surveys  of mission-critical office and divisions and senior officers and compared  them to the results of our 2013 survey. We also compared SEC  responses on training from the 2016 OPM FEVS to that of other  agencies.
Finally, for all of the personnel management practices we assessed, we  reviewed responses from our surveys and from our individual and  structured group interviews, and we included relevant responses to  provide SEC staffs’ perspectives on these practices.
We assessed the reliability of all of the data we used during this review  and determined they were sufficiently reliable for our purposes, which  include describing trends and views on personnel management practices  at SEC. To assess the reliability of the FEVS data, we examined  descriptive statistics, data distribution, and reviewed missing data. We  also reviewed FEVS technical documentation as well as the statistical  code OPM uses to generate the index and variance estimates, and we  interviewed officials responsible for collecting, processing, and analyzing  the data. We used SEC data derived from the Department of the Interior’s  Federal Personnel/Payroll System to construct our sample frames for the  three surveys, test the implementation of various personnel management  practices, and develop summary tables in our appendixes. To determine the reliability of these data, we interviewed SEC staff responsible for  these data to determine how data were collected, what controls existed  over the data, and any limitations on the data. In addition, where possible,  we compared data elements to the original source documents to  corroborate the accuracy of the data where available.
We conducted this performance audit from July 2015 to December 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix III: Office of Personnel Management Federal Employee Viewpoint Survey – Securities and Exchange Commission Staff Demographics, Fiscal Year 2015
paragraphs: The Office of Personnel Management (OPM) conducts an annual survey  of federal employees to obtain their views about their work experiences,  agencies, and leaders. The following tables provide information on  Securities and Exchange Commission (SEC) employee responses to  selected survey questions in fiscal year 2015. Tables 3 through 9 provide  responses by employee group: race, ethnicity, gender, and tenure. For  the demographic variables we tabulate survey responses by, the rate of  missing data ranges from 5 percent to 10 percent of SEC employees.  SEC employees with missing data for a particular demographic variable  are not included in that tabulation.
section_title: Appendix IV: Securities and Exchange Commission Awards for Fiscal Years 2013-2015
paragraphs: The Securities and Exchange Commission (SEC), among other things,  provides cash and time-off incentive awards to motivate staff and  recognize their contributions to the agency. The following tables provide  information on all cash awards for all SEC staff for fiscal years 2013  through 2015. Table 10 provides information on individual cash awards  broken out by supervisory status. Tables 11 through 13 provide  information on individual cash awards broken out by age, gender, and  race, respectively.
The following tables provide additional information on time-off awards for  fiscal years 2013 through 2015. Table 14 provides information on  individual time-off awards broken out by supervisory status, including a  separate breakout by senior officers. Tables 15 through 17 provide  information on individual time-off awards broken out by age, gender, and  race, respectively.
section_title: Appendix V: Securities and Exchange Commission Personnel Management Practices – Training
paragraphs: As part of our review of the Securities and Exchange Commission’s  (SEC) workforce planning practices, we reviewed SEC’s practices related  to employee training for staff who are primarily responsible for  implementing the agency’s mission: the Office of Compliance Inspections  and Examinations; and the Divisions of Corporation Finance,  Enforcement, Investment Management, Economic and Risk Analysis, and  Trading and Markets (hereinafter, mission-critical office and divisions).  We determined that SEC’s practices related to training employees have  been designed and implemented consistent with relevant criteria. Our  prior work notes that one of the core characteristics of a strategic training  and development process is strategic alignment.  Our prior work also  notes that other core characteristics of strategic training and development  include communication from agency leadership, involvement of  stakeholders, a system of accountability, and effective use of resources.  In addition, the Office of Personnel Management’s (OPM) Human Capital  Assessment and Accountability Framework notes, among other things,  that agency leaders and supervisors should sustain a learning  environment that drives continuous improvement, invest in training to help  employees build mission-critical competencies, and use a variety of  learning methods.
SEC’s policies on training and development specify that the intent of SEC  University and its programs is to support the mission of SEC, its strategic  plan, and performance objectives and to enable employees to perform  their current functions at the maximum level of proficiency. SEC’s  training policies also provide specific responsibilities for senior officers,  supervisors, and nonsupervisory staff, including requiring senior officers  to provide fair opportunities for training. In addition, SEC supervisors and  managers are responsible for supporting fair selection for training,  ensuring the training meets the definition of mission-related training, and  ensuring the availability of funds for a variety of internal and external  training. SEC’s collective bargaining agreement specifies the purpose of  training, which is to enable employees to perform their official duties at  the maximum level of proficiency. The collective bargaining agreement  also specifies the responsibilities of the agency, including determining the  training needs, ensuring consideration of employee requests for training  that supports the agency’s strategic plan, and supporting attorney and  accountant opportunities to obtain mandatory continuing education  credits. According to SEC staff in SEC University, they serve as liaisons  with stakeholders from across SEC on the design, implementation, and  evaluation of a variety of training methods to ensure that training is  meeting the needs of the various offices and divisions within the agency.
SEC supervisors we met with and staff we surveyed noted that training  has improved since 2013. When asked to what extent their employees  received training that was applicable and sufficient for them to perform  their jobs, the supervisors we interviewed told us that training for staff and  supervisors equipped staff with the necessary skills, and had improved in  recent years. They also said the training they received as supervisors  was applicable and sufficient for them to do their jobs. In addition, staff  surveyed in 2016 had more positive views on training than in 2013. The  percentage of staff surveyed who agreed that new staff were given  enough guidance and training increased for nonsupervisors from about  38 percent in 2013 to 48 percent in 2016 and for supervisors from  approximately 66 percent to 72 percent in 2016, as shown in figure 11.
In addition, the percentage of staff who agreed (to a moderate or great  extent) that senior officers work to make improvements in training focused  on specific competencies increased for nonsupervisors from about 42  percent in 2013 to 46 percent in 2016, and for supervisors from  approximately 60 percent in 2013 to 62 percent in 2016. SEC staff also  had more positive views on training than staff in other government  agencies. In OPM’s 2016 Federal Employee Viewpoint Survey, an  estimated 66 percent of SEC employees surveyed were satisfied with the  training they received for their present job, which was higher than that of  other federal agencies, with an estimated 53 percent of all respondents  satisfied with training.
section_title: Appendix VI: Ratio of Supervisors and Senior Officers, Fiscal Years 2008–2015
paragraphs: Section 962 of the Dodd-Frank Act Wall Street Reform and Consumer  Protection Act included a provision for us to review whether there is an  “excessive number of low-level, mid-level, or senior-level managers” at  the Securities and Exchange Commission (SEC). We did not find any  standards that have been established for evaluating excessive numbers  of supervisors. Therefore, we are reporting on the ratio of SEC  employees at the various levels for fiscal years 2008 through 2015. Table  18 illustrates the ratio of nonsupervisors to supervisors at SEC. Table 19  illustrates the ratio of nonsupervisors to senior officers, and table 20  illustrates the ratio of supervisors to senior officers.
section_title: Appendix VII: Percentage of Staff Who Left the Securities and Exchange Commission, Fiscal Years 2008- 2015
paragraphs: Among its provisions, Section 962 of the Dodd-Frank Wall Street Reform  and Consumer Protection Act included a provision for us to review  turnover rates within Securities and Exchange Commission (SEC)  subunits. While staff turnover rates could be used to identify potential  areas for improvement and further develop current supervisors, officials  from the Merit Systems Protection Board noted that turnover was not a  good indicator of poor supervision for several reasons. For example, they  said that staff may leave to pursue opportunities with a different employer  or a different career path, or for personal reasons. SEC officials also  indicated that staff facing potential removal or termination often would  resign or retire, rather than going through removal or termination. Tables  21 and 22 show the percentage of staff who left SEC from fiscal years  2008 through 2015 from headquarters and the 11 regional offices,  respectively. Table 23 shows the total number of staff who left SEC  during the same period.
section_title: Appendix VIII: Performance Ratings for Fiscal Years 2013–2015
paragraphs: The Securities and Exchange Commission’s (SEC) current performance  appraisal system is designed to rate employees on a numerical scale  from 1 to 5. However, due to an agreement with the SEC employees  union, bargaining-unit employees are officially rated as either meets  expectations (that is, needs improvement, meets expectations, exceeds  expectations, or greatly exceeds expectations) or unacceptable. Table 24  shows the distribution of performance ratings for fiscal years 2013  through 2015. The initial rating for bargaining-unit staff is on the five-point  scale. The final rating translates that initial rating to either meets  expectations or unacceptable, based on the agreement reached between  SEC and the union.
section_title: Appendix IX: Comments from the Securities and Exchange Commission
subsections: section_title: GAO’s Comments
paragraphs: 1.  We address these issues in our responses to comments 3 and 4.  2.  According to officials from the SEC union and the Office of Human  Resources, the pilot has only been implemented for non- bargaining-unit staff. Moreover, SEC did not provide us with  information on its agency-wide pilot of its new performance  management system, nor did it provide an implementation plan  that identified key milestone dates or schedules to pilot or fully  implement the new performance management system to all  employees in fiscal year 2017.  3.  We recognize that not all staff at SEC may need to coordinate and  collaborate for work-related issues.  However, staff in mission- critical offices and divisions should be enabled to collaborate and  communicate with staff in other offices and divisions. As  acknowledged in our report, the Division of Enforcement created  formal liaisons that other divisions and offices can contact, and  these liaisons help to facilitate cross-divisional communication and  collaboration within the division. Based on our survey results, staff  in the Division of Enforcement more frequently interacted with staff  from other mission-critical offices and divisions.  As SEC  acknowledged in its response, the Division of Economic and Risk  Analysis is similar to the Division of Enforcement in that staff  should be routinely communicating and collaborating with them.  However, unlike the Division of Enforcement, the Division of  Economic and Risk Analysis lacks a mechanism to easily facilitate  cross-divisional communication and collaboration. Our survey  results show that interaction between Division of Economic and  Risk Analysis staff and staff from other mission-critical offices and  divisions is limited.
SEC also expressed concern that our report cited an anecdotal  account from one former employee. However, we found  substantial evidence that siloed communication remains a  challenge at SEC. For instance, 78 of the 187 employees we  interviewed (over 40 percent) cited issues around siloed  communication as an area where SEC needs to improve.  Additionally, of the 1,947 written responses we received to our  survey questions, 597 of them cited various challenges related to  communication and collaboration. We provided examples from  several current and one former employee to illustrate the siloed  communication at SEC.  4.  SEC expressed concern with our recommendation to expand the  responsibilities and authority of the COO or other official or office. We  are not suggesting that an additional layer of management is needed.  Rather, we are recommending that the authority of the COO or some  other official be enhanced in order to ensure that each mission-critical  office and division establish a mechanism or develop procedures to  facilitate communication and collaboration.
section_title: Appendix X: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact above, Triana McNeil (Assistant Director), José  R. Peña (Analyst-in-Charge), Carl Barden, Michelle Batie, Bethany  Benitez, Laura Chase, Pamela Davidson, Tom Gilbert, Jill Lacey, Wati  Kadzai, Steven Lozano, Marc Molino, Janice Morrison, Alexander Ray,  Ginelle Sanchez-Leos, Jerome Sandau, Jennifer Schwartz, Ebonye  Watson, and William White made key contributions to this report.