Department of Homeland Security: Taking Further Action to Better Determine Causes of Morale Problems Would Assist in Targeting Action Plans
Released: 2012-10-31T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: DHS is the third largest cabinet-level department in the federal government, employing more than 200,000 staff in a broad range of jobs. Since it began operations in 2003, DHS employees have reported having low job satisfaction. DHS employee concerns about job satisfaction are one example of the challenges the department faces implementing its missions. GAO has designated the implementation and transformation of DHS as a high risk area, including its management of human capital, because it represents an enormous and complex undertaking that will require time to achieve in an effective and efficient manner. GAO was asked to examine: (1) how DHS's employee morale compared with that of other federal employees, and (2) the extent to which DHS and selected components have determined the root causes of employee morale, and developed action plans to improve morale. To address these objectives, GAO analyzed survey evaluations, focus group reports, and DHS and component action planning documents, and interviewed officials from DHS and four components, selected based on workforce size, among other things.

section_title: What GAO Found
paragraphs: Department of Homeland Security (DHS) employees reported having lower average morale than the average for the rest of the federal government, but morale varied across components and employee groups within the department. Data from the 2011 Office of Personnel Management (OPM) Federal Employee Viewpoint Survey (FEVS)--a tool that measures employees' perceptions of whether and to what extent conditions characterizing successful organizations are present in their agencies--showed that DHS employees had 4.5 percentage points lower job satisfaction and 7.0 percentage points lower engagement in their work overall. Engagement is the extent to which employees are immersed in their work and spending extra effort on job performance. Moreover, within most demographic groups available for comparison, DHS employees scored lower on average satisfaction and engagement than the average for the rest of the federal government. For example, within most pay categories DHS employees reported lower satisfaction and engagement than non-DHS employees in the same pay groups. Levels of satisfaction and engagement varied across components, with some components reporting scores above the non-DHS averages. Several components with lower morale, such as Transportation Security Administration (TSA) and Immigration and Customs Enforcement (ICE), made up a substantial share of FEVS respondents at DHS, and accounted for a significant portion of the overall difference between the department and other agencies. In addition, components that were created with the department or shortly thereafter tended to have lower morale than components that previously existed. Job satisfaction and engagement varied within components as well. For example, employees in TSA's Federal Security Director staff reported higher satisfaction (by 13 percentage points) and engagement (by 14 percentage points) than TSA's airport security screeners.

DHS has taken steps to determine the root causes of employee morale problems and implemented corrective actions, but it could strengthen its survey analyses and metrics for action plan success. To understand morale problems, DHS and selected components took steps, such as implementing an exit survey and routinely analyzing FEVS results. Components GAO selected for review--ICE, TSA, the Coast Guard, and Customs and Border Protection--conducted varying levels of analyses regarding the root causes of morale to understand leading issues that may relate to morale. DHS and the selected components planned actions to improve FEVS scores based on analyses of survey results, but GAO found that these efforts could be enhanced. Specifically, 2011 DHS-wide survey analyses did not include evaluations of demographic group differences on morale-related issues, the Coast Guard did not perform benchmarking analyses, and it was not evident from documentation the extent to which DHS and its components used root cause analyses in their action planning. Without these elements, DHS risks not being able to address the underlying concerns of its varied employee population. In addition, GAO found that despite having broad performance metrics in place to track and assess DHS employee morale on an agency-wide level, DHS does not have specific metrics within the action plans that are consistently clear and measurable. As a result, DHS's ability to assess its efforts to address employee morale problems and determine if changes should be made to ensure progress toward achieving its goals is limited.

section_title: What GAO Recommends
paragraphs: GAO recommends that DHS examine its root cause analysis efforts and add the following, where absent: comparisons of demographic groups, benchmarking, and linkage of root cause findings to action plans; and establish clear and measurable metrics of action plan success. DHS concurred with our recommendations.

section_title: Letter
paragraphs: The Department of Homeland Security (DHS) is the third largest cabinet- level department in the federal government, employing more than  200,000 staff in a broad range of jobs, including aviation and border  security, emergency response, cybersecurity analysis, and chemical  facility inspection. The DHS workforce is situated throughout the nation,  carrying out activities in support of DHS’s mission to (1) prevent terrorism  and enhance security, (2) secure and manage the nation’s borders, (3)  enforce and administer immigration laws, (4) safeguard and secure  cyberspace, (5) ensure resilience from disasters, and (6) provide  essential support to national and economic security.

Since it began operations in 2003, DHS employees have reported having  low job satisfaction. In 2011, for example, DHS’s scores on the Office of  Personnel Management (OPM) Federal Employee Viewpoint Survey  (FEVS)—a tool that measures employees’ perceptions of whether and to  what extent conditions characterizing successful organizations are  present in their agency—and the Partnership for Public Service’s (the  Partnership) rankings of the Best Places to Work in the federal  government, were generally low.

In determining whether a government program is high risk, we consider whether it  involves national significance, a management function that is key to performance and  accountability, or whether there is an inherent or systematic problem, among other things.  Our prior work has identified four high-risk areas for which DHS has primary or significant  responsibilities: (1) Implementing and Transforming DHS, (2) The National Flood  Insurance Program, (3) Protecting the Federal Government’s Information Systems and the  Nation’s Critical Infrastructure, and (4) Establishing Effective Mechanisms for Sharing  Terrorism-Related Information to Protect the Homeland. GAO, Department of Homeland  Security: Progress Made in Implementation and Transformation of Management  Functions, but More Work Remains, GAO-10-911T (Washington, D.C.: Sept. 30, 2010). department’s workforce goals, objectives, and performance measures for  human capital management.

We have previously reported that successful organizations empower and  involve their employees to gain insights about operations from a frontline  perspective, increase their understanding and acceptance of  organizational goals and objectives, and improve motivation and morale. In light of the critical nature of DHS’s mission to protect the security and  economy of our nation and the importance of attracting and retaining  engaged and satisfied DHS employees to perform its work, you asked us  to assess DHS’s efforts to address employee morale.addresses the following questions:

How does DHS’s employee morale compare with that of other federal  government employees?

To what extent have DHS and selected components determined the  root causes of employee morale and developed action plans to  improve morale?

GAO, High-Risk Series: Strategic Human Capital Management, GAO-03-120  (Washington, D.C.: January 2003). with the non-DHS average.above, below, and similar to the average. In addition, we interviewed  representatives of employee groups within the four selected components  to gather employee perspectives on drivers of morale. Details of the  selected component index scores, and their statistical significance, are  reported in appendix I.

The components selected had scores both  To compare DHS’s employee morale with that of other federal  government employees, we analyzed the 2011 FEVS results and  reviewed OPM survey results issued since 2004, the first full year in  which survey data are available. During the course of our analysis, we  interviewed knowledgeable agency officials, reviewed relevant  documentation, tested data for errors, and determined that the FEVS data  are sufficiently reliable for the purposes of this report. As part of this  analysis, we compared 2011 DHS and non-DHS job satisfaction and  engagement score results by several categories of employees, such as  supervisory status, pay grade, and age. We also compared satisfaction  and engagement scores within the selected components by employee  group, where possible. For example, within TSA, we compared  satisfaction and engagement scores reported by Transportation Security  Officers, Federal Security Director staff, headquarters staff, and Federal  Air Marshals.

To determine the extent to which DHS and the selected components  identified the root causes of employee morale and developed action plans  for improvements, we reviewed agency analysis results, interviewed  agency human capital officials and representatives of employee groups,  and evaluated action plans for improving morale. We also compared DHS  and selected components’ morale root cause analyses and related action  plans with available guidance for such efforts.

We conducted this performance audit from October 2011 through  September 2012 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives. Appendix II  contains more detailed information about our scope and methodology.
subsections: section_title: Background
paragraphs: Federal employees are routinely surveyed through OPM’s administration  of the FEVS, which is administered to collect data on federal employees’  perceptions about how effectively agencies are managing their  workforces. The FEVS is a tool that measures employees’ perceptions of  whether, and to what extent, conditions that characterize successful  organizations are present in their agencies, according to OPM. This  survey was administered for the first time in 2002 and then repeated in  2004, 2006, 2008, 2010, 2011, and April through June 2012. The survey  provides general indicators of how well the federal government is  managing its human resources management systems. It also serves as a  tool for OPM to assess individual agencies and their progress on strategic  management of human capital, and gives senior managers employee  perspectives on agency management. Specifically, the survey includes  categories of questions asking employees for their perspectives on their  work experience, work unit, agency, supervisor, leadership, and  satisfaction. OPM intends for agency managers to use the findings to  develop policies and action plans for improving agency performance. In  2011, OPM provided a summary of FEVS findings to DHS. In that report,  OPM summarized DHS’s survey results relative to governmentwide  averages and provided positive and negative response levels for each  survey question. Also included in the report was action planning guidance  for using FEVS results to improve human capital management.

Pub. L. No. 107-295, § 1304, 116 Stat. 2315, 2289 (2002) (codified at 5 U.S.C. §  1103(c)). management, which is focused on agencies having quality people with  the appropriate competencies in mission-critical activities. The FEVS job  satisfaction index is one of the metrics used by OPM to assess whether  agencies are effectively managing the talent management system. The  FEVS provides one source of information for evaluating success on other  HCAAF standards as well by measuring responses to groups of FEVS  questions for four indices. The four index measures are: Leadership and  Knowledge Management; Results-Oriented Performance Culture; Talent  Management; and Job Satisfaction. In addition, in 2011, OPM added an  index to measure employee engagement, which OPM defines as the  extent to which an employee is immersed in the content of the job and  energized to spend extra effort in job performance.

DHS’s OCHCO is responsible for implementing policies and programs to  recruit, hire, train and retain DHS’s workforce. As the department-wide  unit responsible for human capital issues within DHS, OCHCO provides  OPM with a DHS-wide action plan every other year, with the next plan  due in January 2013. OCHCO also provides guidance and oversight to  the DHS components related to morale issues. For example, OCHCO  provides a survey analysis and action planning tool that the components  must use in response to FEVS results to develop action plans for   These plans are to state  improving employees’ positive scores.objectives and identify actions to be taken in response to survey results.  OCHCO also has provided oversight by reviewing and providing feedback  on component action plans.

section_title: DHS Employees Reported Lower Morale than the Rest of the Federal Government, but Morale Varied across DHS Components and Employee Groups
paragraphs: Data from the 2011 FEVS show that DHS employees have lower average  levels of job satisfaction and engagement overall and across most  demographic groups available for comparison, such as pay grade, when  compared with the average for the rest of the federal government. Levels  of satisfaction and engagement vary across components, with some  components reporting satisfaction or engagement above the average for  the rest of the government. Similarly, these measures of morale vary  within components as well, with some employee groups reporting higher  morale than other groups within the same component.
subsections: section_title: DHS Employees as a Whole Reported Lower Satisfaction and Engagement According to Several Measures
paragraphs: As shown in figure 1, DHS employees generally reported improvements  in job satisfaction index levels since 2006 that narrowed the gap between  DHS and the governmentwide average. However, employees continue  to indicate less satisfaction than the governmentwide average.  Partnership analysis of FEVS data also indicates consistent levels of low  employee satisfaction relative to other federal agencies. Similar to its  2011 ranking, 31st of 33 federal agencies, the Partnership ranked DHS  28th of 32 in 2010, 28th of 30 in 2009, and 29th of 30 in 2007 in the Best  Places to Work ranking on overall scores for employee satisfaction and  commitment.

Our analyses of 2011 FEVS results also indicate that average DHS-wide  employee satisfaction and engagement scores were consistently lower  when compared with average non-DHS employee scores in the same  demographic groups. As shown in figure 2, comparisons of DHS with  non-DHS employees by supervisory status, pay group, and tenure  indicate that satisfaction and engagement are lower across many of the   For  DHS groups where statistically significant differences are evident.example, across pay categories DHS satisfaction and engagement were  lower than the scores for the same non-DHS employee pay groups, with  the exception of senior executives, senior leaders, employees with less  than 1 year of tenure, and General Schedule pay grades 1-6.job satisfaction and engagement scores for DHS management and non- management employees were lower than for the same non-DHS  employee groups.

section_title: DHS Took Steps to Determine Root Causes of Morale Problems and Implemented Corrective Actions, but Could Strengthen Its Efforts
paragraphs: DHS and the selected components have taken steps to understand  morale problems, such as holding focus groups, implementing an exit  survey, and routinely analyzing FEVS results. On the basis of FEVS  results, DHS and the selected components planned actions to improve  FEVS scores. However, we found that DHS could enhance its survey  analysis and monitoring of action plan results. In addition, according to  DHS’s Integrated Strategy for addressing the implementing and  transforming high risk area, DHS has begun implementing activities to  address morale but has not yet improved DHS’s scores on OPM’s job  satisfaction index or its ranking on the Partnership’s Best Places to Work  in the Federal Government.
subsections: section_title: DHS and Selected Components’ Have Taken Steps to Understand Morale Problems
paragraphs: DHS’s OCHCO has taken several steps to understand morale problems  DHS-wide. Specifically, since 2007, OCHCO:

Conducted focus groups DHS-wide in 2007 to determine employee  concerns related to morale, which identified employee concerns in  areas of leadership, communication, empowerment, and resources.

Performed statistical analysis in 2008 to identify workplace factors that  drove employee job satisfaction, finding that the DHS mission and  supervisor support, among other things, drove employee job  satisfaction.  Initiated an exit survey, first administered DHS-wide in 2011, to  understand why employees chose to leave their position. The survey  found lack of quality supervision and advancement opportunities were  the top reasons for leaving.

Analyzed 2011 FEVS results, among other things, showing where  lower scores on HCAAF indices were concentrated among several  components—Intelligence and Analysis, TSA, ICE, National  Protection and Programs Directorate, and the Federal Emergency  Management Agency (FEMA).

Launched an Employee Engagement Executive Steering Committee  (EEESC) in January 2012 that will identify action items for improving  employee engagement by September 2012, according to OCHCO  officials.

The selected components also evaluated FEVS results to identify morale  problems and considered additional information sources. For example:

TSA convened a corporate action planning team in March 2011, as  part of its response to FEVS results, which relied on data sources  such as the TSA-administered exit survey, employee advisory groups,  and an online employee suggestion tool, to gain perspectives on  systemic challenge areas and to develop plans to address morale,  according to TSA officials. TSA’s action plan for improving morale,  based on these sources, was completed in July 2012.  ICE considered results of a Federal Organizational Climate Survey  (FOCS), last completed in March 2012, and held focus groups to  gauge the extent to which employees view ICE as having an  organizational culture that promotes diversity.

CBP launched a quarterly online employee survey in 2009 to solicit  opinions on one specific topic per quarter, such as use of career  development resources and how the resources contributed to  employees’ professional growth at CBP.

The Coast Guard relied on an Organizational Assessment Survey  (OAS), last administered by OPM in 2010, to understand employee  morale. The OAS solicits opinions on a range of topics, including job  satisfaction, leadership, training, innovation, and use of resources. It  included civilian and military Coast Guard personnel, but is not  administered governmentwide so comparisons between the Coast  Guard and other federal employees are limited to organizations that  may use the OAS, according to Coast Guard officials.

Appendix III provides more detailed descriptions of DHS’s steps to  address morale problems and selected components’ 2011 FEVS analysis  methods and findings. Appendix IV provides additional information on the  selected components’ data sources beyond FEVS for evaluating root  causes of morale, including a summary of results and how the information  was used by the components.

section_title: DHS and Selected Components Conducted Limited FEVS Analyses
paragraphs: For the 2011 FEVS, DHS and the selected components completed  varying levels of analyses to determine the root causes of low morale.  However, DHS and the selected components conducted limited analysis  in several areas that is not consistent with OPM and Partnership  guidance that lays out useful factors for evaluating root causes of morale  problems through FEVS analysis, as shown in figure 4.

Usage of the three factors described in figure 4 varied across DHS-wide  and component-level 2011 FEVS analyses we reviewed. In some  instances, the factors were partially or not used. For example:

Demographic group comparisons. According to our reviews of  OCHCO’s analyses, OCHCO’s DHS-wide analyses did not include  evaluations of demographic group differences on morale-related  issues for the 2011 FEVS. According to OCHCO officials, DHS’s  Office of Civil Rights and Civil Liberties reviews survey results to  identify diversity issues that may be reflected in the survey, and  OCHCO officials considered these results when developing one of the  current (as of August 2012) DHS action plans to create policies that  identify barriers to diversity. In 2007 and 2009, years in which DHS  administered the Annual Employee Survey (AES), demographic  comparisons were made. For example, on the basis of 2009 AES  data, DHS found no significant demographic differences other than  supervisors’ positive responses to questions were generally higher  than those of non-supervisors and differences among pay grade  levels. Because OPM now administers the survey each year, DHS is  not able to make significant demographic group comparisons because  of the format of the data provided by OPM, according to OCHCO  officials. However, we obtained FEVS data from OPM that allowed us  to make demographic group comparisons. For example, we compared  DHS and non-DHS employee satisfaction and engagement scores  across available demographic groups and found that both satisfaction  and engagement were generally lower for DHS employees, which is  summarized in appendix I, table 5.

For the DHS component analyses we reviewed, TSA and CBP  conducted some demographic analysis. For example, TSA compared  screeners, Federal Security Director staff, Federal Air Marshals, and  headquarters staff on each FEVS dimension (e.g., work experiences,  supervisor/leader, satisfaction, and work/life). As a result, TSA was  able to identify screeners as having survey scores below those of  other TSA employee groups. CBP also compared race, ethnicity,  gender, and program office scores. CBP found that no significant  differences were present in the positive responses to the 2011 FEVS  core questions when comparing race, ethnicity and gender, and found  that Border Patrol employees reported higher job satisfaction than  field operations employees (74 versus 66 percent on the job  satisfaction index). In contrast, the Coast Guard did not conduct  analysis in addition to data that was provided by DHS OCHCO.  Because OCHCO’s data did not include demographic information for  the 2011 FEVS, Coast Guard did not make demographic group  comparisons. ICE and CBP officials stated that they did not have  access to 2011 FEVS data files necessary to conduct more detailed  demographic comparisons. However, as shown in appendix I, we  were able to make various demographic comparisons based on a  more detailed data file provided by OPM, which is similar to a file that  OPM makes available to agencies and the public.

Benchmarking against similar organizations. TSA benchmarked  its FEVS results against results from similar organizations, by  comparing results with CBP, and OCHCO’s DHS-wide analysis  highlighted Partnership rankings data, showing DHS’s position relative  to the positions of other federal agencies as a Best Place to Work.  Similarly, ICE benchmarked its FEVS results overall and for program  offices, such as homeland security investigators, against other DHS  components, including the U.S. Secret Service and CBP. For the  2011 FEVS, CBP performed more limited benchmarking, by  comparing FEVS results with governmentwide averages. According to  CBP officials, when analyzing annual employee surveys prior to 2011,  CBP benchmarked its results against agencies with high positive  FEVS scores, such as the Social Security Administration, the Federal  Bureau of Investigation, the Internal Revenue Service, and the  Nuclear Regulatory Commission. CBP is in the initial planning phase  of a larger benchmarking project that would benchmark CBP against  foreign immigration, customs, and agriculture inspection agencies,  such as the Canadian Border Services Agency and the Australian  Customs and Border Protection Service. If approved, this  benchmarking project is expected to occur in fiscal year 2013,  according to CBP officials. The Coast Guard did not perform FEVS  benchmarking analysis, according to the documentation we reviewed,  but did make OAS-based comparisons between the Coast Guard and  other organizations that use the OAS, according to Coast Guard  officials.

Linkage of root causes with action plans. For both DHS-wide and  selected component action plans, FEVS questions with low scores  were linked with action plan areas. For example, in the DHS-wide  action plan, low scores on employee satisfaction with opportunities to  get a better job in the organization were linked to action plan items for  enhancing employee retention. However, the extent to which DHS  and the components used root causes found through other analyses  to inform their action plans, such as quarterly exit survey results or  additional internal component surveys, was not evident in action plan  documentation (see appendix IV for a description of these additional  root cause analyses). For example,

OCHCO’s DHS-wide action plan was last updated based on 2010  FEVS data and therefore did not rely on data from the DHS 2011  exit survey, since those results were not published until January  2012. Similarly, the EEESC was launched in January 2012 and  therefore its efforts are not yet documented in DHS-wide action  planning documents. According to OCHCO officials, the 2010  DHS-wide action plan includes consideration of results from  OCHCO’s 2008 statistical analysis identifying key drivers of job  satisfaction and results from the 2007 focus groups. However,  linkage to items in the DHS-wide action plan to these results is not  clearly identified because a new action plan template OPM  introduced in 2010 did not provide an area to identify the linkage  between each action and the driver, according to OCHCO  officials. In addition, DHS’s September 2009 action plan indicates  consideration of the 2008 key driver analysis and 2007 focus  group effort that led to a focus on leadership effectiveness  initiatives.

According to CBP and TSA officials, data from other root cause  analysis efforts are not explicitly documented in action plans  developed in response to FEVS results because DHS has not  included linkage of other root cause analysis efforts to actions  items in the FEVS action planning templates used by the  components. TSA officials also stated that other root cause efforts  (see appendix IV) were used to develop TSA’s July 2012 action  plan update. However, the July 2012 plan did not include linkage  of root cause findings other than FEVS results, such as exit  survey results, to action plan items.  ICE officials stated that results from other root cause efforts, such  as its FOCS, have not yet been considered in FEVS-based action  planning but that ICE plans to do so in future efforts to address  morale.

The Coast Guard uses information from its OAS as part of a  process separate from FEVS-based action planning for  addressing morale, so OAS results are not linked to FEVS-based  action plans.

OCHCO and component human capital officials described several  reasons for the variation in root cause analysis of FEVS results. OCHCO  officials described resource constraints and leadership changes within the  OCHCO position as resulting in a lack of continuity in root cause analysis  efforts. For example, one OCHCO official stated that because of resource  constraints, OCHCO has focused more efforts on workforce planning than  on morale problem analysis since 2009. ICE human capital officials stated  that ICE’s human capital services were provided via a contract with CBP  until 2010, when the human capital function became an independently  funded part of the ICE organization. Only since moving to its current  position within ICE has the human capital office been able to devote more  resources to addressing morale issues, according to the officials. CBP  human capital officials stated that for assessing morale issues, CBP uses  both quantitative and qualitative information. However, according to the  officials, qualitative evidence is preferable over quantitative survey  analysis because focus groups and open-ended surveys, such as the  Most Valuable Perspective online survey, allow CBP to better understand  the issues affecting employees. Because of CBP human capital officials’  preference for qualitative information, CBP has not emphasized extensive  quantitative analysis of survey results, such as statistical analysis that  may determine underlying causes of morale problems.

Without a complete understanding of which issues are driving low  employee morale, DHS risks not being able to effectively address the  underlying concerns of its varied employee population. Emphasis on  survey analysis that includes demographic group comparisons,  benchmarking against similar organizations, and linkage of other analysis  efforts outside of FEVS within action plan documentation could assist  DHS in better addressing its employee morale problems.

section_title: DHS and Its Components Completed Action Plans
paragraphs: DHS and the selected components routinely update their action plans to  address employee survey results in accordance with the Office of  Management and Budget’s budget guidance; the DHS-wide plan is  updated every two years, and components update their plans at least  annually. According to OPM’s guide for using FEVS results, action  planning involves, among other things, identifying goals and actions for  improving low-scoring FEVS satisfaction topics such as reviewing survey  results to determine steps to be taken to improve how the agency  manages its workforce. DHS-wide and component action plan goals and  examples of low-scoring FEVS satisfaction topics are listed in table 2.

As part of DHS’s efforts to address our high-risk designation of  implementing and transforming DHS, DHS described a plan for improving  employee morale in its Integrated Strategy for High Risk Management  (Integrated Strategy). In June 2012, DHS provided us with its updated  Integrated Strategy, which summarized the status of the department’s  activities for addressing its implementation and transformation high-risk  designation. In the Integrated Strategy, DHS identified activities to  improve employee job satisfaction scores, among other things. The status  of the activities included ongoing analysis of the 2011 FEVS results,  launch of the EEESC to address DHS scores on the HCAAF indexes,  ongoing coordination between the OCHCO and components to develop  action plans in response to the 2011 FEVS results, and launch of an  online employee survey in the first quarter of fiscal year 2013.

Within the Integrated Strategy action plan for improving job satisfaction  scores, DHS reported that three of six efforts were hindered by a lack of   For example, resources are a constraining factor for DHS’s  resources.Office of the Chief Human Capital Officer to consult with components in  developing action plans in response to 2011 FEVS results. Similarly,  resources are a constraining factor to deploy online focus discussions on  job satisfaction-related issues.

section_title: DHS and Selected Components Generally Followed OPM’s Six Steps for Effective Action Planning but Do Not Have Effective Metrics for Monitoring Efforts
paragraphs: According to our review of the action plans created in response to the  FEVS and interviews with agency officials, DHS and the selected  components generally incorporated the six action planning steps  suggested by OPM, but the agency does not have effective metrics to  support its efforts related to monitoring. (See figure 5.)

We found that, in general, DHS and its components are implementing the  six steps for action planning as demonstrated in table 3 below.

Three attributes relevant to the  linkage—determines whether there is a relationship between the  performance measure and the goals;  clarity—determines whether the performance measures are clearly  stated; and measurable target—determines whether, performance measures  have quantifiable, numerical targets or other measurable values,  where appropriate.

In general, DHS and component measures satisfied the linkage attribute  but did not address the clarity and measurable targets attributes. We  compared DHS and the four components measures of success to the  three attributes and found that all 54 measures of success incorporated  the linkage attribute, 12 of the 54 measures of success did not address  the clarity attribute, and 29 of the 54 measures of success did not  address the measurable target attribute.

As shown in table 4 below, we found that these measures demonstrate  linkage because they align with the action plan goals. However, we  determined that the measures demonstrate neither clarity nor a  measurable target. Specifically, the measures do not demonstrate clarity  because they do not provide enough detail to clearly state the metric used  to measure success. They also do not demonstrate a measureable target  because they do not list quantitative goals or provide a qualitative  predictor of a desired outcome, which would allow the agency to better  determine the extent to which they were making progress toward  achieving their goals.

Officials provided several reasons why their measures of success may fall  short of the attributes for successful metrics. According to OCHCO  officials, OCHCO considers accomplishment of an action item step as a  success and relies on the measures of success listed in its action plan as  a metric for whether the action plan items were implemented. OCHCO  considers whether positive responses to survey questions noted in the  action plan improve over time as the outcome measure for whether action  plans are effective. However, as part of its oversight and feedback on  component action plans, OCHCO does not monitor or evaluate measures  of success for action planning and therefore is not in a position to  determine whether the measures reflect improvement. CBP officials  stated that they monitor the change in FEVS results overall as the intent  of the action planning is to improve their scores on the HCAAF indexes.  Coast Guard officials stated that they rely on qualitative feedback from  employees on action plan items, such as improved training and website  updates, to measure action plan performance. TSA officials stated they  assess action plan results by tracking completion dates for action items  and updating OCHCO on results at least semi-annually, and ICE officials  have stated they have not yet fully developed monitoring efforts to  evaluate job satisfaction action planning because the human capital office  received funding in the summer of 2011 to implement human capital  programs.

We acknowledge that positive responses in survey results and positive  employee feedback are good indicators that action planning is working.  However, until DHS and its components begin to see positive results, it is  important for them to (1) understand whether they are successfully  implementing the individual steps of their action plans and (2) make any  necessary changes to improve on them. By not having specific metrics  within the action plans that are clear and measurable, it will be more  difficult for DHS to assess its efforts to address employee morale  problems, as well as determine if changes should be made to ensure  progress toward achieving its goals. Furthermore, effective measures are  key to DHS’s action plan as it is part of a process that informs the Office  of Management and Budget and OPM of DHS efforts to address survey  results. According to an OPM official responsible for federal action  planning to improve morale, DHS should carefully consider, for each  action step, what success means to the agency, such as increased  employee engagement targets. The official said that when success is  defined, it should not only be clear and measurable, but should also take  into account as many of the different demographic groups evaluated as  possible.

section_title: DHS and the Selected Components Consulted Best Practices
paragraphs: DHS and the selected components have initiated efforts to determine how  other entities approach employee morale issues. DHS officials stated they  have started to review and implement what they consider to be best  practices for improving employee morale, such as the following:

DHS working group—OCHCO leads a survey engagement team that  holds monthly meetings during which action planning efforts from  across the different components are shared and discussed.  Representatives from other federal agencies such as the National  Aeronautics and Space Administration and the Federal Aviation  Administration have also attended these meetings and presented their  action plans for addressing survey results.  Idea Factory—a TSA web-based tool adopted by DHS that empowers  employees to develop, rate, and improve innovative ideas for  programs, processes, and technologies. According to a DHS  assessment, the Under Secretary for Management plans to use this  tool for internal DHS employee communication so as to promote  greater job satisfaction and enhance organization effectiveness.

Component officials we interviewed also stated they have started to  review, implement, and share what they consider to be best practices for  improving morale. For example:  ICE officials stated they consult with other agencies and DHS  components, such as the U.S. Marshal’s Service, when addressing  morale challenges and developing policies and programs. For  example, the U.S. Marshal’s Service has a critical incident response  program for employees encountering a traumatic event and ICE is  exploring adopting a similar program.

TSA officials stated that they reached out to Marriott Corporation,  CBP, and the National Aeronautics and Space Administration to  identify actions for increasing employee rewards and employee  confidence in leadership.

CBP officials stated they have established several ongoing working  groups that routinely meet and share human capital best practices  within the agency. One of these working groups has conducted  benchmarking work with high-FEVS-scoring federal agencies such as  the Social Security Administration, the U.S. Secret Service, the  Federal Bureau of Investigation, the Internal Revenue Service and the  Nuclear Regulatory Commission.

Coast Guard officials stated they share human capital best practices  that may improve job satisfaction with other DHS components such as  (1) their performance appraisal system which was adopted, in part,  DHS-wide; (2) their automated cash award process with FEMA; and  (3) Coast Guard training to supervisors with both DHS headquarters  officials and FEMA.

section_title: Conclusion
paragraphs: Given the critical nature of DHS’s mission to protect the security and  economy of our nation, it is important that DHS employees are satisfied  with their jobs so that DHS can retain and attract the talent required to  complete its work. Employee survey data indicate that when compared to  other federal employees, many DHS employees report being dissatisfied  and not engaged with their jobs. It is imperative that DHS understand  what is driving employee morale problems and address those problems  through targeted actions that address employees’ underlying concerns.  DHS has made efforts to understand morale issues across the  department, but those efforts could be improved. Specifically, given the  annual employee survey data available through the FEVS, DHS and its  components could improve their efforts to determine root causes of  morale problems by comparing demographic groups, benchmarking  against similar organizations, and linking root cause findings to action  plans. Uncovering root causes of morale problems could help identify  appropriate actions to take in efforts to improve morale. In addition, DHS  has established performance measures for its action plans to improve  morale, but incorporating attributes such as improved clarity and  measurable targets could better position DHS to determine whether its  action plans are effective. Without doing so, DHS will have a more difficult  time determining whether it is achieving its goals.

section_title: Recommendations for Executive Action
paragraphs: To strengthen DHS’s evaluation and planning process for addressing  employee morale, we recommend that the Secretary of Homeland  Security direct OCHCO and component human capital officials to take the  following two actions: examine their root cause analysis efforts and, where absent, add the  following: comparisons of demographic groups, benchmarking against  similar organizations, and linkage of root cause findings to action  plans; and establish metrics of success within the action plans that are clear and  measurable.

section_title: Agency Comments and Our Evaluation
paragraphs: We requested comments on a draft of this report from DHS. On  September 25, 2012, DHS provided written comments, which are  reprinted in appendix V, and provided technical comments, which we  incorporated as appropriate. DHS concurred with our two  recommendations and described actions planned to address them.  Specifically:

DHS stated that it will ensure that department-wide and component  action plans are tied to root causes and that the department will  conduct benchmarking against other organizations. DHS also stated  that its ability to conduct demographic analysis is limited due to the  data set OPM makes available to federal agencies. However,  according to OPM, DHS has access to the data necessary for  conducting analysis similar to our comparison of demographic groups.

DHS stated it will review action plans to ensure that each action is  clear and measurable.

We also requested comments on a draft of this report from OPM. On  September 18, 2012, OPM provided a written response, which is  reprinted in appendix VI. OPM’s letter indicated that it reviewed the draft  report and had no comments.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of  Homeland Security, the U.S. Office of Personnel Management, and  interested congressional committees. The report also will be available at  no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report please contact  me at (202) 512-9627 or maurerd@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix VII.

section_title: Appendix I: Statistical Analysis of Employee Morale at Department of Homeland Security and Other Agencies
paragraphs: We conducted a statistical analysis of the 2011 Federal Employee  Viewpoint Survey (FEVS) to assess employee morale at the Department  of Homeland Security (DHS). Our analysis addressed two specific  questions. First, how does morale at DHS and its components compare  with morale at other agencies, holding constant demographic differences  among employees? Second, to what extent is the morale gap between  DHS and other agencies explained by differences in the demographic  composition of the DHS workforce versus other unique characteristics of  the agency or unmeasured demographic factors?

This appendix explains the value of statistical analysis for understanding  the employee morale gap, describes the data and methods we used, and  provides additional details about our findings, which are summarized in  the body of the report. In sum

DHS employees with the same demographic profiles (measured by  FEVS) were about 7 percentage points less engaged and 6 points  less satisfied than non-DHS employees.

Demographic differences (measured by FEVS) between DHS and  other agencies are unlikely to explain the overall morale gap. Unique  features of DHS (or unmeasured demographics) are more likely to be  responsible.

DHS middle managers and employees with 1 to 10 years of tenure at  their components—those hired after the department’s creation—have  lower morale than similar employees at other departments.

Morale varies widely across DHS components, and some have similar  morale as non-DHS agencies. Individual offices can strongly influence  the morale gap at the component level.

The morale gap is smaller for DHS components that existed before  the department was created.
subsections: section_title: Understanding the Morale Gap at DHS
paragraphs: The morale gap between DHS and other agencies may be due to unique  issues within DHS or common issues faced by all agencies in similar  circumstances. Unique issues might include developing an agency-wide  culture, the decisions and composition of senior leaders, and the inherent  uniqueness of homeland security programs. Common characteristics  might include having many law enforcement and front-line customer  service occupations, and having employees dispersed among many  headquarters and field offices.

Determining whether unique or shared issues account for the overall  morale gap is important for understanding the cause of the problem. If  morale at DHS was not uniquely low, compared with morale at agencies  with similar demographics and programs, the agency might learn from  peer agencies facing similar challenges. Alternatively, if morale was lower  at DHS for reasons unique to the agency, DHS might put more emphasis  on understanding its own particular challenges. Distinguishing among  these possible explanations can help develop a solution that is narrowly  tailored to the problem.

Our analysis focused on one group of shared circumstances that might  explain the morale gap: employee demographics. If DHS were more likely  to employ the types of workers who tend to have lower morale across all  agencies of the government, the composition of the workforce might  account for the gap to a greater extent than factors specific to DHS. In  other words, morale at DHS may be no worse than at other agencies  among demographically equivalent employees. Our analysis focused on a  limited number of demographic differences, such as location and age, but  attitudinal differences about pay, benefits, supervision, training,  mentoring, and other human capital issues could be assessed in a similar  way.

We also considered how large of a morale gap there was between  employees in various DHS components and work groups and non-DHS  employees. The gap at the department level can mask groups of  employees with higher or lower morale. Disaggregating morale into small  work groups identifies areas of DHS in which morale may be high or low,  and thus provides sufficiently detailed data for focused solutions to the  problem.

Any analysis of morale in employee surveys is limited by the fact that  associations among the variables of interest may not represent cause- and-effect relationships. Nevertheless, a limited observational analysis  remains useful for evaluating human capital programs. Since federal  agencies cannot easily conduct high-quality randomized controlled trials  of various approaches to managing their employees, the use of  observational methods is common, often in the form of quantitative survey  analyses or qualitative interviews and focus groups. We have previously  found that a pragmatic approach to answering necessary policy  questions, using the best methods and data that are feasible, is widely  supported by academic experts and practitioners in policy analysis.

Moreover, statistical theory has shown that observational methods can  estimate cause-and-effect relationships in certain conditions.

Associations between morale and demographic characteristics are useful  for understanding the operation of human capital programs, when  interpreted cautiously and in the context of all the available evidence. Our  analysis here describes patterns across the demographic groups  identified in the 2011 FEVS and determines whether the aggregate  differences between DHS and other agencies persists among  demographically similar employees. We make no causal interpretations of  these relationships, and our approach is only one that might be valid and  useful.

section_title: The 2011 Federal Employee Viewpoint Survey
paragraphs: The Office of Personnel Management (OPM) provided us with a version  of the 2011 FEVS that included more detailed demographic and  organizational data than the file it released to the public. Specifically, our  file contained the same variables as the public file but identified more  detailed groups of employees. The 2011 survey included responses from  266,376 full-time, permanent federal employees, working for agencies  that, according to OPM, constituted 97 percent of the executive branch  workforce. OPM sampled employees within strata formed by supervisory  status and organizational subgroup (e.g., component and work group).This produced generally large sample sizes even for many small work  groups within components, which allowed us to analyze morale among  small groups of employees with an acceptable degree of precision.

We focused on two types of variables in the FEVS: (1) employee  demographics and (2) OPM’s Employee Engagement and Job  Satisfaction indexes. A series of questions at the end of the survey  collected the demographic data, rather than preexisting administrative  records. OPM reported independently developing and validating the  engagement indexes using factor-analytic procedures, which are common  psychometric statistical methods. The survey items that made up each  index used five-point, Likert-type scales, with “agree/disagree,”  “satisfied/dissatisfied,” or “good/poor” response options.

We used weights provided by OPM to calculate estimates and sampling  variances for all analyses. The weights were the product of the unequal  sampling probabilities across strata and non-response and post- stratification adjustments. Because some strata had relatively small  population sizes—one-quarter with 18 employees or fewer—we corrected  for finite populations.

section_title: Morale Differences Between DHS Employees and Employees at Other Agencies
paragraphs: One explanation for lower morale at DHS is that its employees could be  members of demographic groups that typically have lower morale across  all agencies. If this is true, the cause of morale problems and their  solutions might focus less on factors that are unique to DHS and more on  approaches that apply to any agency with a similar workforce.

Table 5 provides basic evidence to help assess the demographic  explanation. The table presents the average OPM Engagement Index for  several demographic groups in the 2011 FEVS. If engagement problems  at DHS were isolated to particular subgroups of employees, the morale  gap should vary widely across those subgroups. In fact, engagement at  DHS is lower (or statistically indistinguishable from zero) than at other  agencies in each demographic subgroup we analyzed, and the gap  relative to DHS does not vary by large amounts across most subgroups.  However, the gap is somewhat larger among employees who were in  certain subgroups, such as those who had 4 to 10 years of experience  with their components and who worked outside of headquarters.

We developed several statistical models to further assess the  demographic explanation. These models held constant the demographic  profiles of DHS and non-DHS employees, in order to isolate the portion of  the morale gap that was specifically due to non-demographic factors. The  models allowed us to compare morale at DHS and other agencies among  employees who were in the same demographic groups, as measured by  the FEVS.

To avoid methodological complications with modeling latent variables, we  created a binary measure that identified whether a respondent was  engaged or satisfied on each item in the respective scales. Our measure  equaled 1 if the respondent gave positive answers (4 or 5) to each item in  the index and 0 if the respondent gave neutral or negative responses  (1,2, or 3) to at least one item. Collapsing the scale loses some  information, since morale and satisfaction are continuous, latent  variables. However, a collapsed measure provides some degree of  comparability between OPM’s aggregate indices and our individual-level  analysis, since the OPM’s indices also collapse the scale. The differences  among agencies and subgroups of employees are generally similar using  either our measure or OPM’s.

We focused on the associations between broad measures of morale and  fixed demographic characteristics available in the 2011 FEVS. Fixed  demographics and broad measures of satisfaction are not subject to  artificially high correlations that a survey’s design can produce among  attitudinal measures.

section_title: E
paragraphs: ) (1)

section_title: E
paragraphs: ) (2)

Moraleij indicates whether employee i at agency j was engaged or  satisfied, using the binary measure we calculated from the survey items  that make up the OPM indexes (see above). DHS indicates whether the  employee worked for DHS, Demogij is a vector of demographic indicators  (listed in table 6),  Λ is the logistic function, and α and β are vectors of  coefficients that estimate how morale varied among employees in  different demographic groups. We included all demographic factors  measured by the FEVS that plausibly could have predicted morale and  were clearly causally prior to morale. We excluded pay group, however,  because of its high correlation with supervisory status. Model 2 allows  DHS and non-DHS employees in the same demographic groups to have  different levels of morale, as described by  Dβ and  Gβ .We estimated each  model using cluster-robust maximum likelihood methods, with 365 agency  clusters (e.g., Transportation Security Administration ).

Our multivariate analysis found that DHS employees remained an  average of 6.4 percentage points less engaged (+/- 3.2) (see table 6) and  5.5 points less satisfied (+/- 2.2) (not shown) on our scales than  employees at other agencies who had the same age, office location, race,  sex, supervisory status, and tenure. This suggests that measured  demographic differences between employees at DHS and other agencies  do not fully explain the morale gap. Instead, factors that are intrinsic to  DHS, such as culture or management practices, or demographic factors  not measured by FEVS, such as education or occupation, are likely to be  responsible.

We can further explore the roles of demographics and unique DHS  characteristics by performing an Oaxaca decomposition of the results of  model 2, in order to compare DHS with other agencies. Oaxaca  decomposition can assess whether the overall morale gap is explained by  the demographic characteristics of DHS employees, or whether it is  explained by lower morale among DHS employees in the same  demographic groups. In other words, does DHS employ an unusually  large number of workers who tend to have low morale across all  agencies, or do workers with the same backgrounds have uniquely lower  morale at DHS?

As shown in table 6, the model suggests that the demographic profile of  DHS employees (measured by FEVS) tends to slightly increase their  engagement and reduce the gap compared with employees at other  agencies. The demographic characteristics we can observe in FEVS  reduce the overall gaps in the proportion engaged and satisfied on our  scales by 0.1 and 1.0 percentage points, respectively. Instead, the  morale gap is better explained by unique differences in morale between  DHS and other agencies among demographically similar employees.  Such intrinsic differences increase the gaps in the proportion engaged  and satisfied by 6.4 and 5.5 percentage points, respectively. If the  demographic profile of the DHS workforce did not change, but DHS could  achieve the same levels of morale as other agencies from the same types  of employees, our model predicts that DHS employees would not have  lower morale than employees at other agencies.

DHS employees with lower-level positions and component tenure were  among those with lower morale, relative to employees in other agencies.  As shown in figures 6 and 7, our measures of engagement and  satisfaction generally increased with seniority and decreased with tenure,  among employees at DHS and other agencies. At DHS, however, morale  increased more slowly as employees gained more seniority, and it  declined more quickly as they spent more time at the agency. For  example, the average newly hired employee at DHS and similar  employees at other agencies had statistically indistinguishable levels of  engagement. By their sixth years, however, satisfaction for the DHS  employee declined to an average of 18 percentage points, whereas  satisfaction for the non-DHS employees declined to an average of only 26  percentage points. A similar pattern exists with respect to supervisory  status (see figures 6 and 7). These patterns are particularly important for  explaining the overall morale gap, because DHS had about 30 percent  more supervisors and about twice as many people with 6 to 10 years of  component tenure (as a share of all employees), compared with people at  other agencies (according to FEVS).

Low employee morale is not a uniform problem throughout DHS. As  shown in table 7, engagement varies widely across components within  the department, with employees in some components not being  significantly different from the average employee at non-DHS agencies.  These components include the U.S. Coast Guard (Coast Guard), Federal  Law Enforcement Training Center (FLETC), Management Directorate  (MGMT), and U.S. Secret Service (USSS). Job satisfaction at these  components also matches or exceeds that found at other agencies (not  shown in table 7).

DHS has a number of components whose employees have substantially  lower morale than employees at other agencies and elsewhere in the  department. The large share of DHS employees working in these  components accounts for the overall morale gap between DHS and other  agencies.

Components with lower morale include Federal Emergency Management  Agency (FEMA), Immigration and Customs Enforcement (ICE),  Intelligence and Analysis (IA), National Protection and Programs  Directorate (NPPD), Science and Technology (ST), and the TSA. The  engagement scores of these components range from 9.1 to 13.9  percentage points lower than the average score for non-DHS agencies  (see table 7). As a group, these components make up 46 percent of the  employees interviewed for the FEVS. Consequently, the components with  substantially lower morale have a large influence on the gap relative to  the rest of the government, despite the fact that morale at many smaller  DHS components is no worse.

Morale at some of the less engaged and satisfied components is, in turn,  strongly influenced by particular employee workgroups (see table 7). For  example, the average engagement at TSA is 12.8 percentage points  (apart from rounding) lower than at non-DHS agencies. Within TSA,  however, the collectively large groups of air marshal, law enforcement,  and screening workers account for much of the overall difference. A  similar pattern applies to the enforcement, removal, and homeland  security investigation staffs at ICE, the field operations staff at CBP, and  the Federal Protective Service. Such variation within components further  suggests that the morale gap is isolated to particular areas within DHS  that account for a large proportion of its workforce.

At other components, morale is more uniformly lower across most offices.  Average engagement at all work groups within FEMA is 5.8 to 17.7  percentage points lower than the non-DHS average, with the exception of  two regional offices and the offices of the Administrator and Chief of Staff.  The components of ST and IA also have more consistently low morale  across work groups.

One explanation for why morale varies across components focuses on  the length of time each organization has existed. Components that  existed prior to the creation of DHS may have had more time to develop  successful cultures and management practices than components that  policymakers created with the department in 2003. As a result, the  preexisting components may have better morale today than components  with less mature cultures and practices.

To assess this explanation, we analyzed morale among two groups of  components, divided according to whether the component was  established with the creation of DHS or existed previously (see table 8).  We considered three components to be preexisting—FLETC, USSS, and  the Coast Guard—and the rest to be newly created. Because TSA was  created about 2 years before DHS, we included it with components that  were created with DHS.

Our analysis shows that employees at the more recently created  components were less engaged and satisfied on average than employees  at the preexisting components and at non-DHS agencies. For the  preexisting components, engagement was about 2.2 percentage points  higher than at the rest of the government, and the difference in  satisfaction was small (less than 1.4 percentage points). In contrast,  engagement and satisfaction at the more recently created components  were about 8 and 5.1 percentage points lower than at the rest of the  government, respectively.

We developed a statistical model to confirm whether the differences  among components persist, holding constant demographic differences  among their employees. In an alternative version of model 1 above, we  replaced DHS with a vector of variables indicating whether the employee  worked for DHS components or at an agency other than DHS. All other  parts of the model were identical.

The model estimates generally confirmed the differences in engagement  between non-DHS and DHS component employees in the raw data (see  table 9), with two exceptions. The model estimated that, holding constant  demographic differences, employees in the Management Directorate and  Office of the Secretary were 6.9 and 7.7 percentage points less engaged  on average than employees in non-DHS agencies. This suggests that the  engagement gap for employees in these offices is more similar to the gap  at other offices, holding constant the demographic differences among  offices measured by FEVS. The model estimated that differences in  satisfaction between the components and non-DHS agencies were  generally similar to such differences in engagement (see table 9).

The fact that differences among components remained, even among  demographically equivalent employees, suggests that either unmeasured  demographic variables or intrinsic characteristics of the components are  responsible for the differences in morale.

Our analysis discussed in this appendix has a narrow scope: assessing  whether demographic differences among employees explain the morale  differences across DHS and non-DHS employees. Consequently, DHS or  others could expand and improve upon our findings.

Future work could examine whether attitudinal differences among  employees at DHS and other agencies explain the overall morale gap, in  addition to demographic differences. The 2011 FEVS measures  employee attitudes about pay, benefits, health and safety hazards,  training, supervisors, and other issues that could vary meaningfully  between employees at DHS and other agencies and, therefore, explain  why DHS has lower morale. One might include these factors in a  decomposition similar to the one we performed in this appendix. This  could further assess how factors unique to DHS and factors that are  common across all agencies explain the overall morale gap.

A broader attitudinal analysis likely would require the use of more  sophisticated statistical methods for estimating the values of and  relationships among latent variables. The broad measures of morale we  analyze in this appendix, such as the OPM Employee Engagement index,  are made up of responses to questions on smaller dimensions, such as  leadership and supervision. To avoid simply replicating the correlations  that were used to create the indexes, latent variable models could be  useful to examine the relationships among these concepts and compare  morale on latent scales between DHS and non-DHS agencies. This was  beyond the scope of our work.

section_title: Appendix II: Scope and Methodology
paragraphs: The objectives for this report were to evaluate (1) how DHS employee  morale compares with that of other federal government employees and  (2) to what extent DHS and its selected components determined the root  causes of employee morale and developed action plans to improve  morale.

To address our objectives, we evaluated both DHS-wide efforts and  efforts at four selected components to address employee morale—CBP,  ICE, TSA, and the Coast Guard. We selected the four DHS components  based on their workforce size and how their 2011 job satisfaction and  engagement index scores compare with the non-DHS average. The  components selected had scores both above, below, and similar to the  average: TSA—below average on both indexes, constituting 25 percent of  the DHS workforce; ICE—below average on both indexes, accounting for  9 percent of the DHS workforce; CBP—at the non-DHS average for  satisfaction and below on engagement, representing 27 percent of the  DHS workforce; and the civilian portion of the Coast Guard—at the non- DHS average for satisfaction and above on engagement, composing 4  percent of the DHS workforce. Together these components represent 65  percent of DHS’s workforce.

To evaluate how DHS’s employee morale compares with that of other  federal government employees, we analyzed employee responses to the  2011 FEVS. We determined that the 2011 FEVS data were reliable for  the purposes of our report, based on interviews with OPM staff, review  and analysis of technical documentation of its design and administration,  and electronic testing. We used two measures created by OPM—the  employee job satisfaction and engagement indexes—to describe morale  across the federal government and within DHS. We calculated these  measures for various demographic groups, DHS components, and work  groups, in order to compare morale at DHS and other agencies among  employees who were demographically similar, in part using statistical  models. Appendix I describes our methods and findings in more detail. In  addition, we interviewed employee groups about morale to identify  examples of what issues may drive high and low morale within DHS. We  selected the employee groups based on the size of the employee group  within each selected component, ensuring we met with employees from  employee groups that composed significant proportions of FEVS  respondents, such as screeners from TSA (61 percent of TSA  respondents) and homeland security investigators from ICE (33 percent  of ICE respondents). The comments received from these interviews are  not generalizable to entire groups of component employees, but provide  insights into the differing issues that can drive morale.

To determine the extent to which DHS and the selected components  identified the root causes of employee morale and developed action plans  for improvements, we reviewed analysis results, interviewed agency  human capital officials and representatives of employee groups, and  evaluated action plans for improving morale. To identify criteria for  determining effective root cause analysis using survey data, we reviewed  both OPM and Partnership for Public Service guidance for action planning  based on annual employee survey results. On the basis of these  guidance documents, we identified factors that should be considered in  employee survey analysis that attempts to understand morale problems,  such as use of demographic group comparisons, benchmarking results  against results at similar organizations, and the linking results of root  cause analyses to action planning efforts. We evaluated documents  summarizing DHS-wide and selected component root cause analyses of  the 2011 FEVS to determine whether the factors we identified were  included in the analyses. In addition, we interviewed DHS officials who  conducted the analyses in order to fully understand root cause analysis  efforts. To identify criteria for determining agency action plans we  reviewed OPM guidance for using FEVS results and previous GAO work  indicating agencies’ success in measuring performance. On the basis of  these guidance documents, we identified OPM’s six steps that should be  considered in developing action plans and identified three attributes that  were relevant for measuring action plan performance—linkage, clarity,  and measurable target. We compared the action plans with these criteria  to determine whether these items were included in the action plans. In  addition, we interviewed DHS and component officials to identify efforts to  leverage best practices for improving morale.

We conducted this performance audit from October 2011 through  September 2012 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.

section_title: Appendix III: DHS and Selected Component Steps Taken to Determine Root Causes of Morale Problems
subsections: section_title: DHS Efforts to Determine Root Causes of Morale Problems
paragraphs: Since 2007 DHS’s Office of the Chief Human Capital Officer (OCHCO)  has completed several efforts to determine root causes of morale DHS- wide.

Focus groups. In 2007 OCHCO conducted focus groups to determine  employee concerns related to employee morale. DHS’s focus group effort  probed for insights into four areas—(1) leadership, (2) communication, (3)  empowerment, and (4) resources—and highlighted concerns raised by  focus group participants in each of those areas. For example, within the  leadership area, OCHCO’s focus group analysis found that the Customs  and Immigration reorganization was a topic discussed by many of the  U.S. Customs and Border Protection (CBP), U.S. Immigration and  Customs Enforcement (ICE), and Citizenship and Immigration Services  (CIS) personnel, especially what they felt was a lack of mission  understanding on the part of their managers. According to the analysis,  non-supervisory participants expressed dissatisfaction with the  combination of three types of inspection functions to present “one face at  the border.”

One Face at the Border  For operations at ports of entry, in September 2003 CBP issued its plan for  consolidating the inspection functions formerly performed by separate inspectors from  the three legacy agencies—customs inspectors from U.S. Customs, immigration  inspectors and Border Patrol from the former Immigration and Naturalization Service,  and the agriculture border inspectors from the Department of Agriculture’s Animal and  Plant Health Inspection Service. The plan, referred to as “One Face at the Border,”  called for unifying and integrating the legacy inspectors into two new positions—a CBP  officer and a CBP agricultural specialist. The new CBP officer would serve as the  frontline officer responsible for carrying out the priority anti-terrorism mission as well as  the traditional customs and immigration inspection functions while also identifying and  referring goods in need of a more extensive agricultural inspection to the agricultural  specialist. CBP anticipated that having a well-trained and well-integrated workforce that  could carry out the complete range of inspection functions involving the processing of  individuals and goods would allow it to utilize its inspection resources more effectively  and enable it to better target potentially high-risk travelers. Together, CBP envisioned  the result to be more effective inspections and enhanced security at ports of entry while  also accelerating the processing of legitimate trade and travel.

Focus group results were distributed to DHS components for  consideration in action planning efforts, according to OCHCO officials.  CBP, CIS, TSA, the Federal Emergency Management Agency (FEMA),  and the Federal Law Enforcement Training Center each addressed at  least one of the focus group results relating to leadership, communication,  empowerment, or resources in subsequent action plans, according to  OCHCO officials.

Statistical analysis. In 2008 OCHCO performed statistical analysis of  Federal Employee Viewpoint Survey (FEVS) data, beyond examining  high- and low-scoring questions, in an effort to determine what workplace  factors drove employee job satisfaction. Specifically, the analysis involved  isolating which sets of FEVS questions most affect employee job  satisfaction. The analysis found that five work areas identified in FEVS  questions drive employee job satisfaction: (1) performance and rewards,  (2) supervisor support, (3) physical conditions and safety, (4) senior  leadership effectiveness, and (5) the DHS mission. According to OCHCO  officials, DHS components were encouraged to conduct follow-up  discussions at the lowest possible organizational level based on  component survey scores in each of the five work areas. However,  OCHCO officials stated that they are not aware of any results of this effort  because OCHCO did not track or follow-up with the components on the  effect of key driver discussions that may have occurred. In addition,  increased emphasis on supervisor performance management training  was also implemented as a result of the analysis, according to OCHCO  officials.

Exit survey. In 2011, DHS began administering an exit survey to  understand why employees choose to leave their DHS positions.  Specifically, according to OCHCO officials, the DHS exit survey was  designed to determine where departing employees were moving both  inside and outside of DHS, to identify barriers related to diversity, to  identify reasons that veterans may be leaving DHS, and to capture  feedback from interns. The 2011 exit survey found, among other things,  that 27 percent of departing employees who responded to the exit survey  were staying within DHS or moving to a different position, and an  additional 12 percent of respondents were retiring. Lack of quality  supervision and advancement opportunities were the top reasons  responding employees indicated for leaving their positions. Exit survey  results are shared with DHS components on a quarterly and annual basis.  2011 FEVS analysis. For the 2011 FEVS, DHS’s OCHCO evaluated the  results by comparing Human Capital Assessment and Accountability  Framework (HCAAF) index results by component. The analysis showed  where the lowest index scores were concentrated. As shown in figure 8,  lower scores across the indexes were concentrated among several  components, including Intelligence and Analysis, Transportation and  Security Administration (TSA), ICE, National Protection and Programs  Directorate, and FEMA.

The analysis also determined how DHS’s scores on the four indexes  trended over time and compared with governmentwide averages. As  shown in figure 9, DHS-wide scores have generally trended upward over  time, but continue to lag behind governmentwide averages for each  index.

Employee Engagement Executive Steering Committee (EEESC). In  January 2012 the DHS Secretary directed all component heads to take  steps to improve employee engagement through launch of the EEESC.  According to OCHCO officials, the EEESC was launched in response to  congressional concerns about DHS employee morale and the Partnership  for Public Service results showing DHS’s low placement on the list of Best  Places to Work. The EEESC is charged with serving as the DHS  corporate body responsible for identifying DHS-wide initiatives to improve  employee engagement, oversee the efforts of each DHS component to  address employee engagement, and provide periodic reports to the  Under Secretary for Management, Deputy Secretary, and Secretary on  DHS-wide efforts to improve employee morale and engagement.  Specifically, the Secretary made the following directives to component  heads: develop and assume responsibility for employee engagement  improvement plans,  identify and assign specific responsibilities for improved employee  engagement to component senior executive performance objectives,  identify and assign a senior accountable official to serve on the  EEESC,  conduct town hall meetings with employees, attend a Labor-Management Forum meeting, and provide monthly reports on actions planned and progress made to the  Office of the Chief Human Capital Officer.

As of August 2012, each of the Secretary’s directives were completed,  with the exception of assigning responsibilities for improved employee  engagement to Senior Executive performance objectives, which DHS  plans to implement in October 2012 as part of the next senior executive  performance period. The EEESC met in February 2012, and component  representatives shared their latest action plans and discussed issues of  joint concern. In preparation for the 2012 FEVS, the EEESC released a  memorandum from the Secretary describing the responsibilities of the  EEESC, highlighting department actions, and encouraging employee  participation in the FEVS, which began in April 2012. The EEESC also  agreed that a corresponding message should be released from  component heads outlining specific component actions taken in response  to past survey results and encouraging participation in the next survey. In  an April 2012 EEESC meeting, the Partnership for Public Service  provided a briefing describing the Best Places to Work in the Federal  Government rankings and best practices across the government for  improving morale scores. The EEESC members also discussed methods  for improving the response rates for the upcoming survey and engaged in  an action planning exercise designed to help identify actions for  department-wide deployment, according to OCHCO officials. As of  August 2012, EEESC action items were in development and had not  been finalized. According to OCHCO officials, the EEESC plans to decide  on action items by September 2012, but a projected date for full  implementation has yet to be established because the actions have not  been decided upon.

section_title: Components Have Also Conducted Some Root Cause Analyses Using FEVS Results
paragraphs: In addition to the DHS-wide efforts, the components we selected for  review—ICE, TSA, the U.S. Coast Guard (Coast Guard), and CBP— conducted varying levels of analyses regarding the root causes of morale  issues to inform agency action planning efforts. The selected components  each analyzed FEVS data to understand leading issues that may relate to  morale, but the results indicated where job satisfaction problem areas  may exist and do not identify the causes of dissatisfaction within  employee groups. A discussion of the four selected components’ 2011  FEVS analysis and results are described below.

TSA. In its analysis of the 2011 FEVS, TSA focused on areas of concern  across groups, such as pay and performance appraisal concerns, and  also looked for insight on which employee groups within TSA may be  more dissatisfied with their jobs than others by comparing employee  group scores on satisfaction-related questions. TSA compared its results  with CBP results, as well as against DHS and governmentwide results.  When comparing CBP and TSA scores, TSA found that the greatest  differences in scores were on questions related to satisfaction with pay  and whether performance appraisals were a fair reflection of  performance. TSA scored 40 percentage points lower on pay satisfaction  and 25 percentage points lower on performance appraisal satisfaction. In  comparing TSA results with DHS and governmentwide results, TSA found  that TSA was below the averages for all FEVS dimensions. TSA also  evaluated FEVS results across employee groups by comparing  dimension scores for headquarters staff, the Federal Air Marshals,  Federal Security Director staff, and the screening workforce. TSA found  that the screening workforce scored at or below scores for all other  groups across all of the dimensions.

ICE. In its analysis of the 2011 FEVS, ICE analyzed the results by  identifying ICE’s FEVS questions with the top positive and negative  responses. ICE found that its top strength was employees’ willingness to  put in the extra effort to get a job done. ICE’s top negative result was  employees’ perceptions that pay raises did not depend on how well  employees perform their jobs. ICE also sorted the primary low-scoring  results into action planning themes, such as leadership, empowerment,  and work-life balance. ICE found, among other things, that employee  views on the fairness of its performance appraisals were above DHS’s  average but that views on employee preparation for potential security  threats were lower. When comparing ICE’s results with average  governmentwide figures, ICE found, among other things, that ICE was  lower on all of the HCAAF indexes, including job satisfaction. According  to ICE human capital officials, future root cause analysis plans for the  2012 FEVS are to benchmark FEVS scores with those of similar law  enforcement agencies such as the Drug Enforcement Agency; Federal  Bureau of Investigation; Federal Law Enforcement Training Center;  United States Secret Service; Alcohol, Tobacco and Firearms, and the  U.S. Marshals.

CBP. In its analysis of the 2011 FEVS, CBP focused its analysis on  trends since 2006. For example, the analysis showed that CBP increased  its scores by 5 or more percentage points for 36 of the 39 core FEVS  questions. CBP highlighted its greatest increases in HCAAF areas, such  as results-oriented performance, which showed a 21 percent  improvement over 2006 responses to the question—my performance  appraisal is a fair reflection of my performance. The analysis also  identified areas in greatest need of improvement, which showed progress  since 2006 but continued low scores, such as questions on dealing with  poor performers who cannot or will not improve (28 percent positive),  promotions based on merit (28 percent positive) and differences in  performance are recognized (34 percent positive).

Coast Guard. In its review of high and low 2011 FEVS responses, the  Coast Guard identified employee responses to two questions that  warranted action planning items—(1) How satisfied are you with the  information you receive from management on what’s going on in your  organization (53 percent positive) and (2) My training needs are assessed  (51 percent positive).additional FEVS analyses that were used to inform action planning.

section_title: Appendix IV: Selected Components’ Data Sources for Evaluating Morale, Other than the Federal Employee Viewpoint Survey
paragraphs: Appendix IV: Selected Components’ Data  Sources for Evaluating Morale, Other than the  Federal Employee Viewpoint Survey Purpose   Identify why employees leave the  agency and where they are going.

Summary of results and how used  The number of exit survey respondents  from ICE was too low to identify any  results and have not been used to  address morale as of June 2012,  according to ICE officials.

Last conducted in March 2012, the  FOCS is a data-gathering tool for  addressing the extent to which  employees perceive their  organizational culture as one that  incorporates mutual respect,  acceptance, teamwork, and  productivity among individuals who are  diverse in the dimensions of human  differences. Additionally, ICE conducts  focus groups and individual one-on-one  interview sessions to obtain clarifying  information pertaining to the FOCS  results and written comments.

The survey showed low employee  perceptions of ICE as an organization  where people trust and care for each  other, relative to the federal average,  according to ICE officials. The results  from the FOCS and feedback from the  focus groups and individual one-on-one  interview sessions are provided to ICE  program offices with recommended  strategies to improve the program  office’s organizational climate.

Conducted in 2007, focus groups were  launched in response to the 2006  annual employee survey results, which  showed CBP below DHS and  governmentwide averages.

The focus groups identified employees’  perceived problems in specific work  environment areas, such as leaders  lacking supervisory or communication  skills.  Among other things, the issues  identified by focus group participants  allowed CBP to develop action plans  that addressed these issues, according  to CBP officials.

Most Valuable Perspective  online survey (MVP)

Launched in 2009, this survey was  implemented to solicit employee  opinions on one topic per quarter as a  mechanism for gathering further  insights on FEVS results. The MVP  was implemented as a continuation of  the CBP focus groups completed in  2007.

In the July 2012 MVP, which solicited  employee preferences for future CBP  webcasts to employees, employees  suggested retirement planning and  financial management as their top two  preferences. CBP’s action plan  planning process in response to FEVS  results includes consideration of MVP  results, according to CBP officials.

Data source  U.S. Office of Personnel  Management Organizational  Assessment Survey (OAS)

Purpose   Beginning in 2002, in order to provide  the granularity, detail, and reliability  needed to ensure the best  organizational value, the Coast Guard  adopted the OAS as its primary  personnel attitude survey, according to  Coast Guard officials. The OAS is  administered to military (active and  reserve) and civilian personnel  biennially.

Summary of results and how used  OPM’s report to the Coast Guard on  the 2010 OAS results identified seven  strong organizational areas (diversity,  teamwork, work environment,  leadership and quality, communication,  employee involvement and supervision)  and three areas for improvement  (innovation, use of resources, and  rewards/recognition).  Coast Guard unit commanders and  headquarters program managers use  the OAS to support overall Coast  Guard improvement. This improvement  is achieved by feeding results of the  OAS to Coast Guard Unit Commanders  and Program Managers who then use  OAS results in conjunction with other  information as part of routine unit and  program leadership and management.

Identify why employees leave the  agency, launched in 2005.

Top reasons for leaving overall were  personal reasons, career  advancement, management, schedule,  and pay. Each quarterly report includes  actions managers should take to  reduce turnover. A real-time reporting  system is also available for each airport  and office within TSA so managers can  gain access to their results and use  them to reduce turnover and make  improvements, according to DHS  officials.  Results from the exit survey were also  used by TSA officials in updating TSA’s  action plan, according to TSA officials.  However, the July 2012 action plan did  not link exit survey findings to action  items.

An online tool for gathering employee  suggestions for agency improvement.  Each week, approximately 4,000 TSA  employees log on to rate, comment, or  search, or to submit ideas of their own.  The Idea Factory team reviews all  submissions and uses Idea Factory  challenges to implement solutions to  issues.

Results were not available for our  evaluation.

Results were not available for our  evaluation.
subsections: section_title: Component
paragraphs: Purpose   Provides informal problem resolution  services with the mission of promoting  fair and equitable treatment in matters  involving TSA, according to TSA  officials. The Ombudsman assists  customers by identifying options,  making referrals, explaining policies  and procedures, coaching individuals  on how to constructively deal with  problems, facilitating dialogue, and  mediating disputes.

Summary of results and how used  Results were not available for our  evaluation.

Each airport and TSA headquarters  has an employee advisory council  made up of elected members who work  on understanding and addressing a  variety of workplace issues.

Results were not available for our  evaluation.

section_title: Appendix V: Comments from the Department of Homeland Security

section_title: Appendix VI: Comments from the U.S. Office of Personnel Management

section_title: Appendix VII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Dawn Locke (Assistant Director),  Sandra Burrell (Assistant Director), Lydia Araya, Ben Atwater, Tracey  King, Kirsten Lauber, Jean Orland, Jessica Orr, and Jeff Tessin made key  contributions to this report.


Question-summary annotations: questions: question: What steps has DHS taken to combat employee morale problems?
answer: DHS has taken steps to determine the root causes of employee morale problems and implemented corrective actions, but it could strengthen its survey analyses and metrics for action plan success. To understand morale problems, DHS and selected components took steps, such as implementing an exit survey and routinely analyzing FEVS results.
child_questions: question: What actions have the reviewed components taken?
answer: Components GAO selected for review--ICE, TSA, the Coast Guard, and Customs and Border Protection--conducted varying levels of analyses regarding the root causes of morale to understand leading issues that may relate to morale.
child_questions: question: How could these efforts be improved?
answer: Specifically, 2011 DHS-wide survey analyses did not include evaluations of demographic group differences on morale-related issues, the Coast Guard did not perform benchmarking analyses, and it was not evident from documentation the extent to which DHS and its components used root cause analyses in their action planning.

question: How does DHS measure the success of the action plans?
answer: In addition, GAO found that despite having broad performance metrics in place to track and assess DHS employee morale on an agency-wide level, DHS does not have specific metrics within the action plans that are consistently clear and measurable.
child_questions: question: How does this negatively affect DHS?
answer: As a result, DHS's ability to assess its efforts to address employee morale problems and determine if changes should be made to ensure progress toward achieving its goals is limited.

questions: question: How does DHS compare to other cabinet-level departments in the federal government?
answer: DHS is the third largest cabinet-level department in the federal government, employing more than 200,000 staff in a broad range of jobs.

question: How satisfied are DHS employees with their jobs?
answer: Since it began operations in 2003, DHS employees have reported having low job satisfaction.
child_questions: question: How important is this lack of job satisfaction?
answer: DHS employee concerns about job satisfaction are one example of the challenges the department faces implementing its missions.

question: What was GAO asked to review?
answer: GAO was asked to examine: (1) how DHS's employee morale compared with that of other federal employees, and (2) the extent to which DHS and selected components have determined the root causes of employee morale, and developed action plans to improve morale.
child_questions: question: How did GAO collect data for this report?
answer: To address these objectives, GAO analyzed survey evaluations, focus group reports, and DHS and component action planning documents, and interviewed officials from DHS and four components, selected based on workforce size, among other things.