# Presentation: 047241

**The Impact of Social/Cultural Factors on Tornado Warning Performance**
**Dr. Stephan Smith  **
**National Weather Service**
**Dr. Lou Mischkind and Shawn Del Duco**
**Sirota Corporation**

**“****Science does not float upon the winds, **
**but is inevitably grounded ****in human relations.”****Professor Gary Fine**

## o

- Constant = -131.1915
- Coefficient = 0.0708
- Rsqr = 0.053

***U95***

***Trend***

***Actual***

***L95***

- T-value for slope = 0.53
- 2-tailed t-test 95% CI w/ 5 degrees of freedom = 2.57

**o**

**o GPRA GOAL**

## o

- Constant = 10.0956
- Coefficient = -0.0047
- Rsqr = 0.127

- T-value for slope = -0.85
- 2-tailed t-test 95% CI w/ 5 degrees of freedom = 2.57

***U95***

***Trend***

***Actual***

***L95***

**o**

**o GPRA GOAL**

## o

- Constant = -31.8362
- Coefficient = .0163
- Rsqr = 0.623

- T-value for slope = 2.87
- 2-tailed t-test 95% CI w/ 5 degrees of freedom = 2.57

***U95***

***Trend***

***Actual***

***L95***

**o**

**o GPRA Goal**

## Questions asked by National Weather Service Management

- How do we improve our tornado warning verification statistics?
- How will we meet our GPRA goals for tornado warnings?
- What do we invest our resources in?

## Examine  the Top Ten Offices

## *

- Constant = -131.1915
- Coefficient = 0.0708
- Rsqr = 0.053

***U95***

***Trend***

***Actual***

***L95***

- T-value for slope = 0.53
- 2-tailed t-test 95% CI w/ 5 degrees of freedom = 2.57

*** Top Ten WFO’s**

**o**

**o GPRA GOAL**

## *

- Constant = -31.8362
- Coefficient = .0163
- Rsqr = 0.623

- T-value for slope = 2.87
- 2-tailed t-test 95% CI w/ 5 degrees of freedom = 2.57

***U95***

***Trend***

***Actual***

***L95***

*** Top Ten WFO’s**

**o**

**o GPRA GOAL**

## *

- Constant = 10.0956
- Coefficient = -0.0047
- Rsqr = 0.127

- T-value for slope = -0.85
- 2-tailed t-test 95% CI w/ 5 degrees of freedom = 2.57

***U95***

***Trend***

***Actual***

***L95***

*****

*** Top Ten WFO’s**

**o**

**o GPRA GOAL**

## Why are the Top Ten Offices doing so well?

- Better Science and Technology?  No.
- Easier Verification?  No.
- Easier Storms?  No.
- More Practice?  No.
- Better at Managing Diversity???

## Hypothesis

- A tornado warning is arguably the most challenging of all
- products issued by NWS forecasters.  Sustained, high
- performance in tornado warnings, requires a highly-trained,
- dedicated staff who can work well as a team under very stressful
- conditions.
- If the benefits of an inclusive, supportive, and flexible
- work environment, where managers foster personal
- development, teamwork and empowerment, among their
- employees, are to be seen, it would be in tornado warning
- verification statistics.
**Offices that score high in tornado warning verification **
**statistics will also score high in SFA results compared **
**to offices with poor tornado verification statistics.**

## Methodology

- Obtained 2000 & 2001 Tornado Warning Statistics for each NWS forecast office
- Ranked all forecast offices by skill
- Requested a special aggregate report of NOAA’s all-employee survey (SFA) for the Top 10 and Bottom 10 forecast offices in skill
- Compared the two reports for significant differences

## Supervision

## Supervision

## Supervision

## Supervision

## Fairness

## Fairness

## Fairness

## Fairness

## Communication

## Innovation

## Supplemental Questions

## Supplemental Questions

## Supplemental Questions

## Leadership

## Leadership

## Employee Involvement

## Additional Experiences

## EEO

## Findings from Phone Interviews of Top Ten Forecast Offices

- Manager leadership demonstrated through action (working shifts, severe weather) often in a subordinate role
- Managers do not micro-manage severe weather operations
- Some managers had anti-role models
- Managers back up their forecasters’ decisions
- Managers foster open dialog
- Managers are careful in hiring people to enhance the existing team (consider both skills and personality)
- Office commitment to improvement

## Findings from Phone Interviews of Top Ten Forecast Offices

- Managers work closely with the union representatives
- Managers support family/personal needs
- Offices seem to have low staff turnover
- Managers have strong focus on making the work satisfying and enjoyable for their employees
- Managers work to make sure that all employees are appreciated (both mets and non-mets)
- Managers reward quickly and often
- Management team support manager’s goals

## NWS Forecast Offices and Regions *

**** Offices west of the Rockies were excluded from our analyses, as well as those reporting fewer than five tornado events in 2001/2002 and those with fewer than five employees completing the survey.***

## Weather Forecast Office (WFO) Culture

- Characteristics
- “Family” unit of 20-30 people, isolated from other offices
- Relatively homogenous in gender, ethnicity, age
- Experienced (10-25 yrs)
- Unionized – can be contentious
- High value placed on Science and Technology and Dedication to Mission

## Which Storms are Tornadic?

**Has the radar identified a strong storm?**
**Are there signs of rotation ****in the storm?**
**Are there spotter reports ****of a funnel with the storm?**
**Have other storms ****in the area produced tornadoes?**
**Where is the storm heading?**
**Will it remain tornadic?**

***Research Question:  How do we explain these CSI performance differences for tornado warnings across WFO’s?***

**Poor Performance**

**Better Performance**

**Number of Weather ****Forecasting Offices**

- Variation in Critical Success Index (CSI) Scores Across Offices

**.445 - .488**

**.403 - .445**

**.360 - .403**

**.318 - .360**

**.275 - .318**

**.233 - .275**

**.190 - .233**

**.148 - .190**

**.105 - .148**

**.063 - .105**

**10**

**8**

**6**

**4**

**2**

**0**

## Regression Analysis

**Unknown factors**

**Accommodations ****for disabled**

*** Results based upon stepwise regression analysis**

**Performance appraisal on schedule**

**Sick leave**

**Know process for voicing complaint through union**

***Nearly half of the differences in WFOs’ performance ******are accounted for by four variables:***

## Conceptual Model

**CSI**
**(Performance)**

**Sick leave**

**Accommodations for disabled**

**Performance appraisal on **
**   ****schedule**

**Know process for voicing complaint through union**

**0.283***

**0.310***

**0.263***

**-0.355***

*** Values are Standardized Beta coefficients**

**HR data**

**Survey data**

***The most important factors in tornado warning performance reflect managerial effectiveness: Performance Orientation and Employee Relations***

## Highest SFA Correlates of Sick Leave(R= -0.30 to -0.45; Significance: 0.01 )

**1.****	****Supervisor behavior **
**2.****	****Work group cooperation and teamwork **
**3. ****	****Emphasis on performance and diversity**
**	**

- THE SICK LEAVE FACTOR

## THE SICK LEAVE FACTOR (cont.)

**	****People can become ill for all sorts of reasons and one would expect to find *****NO CORRELATION***** between these factors and sick leave.  **
**	****That fact that they are *****STRONGLY NEGATIVELY CORRELATED WITH SICK LEAVE***** suggests that in offices where employees are unhappy about their supervisor’s behavior, the lack of cooperation/teamwork and the lack of emphasis on diversity and performance, employees are taking more *****false sick leave***** and/or *****getting sick because of their work environment*****.    This in turn adversely affects office tornado warning performance.**

## NWS Tornado Warning Performance

**CSI Score**

**CSI Trend Line**

**2007 ****NWS Goal**

**X**

## NWS Tornado Warning Performance

**CSI Score**

**2007 ****NWS Goal**

***WFOs performing best on cultural variables have reached the NWS goal four years ahead of schedule:***

**Top Third of WFOs on Culture Index***

**Bottom Third of WFOs on Culture Index***

***The Culture Index comprises the following items: Performance** **appraisal on schedule, Accomodations for disabled and Know process for voicing complaint through union**

**X**

**.21**

**.34**

## Conclusions

**Improvements in technology are extremely important to improve tornado warning performance. They promise to raise the performance of *****all***** offices. **
**In addition, we have found that leadership in *****individual *****National Weather Service offices also has a demonstrable impact on performance. **
**In fact, the quantitative goal of excellence the National Weather Service has set for itself could be achieved by attending to these cultural variables alone. **

## Conclusions (cont.)

**Where diversity is managed better, a better job is done carrying out the National Weather Service mission!**
**Where diversity is managed better, the cost of carrying out the National Weather Service mission is reduced!**
**SFA 2002 results show that the National Weather Service should focus on aligning the management practices in the all our offices with the NOAA Diversity Vision.  An improvement strategy based only on science and technology without an aggressive managing diversity component is likely to fall short of the mark. **
**This is the Business Case for Managing Diversity for NOAA’s National Weather Service**

- Highly Accurate  Warnings
- Low Sick Leave

- High Performance
- Workgroup

- Low Performance
- Workgroup

- CULTURE
- Flexible Policies/Procedures
- Teamwork
- Open Communication
- Focus on Performance
- Goals Set and Tracked
- Strong Customer Orientation
- Emphasis on Innovation
- Trust and Respect
- Good Relations with Union

- CULTURE
- Rigid Policies/Procedures
- Unresolved Conflict
- Climate of Fear
- Lack of Empowerment
- Poor Sense of Goals
- Lack of Customer Focus
- Resistance to Change
- Ignorance of Diversity Issues
- Poor Relations with Union

- Moderately Accurate  Warnings
- High Sick Leave

## Bottom Line:

- The key to NOAA success lies with you,
- its middle managers.
- You are on the front lines leading NOAA employees.
- You can tap into NOAA’s unrealized potential.
- You can make the NOAA Diversity Vision a reality.
- NOAA’s Managers can make a BIG DIFFERENCE!
- And the PAYOFF IS GREAT!

## Question

- Knowing  that Diversity Issues have a significant
- impact on our ability to perform NOAA’s Mission
- and we are being held accountable to meeting our
- GPRA performance measures...
- What are we going to do about it?
- What are *YOU* going to do about it?