# Presentation: 668023

## 2004 Customer Satisfaction Measurement Program 

**2004 Customer Satisfaction Measurement Program **
- National Weather Service
- December 7, 2004

## Who is CFI Group?

**Founded in 1988; headquartered in Ann Arbor, Michigan**
**13 offices, 150 full-time consultants and researchers worldwide**
**CFI system is the source for University of Michigan’s American Customer Satisfaction Index (ACSI)**

## What is the ACSI?

**The only standardized measure of customer satisfaction****covering more than 200 companies in 40 industries; about 75% of the U.S. economy**
- Including over 70 U.S. Federal Government agencies
- Next results for government released December 2004
**A quarterly snapshot of the national economy’s health **
**Results from published quarterly**
- For more information, see www.theacsi.org

## So what exactly is satisfaction?Our Philosophy

**Satisfaction is not just black and white, it’s a matter of degree**
**Knowing the score is nice... but you need to understand the underlying drivers of satisfaction in order to improve satisfaction**
**Satisfied customers are critical to the survival of any organization**
- Financial rewards in the private sector
- Leads to trust, economic benefits and potentially saving lives, as is the case with NWS
**Like you, our work is based in science**

## A Bit About the Math

***Just kidding....***

**Notes:**
The theoretical set up for a three block (3 component) model.
Most CFI Group models have multiple predictor variables and different dependent variables (CSI, Recommend, Loyalty, etc.). Those models are much more complicated versions of the three block model illustrated here.

## Survey Results

## NWS is committed to Customer Satisfaction

**Four segments measured in 2003**
- Aviation (pilots and dispatchers)
- Emergency Managers
- Marine/Tropical
- Media
**Three more segments measured in 2004**
- Hydrology (complete)
- Climate (in process)
- General public (to field in December/January)

## Hydrologic Services Program SurveyBackground

**Goals**
- Measure customer satisfaction with current products and services
- Gain insight for future areas of focus
**Administered via the web, September/October 2004**
- The NWS provided survey link to various customers
- Survey posted on the NWS web pages
- 2,345 English version responses
- 7 Spanish version responses

## Hydrology Receives High MarksSimilar to other NWS scores; above Fed Gov ACSI

**Scores on 0-100 scale**

## Praise from respondents ...

**Respondents recognize the NWS for the invaluable, life-saving information they provide, and your commitment to continuous improvement...**

_*I'd be lost without your valuable information to forward to all my local EMS directors and public safety officials*_*.  They depend on my providing them with this information throughout the year.*

## Who are the survey respondents? Considerable number of your customers are “personal users”

## Satisfaction varies by role Some groups less satisfied

**Customer Satisfaction Index**

**n=59**

**n=632**

**n=934**

**n=82**

**n=149**

**n=117**

**n=42**

**n=51**

**n=39**

**n=182**

**n=21**

**Notes:**
Note that some of the segments have low sample size.  

## How do customers get information? Vast majority get information via the web; television also a source 

**Notes:**
51% of personal users have NOAA weather radio
Phone: 33% of EM, 2% of personal

## What information are respondents using? Flood and precipitation dominate 

**Notes:**
EM use more of everything

## What do customers think about products/services?All products perform well

**Notes:**
For each component (light blue rectangle) 3-5 questions is asked.  
Clarity, Conciseness, Timeliness, Meets needs, Accuracy, Organization

Impacts: change in sat resulting from 5-point change in component score: Flood 1.7, Precip 1.1, Water Supply 0.9, Rec 0.7, River 0.5, Drought 0

Customer Satisfaction Index is three questions:
Satisfaction overall
Satisfaction compared to expectations
Satisfaction compared to ideal

## More praise from respondents ...

***I think that NWS is doing a great job of keeping up-to-date with its presentation of data . . . ***_***the data is being presented in an easier to understand format than in the past***_***.**** *
***... ******information ... as accurate as the data allows. ***_***Our local decisions that have been based on this information have been greatly aided and influenced by their work.***_

## Format / Graphics

## Customer opinions of graphics varyMore complicated ‘river stage’ graphics score lower

**Score based on three questions: visual appeal, ease of understanding, tells me what I need to know about . . . **

**Significantly **
**Lower Scores**

**“****Most, if not all of the graphics on the last few pages were way too difficult to understand much less interpret ... I don't even have the time to try and understand them so whatever you can do **_**must be much simpler**_**.”**

## Customer opinions of graphics varyLow Scoring Graphics

- Customer opinions of graphics vary*Low Scoring Graphics*

**Weekly Probabilities**
***Score: 70***

**Exceedance Probability**
***Score: 68***

**Hydrograph w/Error Bands**
***Score: 69***

**“****Most, if not all of the graphics on the last few pages were way too difficult to understand much less interpret ... I don't even have the time to try and understand them so whatever you can do **_**must be much simpler**_**.”**

**Notes:**
visual appeal, ease of understanding, tells me what I need to know about....

## Customer opinions of graphics varyHigh Scoring Graphics

**WFO Map**
***Score: 86***

**National Precipitation Analysis**
***Score: 83***

**Notes:**
visual appeal, ease of understanding, tells me what I need to know about....

## Summing it up

**Customers are generally happy**
**Flood Information and Precipitation Information are most used, and most critical to overall satisfaction **
**Different groups have different needs**
- Consider different products targeted to different user groups
**Simplify graphics where possible, particularly those related to uncertainty/probability**
- Uncertainty/probability graphics less clear, do not provide information as well as other graphics; revision is necessary

## Contact information

- Sheri Teodoru
- CFI Group
- 734-623-1318
- steodoru@cfigroup.com

## Appendix

## Scores for GraphicsWFO Map

_**Average:  86**_
**86:  Visual Appeal  n=2295**
**86:  Ease of Understanding  n=2291**
**86:  Tells me what I need to know about river conditions n=2292**

## Scores for GraphicsNational Precipitation Analysis

_**Average:  83**_
**84:  Visual Appeal  n=2247**
**83:  Ease of Understanding  n=2234**
**83:  Tells me what I need to know about national precipitation n=2211**

**58% currently use the combined NWS rain gauge and radar data within the national precipitation analysis**

## Scores for Probability GraphicsWeekly probabilities

_**Average:  70**_
**71:  Visual Appeal  n=2118**
**69:  Ease of Understanding  n=2111**
**71:  Tells me what I need to know about a given river stage n=2081**

**71:  EMS**
**71:  Personal Use**
**70:  All Others**

## Scores for Probability GraphicsHydrograph with error bands

_**Average:  69**_
**69:  Visual Appeal  n=2125**
**68:  Ease of Understanding  n=2122**
**71:  Tells me what I need to know about a river stage n=2081**

**70:  EMS**
**69:  Personal Use**
**69:  All Others**

***Scores for probability graphics are significantly lower***

## Scores for Probability GraphicsExceedance probability

_**Average:  68**_
**69:  Visual Appeal  n=2105**
**65:  Ease of Understanding  n=2102**
**69:  Tells me what I need to know about river stages n=2047 **

**69:  EMS**
**67:  Personal Use**
**66:  All Others**

## GraphicsWhat respondents had to say ...

**Data Precision**
*Graphical data needs to be *_*updated more often*_*.  Now it is updated once every twenty-four hours.  I think the NWS is on the right track with the idea to modernize.*
*I need to know water levels, and flood stage levels in *_*‘near real time’*_* and I *_*do not need all of this frilly stuff*_*.*
**Usability**
*Because we deal with the public on a daily basis, the information provided must be easily understood at a *_*reading level of 4th grade*_*. *