# Presentation: 060022

## National Weather Service

**National Weather Service**
**Marine/Tropical Survey**
**Presentation of****Customer Satisfaction Results**
- March 16, 2004

## Introduction

## Snapshot of the American Customer Satisfaction Index (ACSI)

- The #1 national economic indicator of customer satisfaction
- Compiled by the National Quality Research Centerat University of Michigan since 1994 usingmethodology licensed from CFI Group
- Measures 30 industries, 180+ organizations  covering 75% of the U.S. economy
- Over 70 U.S. Federal Government agencies have used ACSI to measure more than 120 programs/services
- Advanced methodology quantifiably measures and links satisfaction levels to performance and prioritizes actions for improvement
- Results from all surveys are published quarterly in *The Wall Street Journal*

## Scope of the 2003 NWSCustomer Satisfaction Program

**Four segments measured in 2003**

## Marine Customer Satisfaction Results

## Project Background

**Target segment:** Users of Marine and Coastal, Tropical, and/or Great Lakes products and services, including recreational boaters, coastal residents, beachgoers, and professionals.
**Survey content:** Survey designed over July and August 2003, with feedback from process owners in NWS (Silver Spring and regions).
**Survey method: **The survey was conducted online and hosted on CFI Group’s secure server.
**Customers contacted: **NWS publicized the survey to interested parties and provided links to the survey on multiple web sites, including the National Hurricane Center.
**Response totals:** The online survey accepted responses from September 9 through October 6, 2003.  Completed a total of 6298 interviews.

## Completed Surveys, by Date

**Completed Surveys, by Date**

**Notes:**
The response rate to the web link was much higher than anticipated, spurred in part by Hurricane Isabel. In order to ease pressure on the survey, and to make sure more non-hurricane-related responses were collected, the link was removed from the NHC site.

## Who Are the Marine Respondents?Services Used

**Notes:**
Tropical and Marine/Coastal services are by far the most popular among the respondents. 

## Who Are the Marine Respondents?Primary Operating Area

**Notes:**
Relatively sparse representation of Alaska/Pacific/West Coast. Note that Alaska had a total of 11 responses. Internet penetration in Alaska may be lower than for the other regions.

## Who Are the Marine Respondents?Primary Marine Activity

**Who Are the Marine Respondents?*****Primary Marine Activity***

**Notes:**
Boaters/residents/beachgoers are the most common types of respondent.

## Who Are the Marine Respondents? Most-Used Products: Marine and Coastal

**Notes:**
Chart shows the proportion of Marine/Coastal service users who use the products. High Seas and Surf Zone forecasts are the least-used product among the sample.

## Usefulness of Dissemination Methods

**Usefulness of Dissemination Methods**

**97% of respondents rate land methods**
**58% of respondents rate sea methods**

**Notes:**
For sea users, broadcast methods (NOAA radio, U.S. Coast Guard) are the most popular methods for getting weather information. All other methods are rated as much less useful.

For land users, internet methods dominate, followed by weather radio and local/cable tv/radio. U.S. Coast Guard broadcasts are much less critical for the land users. Note that the high score for Internet access may be due in part to the survey being conducted via the internet.

Overall, EMWIN and NOAA weather wire/FOS are seen as less useful. Phone, private vendor, radiofax, NAVTEX, and Immarsat are seen as much less useful. Keep in mind that these scores are overall, and there may be key segments where the ‘less useful’ methods are critical for operations.

## CSI Comparison across Government Agencies

**Customer Satisfaction Index (CSI) scores are based on three questions:****    *****•  Overall satisfaction with products and services******    •  Products and services compared to expectations******    •  Products and services compared to ideal ***

**Notes:**
All NWS segments score very well. The green bars represent comparisons to ACSI scores to ‘business-to-consumer’ type government agencies.

## Satisfaction ScoresServices Used

**Notes:**
Scores are good across the board. Great Lakes and Marine/Coastal trail slightly.

## Satisfaction ScoresPrimary Operating Area

**Notes:**
Scores are high across all operating areas. No score is reported for Alaska due to the low sample size. Tropical Atlantic/Atlantic/Gulf of Mexico users report the highest satisfaction.

## Satisfaction ScoresPrimary Marine Activity

**82**

**81**

**75**

**73**

**81**

**81**

**79**

**79**

**78**

**78**

**Law enforcement**

**Passenger transport**

**Coastal resident**

**Beachgoer**

**Science/research**

**Search and rescue**

**Freight transport**

**Other**

**Recreational boater**

**Commercial fishing**

**1% of resp.**

**34% of resp.**

**15% of resp.**

**1% of resp.**

**2% of resp.**

**4% of resp.**

**12% of resp.**

**29% of resp.**

**1% of resp.**

**1% of resp.**

**Notes:**
Again we see somewhat lower scores for boating/fishing. Residents and beachgoers are very satisfied.

## NWS Marine CSI Model

**Notes:**
Inputs represent the main drivers of customer satisfaction (CSI). All components are measured with questions from the survey, some of which are included in the picture (not all questions are included due to space limitations).

## NWS Marine Model

**Confidence in**
** ****NWS**

**Take Action**

**Contact NWS**

**INPUTS**

**SATISFACTION**

**OUTPUTS**

**Great Lakes Marine Forecasts**

**Great Lakes Short-Fused**
**And Other Forecasts**

**Great Lakes Ice Forecasts**

**NHC Forecasts**

**Tropical Warnings and Watches**

**Customer**
**Satisfaction**

**Hurricane Local Statements**

**Coastal /Offshore/High Seas Forecasts**

**Marine/Coastal Short-Fused and **
**Other Forecasts**

**Coastal Flood and Surf Zone Information**

**Marine Forecast Accuracy**

**General Forecast Performance**

**General Forecast Availability**

**General Forecast Verification**

**Notes:**
Note that a zero impact does not imply “not important”. It only means that at this time, improvements in that area will not have a direct effect on Customer Satisfaction; other areas are more in need of improvement. An example of a very important but low impact area is safety on an airline. Safety is of paramount importance, but has little impact on customer satisfaction because safety expectations are being met very well.

## Priority Matrix: Where to Focus Attention

**Notes:**
The Priority Matrix sorts the components by score and impact, leaving higher impact, poorer performing areas in the lower right-hand quadrant of the matrix.  These are ‘Top Priority’; they are the areas most in need of remediation (in this case, Marine Forecast Accuracy and General Forecast Verification).  The ‘Building on Success’ section represents areas that perform well yet still have substantial leverage on satisfaction.  These aspects may be more difficult to improve because of their relatively high scores, but any improvements would pay dividends.  The ‘Maintain’ quadrant contains higher scoring, lower impact components.  These areas are performing reasonably well, but at the current time are not large drivers of media personnel customer satisfaction.  The quadrant in the lower left corner, ‘Monitor’, contains components with low scores and low impacts.  These are areas that may prove to be more critical when other areas are remedied, but at this time they do not have substantial leverage on satisfaction.

General Forecast Verification represents only a single question on the questionnaire (number of times an event occurs within 6 hours of being predicted, rated from Never to Always). And while the score of 69 is low compared to other areas, in absolute terms the score may be at or near its practical peak.

Great Lakes Ice Forecasts receive the lowest scores, but their impact on CSI is low.

## Marine Forecast AccuracyImpact on CSI: 0.9

**Notes:**
Customers are reasonably positive on forecast accuracy. Wave forecasts are seen as the least accurate.

## NHC ForecastsImpact on CSI: 0.9

**The usefulness of additional probabilistic information is rated an 86 – very useful**
**When asked which area needs improvement most, 51% of respondents say ‘Track’. No other feature is mentioned more than 11% of the time.**

**Notes:**
All scores are excellent (80+); ease of understanding is the only possible weakness. The high impact for NHC Forecasts could possibly be related to the fielding of the survey during Isabel.

## General Forecast PerformanceImpact on CSI: 0.7

## Coastal/Offshore/High Seas ForecastsImpact on CSI: 0.6

## Tropical Warnings and WatchesImpact on CSI: 0.5

**Most-mentioned minimum desired probability associated with a watch: 30%**
**Most-mentioned minimum desired probability associated with a warning: 70%**
**50% of respondents prefer break points over county boundaries, and 28% are indifferent**

## General Forecast AvailabilityImpact on CSI: 0.4

**Notes:**
Update frequency was also an issue in the NWS Media study.

## Marine and Coastal Short-Fused/OtherImpact on CSI: 0.4

## Coastal Flood and Surf Zone InfoImpact on CSI: 0.0

## Other Aspects of NWS Performance

**Notes:**
Respondents are OK with the use of the English units, though they recognize the limited availability of other languages. The 500 millibar and wave period forecasts are not seen as very useful.

## What Customers Want

**Customers express a clear preference for graphical forecasts.**
**A constantly-evolving forecast is preferred over a regularly-scheduled forecast, 79% to 21%.**

**Notes:**
Respondents were asked which new products they preferred; they could select one or all. Graphical forecasts are the most popular.

## What Customers Want (cont.)

**When asked if they want any weather information not currently provided by NWS, 82% say no.**
**Needs are varied, from better barometric readings to more accurate lightning forecasts to better wave forecasts.**

**Notes:**
Customers rated their preference for the different elements of the new suite of products. 

## Recommendations

- Keep up the good work! (e.g., Isabel forecasting)
- Improve perceived forecast accuracy by increasing the number of official forecasts
- Give customers more information to explain the forecast data
- Educate customer regarding reasonable expectations, especially with wave height and period forecasts
- As in the Media survey, improve the consistency of Area Forecast Discussions and terminology use
- Ensure storm track is a top priority for tropical/hurricane forecasts
- Focus on graphical products, but keep in mind that radio is still a dominant method of information acquisition
- Examine possible causes of lower scores for Great Lakes Ice and Coastal Surf Zone forecasts
- Consider how different regions/users may respond to the survey differently
- These overall findings will have varying applicability to specific sub-populations

## Discussion

## For More Information

- Brian LaMarre, NOAA
- Marine Program Manager
- (301) 713-1677 x108
- Brian.LaMarre@noaa.gov
- Doug Helmreich, CFI Group
- Senior Consultant
- (734) 623-1321
- dhelmreich@mail.cfigroup.com
- Bernie Lubran, Federal Consulting Group
- Project Manager
- (202) 906-5642
- bernie.lubran@ots.treas.gov