# Presentation: 039767

## Developing an Air Quality Forecasting Program

- Key elements of forecasting programs
- Forecast audience
- Resources needed to get the job done
- Developing air quality forecast tools
- Verification

## Key Elements – Getting Started (1 of 2)

- In the beginning:
- Identify your organization's commitment
- Evaluate resources:  budget, manpower, and time availability
- Identify forecast target audience and their needs
- Define a goal for the forecasting program
- Health-based notification
- Voluntary action days
- Mandatory emissions abatement measures
- Consider these issues
- Size of forecast domain
- Population affected
- Pollutants to forecast
- Industries to be controlled
- Smog transport

## Define resources
In-house staff
Contractors
Infrastructure
Set realistic forecasting goals
Public outreach is #1	
Consistent, reliable product
Don’t expect to be correct all of the time
Perception counts (e.g., visibility reductions)	
Set reasonable forecast accuracy goals

- In-house staff
- Contractors
- Infrastructure
- Set realistic forecasting goals
- Public outreach is #1
- Consistent, reliable product
- Don’t expect to be correct all of the time
- Perception counts (e.g., visibility reductions)
- Set reasonable forecast accuracy goals

- Key Elements – Getting Started (2 of 2)

## Key Elements – Program Sophistication

- Basic programs
- Climatology and persistence
- Warn public of poor air quality
- Mid-range programs
- Use of an objective method (i.e., regression) combined with persistence or climatology
- Use of meteorological forecast data
- Invoke action days
- Make agricultural/wildland burning decisions
- May require interagency coordination
- High-end programs (like Mid-range except)
- Use of multiple forecasting methods
- May incorporate rule-defined emissions abatement actions with compliance

## Key Elements – Forecasting Timeline

- Daily vs. weekdays with extended weekend forecast
- Seasonal
- Spring/summer ozone (6 months)
- Winter CO and NO2 (3-4 months)
- Year-round PM10 & PM2.5
- Spring:  windblown dust
- Summer/fall:  photochemical particulates
- Fall:  forest fires and agricultural burning
- Winter:  wood smoke

## Forecast Audience

- Internal:  identify the chain of communication
- Air Pollution Control Officer (forecast is made in their name)
- Air quality outreach (public information, media links)
- Information management (communication network– fax, internet, telemetry, etc.)
- Compliance personnel
- External:  target to forecast audiences
- Public through AQI & action day notices
- Schools & day care (outdoor programs)
- Cities & hospitals
- Industry (for emissions abatement actions)
- Media (the longest weather segment on TV is less		 than 3 minutes – air quality is a sound-bite)

## Forecast Audience – Message Formats

- Forecast messages are typically split by level of user understanding and desired response
- Routine message (Daily Forecast)
- Consistent format in appearance
- Consistent content (the message can be repeated)
- Standardized time of issuance (e.g., 11:00 a.m. daily)
- Special messages (Departure from routine)
- Urgent forecast update/alert imminent
- Smoke statement/wildland fires or wood smoke
- Fumigation
- Dust storms

## Forecast Audience – Message Content

- Dependent on target audience
- Key elements for schools and the public
- Identify pollutant
- Identify locations impacted
- Forecasted air quality category/alert level
- Forecasted AQI
- Duration of impact (beginning & end time)
- Recommended actions to be taken
- Industry – actions to be implemented (yes or no)
- Media – keep it simple, just AQI category

## Forecast Audience – General Forecast

-                    SOUTH COAST AIR QUALITY MANAGEMENT DISTRICT
-                            DAILY AIR QUALITY FORECAST
-                           VALID:  WED., FEB.  2, 2000
- --------------------------------------------------------------------------------
- SRA         AREA            1-HR   8-HR    8-HR   24-HR  24-HR  24-HR   MAX
- NUMBER                     OZONE  OZONE      CO    PM10  PM2.5   NO2    AQI
-                              PPM    PPM     PPM   UG/M3  UG/M3   PPM
- --------------------------------------------------------------------------------
- Los Angeles County:  South Coast Air Basin
-  1   Central LA Co           .03    .02     4.5     54     37    .09     93
-  2   NW Coastal LA           .05    .04     6.9     53     36    .09     91
-  3   SW Coastal LA           .04    .03    13.1     52     36    .08    165
-  4   S Coastal LA            .04    .03     5.1     52     36    .07     91
-  5   Southeast LA Co         .04    .03     3.8     60     39    .07     97
-  6   W San Fernando Vly      .05    .04     4.2     44     32    .05     83
-  7   E San Fernando Vly      .01    .01     7.9     44     32    .10    100
-  8   W San Gabriel Vly       .03    .02     4.6     51     25    .06     69
-  9-1 E San Gabriel Vly-1     .03    .03     3.1     49     23    .04     65
-  9-2 E San Gabriel Vly-2     .04    .03     4.5     53     26    .05     71
- 10   Pomona Walnut Vly       .04    .03     3.8     58     28    .09     90
- 11   S San Gabriel Vly       .05    .04    11.4     38     18    .09    140
- 12   S Central LA Co         .04    .03     2.1     35     28    .05     75
- 13   Santa Clarita Vly       .05    .04     2.1     35     17    .02     53
- 15   San Gabriel Mts         .04    .03     3.0     49     23    .06     65

## Forecast Audience – Focused Forecast (1 of 2)

## Forecast Audience – Focused Forecast (2 of 2)

## Forecast Audience – Media Summary

| Area | AQI | Category | Pollutant |
| --- | --- | --- | --- |
| Coastal | 45 | Good | Carbon Monoxide |
| Metropolitan | 75 | Moderate | Nitrogen Dioxide |
| Inland Valleys | 125 | Unhealthy-Sensitive | Ozone |
| Low Desert | 105 | Unhealthy-Sensitive | PM10 and PM2.5 |

## Resources – Staffing

- Getting started (problem dependent)
- 2 staff members
- 1⁄2 to 1 day forecaster time commitment
- Mature program
- 2-3 staff members
- 1⁄4 to 1⁄2 day forecaster time commitment
- Other internal staff
- Monitoring & telemetry (full-time operations but		 variable degree of commitment)
- Outreach & communications, 1⁄4 to 1⁄2 person per day

## Resources – Daily Operational Manpower Requirements

## Resources – Infrastructure

- Old Days
- Teletype machines (NWS data services)
- Wet paper facsimile
- Telephones
- Radio broadcast
- Special monitoring (sounding program)
- Today
- Internet (data/maps/NWS products/AIRNow)
- Contracted weather vendors
- Auxiliary monitoring (e.g., Doppler radar, etc.)

## Resources – External Communications

- Web site to host forecasts
- Automated voice recording capability
- Links to other agencies
- Remote data transfer (AIRNow/state agencies)
- Post forecast to electronic mailboxes
- Media outreach porthole
- Special message formats
- Access to selected host computers

## Developing Forecast Tools – What’s Good For My Application?

- Start simple and build over time
- Goal to establish reliable product (it’s not necessary to be totally accurate)
- Assess weekday/weekend effect
- Persistence-based forecast is easiest to implement and will satisfy many applications
- Persistence, time series, and climatology will never identify a significant change in air quality
- Analog point systems, regression, neural networks, and pattern recognition require time to develop and validate but are usually more accurate than persistence

## Developing Forecast Tools – General Process (1 of 2)

- Develop understanding by reviewing past episodes
- Large-scale weather patterns
- Transport
- Special local phenomena (emissions anomalies)
- Develop prediction database
- Meteorological data
- Air quality data
- Ideally 3-5 years of data (more data may bridge trend, less data may be insufficient for analysis)
- Sort data for independent validation and quality assurance

## Developing Forecast Tools – General Process (2 of 2)

- Design a model that fits your forecast profile
- Consider your infrastructure
- Consider your forecasters
- Start with same-day algorithms
- Reinforce understanding of past episodes
- Use direct meteorological predictions to extend to next-day forecasts
- Build day-in-advance algorithms
- There is no set blueprint – use all reasonable and understandable methodologies
- Link to available prognostic products (NWS model output, MOS, MM5, etc.)

## Developing Forecast Tools – Regression

## Developing Forecast Tools – Pattern Recognition

## Developing Forecast Tools – Persistence and AQ Trends

## Developing Forecast Tools – Data Considerations

- Develop the forecast tool using data that are routinely available
- Establish data communications protocol
- Identify and set Internet links
- Use data vendors
- Develop monthly meteorological climatology for backup
- Note that monitored air quality data don’t need extensive Q/A for daily forecasting

## Developing Forecast Tools – Importance of Forecast Data

## Verification 

- You are typically as good as your last forecast!
- Daily verification
- Can identify systematic problems
- Can identify mistaken analysis of events
- Can identify problems with data
- Provides opportunity for mid-season procedure 	corrections
- Seasonal verification
- Identifies if model/methodology is appropriate
- Benchmarks performance of models and forecasters
- Is a tool for evaluating emissions trends

## Verification – Techniques

- Categorical: Compare observed and forecasted categories (AQI, Ozone Action Day, etc.)
- Percent correct (PC) - Percent of forecasts that correctly predicted the categories
- False alarm (FA) - Percent of times a forecast of the category did not actually occur (“crying wolf”)
- Probability of detection (POD) - Percent of target category days correctly predicted
- Discrete: Compare observed and forecasted concentrations
- Accuracy - Average closeness between the forecasted and observed concentrations
- Average Absolute Error - Average absolute closeness between the forecasted and observed concentrations
- Bias - Indicates, on average, the tendency to over or underpredict the concentrations
- See full discussion in the Ozone and PM2.5 Forecasting Guidance Document (U.S. Environmental Protection Agency, 2003).

## Verification – Contingency Matrix

** ****	****         ****	****	****	****         ****(n11 + n12 + n15 + n16) **
**	****Accuracy  = 100*   ------------------------------------------------------------------------------------------------**
**	****	****              ****( n3 + n4 + n7 + n8 + n9 + n10 + n11 + n12 + n13 + n14 + n15 + n16)**

## Verification – Acceptable Error

- Forecast should be unbiased - equal numbers of over and underpredictions
- Occasional big misses are expected
- Review for cause of error
- Timing may be the limiting factor
- Repeated bias in one direction (high or low) suggests a systematic problem
- Agency policy may impact the bias
- Forecaster error should be differentiated from prediction model error

## Verification – Performance Targets

- The average absolute error should be approximately 10% of the maximum observed concentration
- Percent Correct is threshold dependent
- Start with 50% - 67%
- Optimally 80% - 90%
- Probability of Detection:  60% - 70%
- False Alarm:  30% - 40%
- The statistics may be misleading if the threshold is set too high or too low - high score
- Changes in forecast performance suggest changes in the air quality trend and it may be time to retool

## Summary

- Developing an Air Quality Forecasting Program
- Identify your organization's commitment
- Evaluate resources:  budget, manpower, and time availability
- Identify forecast target audience
- Develop a message format
- Set realistic forecasting goals
- Start simple and build experience and confidence
- Remember that the forecast is issued daily, and is the agency’s most visible product

- Next steps – Daily Forecast Operations
- Questions