# Presentation: 305866

## Particle Pollution Forecasting: An Initial Performance Evaluation for CharlotteScott JacksonNorth Carolina Division of Air Quality

**                    ****Background **On October 1, 2003, the North Carolina Division of Air Quality (NCDAQ) began forecasting particle pollution (PM 2.5) in the Charlotte Metropolitan area.
- NCDAQ issues 1-day forecasts Monday-Friday for the Tuesday-Saturday time period.  Friday’s forecast (covering 3 days) includes the Sunday-Monday time period.

**                    ****Motivation**
- Particle pollution has become an important component of air quality forecasting over the past few years.  The NCDAQ has been forecasting ozone since 1997, however, particle pollution forecasting is new to the agency.  In order to increase forecast accuracy, it is important to evaluate initial forecast performance.  A comparison of human forecaster skill versus other forecasting techniques is also good metric for improving accuracy.  To assess the current skill of the forecast program, the following questions were asked:
-  How are we performing versus a persistence forecast?
-  Does accuracy differ between one, two, and three day   forecasts?
-  How are we performing in relation to statistical forecast   methods, specifically the MARAMA/SAI CART tool?

**Conclusions**
-   Persistence forecasting provides decent accuracy in   the winter season when there are less dramatic    changes in particle pollution levels.
-  The relative success of persistence forecasting can be    attributed to its ability to “know” the concentration   from the previous day.  A human forecaster must    create their forecast at 15:00 before the day’s 24-   hour average concentration is complete.
-   Accurately projecting the current day’s observed    value can improve the human forecaster’s skill.
-  There is an improvement in skill for   next day forecasts over 2nd/3rd day predictions.

**Classification and Regression Tree (CART) Tool**
- A collaboration between the Mid-Atlantic Region Air Management Association (MARAMA) and Systems Applications International (SAI) provided NCDAQ with a particle pollution forecast tool.
- 1999-2002 meteorological and fine particle data were used to correlate certain meteorological conditions with fine particle concentrations.
- The CART tool provides 1-day predictions using model forecast surface and upper air meteorological data.    NCDAQ used the following forecast data as input:

**Forecast Verification Process**
**1)** NCDAQ’s forecasts are verified based on a 24-hour average     concentration beginning at 12 a.m. (midnight) and ending at      11:59 p.m. on the forecast day.
**2) ** 24-hour averages are calculated using hourly data from two     continuous (TEOM) PM 2.5 monitors in the Charlotte forecast area.

**                            ****Forecast Statistics**
- Statistics are based on an Air Quality Index (AQI) forecast** **versus the observed AQI.  Forecasting the
- observed AQI color code was considered a “hit.”  Not forecasting the correct AQI color code was a considereda “miss.”  The evaluation took place over a 129 day period, from October 1, 2003 – February 6, 2004.

**Charlotte Forecast Area**

**3)**  Monitored data are      discarded if greater      than 25% (6 hours)     of the 24-hour period     are missing or     erroneous.

**4)** Valid 24-hour     averages are     converted from     μg/m3 to AQI     using the table* on     left.

***Table shown here is an excerpt****  from a more complete version**

**Future Work**
-   Further evaluation in spring and summer months   that typically have higher fine particle concentrations.
-   Determine an accurate method for projecting the   current day’s observed value for use in making the   next day’s forecast.

**Garinger TEOM Monitor**
**Montclaire TEOM Monitor**

**     **_**Forecast Period Details**_
**Persistence - Next Day: ** A persistence forecast that predicts the next day’s AQI value
**Human - Next Day:**  All NCDAQ next day forecasts (includes CART tool influence)
**Human - Next, 2****nd**** and 3****rd****: ** All NCDAQ forecasts during the 129 day evaluation period
**Human - 2****nd**** Day:**  NCDAQ two day forecast only
**Human - 3****rd**** Day: ** NCDAQ three day forecast only
**CART only - Next Day:**  CART tool forecast of AQI color code only
**          **_**Stats Definitions**_
**Count** – Number of days forecasted by particular method
**Accuracy** – Percentage of days where color code was accurately forecast.  Higher numbers are better.
**Mean Absolute Error** – Measures the average “closeness” between the forecast and observed AQI values
**Bias** – Average under-prediction or over-prediction. Values near zero are best