# Presentation: 214051

## A Probabilistic-Spatial Approach to the Quality Control of Climate Observations 

- Christopher Daly, Wayne Gibson, Matthew Doggett, Joseph Smith, and George Taylor
- Spatial Climate Analysis Service
- Oregon State University
- Corvallis, Oregon, USA

## Traditional QC Systems are Categorical and Deterministic

- Data subjected to categorical quality checks
- Designed to uncover mistakes
- Validity determined from test results
- Mistake = flag / toss
- No mistake = no flag / keep

**Designed to Work With **
**Human Observing Systems**

## Alien Electronic Devices are Invading the Climate Observing World!

***They’re Everywhere!***
***They’re Everywhere!***

## Electronic Sensors and Modern Applications Create Challenges for Traditional QC Systems

- Errors tend to be continuous drift, rather than categorical mistakes
- Increasing usage of computer applications that rely on climate observations

- Continuous estimates, rather than categorical tests, of observation validity
- Quantitative estimates of observational uncertainty, not just flags

_*Situation*_

_*Need*_

## More Challenges...

- Range of applications is increasingly rapidly, and each has a difference tolerance for outliers
- Data are often more voluminous and disseminated in a more timely manner

- Probabilistic information from which a decision to use an obs can be made, not up-front decision
- Automated QC methods

_*Situation*_

_*Need*_

## An Opportunity

- Advances in climate mapping technology now make it possible to estimate a reasonably accurate “expected value” for an observation based on surrounding stations.
- Assumption: Spatial consistency is related to observation validity

## Useful Characteristics for a Next-Generation Climate QC System

- continuous
- quantitative
- probabilistic
- automated
- spatial

## PRISM Probabilistic-Spatial QC (PSQC) System for SNOTEL Data

- Uses climate mapping technology and climate statistics to provide a continuous, quantitative confidence probability for each observation, estimate a replacement value, and provide a confidence interval for that replacement.

-   Start with daily max/min temperature for all   	SNOTEL sites, period of record
-   Move to precipitation, SWE, soil 	temperature and moisture
-   Develop automated system for near-real 	time operation at NRCS

## Climatological Grid Development

- PRISM must produce a high-quality estimate of temperature at each SNOTEL station each day
- Highest interpolation skill obtained by using a high-quality predictive grid that represents the long-term climatological temperature for that day, rather than a digital elevation grid
- Climatological grid: 0.8 km resolution, 1971-2000

*4 km*

*0.8 km*

## Oregon Annual Precipitation

- Leveraging Information Content of High-Quality Climatologies
- to Create New Maps with Fewer Data and Less Effort

- Climatology used in place of DEM as PRISM predictor grid

**Notes:**
This method is being used operationally at SCAS to create near real time climate maps for each month, 7-14 days after the end of that month.  In the above example, July 2003 (right) was one of the warmest on record in the West. Even so, the local patterns of temperature bear a striking resemblance to the long-term climatological average for July (left).  

## PRISM Regression of “Weather vs Climate”

- 20 July 2000 Tmax vs 1971-2000 Mean July Tmax

**Notes:**
This is a scatterplot showing how this method can be used for daily interpolation, as well.  On the x-axis is the mean July temperature, and it is being regressed with the daily maximum temperature for July 20, 2000.  This plot, for a location new Mount Hood in Oregon, shows that even though the temperature is nearly 5C above the mean, the spatial patterns are remarkably similar.  

## Generates gridded estimates of climatic parameters

Moving-window regression of climate vs. elevation for each grid cell
Uses nearby station observations

Spatial climate knowledge base (KBS) weights stations in the regression function by their climatological similarity to the target grid cell

- Moving-window regression of climate vs. elevation for each grid cell
- Uses nearby station observations
- Spatial climate knowledge base (KBS) weights stations in the regression function by their climatological similarity to the target grid cell

**PRISM**

**Parameter-elevation Regressions on Independent Slopes Model**

**Notes:**
PRISM stands for “Parameter-elevation Regressions on Independent Slopes Model.”  It generates gridded estimates of climate parameters such as temperature and precipitation.  The model is basically a moving-window regression of climate vs. elevation that is calculated for each grid cell in a digital elevation grid.  Stations surrounding the grid cell provide data points for the regression.  The real heart of the model, and what makes it unique, is the extensive spatial climate knowledge base that calculates station weights upon entering the regression function.  These weights are based on each station’s climatological similarity to the grid cell being estimated (target grid cell).  

## PRISM KBS accounts for spatial variations in climate due to:

Elevation
Terrain orientation
Terrain steepness
Moisture regime
Coastal proximity
Inversion layer
Long-term climate patterns

- Elevation
- Terrain orientation
- Terrain steepness
- Moisture regime
- Coastal proximity
- Inversion layer
- Long-term climate patterns

**PRISM**

**Parameter-elevation Regressions on Independent Slopes Model**

**Notes:**
The knowledge base and resulting station weighting functions, currently account for spatial variations in climate due to: 
Elevation, provided by a digital elevation grid
Terrain orientation, from topographic facet grids a six different scales; 
Terrain steepness, from a terrain profile grid; 
Moisture regime, from output from the PRISM straight-line trajectory model that calculates relative moisture loss from the coast to inland pixels; 
Coastal proximity, from output from the PRISM coastal trajectory model that finds the optimum trajectories from the coast to inland pixels; and 
Inversion layer, from an inversion height grid.  

In addition, PRISM can substitute a climate grid for the elevation grid as the predictor in the regression function, thereby creating moving-window relationships between climate and climate.  This is sometimes referred to as “climatologically aided interpolation.”  An example of this is the creation of monthly time series grids (e.g., January 1982) using a high-quality, 1961-90 mean monthly grid as the predictor, rather than an elevation grid.  Results are monthly grids that are robust to changes in station data density over time.

## PRISM Moving-Window Regression Function

**PRISM Moving-Window Regression Function**

- 1961-90 Mean April Precipitation, Qin Ling Mountains, China

- Weighted linear
- regression

**Notes:**
Here is output from the PRISM Graphical User Interface (GUI), showing a precipitation-elevation regression function for the north slope of the Qin Ling Mountains in central China.  The red square is the target pixel, and the black dots are the surrounding station locations.
On the scatterplot, the blue dots represent the stations, plotted by elevation on the x-axis and precipitation on the y-axis.  The size of each dot represents the relative weight of the station in the regression function.  

## Rain Shadows: 1961-90 Mean Annual Precipitation

- Oregon Cascades

- Portland

- Eugene

- Sisters

- Redmond

- Bend

*Mt. Hood*

*Mt. Jefferson*

*Three Sisters*

**N**

**350 mm/yr**

**2200 mm/yr**

**2500 mm/yr**

**Dominant **
**PRISM KBS**
**Components**
**Elevation**
**Terrain orientation**
**Terrain steepness**
**Moisture Regime**

**Notes:**
Here are some examples of PRISM at work.  The Oregon Cascades are a good example of a significant rain shadow; mean annual precipitation drops from 2200 mm/yr at the crest of the Cascades, to only 350 mm/yr just down the hill to the east.   The components of the PRISM knowledge base that play the greatest role in station weighting are elevation, terrain orientation, terrain steepness, and moisture regime. This PRISM mean annual precipitation map shows the wettest areas in white and magenta, moderate in blues and green, and driest in yellow and orange. 

## 1961-90 Mean Annual Precipitation, Cascade Mtns, OR, USA

**Notes:**
To illustrate the ability of PRISM to isolate the windward (wet) side of a mountain range from the leeward (dry) side, we use the PRISM graphical user interface to model mean annual precipitation along the crest of the Oregon Cascades at Santiam Pass.  Here we see that when we run the model on the west (wet) side at the location of the red dot, PRISM preferentially uses stations (black pluses) on the same side of the crest and predicts a relatively large amount of precipitation (2108 mm per year) from the precipitation-elevation regresssion function.  

## 1961-90 Mean Annual Precipitation, Cascade Mtns, OR, USA

**Notes:**
Now when we click on the eastern (dry) side of the crest, PRISM weights stations on the eastern side more highly, and rejects more numerous (and nearby) stations on the western side, thus predicting a relatively small amount of precipitation (618 mm per year) from the precipitation-elevation regression function.   

## Coastal Effects: 1971-00 July Maximum Temperature

- Central California Coast

- Monterey

- San Francisco

- San Jose

- Santa Cruz

- Hollister

- Salinas

- Stockton

- Sacramento

*Pacific Ocean*

- Fremont

**N**

- Preferred
- Trajectories

**Dominant**
**PRISM KBS **
**Components**
**Elevation **
**Coastal Proximity**
**Inversion Layer**

- 34°

- 20°

- 27°

- Oakland

**Notes:**
The central California coast is well-known for its extreme gradients in maximum temperature in summer, caused by the interaction of the warm land mass and the adjacent cool Pacific Ocean.   Coastal proximity is estimated with the PRISM coastal influence trajectory model, which performs a cost-benefit path analysis to find the optimum path marine air might take, given prevailing winds and terrain.  Penalties are assessed for moving uphill, and for the length of the path, requiring the optimal path to be a compromise between the shortest path, and path of least terrain resistance.  This PRISM July minimum temperature map shows the coolest temperatures in magenta and blue, moderate in green and yellow, and warmest in orange. 

## Inversions – 1971-00 July Minimum Temperature 

- Inversions – 1971-00 July Minimum Temperature
- Northwestern California

- Ukiah

- Cloverdale

- Lakeport

- Willits

*Clear     Lake*

*Pacific Ocean*

*Lake Pilsbury.*

**N**

**Dominant**
**PRISM KBS **
**Components**
**Elevation **
**Inversion Layer**
**Topographic Index**
**Coastal Proximity**

- 12°

- 17°

- 9°

- 16°

- 10°

- 17°

**Notes:**
Inversions occur frequently on clear, calm, summer mornings.  In northwestern California, summer minimum temperatures are highly inverted.  Low temperature in the valleys are five or more degrees Celsius lower than those on the surrounding highlands.   Such variability must be taken into account when assessing suitability for various agricultural uses, such as grape growing.   This PRISM July minimum temperature map shows the coolest temperatures in blue, moderate in green and warmest in yellow.  

## Definition of CP: Given the difference between an observation and an expected value (residual), CP is the probability that another observation and expected value from the same time of year would differ by at least as much

Residual distribution
+/- 15 day, +/- 2 year window = 5 yrs, 31 days each (N~155)

- Residual distribution
- +/- 15 day, +/- 2 year window = 5 yrs, 31 days each (N~155)

- PRISM PSQC SystemConfidence Probability (CP)

## Confidence Probability Takes into Account Uncertainty in the System

**P-value is higher for a given deviation from **
**the mean when *****S******x***** is large (low skill)**

*X* = Residual (*P-O*)

- Low Overall Skill

- High Overall Skill

## Interpreting Confidence Probability

- Continuous values from 0 – 100%
-  0% = highly spatially inconsistent observation, reflected in a PRISM prediction that is unusually different than the observation
- 100% = highly consistent observation, reflected in a PRISM prediction that is relatively close to the observation
- Guidelines to date
- CP > 30: Use observation as-is
- 10 < CP < 30: Blend prediction and observation
- CP < 10: Use prediction instead of observation

## PRISM PSQC Process1.  Create Database Records

- Goal:  Enter daily tmax/tmin observations for all networks into database and prepare data
- Current Actions:
- Ingest daily tmin/tmax observations from SNOTEL, COOP, RAWS, Agrimet, ASOS, and first-order networks.
- Shift AM COOP observations of tmax to previous day  (assumes standard diurnal curve, which does not always apply).
- Convert units to degrees Celsius.

## PRISM PSQC Process2.  Single-Station Checks

- Goal:  Take all QC actions possible at the single-station level, before entering the spatial QC process.
- Current Checks:
- Temperature observation is well above the all-time record maximum or well below the all-time record minimum for the state – flag set and CP set to 0
- Maximum temperature is less than the minimum temperature – flag set and CP set to 0
- First daily tmax/tmin observation after a period of missing data – flag set and CP set to 0 (COOP only?)
- More than 10 consecutive observations with the same value (<+/-1F COOP, <+/-0.1C others), or more than 5 consecutive zero values, is a definite flatliner – flag set and CP set to 0
- 5-10 consecutive observations with the same value is a potential flatliner, to be assessed by the spatial QC system – flag set and CP unchanged

## PRISM PSQC Process3.  Spatial QC System

- Goal:  Through a series of iterations, gradually and systematically “weed out” spatially inconsistent observations from consistent ones
- Overview:
- PRISM is run for each station location for each day, and summary statistics are accumulated
- Once all days have been run, frequency distributions are developed and confidence probabilities (*CP*) for each daily station observation are estimated
- These *CP* values are used to weight the daily observations in a second iteration of PRISM daily runs
- Obs with lower *CP* values are given lower weight, and thus have less influence, in the second set of PRISM predictions, and are also given lower weight in the calculation of the second set of summary statistics
*CP* values are again calculated and passed back to the daily PRISM runs
- This iterative process continues for about 5 iterations, at which time the *CP* values have reached equilibrium

## QC Iteration

- For each station-day:
- Run PRISM for each station location in its absence, estimating its obs for each day
- PRISM omits nearby stations, singly, and in pairs, to try to better match observation
- Prediction closest to obs is accepted
- Raw PRISM variables:  Observation (O), Prediction (P), Residual (R=P-O), PRISM Regression Standard Deviation (S)
- Once all station-days are run:
- Calculate summary statistics for each station for each day
-  Mean and std dev of O (Os), P (Ps), R (Rs), and S (Ss)
- +/- 15 day, +/- 2 year window = 5 yrs, 31 days each (N~155)
- 5-day running Standard Deviation (RunSD) as a measure of day-to-day variability (time shifting)
- Potential flatliners: calculate V, the ratio of station’s RunSD (set to 0.3) to that of surrounding stations
- Determine “effective” standard deviation for frequency distribution
- Sigma = Max ( Rs, S, Ss, RunSD, 2 )
- Calculate probability statistics for O, P, R, S, and V for each day
- Probability statistics are p-values from z-tests
- Residual Probability (RP) used as an estimate of overall Confidence Probability (CP) for an observation
- Except in the case of potential flatliners, where CP = min(RP,VP)
- CP used to weight stations in next iteration

## Drifting sensor : MCKENZIE PASS (21E07S)

- Observations and CP values, Date: 1996-02-08

**Drifting sensor** : MCKENZIE PASS (21E07S)

## Drifting sensor : MCKENZIE PASS (21E07S)

- Climatology vs Observation and Prediction, Date: 1996-02-08

**Drifting sensor** : MCKENZIE PASS (21E07S)

## Warm Bias: SALT CREEK FALLS (22F04S)

- Observations, Date: 2000-07-14

## Warm Bias: SALT CREEK FALLS (22F04S)

- Anomalies and CP values,   7-21 July 2000

- Warm Bias: SALT CREEK FALLS (22F04S)

- 14 July

## Warm Bias: SALT CREEK FALLS (22F04S)

- Scatter Plot: Climatology vs Observation, 14 July 2000

- 22F04S

- Odell Lake
- COOP

## Tmax Observations, Date: 2000-07-14

- Tmax Observations, Date: 2000-07-14

## Computing Obstacles

- Computing – currently takes about 60 hours to run PRISM PSQC system for SNOTEL sites in the western US
- 14-processor cluster
- Disk space – we now have > 1 TB, but will probably need more
- Funds are insufficient to “do it right”

## Issues to Consider

- How far can the assumption be taken that spatial consistency equates with validity?
- Are continuous and probabilistic QC systems useful for manual observing systems?
- Can a high-quality QC system ever be completely automated?