# Presentation: 490072

## NEXRAD DATA QUALITY ASSURANCE

**An Update**

**Mark Isaminger and Dave Smalley**
**MIT Lincoln Laboratory - Weather Sensing Group**
**28 May 2003**

## Background/Review

Algorithm Description

Examples of DQA editing
Artifact detection
Anomalous propagation
Utility of REC

Conclusions

**Algorithm Description**
**Examples of DQA editing**
**Artifact detection**
**Anomalous propagation**
**Utility of REC**
**Conclusions**

**Outline**

**• ****Artifacts not limited to “bull’s-eye” patterns **
**	****–****	****Wedges/ spikes/ strobes**
**	****–****	****Other circular patterns (starbursts)**
**• ****Typical sources **
**	****– ****WSR- 88D equipment failure**
**	****	****	****AGC problems/Receiver failure/Interference**
**	****	****	****Three-body scattering**
**	****– ****FAA, NWS, and DOD radars**
**	****	****	****Aircraft**
**	****	****	****Ground based, wireless, cable TV, and other ****	****	****transmitter transmissions**
**	****	****	****Solar**
**• ****Many characterized by a radial constant power function signature**

**WSR 88D Artifacts**

**Notes:**
Three-body scattering is an internal issue but not an equipment failure.

**Build 3 – Data Quality Assurance (DQA) **

**Purpose is to provide “cleaned-up” reflectivity for use in FAA precipitation products such as composite reflectivity and high resolution VIL**
**To include AD (Artifacts Detection) and AP editing**
**AD editing adapted from Unisys algorithm for WARP**
**AP editing to be full MIT/LL AP Editing scheme**
**WARP/ITWS interest:  use to generate improved composite reflectivity products**
**CIWS:  use AD edit portion for editing of vertical integrated liquid (VIL) product from individual radars before VIL mosaic formation**
**NWS Hydrology algorithms**
**Others:  use as input for future NEXRAD algorithms**

**Notes:**
CIWS has one advantage over these other systems in terms of its ability to mosaic,
which could also serve to mitigate DQ concerns.
MIGFA could certainly benefit by improving the DQ.

## DQA Algorithm Flow – AD Removal

- Source:  Logic based on Unisys’ WARP artifact detection design document.
- MIT/LL modified for application to all tilts in volume and added exclusion checks.

**Notes:**
Twenty km segment size is elevation dependant.
Ignore the data in close to the radar due to high background and AP/Noise concerns.
Does not edit if the radial also contains wx.
Does a great job removing the “starburst” and “ bulls-eye” patterns.

## MEM Noise Strobe

**Notes:**
High constant power returns such as this should be edited.
The issue is whether or not the wx returns early in the radials would degrade the performance.

## Sun Strobe Artifact Editing

**Notes:**
Strobe not associated with wx, so it is edited.
Wx returns are not edited.

## Tilt Analysis of DQA and REC (0.5 degrees)

**Tilt Analysis of DQA and REC (0.5 degrees)**

**KMLB 08/13/02**
**1909 UTC**
**0.5****o**** Tilt**
**Reflectivity Factor**

**In this case, the lowest few tilts of a VCP 11 volume were uncontaminated.  This base tilt is representative of those tilts with small convection west of the radar.  Note that REC assigns at least 50% likelihood of clutter to many of the edges of these small storms.**

**After DQA**

**As Received**

**Notes:**
REC assigns a high % of the wx returns as AP!!
Wx returns not edited out by DQA.

## Tilt Analysis of DQA and REC (3.4 degrees)

**Tilt Analysis of DQA and REC (3.4 degrees)**

**KMLB 08/13/02**
**1909 UTC**
**3.4****o**** Tilt**
**Reflectivity Factor**

**In the middle tilts of this volume, technician error results in contamination being introduced.  The DQA algorithm readily mitigates it since it has a constant power function (CPF) artifact signature.  The REC algorithm assigns a low likelihood that this is AP and does not recognize it as CPF artifacts.**

**After DQA**

**As Received**

**REC % likelihood AP**

**Notes:**
Do NOT have any hard numbers on REC vs. DQA for AP cases.
REC not designed to edit artifacts.

## High Resolution VIL Product

**KMLB 08/13/02 1909 UTC**

**The Data Quality Assurance (DQA) algorithm clearly provides benefit to the HRVIL product by greatly reducing the obscuration of weather by the mid-volume contamination of the radar data.**

**Without DQA**

**With DQA**

**Notes:**
Still some contamination though.

## Unedited Noise Speckles (CIWS)

**Notes:**
Speckle or point target filter might work in this case a.k.a. TDWR.

## Unedited Strobe (CIWS)

**Notes:**
Radial also has wx.

## AP-Editing Modifications

**Main difference between baseline NEXRAD editing algorithm and that used in the ITWS/CIWS prototypes is the use of a median versus a “scatter” filter.**
**this shortcoming has been rectified somewhat by changing several adaptable parameters**
**Further elimination of AP residue will be realized by the AD editing function.**
**Utility of the REC to further improve AP editing (especially when velocity products are missing) is to be assessed**

**Notes:**
Use of HRVIL product will help FAA applications as well.

## KNQA NEXRAD Nocturnal AP Episode

## KNQA Storm Total Rainfall for an AP Episode

**Notes:**
What if the AP was co-located with wx returns?
Accuracy of the data then becomes questionable.

## Comparison of AP-Editing Performance (MIAWS vs. ITWS)

**Comparison of AP-Editing Performance (MIAWS vs. ITWS)**

**Notes:**
Based on CR product.
Level 1 thrown out due to clear air contamination.

## Comparison of AP-Editing Performance (MIAWS vs. ITWS)

## Weather Editing

**Notes:**
Generally an issue with stratiform and not convective weather.
Overall percentage of wx editing would be quite low I.e < one percent.
This case is from SGF.

**CIWS AP-Editing Experience**

**030509 > Strong AP returns (levels 5/6) in KLSX data.**
**030421 > Widespread and long-lived regions of weak-severe AP in ME, PA, MA, and NY.**
**030328 > Scattered (level 2-5) AP returns across southern NJ that persisted.**
**030324 > AP along the southern NJ coastline for more than one hour.**

**Notes:**
The point here is that even with HRVIL and basedata cleanup techniques, AP residue can still be problematic at times.
Even so, the frequency of AP contamination is down.
KLSX = St. Louis

## CIWS AP Case on 030421

**Notes:**
This is an example of distant AP not being removed due to loss of V/SW data.
Adding radars to the edges of the mosaic would just push the contamination farther out.

**Conclusions**

**Artifact Detection (AD) and Removal**
**Successful removal of common artifacts**
**Non-CPF artifacts go undetected**
**Utilize other techniques such as point target filtering**
**Anomalous Propagation (AP) Detection and Removal**
**Good AP removal when 3 radar moments present**
**REC may help when only reflectivity is available**
**Minor editing of low reflectivity weather returns**
**NEXRAD Open RDA should improve AP mitigation**
**Future APR techniques might incorporate multiple tilt analysis**
**Mosaic processing could detect AP and provide feedback to the radar operator to utilize the clutter filters**
**Satellite data and/or echo tops?**

**Notes:**
Need quantitative numbers for REC and DQA for AP cases.
Qualitative observations so far show no advantage of REC over DQA.
Open RDA will employ engineering techniques to mitigate AP.
Clean-up noise strobes during sunrise/sunset even if there is wx.