This section contains descriptions of the three quality control
algorithms currently implemented by the Wind Profiler Demonstration
Network's Hub software.  The three algorithms are, in order of
execution:
     1. Consensus Averaging
     2. Time-Height Median Check
     3. Vertical Shear Check
The following descriptions are intended to provide a general
understanding of the algorithms.  The details of the algorithms are
available from the Profiler Program in the form of documented
FORTRAN source code.
                           CONSENSUS ALGORITHM
This algorithm operates on the ten 6-minute radial velocities (i.e.
lst moments) from the profiler radar for a given hour.  For each
gate of each beam (east, north, vertical), the ten 6-minute radial
velocities are examined to determine whether they are within a
predetermined delta velocity of each other.  The number of
velocities which are within this delta velocity is called the
consensus number.  The consensus is said to have been passed when
the consensus number is greater than or equal to 4.  If the
consensus is passed, those velocities which comprise the consensus
are then summed and divided by the consensus number.  This results
in a consensus-averaged velocity.
In addition to determining the consensus-averaged velocity, a
significant portion of the algorithm determines for the vertical
velocity.  This is done differently for the Low and High Modes as
described below.
                               HIGH MODE
The consensus number is determined for the ease, north and vertical
beams' radial velocities.
CONSENSUS PASSED:
The 0th, 1st and 2nd moments of the 6-minute periods which comprise
the consensus-averaged data are summed and divided by the consensus
number.  In addition, the correction for the antenna elevation is
made by dividing by the cosine of the antenna elevation angle.  It
is not necessary to have a consensus on the vertical beam in order
to produce a wind in the high mode.
CONSENSUS FAILED:
If the consensus number is less than 4, then an indicator is set
for the appropriate gate indicating that no wind could be produced,
i.e. there was no consensus.
CORRECTION OF WIND COMPONENTS FOR VERTICAL VELOCITY:
A correction is made to the U and V wind components for the
vertical velocity of the wind at a particular gate in the event
that all of the following are true:
     1. A wind was produced, i.e. both east and north beams passed
        consensus.
     2. Vertical velocity passed consensus.
     3. Absolute value of consensus-averaged vertical velocity is
         greater than 0.15 m/s.
                               LOW MODE
The Low mode consensus processing is more complicated than that of
the high mode.  This was done in an attempt to properly correct the
wind components for the vertical velocity in the presence of
precipitation.
The consensus number of the vertical beam is determined.  If the
consensus number is greater than or equal to 4, the 0th, 1st and
2nd moments for the vertical Low are summed and divided by the
consensus number.
The consensus numbers of the east and north beams are determined.
Then each 6-minute radial velocity of these two beams is corrected
for the vertical velocity using the vertical radial velocity for
the same 6-minute period.  The absolute value of the vertical
velocity must be greater than 0.15 m/s.  The correction is as
follows:
where:  Vel_radial = east or north 6-minute radial velocity
        W_radial   = vertical 6-minute radial velocity
Then the consensus numbers for the corrected velocities for the
east and north beams are determined.  At this point, there are 4
consensus numbers: east and north uncorrected,east and north
corrected.
It is then left to determine which (corrected or  uncorrected
velocities will be used to calculate the hourly averaged value for
the 1st moments.  The 0th and 2nd moments consensus averages will
be determined by using the consensus number that corresponds to the
method that has the higher minimum consensus number for the oblique
beams.
For example, given the following consensus numbers:
     Uncorrected east, Uncorrected north:  7, 8.
     Corrected east, Corrected north:      6, 9.
     Uncorrected radial velocities would be used.
Although the north beam has a higher consensus number using the
corrected method, the minimum consensus number for the uncorrected
velocities ( = 7) is greater than the minimum consensus number for
the corrected velocities ( = 6).
NO WIND PRODUCED:
For no wind to be produced in the low mode, one of the following
must be true:
     1. There is no vertical consensus.
     2. One or both of the oblique beams did not pass consensus for
       both methods, i.e. need to pass consensus on both beams
      for at least one method.
                           TIME-HEIGHT MEDIAN CHECK
This algorithm was developed by Keith Brewster of PROFS.  See the
document, PROFS Operational Profiler Quality Control 24 April 1987
revision for the details (Attachment 1).  The WPDN HUB implements
the identical algorithm except for the following minor difference:
In the high mode, 3 gates above and 3 gates below the current gate
for each hour are used in the calculation of the median value.
This was done in order to take into account the redundancy in the
high mode range gates.
                             Attachment 1
The PROFS operational profiler quality control system (also
referred to as "combined quality control") is a multi-stp process,
consisting of modules run by the Wave Propagation Lab (WPL) and
PROFS. Currently, the processing is done for specific workstation
products and data are not stored in the PROFS database with the
quality control flags (except for the WPL consensus averaging).
The quality control is designed to detect spurious values and does
not in any way filter out statistical or small-scale meteorological
"noise".  This also means that erroneous data that lie within
normal statistical noise of the data will be retained.
     Order of processing:
          1. Consensus Averaging (WPL)
          2. Time-Height Median Check
          3. Vertical Shear Check on each mode individually
          4. Vertical Shear Check on combined profile
1. Consensus Averaging
The consensus averaging is done by the WPL profiler data processing
system that produces the horizontal winds from the radial
velocities of the three profiler beams. The samples of radial
velocities for each beam acquired during the hour (ten samples per
beam per hour) are assembled and those that agree within a preset
threshold (s meters per second) are averaged to form a consensus
radial velocity average.  The radial velocity averages are then
geometrically combined to form the horizontal wind direction and
speed.  Should there be less than four samples from the ten that
meet the agreement criterion, no average radial velocity is
computed, and if there are any radial velocities missing no
horizontal wind is computed.  In other words there must be a
consensus on each of the beams at that particular gate or a
horizontal wind is not computed.  These consensus averaged values
are the data are stored in the PROFS databases and are the raw data
that are used for the subsequent quality control (for workstation
products) or forwarded to other users.
2. Time-Height Median Check
The median check consists of several steps.  First data are
assembled from two previous hours and the current hour.  Then, for
each data point in the "current hour", the program searches for the
eight points "surrounding" it in time and height.  The median of
theses values (done separately for each wind component) is compared
to the observation.  If the median and the observation differ by
more than a threshold, the median is recomputed using just the last
hour of data (not the two previous hours).  If there still is
disagreement, the observation is flagged bad.
     Median Check Threshold
          Threshold = a h2 + b h + c
          where a = -7.89 x 10-8 s-1m-1
                b = 1.54 x 10-3 s-1
                c = 9.50        ms-1
          Threshold is in meters/s and h is height above sea level
         (m). This is a quadratic fit to the values:
               threshold      height ASL
                 9.5 ms-1         0 m
                17.0 ms-1      9000 m
                14.0 ms-1     16000 m
          and was designed for use from 2000 m ASL to 18000 m ASL.
          Threshold based on speed:
          threshold = 0.4 * spd, where spd is the average of the
                                 observed wind component and the
                                component determined by the median
          The threshold used is the maximum of these two methods.
The threshold is increased if the data from previous hours are from
several hours ago.  The average time of the points used in the
median computation is found and issued to adjust the threshold.
     Factor to multiply threshold by based on time difference
          factor = 1 + 0.18 (DT - 1.)
          where DT is time difference in hours between data and
median points except when that difference is less than 1 hr.  Then,
DT = 1.
The threshold outlined above is quite large as this step is
currently used just to identify those points which are extreme
outliers, particularly those caused by precipitation contamination
in one or more of the beams.  The idea is to prevent the discarding
of dramatic winds shifts in time caused by strong short-waves or
fronts; the incidence of such erroneous flagging is extremely
small.
Future enhancements
New processing performed by the HUBOPS computer is sending PROFS
the number of samples which "agreed" to form the consensus average
for each gate.  These numbers will be used to further refine the
threshold.  Smaller numbers of agreeing samples will result in a
tighter threshold.  We will also look at newly available
reflectivity data to see if they will be useful.  Exceptionally
high (perhaps caused by solid target like birds or by
precipitation) or low reflectivities could be used to decrease the
threshold.
3 and 4. Vertical Wind Shear Check
This step is performed twice on each wind profile.  First it is
applied to each mode of data separately (there are two or three
operating modes at each site), then its applied to a "combined"
profile consisting of all modes meshed together.  The program
begins from the lowest reported level and steps upward.  At each
point the full wind vector shear (between a "lower" gate and the
gate being tested -- the "upper" gate) is computed and compared to
a threshold.
Establishing the threshold is complicated, allowing for:
     1. Differences in expected shear with wind velocity
     2. Apparent shear caused by statistical uncertainty in the
      wind measurements for two closely spaced observations
     3. Increased shear expected in the vicinity of the jet core
Threshold calculations
     a. Vector difference threshold:     10 m/s
     b. Vector difference threshold based on wind speeds:
          0.42 * spd, where spd is average wind speed of the tow
        gates
          vecthresh is maximum of a and b.
     c. Gate-to-Gate shear threshold:  0.018  (m/s)/m
     d. Vector difference shear threshold:  vecthresh/600
          600 is chosen to make a approx equal to c
     e. "Random error" shear threshold: 7 9m/s) / dh
          where dh is height separation between obs
           shear threshold is maximum of c, d, and e above and is
          used to compare to the vector shear between the two
       gates
     Increase in threshold for the case of unidirectional shear to
    allow for more shear in the vicinity of the jet:
          shear threshold = unifactor * shear threshold
           unifactor = 2   if the directional difference between
                           the gates is less than 20 degrees
           unifactor = 0.5 if the directional difference between
the                              gates is greater than 70 degrees
           unifactor = 1   for all other cases
Should the vector shear between gates exceed that computed by the
above method, the winds are checked ut by each component.  The
gate-to-gate component differences are calculated and the
difference between the upper gate and the next gate is checked.  If
the ratio of the first difference (upper-lower) and the second
difference (next-upper) is greater than -2.0 and less than -0.5
(i.e., approximately -1) then it is assumed that the upper gate is
bad, and it is flagged.  (Note: if one component meets the
criteria, the entire observation is flagged "bad").  Once a gate is
flagged, it is NOT used for checking other gates above it.  If the
ratio criteria are not met, tow possibilities exist: 1) there is a
frontal zone or source of large scale shear between the two gates,
2) there are two adjacent bad winds (the upper gate and the next
gate).  To determine if case 1 is true the large sale shear is
examined by fitting a line (to the wind component) between the
lower gate and the component observation three gates above it.  The
value determined from interpolating that line to the height of the
upper gate and to the next gate (i.e., two gates above the lower
gate) are compared to the observed values at those gates.  If the
tow observed values differ from their interpolated counterparts by
more than 10 meters per second and the difference for both is in
the same direction (i.e., both greater or less than the
interpolated values), they are both flagged "bad".
Special processing is done if shear exceeds the threshold and the
lower gate is the first gate with good data. In this case the wind
components from the next two gates are the extrapolated linearly
down to the height of the first gate.  If the extrapolated value is
reasonable, it is compared to the observation.  If the difference
between the two values is greater than 8.75 meters/sec, the first
gate is flagged bad.  This test is only done for "meshed" profiles,
with the assumption that a better test of whether the observation
is good or bad can be done after meshing (i.e. the point may no
longer be an end point).
There is also special handling for the top data level (again only
for "meshed" profiles). Here, the same extrapolation and comparison
as in the case of the lowest gate is done, and the  threshold used
is also 8.75 meters/sec.
Future Enhancements
As with the median check (see Section 2) the newly acquired
consensus numbers and the reflectivities may be used to further
refine the threshold and/or to refine the decision process when it
is difficult to discern which of the winds causing extreme shear
should be flagged.
The increase in thresholds to allow for shear near jets has caused
a few problems of bad data being accepted aloft where dh is fairly
large.  Further refinement of the vecthresh parameter may be
required.
Detection of spurious values in the first few gates of the profile
caused by electronic noise in the antenna and receiver has proved
to be very difficult.  In face the majority of spurious values that
the system fails to recognize are there noise contaminated winds.
This problem could be attacked in several ways.  The consensus
numbers and reflectivities may provide more evidence, and a method
of "blacklisting" could be imposed that would decrease the
threshold for those gates know to be chronically bad (this would
introduce a site-dependent condition on the processing-- something
that has been avoided thus far).  Finally, a better method of
extrapolation of data  from the next higher gates where the noise
affect is considerably less, will be tried.  This extrapolation
could also be converted to an interpolation between the profiler
data and surface data.  Surface observations taken at the profiler
site would be most useful in this effort, but lacking that,
standard surface observations could be employed.  Using standard
surface observations would have the drawback of introducing another
data source, which itself could have errors and may be missing at
times.
                      VERTICAL SHEAR CHECK ALGORITHM
This algorithm was developed by Keith Brewster of PROFS.  The
algorithm is detailed in Attachment 1.  The WPDN Hub has
implemented the algorithm as developed except for two minor changes
which were desirable due to the differences between the network 404
MHz profilers and the profilers for which the algorithm was
originally developed.  The changes were made by Jack Harlan and
Russ Chadwick after discussions with Keith Brewster.
     The changes are as follows:
          1. The algorithm is performed on each mode separately.
            There is no combined-mode checking.
          2. The precision of the radar, 0.1 m/s for radial
              velocities, is taken into account when any velocity
              thresholds are used to determine whether a wind
vector               passes the shear check.
In addition, 3 of the constants in Attachment 1 had been updated in
the source code that Keith Brewster provided to the Profiler
Program.  They are as follows:
     Under "Threshold calculations "a", 10 m/s was changed to 9 m/s
     Under "Threshold calculations "b", 0.42 was changed to 0.4
     Under "Threshold calculations "c', 0.018 was changed to 0.016
Preliminary Checking:
the data for each mode is selected such that all gates which have
failed either the consensus or the time-height median check are
eliminated.  However, gates which did not have enough surrounding
data points to perform the median check are retained.
General Overview:
Each mode is traversed from the lowest gate to the highest.  The
first gate is assumed correct and the next gate is compared to it.
If the second gate is found to be "failed", then an extrapolation
is made using the slope between the third and fourth gates and the
height difference between the first and second gates.  If the
second gate is then determined to be correct, the first gate is
flagged as "failed".  This is continued to the top of the mode or
the last gate in the mode for which there is data.
Both the vector difference and the slope are calculated using the
current gate and the gate below )the gate below is checked to see
if it passed the shear check).  If either of these quantities is
greater than a calculated threshold, then the gate is flagged
"failed".  In the case of a failure, an interpolation is made
between the gate below and the current gate.  The differences are
checked against a constant threshold and if greater, the gate is
flagged as "failed".  If the differences are smaller than another
constant threshold, the gate's flag is changed to "passed"
Step-By-Step:
     Calculate the Slope between Gates for each Wind Component
     1. Find u, v component differences between current gate and
        previous one:  UDIFF, VDIFF
     2. Find height difference between current gate and previous
        one
     3. Divide each component difference by the height difference -
        UPRIME, VPRIME
     4. Find magnitude of slope: SQRT(UPRIME**2  +  VPRIME**2).
     Calculate the Vector Difference between Gates
     5. Find magnitude of vector difference:
        SQRT(UDIFF**2 + VDIFF**2)
     Determine whether Unidirectional Shear Condition
     6. Find directional difference between previous gate and one
        ABOVE current gate
     7. If directional difference < 20 degrees:
          a. Calculate mean direction using previous gate and one
            above current gate = DIRMEAN.
          b. Find difference between DIRMEAN and direction of
         current gate = DIRDIFF.
     8. Else if directional difference >+ 20 and previous gate is
       not gate #1:
          a. Find directional difference between current gate and
            one above = DIRDIFF.
     9. Else if directional difference >= 20 and previous gate is
       gate #1:
          a. Find directional difference between current gate and
            gate #1 = DIRDIFF.
          b. DIRDIFF = maximum of 21.0 and DIRDIFF
     10. If DIFDIFF =< 20 degrees:
          a. Shear_factor  = 2.0.  Allows for greater differences
            due to unidirectional shear
     11. If DIFDIFF >= 60 degrees:
          a. Shear_factor = 0.5
     12. If DIFDIFF between 20 and 60:
          a. Shear_factor = 1.0
     Calculate Thresholds for Vector Differences
     13a. Multiply the mean speed of current gate and previous gate
            by 0.4, empirically determined constant.
          b. Vector Threshold = shear_factor * MAX(9.0, result of
             13a.) in meters/second.
     Calculate Thresholds for Slop Differences
     14a. Divide 13a. by 600
       b. Divide "7" by height difference of current and previous
          gates
       c. Slop Threshold = shear_factor * MAX(14a,14b,0.16)
     Perform the Shear Check
     15. If Slope (Step 4) is greater than Slope Threshold or
vector           difference (Step 5) is greater than Vector
Threshold, then          gate fails shear check.
     16. If a gate fails and the height difference between current
         gate and previous gate is greater than 250 meters, then
        perform the following interpolation:
          a. Find slope between previous gate and 2 gates above
             current for each component.
          b. Find difference between current gate and previous gate
             interpolated to current gate height.
          c. Interpolate previous gate's component to one gate
             above current gate using slop calculated in 16a. Do
             this for each component.
          d. Square each value from 16b.
          e. Square each value from 16c.
          f. If 16b > 81 (m/s)**2 and 16e. > 81 (m/s)**2 and
             (16c./16b) > 0, then gate still fails.
          g. Else if 16d. and 16e. are each < 16 (m/s)**2 then gate
             passes shear check.
