DRAFT - NWS Technical Procedures Bulletin No. xxx

The RUC 3-d variational analysis (and post-processing modifications)

#### **13 November 2023**

Stanley G. Benjamin, Dezső Dévényi, and Stephen S. Weygandt

NOAA Research, Forecast Systems Laboratory, Boulder, CO

Geoffrey S. Manikin

Environmental Modeling Center, National Centers for Environmental
Prediction, Camp Springs, MD

**Abstract**

An upgrade to the NCEP Rapid Update Cycle analysis from an optimal
interpolation (OI) technique to a 3-d variational (3DVAR) technique is
planned for May 2003. The RUC 3DVAR analysis uses the same hybrid
isentropic-sigma coordinate as used in the RUC OI analysis (and RUC
forecast model). The RUC 3DVAR analysis has been in testing at NOAA-FSL
over the last few years and at NCEP in parallel and retrospective cycles
since November 2002. Overall, the RUC 3DVAR provides

-   Slight improvement or about equal skill overall in 3-h and 12-h
    forecasts compared to those from the previous RUC OI analysis as
    verified against rawinsondes.

-   Closer fit to observations than the RUC OI analysis.

-   A smoother analysis increment (correction to 1-h forecast field) and
    quieter short-range forecasts.

-   Capability for assimilation of indirect observations such as radial
    winds, satellite radiances, and wind speed.

A few minor changes to the RUC post-processing are also being
implemented with this change package, including addition of 0-1 km
helicity and some small fixes.

1.  **Introduction**

A 3-dimensional variational analysis (3dVAR) technique has been
developed for application in the Rapid Update Cycle (RUC). The goals for
the implementation of the RUC 3DVAR are to:

-   Introduce the capability for assimilation of indirect observations
    such as satellite radiances, radar radial winds, wind speed, and GPS
    wet delay (correlated with precipitable water).

-   Avoid known problems with the optimal interpolation (OI) analysis,
    especially small-scale noise from truncation associated with
    observation selection for given groups of grid points. With the
    3DVAR solution, all observations affect all grid points. No data
    thinning is needed (at present) for the 3DVAR solution, even though
    it is necessary with OI, so all observations now get used.

-   Provide equal or improved forecast skill at forecast projections of
    3 h and 12 h. The need for equal or improved forecast skill at a 3-h
    projection is unique for the RUC and less important for 3DVAR
    implementations in the EDAS or GDAS where there is not a user focus
    on short-range forecasts.

-   Match the computational time used for the RUC OI analysis.

This Technical Procedures Bulletin provides an overview of the RUC 3dVAR
and results from parallel and retrospective testing within the RUC.
Other information on the RUC 3DVAR analysis can be found in a journal
paper specifically about the RUC 3DVAR (Devenyi and Benjamin 2003, see
link to online PDF in reference list).

2.  **A description of the RUC 3-d variational analysis solution**

The implementation of the RUC 3DVAR is simplified by the previous
implementation with the RUC20 in April 2002 of a unified analysis
framework that can use solvers from either OI or 3DVAR techniques. This
analysis framework described in the previous RUC20 TPB (Benjamin et al.
2002) is unchanged in the 3DVAR implementation, including ingest and
preprocessing of observations, the calculation of observation-background
differences (innovations), quality control of observations, types of
observations assimilated, assimilation of GOES cloud data, and variables
analyzed. The sequence of analysis steps in the RUC 3DVAR is the same as
with the RUC OI analysis. (RUC20 TPB -- Benjamin et al. 2002, Devenyi
and Benjamin 2003).

The RUC 3DVAR analysis uses the same hybrid isentropic-sigma coordinate
as used in the RUC OI analysis (and RUC forecast model). More
information on the isentropic effect on the analysis is given by Devenyi
and Benjamin (2003) and Benjamin et al. (2003a). The effect of a
quasi-isentropic coordinate on the RUC forecast model is described by
Bleck and Benjamin (1993) and Benjamin et al. (2003b).

The background error spatial covariances in the RUC analysis (3DVAR or
OI) are defined in a quasi-isentropic space (Benjamin et al. 2002).
Thus, the influence of observations in correcting a background 1-h
forecast is adaptive depending on the 3D thermal structure in the
vicinity of the observations. The design of the three-dimensional
variational (3DVAR) version of the RUC analysis closely follows that
applied in the RUC OI analysis. A detailed description of RUC 3DVAR is
given in Devenyi and Benjamin (2003); here we summarize only its basic
features. Some of the key characteristics of the RUC 3DVAR are as
follows:

-   The 3DVAR procedure produces a correction (an analysis increment) to
    the background 1-h forecast, which is added to that background
    field, yielding the new analysis

-   The analysis results from minimizing the standard form of a cost
    function (with penalty terms for 'distance' from observations and
    background).

-   The control variables are streamfunction (ψ) and velocity potential
    (χ) (both scaled by grid spacing), unbalanced height, virtual
    potential temperature, and the natural logarithm of water vapor
    mixing ratio.

-   The analysis is performed on a 56-level modification of the 50
    native RUC hybrid sigma-isentropic model levels. The isentropic
    projection for the RUC 3DVAR is accomplished through mapping
    observations (and innovations) to *k* space (56 levels) using the
    3-d pressure of the hybrid isentropic-sigma background 1-h RUC
    forecast. Thus, the isentropic projection is present in the RUC
    3DVAR only in regions of the 3-D background resolved as isentropic
    levels (typically upward from 150-300 hPa above the surface). The
    actual variational solution is fully generalized in 3-D (*i,j,k*)
    space and has no isentropic or other coordinate dependencies.

-   The analysis is performed on a coarser resolution horizontal grid
    (grid spacing of 80 km), and the coarser resolution analysis
    increment is interpolated back to the fine resolution (20-km) grid.

In the multivariate mass/wind analysis step, balancing is provided by
linear regression using regression coefficients computed from a history
of previous RUC runs (6-h and 12-h forecasts valid at the same time)
using the so-called NMC method (Parrish and Derber 1992) in which
forecast differences are used as a proxy for forecast errors to obtain
correlations between streamfunction and balanced height. At present, no
cross-correlation between streamfunction and velocity potential is used.

The background error correlations are approximated by convex linear
combinations of digital Gaussian filters with different filter scales, a
method developed at NCEP (for details see Purser et al. 2001). Based on
this filtering technique (also used for the Eta/EDAS 3DVAR analysis),
approximate convolutions of a SOAR correlation function are obtained for
different variables and vertical levels, and increment fields are
computed. Different numerical experiments were performed to obtain the
optimal combination of filter weights. The full background error
covariance matrix is applied in preconditioning (Derber and Rosati
1989). Minimization is accomplished using a simple conjugate gradient
method.

The cycling of 3-D cloud hydrometeor mixing ratios (cloud, ice, rain,
snow, graupel) modified by GOES cloud-top pressure is also accomplished
in the RUC 3DVAR analysis, as is the cycling of land-surface variables
(multi-level soil moisture, temperature, snow water equivalent, snow
temperature) from the previous RUC 1-h forecast. The treatment of
hydrometeor and land-surface variables is unchanged from the previous
RUC OI analysis.

3.  **Use of observations in 3DVAR vs. OI**

A key advantage of the 3DVAR formulation compared to OI is a more
complete and more 'global' use of the observations. Optimal
interpolation analyses must select a group of observations to be used
for different groups of grid points (called 'volumes'). This introduces
discontinuities in the analysis increment (difference from background
forecast) at the boundaries of the grid-point volumes. The 3DVAR
analysis avoids this problem by using all observations at each grid
point. This smoothing effect is evident in vertical cross-sections of
the analysis increment of wind vector magnitude from the OI and 3DVAR
analysis for a given case, as shown in [Fig. 1](#Fig1). Stronger
horizontal and vertical discontinuities are evident in the OI analysis
increment.

The smoother analysis increment using the 3DVAR analysis leads to less
noisy short-range forecasts from the RUC model, as shown in [Fig.
2](#Fig2). The RUC model applies a digital filter initialization (DFI,
Lynch and Huang 1992) before the forward integration in each model run,
but the 3DVAR analysis provides a slightly quieter 1-h model forecast
even after DFI application. The reduction of noise is much more
pronounced if DFI is not applied.

The 3DVAR analysis also avoids another shortcoming of the OI analysis,
the need for observation thinning or "superobbing" (combining) of
observations. Since OI is based on inversion of matrices consisting of
expected forecast error covariance between different observation sites,
data thinning is necessary for closely spaced observations
(predominantly aircraft and surface observations). This data thinning is
not necessary in the 3DVAR analysis, and the full use of all
observations (including an observation density function) leads to a more
representative analysis solution.

4.  **Statistical evaluation of parallel RUC cycles (operation-OI vs.
    3DVAR)**

Parallel RUC cycles using the 3DVAR analysis compared to the operational
OI analysis were conducted at NCEP for a cold-season period (74 days --
18 November 2002 -- 30 January 2003) and a warm-season period (28 days -
8 August 2002 -- 5 September 2002). These results were evaluated via
statistical comparison of the forecasts from the different cycles with
rawinsonde observations (wind, temperature, height, relative humidity).
A limited comparison was also made against METAR observations. In
addition, subjective day-to-day comparisons were made using a real-time
web site established at NCEP (examples in next section).

a.  Analysis fit to observations

Verification against rawinsonde data showed that the 3DVAR analysis
generally fits those observations more closely than the OI analysis.
This is true for wind, temperature, height, and relative humidity ([Fig.
3](#Fig3)). Additional verification against METAR observations shows
that the 3DVAR has an equal or slightly closer fit to those surface
observations (no figure shown).

b.  Forecast error statistics

We first discuss results from the cold-season test period from Nov 2002
-- Jan 2003. Wind forecasts from the 3DVAR cycle show slight
improvements at all levels over the OI-operational cycle for 3-h
forecasts ([Fig. 4](#Fig4)). For 12-h forecasts, the 3DVAR cycle gives
slight improvement in the upper troposphere and slightly poorer results
in the lower troposphere. For temperatures, the 3DVAR and OI cycles gave
very similar forecast errors at both 3-h and 12-h projections ([Fig.
5](#Fig5)). A complete set of verification statistics from this period
is available at <http://www.emc.ncep.noaa.gov/mmb/ruc2/stats/>. \[In
these figures, the 3DVAR result is the dotted line, also labeled as
RUC20/236.\]

Verification against rawinsonde observations was also performed for the
warm-season test period from Aug- Sept 2002. These results are also
available online at
<http://www.emc.ncep.noaa.gov/mmb/ruc2/summerretrostats/>. Generally,
the results were very similar, but with 3DVAR forecasts showing a
slightly stronger improvement over the OI-based forecasts in this
warm-season test than in the cold-season period.

5.  **Forecast examples**

In our subjective examination of fields from the 3DVAR and
OI-operational cycles during the cold-season test period, analysis
fields from the 3DVAR cycle often showed a slightly smoother appearance
than those from the OI cycle,. The fields examined included sea-level
pressure, 2-m temperature and dewpoint, 10-m wind, CAPE (convective
available potential energy) and CIN (convective inhibition),
storm-relative helicity, lifted index, 925 mb height/RH, 700 mb
omega/wind, 500 mb height/vorticity, 250 mb wind, soil moisture
availability, and visibility. Consistent with statistical comparisons,
differences in forecast fields between the 3DVAR and OI cycles were
generally small.

The RUC 3DVAR analysis is, overall, able to maintain fidelity to
sounding structures at least as well as the previous OI analysis. An
example of this is shown in [Fig. 6](#Fig6), with a double inversion
structure from the observed Shreveport, LA rawinsonde sounding for 1200
UTC 29 April 2003. The sounding taken from the RUC 3DVAR grid point data
was able to match the observed structure fairly well.

A comparison of 500 mb height/vorticity fields for analysis ([Fig.
7a](#Fig7)) and 12-h forecast (Fig. 7b) shows that the 3DVAR analysis is
slightly smoother than the OI analysis but that differences are smaller
in the 12-h forecast. A comparison of 3-h sea-level pressure (SLP)
forecasts ([Fig. 8a](#Fig8)) shows similar features between the 3DVAR
and OI cycles, but a slightly strong surface low from the 3DVAR cycle
over the Atlantic Ocean southeast of New England and a slightly stronger
cold front trailing to the southwest across Florida. This stronger front
resulted in a more organized precipitation field in the 3DVAR forecast
(Fig. 8b).

More comparisons may be viewed online from the last date of comparison
products from the cold-season test period (4-5 February 2003) at
<http://wwwt.emc.ncep.noaa.gov/mmb/ruc2/para/>.

6.  **Post- processing changes**

A few changes are being made to the RUC post-processing with the 3DVAR
change package, including:

-   Addition of 0-1 km storm-relative helicity (in addition to the
    existing 0-3 km value).

-   A fix to the precipitation type algorithm such that snow will be
    indicated when rimed snow or graupel is falling at temperatures
    between 2-4 C.

-   A change to the combined hydrometeor mixing ratio criterion for a
    cloud level in the cloud fraction field to match that used by the
    cloud base and top computations (10^-7^ g/g).

-   Removal of 0-h (analysis) precipitation from output.

7.  **Summary**

A three-dimensional variational analysis is being implemented into the
operational Rapid Update Cycle (RUC) mesoscale assimilation/forecast
system running at NCEP. This analysis technique is being implemented to
replace a previous optimal interpolation technique in the operational
RUC running at NCEP. The RUC 3DVAR analysis fits observations more
closely than the OI but with a smoother analysis increment field,
leading to quieter short-range (1-3h) forecasts. Overall, forecast skill
is similar to that from the OI analyses with a slight improvement in
upper-level winds. The 3DVAR formulation allows all observations to be
used without any data thinning and will allow the future assimilation of
indirect observations such as radial winds, satellite radiances, and
wind speeds.

Finally, the adaptivity of the isentropic-sigma coordinate for
maintaining air-mass continuity is advantageous in the RUC 3DVAR
analysis and unique among limited-area variational methods.

**8. Acknowledgments**

We are grateful to Dave Parrish, Jim Purser, and Wan-Shu Wu at NCEP for
their helpful discussions on 3DVAR concepts and for providing some
software. We also thank John Ward, Dave Michaud, and other colleagues at
NCEP Central Operations for their work on the operational
implementation. This work has been supported by NOAA and the U.S.
Federal Aviation Administration's (FAA) Aviation Weather Research
Program.

**9. References**

Benjamin, S.G., J.M. Brown, K.J. Brundage, D. Dévényi, G.A. Grell, D.
Kim, B.E. Schwartz, T.G. Smirnova, T.L. Smith, S. S.Weygandt, and G.S.
Manikin, 2002: RUC20 - The 20-km version of the Rapid Update Cycle. *NWS
Technical Procedures Bulletin No. 490*. \[FSL revised version available
through RUC web site at http://ruc.fsl.noaa.gov\]

Benjamin, S.G., D. Dévényi, S.S. Weygandt, K.J. Brundage, J.M. Brown,
G.A. Grell, D. Kim, B.E. Schwartz, T.G. Smirnova, T.L. Smith, and G.S.
Manikin, 2003a: An hourly assimilation/forecast cycle: The RUC. *Mon.
Wea. Rev*., **131,** submitted for publication.

Benjamin, S.G., G.A. Grell, J.M. Brown, T.G. Smirnova, and R. Bleck,
2003b: Mesoscale weather prediction with a hybrid
isentropic/terrain-following coordinate model − the RUC model. *Mon.
Wea. Rev*., **131**, submitted for publication.

Bleck, R. and S.G. Benjamin, 1993: Regional weather prediction with a
model combining terrain-following and isentropic coordinates. Part I:
Model description. *Mon. Wea. Rev.,* **121**, 1770-1785.

Derber, J.C., and A. Rosati, 1989: A global oceanic data assimilation
system. *J. Phys. Oceanogr.,* **19**, 1333-1347.

Dévényi, D., and S.G. Benjamin, 2003: A variational assimilation
technique in a hybrid isentropic-sigma coordinate. *Meteor. Atmos.
Phys.,* **82**, 245-257. PDF available at

<http://link.springer-ny.com/link/service/journals/00703/bibs/3082001/30820245.htm>.

Lynch, P. and X.-Y. Huang, 1992: Initialization of the HIRLAM model
using a digital filter. *Mon. Wea. Rev.,* **120**, 1019-1034.

Parrish, D.F, and J.C. Derber, 1992: The National Meteorological
Center\'s spectral statistical interpolation analysis system. *Mon. Wea.
Rev.,* **120**, 1747-1763.

Purser, R.J., W.-S. Wu, D.F. Parrish, and N.M. Roberts, 2001: Numerical
aspects of the application of recursive filters to variational
statistical analysis with spatially inhomogeneous covariance. *NOAA/NCEP
Office Note 431,* 34 pp.

![](media/image1.png){width="6.875in" height="2.6041666666666665in"}

![](media/image2.png){width="6.875in" height="2.59375in"}

[]{#Fig1 .anchor}Figure 1. Vertical cross-section (west-east across RUC
domain) of magnitude of vector wind (u and v) analysis increment for
1200 UTC 15 Jan 2002 using a) OI analysis, and b) 3DVAR analysis.
Contour interval is 2.0 m s^-1^. Vertical axis is RUC native coordinate
level (k), and horizontal orientation is west-east at approximately
40ºN. Figure is used qualitatively only. Discontinuity near k=5 is from
use of surface data only through lowest 5 levels in OI analysis and
mapping of native levels 1-4 into single level in 56-level 3DVAR
solution.

![](media/image3.png){width="6.75in" height="5.820138888888889in"}

[]{#Fig2 .anchor}Figure 2. Noise parameter (mean absolute surface
pressure tendency) over a single time step (30 s) in RUC model with 3-D
variational or optimal interpolation analysis, both with and without
application of digital filter initialization (DFI). For case with
initial conditions at 1200 UTC 19 November 2002.

![](media/image4.png){width="3.25in" height="2.767361111111111in"}

![](media/image6.png){width="3.375in" height="2.80625in"}

[]{#Fig3 .anchor}Figure 3. RUC analysis fit to rawinsonde observations
for both optimal interpolation and 3-D variational versions for period
from 18 Nov 2002 -- 14 Jan 2003, using NCEP rawinsonde quality control
flags. a) RMS vector difference or standard deviation difference between
observations and analyses. a) wind (m s^-1^), b) height (m), c)
temperature (degrees Celsius), d) relative humidity (percent RH). All
verification using 40-km gridded data post-processed from the 20-km RUC.

![](media/image8.png){width="4.751388888888889in"
height="5.113888888888889in"}

[]{#Fig4 .anchor}Figure 4. Same as Fig. 3, but for 3-h and 12-h RUC wind
forecast errors, also verified against rawinsonde observations.

![](media/image9.png){width="5.030555555555556in"
height="5.082638888888889in"}

[]{#Fig5 .anchor}Figure 5. Same as Fig. 4 but for temperature forecasts.

![](media/image11.png){width="3.051388888888889in"
height="3.7493055555555554in"}

![](media/image13.png){width="2.7381944444444444in" height="3.875in"}

![](media/image14.png){width="2.827777777777778in"
height="3.623611111111111in"}

[]{#Fig6 .anchor}Figure 6. Sounding intercomparison for Shreveport, LA,
1200 UTC 29 April 2003. Top right -- RUC 1-h background forecast, top
left -- observed radiosonde profile, bottom left -- RUC OI analysis, and
bottom right -- RUC 3DVAR analysis. Both OI and 3DVAR analyses used the
same background field in this experiment.

![](media/image15.png){width="4.572916666666667in"
height="7.018055555555556in"}

[]{#Fig7 .anchor}Figure 7. 500-hPa height and vorticity products from
RUC OI-operational (top) and RUC 3DVAR cycles from 1200 UTC 6 December
2002. a) analysis, b) 12-h forecast valid 0000 UTC 7 December 2002.

![](media/image16.png){width="4.498611111111111in"
height="7.018055555555556in"}

Figure 7b.

![](media/image17.png){width="3.2291666666666665in"
height="5.123611111111111in"}

![](media/image18.png){width="3.3520833333333333in" height="5.125in"}

[]{#Fig8 .anchor}Figure 8. RUC 3-h forecasts of a) sea-level pressure
and b) 3-h precipitation from RUC OI-operational (top) and RUC 3DVAR
cycles, valid 0300 UTC 6 December 2002.