# Presentation: 551343

## Recent and Proposed Upgrades for the NOAA/NCEP Short Range Ensemble Forecast (SREF) Systems Jeff McQueen, Jun Du, Binbin Zhou, Geoff DiMego, Dusan Jovic, Matt Pyle and  Hui-Ya Chuang Monday, November 13, 2023

**Notes:**
Thanks to all team members for enduring this difficult upgrade to try and add more physics diversity to the SREF system

Add SREF Talagrand and missing rate performance by season:
Sept-Nov 2004
Dec-Feb 2005
Mar-May 2005
June-August 2005
Also add sub-member performance:
All Eta-BMJ
All Eta-KF
 All RSM

2 m temperature
500 mb winds
700 mb RH

NCO for enduring the frequent corrections to the parallel system
SPC detailed evaluations

## Outline

**Background for Change **
**Recent Implementations**
**Proposed Implementation:****	**
***Additional WRF members***
***Additional Outputs***
***Additional visualization***
**NCO requirements**
**Rationale for additional members**
**WRF Configuration**
**Results**
**Summary and Issues**

## Background of Change

**FY2004 AOP called for WRF **_***Ensemble***_** to be implemented in HiResWindow by Q4 – but we could only implement the two cores [WRF-NMM and WRF-ARW] on the old computer frost/snow**
**Set ourselves a goal to put the 6 member WRF ensemble into HiResWindow in Q3 FY2005**
**In April/May 2005 we decided **_**instead**_** to**
***Increase resolution of the two WRF core runs to ~5 km and run explicitly with no parameterized convection – implemented 28 June 2005***
***Commit to add 6 member WRF ensemble into SREF***
**Plans were already underway for minor SREF upgrades that were completed in August 2005 leaving not enough time to meet original target of Q4 2005 for WRF members**

## SREF Team

**System Integration/Operations****:****	****	****Jun Du & NCO**
**Physics Diversity Configuration:****	****	****B. Ferrier (NAM) ****	****	****	****	****	****	****	****	****	****	****	****	****	****	****	****	****Jonqil Han (RSM)**
**	****	****	****	****	****	****	****	****	****	****	****	****D. Hou(stoc.phys****)**
**Product Generation/Visualization****:**
**Standard Suite:****	****  ****	****	****	****	****	****Binbin Zhou, Jun Du**
**Aviation:****	**** ****	****	****	****	****	****	****Binbin Zhou**
**Severe Weather: ****	****	****	****	****	****G. Manikin, D. Bright, P. Manousous**
**Verification:**
***Model to Observations (Det/Prob):******	******	******H. Chuang, M. Baker,  E. Rogers, Y. Lin, J. McQueen***
***Model to analysis (Det/Prob):******	******	******	******B. Zhou******	***
***Case Studies:******	******	******	******	******	******	******G. Manikin, R. Grumm***
**Calibration:**
***Bias Correction:******	******	******	******	******	******	******J. Du, B. Coi******	***
***Bayesian Model Averaging:******	******	******	******Mark Raulston (PSU), Zoltan Toth***
**High Res Ensembles (WRF):****	****	****	****G. DiMego, D. Jovic****, ****	****	****	****	****	****	****	****	****	****	****	****	****	****	****	****E. Rogers, H. Chuang**
**Ensemble Transforms (Future):****	****	****	****M. Wei, Z. Toth**
**Training:****	****	****	****	****	****	****	****	****	****B. Bua**

## Recent SREF Improvements

**August 2004**
***48 km to 32 km horizontal resolution***
***Increased to 60 levels in Eta model members ***
***Enhance SREF Physics Diversity ***
***Scaled Breeding System ***
***August 2005***
***Upgrade 10 Eta members to Initialize land states from 1 km NDAS fields***
***All member SSTs initialized with new NESDIS real-time analyses***
***Enlarged 5 Regional Spectral Model (RSM) Member domain (to cover Alaska) and improved optimization***
***Lateral boundary conditions from MREF 3 hour forecasts (previously 9 hour)***

## Recent SREF ImprovementsAugust 2005 (continued)

**Extended forecasts from 63 to 87 hours**
**Additional Outputs**
***Alaska 45 km and Eastern Pacific 45 km grids***
***Individual member, mean, spread and probability fields***
***BUFR sounding outputs from all 15 members***
**Implement common WRF postprocessor for  RSM members**
**Verification**
***Produce ranked histogram, spread verification statistics from unified NCEP FVS***
**Dissemination**
***NCO SREF web page implemented for basic mean, spread fields***

## NCO Requirements

**109 nodes for 44 minutes, 2x/day**
**     ****(09, 21 UTC)**
**Remove NetCDF dependencies **
**Convert to NCO directory, I/O formats**
**Reliable products for all regions**

## Proposed Upgrades

**Common ensemble product generator**
***Create mean, spread, prob. products if some members fail***
**Additional outputs for Winter Weather Experiments and AWIPS OB7**

| WRF Core | Physics Suite | Breeding | Initial /Lateral Boundaries | Land States |
| --- | --- | --- | --- | --- |
| NMM V2 
40L50 | EMC | Ctl, n5, p5 | GFS 1⁄2o PGRB
MREF pair 5 | Truncate from NDAS-12 km |
| ARW V2
45L35 | NCAR | Ctl, n1, p1 | GFS 1⁄2o PGRB
MREF pair 1 | Truncate from NDAS |

## Rationale for WRF Configuration

**Multi-core: **
***Previous studies by Du (2004), Buizza and Toth (2005)***
***Multi-physics by Stensrud (2000), McQueen (2005)...******	******showed improved spread especially in warm season***
***Proposed increase to ***
**4 model cores (Eta, RSM, WRF-NMM, WRF-ARW)**
**6 different physics configurations**
**GFS initial atmospheric conditions (used for WRF members):**
***Near equal number of members initialized by EDAS (10) and GDAS (11)***
***WRF SI can only ingest pressure grids***
***WRF GSI not available for North American Initial conditions yet***
**NDAS Land States:**
***WRF NOAH LSM uses same land-use types as in NDAS***
***(24 soil types, 13 land use types)***

**Notes:**
GSI: so we are entering unchartered territories...we usually initialize off NDAS.

## SREF 2004 Implementation (multi-physics) Spread Plots (Summer 2004)

- SLP

- 500H

- 850T

- 850U

**SREF 2004 Implementation (multi-physics) *****Spread Plots (Summer 2004)***

**Notes:**
Now lets switch gears for probabilistic plots:
	Improved system spread in all fields

## SREF Current System

_***Physics Members***_

_***Model***__***	***__***	***__***Res (km)***__***	***__***   Levels***__***	***__***Members     Cloud Physics***__***	***__***	***__***Convection***_
**RSM-SAS****	****  ****45****	****	****	****28****	****Ctl,n,p    ****	****GFS physics****	****       ****	****Simple Arak-Schubert**
**RSM-RAS****	****  ****45****	****	****	****28****	****n,p****	****	****GFS physics ****	****	****Relaxed Arak-Schubert ****	****	**
**Eta-BMJ****	****  32****	****	****	****60****	****Ctl,n,p****	****Op Ferrier****	****	****	****Betts-Miller-Janjic****	**
**Eta-SAT****	****	****  32****	****	****	****60****	****n,p****	****	****Op Ferrier****	****	****	****BMJ-moist prof**
**Eta-KF****	****	****  32****	****	****	****60****	****Ctl,n,p ****	****Op Ferrier****	****	****	****Kain-Fritsch****	**
**Eta-KFD****	****  32****	****	****	****60****	****n,p****	****	****Op Ferrier****	****	****	****Kain-Fritsch**
**	****	****	****	****	****	****	****	****	****	****	****	****	****	****with enhanced ****	****	****	****	****	****	****	****	****	****	****	****	****	****	****detrainment **

***Adjust conv. Params to account for known biases:***
***e.g:  Biases in Convective initiation timing***
***Implemented into NCEP Operations on August 17, 2004***

**Notes:**
3 baseline convective param

## Proposed WRF members

| Core | 3 NMM members |  3 ARW members |
| --- | --- | --- |
| Horizontal | 40km  | 45 km |
| Vertical | 50 hybrid sigma-P levels | 35 Mass levels |
| Adv/Physics Time Step
Computer usage | 110/600 sec
(4 nodes/member) | 108/200 sec
(5 nodes/member) |
| Diffusion | Increased Smagorinsky deformation | Vertical damping |
| Physics | NOAH LSM
MYJ TKE PBL
BMJ Convection
Ferrier Microphysics | NOAH LSM
MRF 1st order PBL
Kain-Fritsch Convection
Ferrier Microphysics |

## 						Ensemble Products

**Prob. THI>75 F**

**Mean/Spread Surface Pressure**

**Mean/Spread 24 h snowfall**** **

**Dominant Precip Type**

**Notes:**
Also prob. Scattered, broken, overcast
Hurricane isabel uncertainties

## Additional GRIB Outputs for OB7/ WWE

_**Means/Spreads**_
**Heights at ****1000****, 850, 700, 500, 250 mb**
**U+V at ****1000****, 850, ****700, 500****, 250 mb & 10 m **
**Temperature 850, 700, ****500 mb & 2 m**** **
**Dew Point (RH) 850, 700, 500 mb & 2 m**
**QPF at 3, 6, 12 and 24 hour totals**
**12-hr Snowfall**
**Sea Level Pressure**
**Precipitable Water**
_**Probabilistic Fields**_
**3-hr/6-hr QPF****	****	****	****	****GE .01”, .25”, .50”, 1.0” **
**12-hr/24-hr QPF ****	****	****	****GE 01”, .25”, .50”, 1.0”, 2.0” **
**12-hr Snowfall ****	****	****	****	****GE 1”, ****4”, 8”, 12”**** (have 2.5, 5, 10, 20”)**
**Temperature at 2 m & 850 mb****	****LE 0****o****C **
**10 m Wind**** ****	****	****	****	****GE 25 kt, 34 kt, 50 kt**
**CAPE ****	****	****	****	****	****GE 500, 1000, 2000, 3000, 4000**
**Lifted Index ****	****	****	****	****LE 0, -4, -8**
**Surface Visibility ****	****	****	****LE 1 mi, 3 mi**
**Cloud Ceiling* ****	****	****	****	****LE 500 ft, 1000 ft, 3000 ft**
**Probability of precipitation types (have rain, frozen, & freezing)**
**6-hr/12-hr/24-hr QPF Best Category**

**Notes:**
Cloud ceiling put Binbin’s code into Eta post then ensemble product generator

## Ensemble Mean Errors  CONUS Errors by Forecast hr (Nov. 2005)

**Ensemble Mean Errors ***** CONUS Errors by Forecast hr (Nov. 2005)***

**2 m Temp**

**RMSE**

**BIAS**

**10m Wind**

**Notes:**
Slightly more diurnal cycle in SREF-21 for temps (green line)

## Ensemble Mean  CONUS Errors w/ Height at 48 hours (Nov. 2005)

***Ensemble Mean  CONUS Errors w/ Height at 48 hours (Nov. 2005)***

**RMSE**

**BIAS**

**UPA Winds**

**Notes:**
Large Improvement for 2 m Temperature ~ 0.8 C at 36 hours (21z) daytime max temperature.
This would mean a lot for energy communtiy and load distribution/forecasting
Improvement over the whole conus.
We also see an improvement in RH, that drives THI index used to estimate loads.

## SREF Performance48 h forecast Spread (Nov. 2005)

**SREF Performance*****48 h forecast Spread (Nov. 2005*****)**

**UPA Winds**

**UPA Temp**

**CONUS****	****	****      **
**CONUS-21****	****	****     **
**EAST-15**
**EAST-21**
**West-15**
**West-21****	****	****      **

***SREF-21 improved over SREF-15***
*** ******Temperature:***
***Spread is smallest in West and near Tropopause***
***Winds:***
*** ******Spread is greatest in West and near Tropopause***

**UPA Winds**

**Notes:**
Forecast Spread:  Temp West spread is very small compared to East esp 

## SREF PerformanceStatistical Consistency (Nov. 2005)

**2 m Temp**

**= Mean Squared Error/VarianceError**
**best ~ 1 (Eckel and Mass, 2005)**
***SREF-21 improved over SREF-15***
***WRF subset yields lowest SC compared to Eta subsets***

**10 m Winds**

**Notes:**
MSE/VAR=1
Constistency very large at early time.  MSE is too large and variance is too small
Eta members have very large errors vs spread at upper altitudes.

## Statistical Consistency (Nov. 2005)

**UPA Temp**

**UPA Winds**

**= Mean Squared Error/VarianceError**
**best ~ 1 (Eckel and Mass, 2005)**
***SREF-21 improved over SREF-15***
***WRF subset yields lowest SC compared to Eta subsets***

**Notes:**
MSE/VAR=1
Constistency very large at early time.  MSE is too large and variance is too small
Eta members have very large errors vs spread at upper altitudes.

## SREF Operational PerformanceOutlier Percentage 48 h forecasts (November 2005) 

**SREF Operational Performance****Outlier Percentage ****48 h forecasts (November 2005)**** **

**850 mb  Temp**

***Outlier percentage reduced for SREF/21 system***
***WRF sub-member  agree best w/ obs as compared to Eta and RSM sub-members***

**Notes:**
Talagrand each sub member has similar performance.  Perhaps, rsm (green) shows some imprrovement
In upper level winds as compared to the Eta system.
First set...ensemble were warmer than the obs.
2nd set...ensembles were within the analysis range...the higher the better
3rd set....enmbles cooler than the obs
4th set....outlier percentabe....WRF-subs (cyan) have the lowest compared to Eta and RSM...agrees with SPC results

## SREF Operational PerformanceTalagrand Ranked Histograms 48 h forecasts (November 2005) 

**850 mb Temp**
**SREF/15**

**850 mb Temp**
**SREF/21**

***Poorer performance over Western Region***

**Notes:**
Talagrand each sub member has similar performance.  Perhaps, rsm (green) shows some imprrovement
In upper level winds as compared to the Eta system.

## SREF Operational PerformanceTalagrand Ranked Histograms 48 h forecasts (November 2005) 

**SREF Operational Performance****Talagrand Ranked Histograms ****48 h forecasts (November 2005)**** **

**250 mb winds**
**SREF/21**

**250 mb winds**
**SREF/15**

***Poorer performance over Western Region***

**Notes:**
Talagrand each sub member has similar performance.  Perhaps, rsm (green) shows some imprrovement
In upper level winds as compared to the Eta system.

## SREF Operational PerformanceETS and RPSS scores for the 12h-accumulated precipitation Ensemble mean fcst and prob fcst (over 5 categories) 15-mem vs 21-mem results in Oct and Nov 

**ETS for 0.1”**

**RPSS over   5 Categories**

**15-mem**

**21-mem**

**Notes:**
Talagrand each sub member has similar performance.  Perhaps, rsm (green) shows some imprrovement
In upper level winds as compared to the Eta system.

## SREF Cold Season Case StudyNovember 20, 2005 09 Z Run – MSLP (48 h forecast)

**SREF Operational**

**SREF Parallel**

***Slightly stronger low when WRF members are added***

**Notes:**
Positive effects of convective physics diversity even in cold season

## SREF Cold Season Case StudyNovember 20, 2005 09 Z Run –Prop of Precip

**Verification**

**SREF Operational**

**SREF Parallel**

***No major change to performance in this case***

**Notes:**
Positive effects of convective physics diversity even in cold season

## SREF Cold Season Case StudyNovember 20, 2005 09 Z Run

**SREF  850 mb T<0 **
**12 UTC 11/22/05**

**Probability of  snow**
**18 UTC 11/22/05**

***Good demarkation of rain-snow line***

**Notes:**
Positive effects of convective physics diversity even in cold season

## SREF Cold Season Case StudyNovember 20, 2005 09 Z Run: 24 h precip

**NMM**

**ARW**

**BMJ**

**KFC**

**RSM**

**OBS**

***WRF/NMM and ARW add diversity and greater spread***

**Notes:**
Valid Nov. 21a at 12 Z (27 hr forecast)
ARW and NMM yield different precip forecasts than KFC and BMJ sets.   ARW much wetter than NMM

## NWS Regional Comments

- ER: Interested in Participating in the evalaution but found the EMC web too troublesome (Jeff Waldstreicher)
- CR: WFO/Goodland, KS & WFO/Pleasant Hill, MO participated but feedback not received in time

## NCEP Service Center Evaluations HPC Pete Manousos

**Real-Time Parallel Run Subjective Evaluation:** *HPC compared the 21 member SREF and the operational 15 member SREF over the 4 week period Nov 1 – 28 2005.  **We ***did not note any significant differences*** in phase or amplitude in mean mass fields – nor significant differences in QPF.  Careful scrutiny revealed the ***spread*** from the 21 Member SREF tended to be ***a bit greater*** than the operational SREF, ***but*** this had ***no operational*** ***impact***.  *
***NOTE ! HPC was very disappointed in the web interface to evaluate the output****** (see below).***

## NCEP Service Center Evaluations HPC Pete Manousos

***Very frustrating evaluation for two reasons:***
- Retrotrospective runs from requested events and in different seasons were not available... and thus due to this reason alone (output evaluated was only over the month of November), we can only recommend a “thumbs sideways”. A sound evaluation incorporates runs from different seasons (warm/cold) and real-time parallel runs.
- Evaluation of SREF real time parallel data via the web was poor as usual.  At a minimum - comparison plots of the operational and parallel for basic fields (height, temp, rh, PW, qpf) should have been made available.  The template to follow are NAM eval web pages.  This is a reiteration of an old request.  Not having this capability is severely limiting our ability to provide a sound evaluation of real-time SREF output via the web.
*Additionally the current web interface locks up frequently and does not indicate to the user if it is still loading data.  A better interface is requested.*

## NCEP Service Center Evaluations - HPC

**Given*** *
- the preliminary verification statistics provided by Jun Du (showing improvement over the operational version of SREF)
- the lack of significant difference noted by HPC during the evaluation period
- the _lack of retrospective runs_
- the short period to evaluate output
*HPC offers a “***thumbs sideways***” for this upgrade*
*Recommendation:*
*Implement as proposed (***Thumbs sideways***)*

## NCEP Service Center Evaluations SPC Steve Weiss/David Bright

**Real-Time Parallel Run Subjective Evaluation:**
*We began ingesting the parallel SREF into N-AWIPS on October 31, and have been processing the test data on a very reliable basis. Our evaluation has consisted of comparing selected SREF products from the 15 member operational and 21 member parallel runs during November that directly relate to severe weather forecasting.  During this month, **there have been four very active tornado outbreak periods** (November 5-6, November 12, November 15, and November 27), and this has allowed us to evaluate the parallel SREF on several strongly forced cool season severe weather days characterized by low CAPE/high shear environments.*
* **We examined a small number of basic and derived SREF-based parameters that are examined routinely by our forecast staff:**  2m dew point, surface CAPE, 0-6 km shear, precipitation, **combination or joint probabilities of precipitation-CAPE-shear exceedance, and locally generated calibrated thunderstorm and severe thunderstorm probabilities. ** For the basic parameters, we examined products of probability, mean-spread, max-min, and median/union-intersection.  *
* *

## NCEP Service Center Evaluations SPC Steve Weiss/David Bright

*The max-min products showed that the addition of 6 WRF members tended to **produce larger (or smaller) extreme values compared to the operational SREF, indicative of an increased spread**.  This suggests that the number of observed outliers should be reduced in the 21 member SREF.*
*The parallel SREF intersection (or grids points where all members exceed a threshold value) showed a tendency to contract in area, whereas the union (or grid points where at least one member exceeds a threshold value) showed little change from the operational SREF.  **This is consistent with an increase in variance but not an excessive increase in spread.*
*The combined probability products in the parallel SREF identified similar areas as the operational SREF, but **probability values were typically of lower value.  This is also indicative of an increase in parallel system spread**.  **This may require a small “human recalibration” by SPC forecasters when interpreting the combined probabilities in the new SREF**, especially near the northern extent of severe potential in return flow situations where the probability of moisture/CAPE exceedance can exhibit lower values compared to the operational SREF. *
*The calibrated severe probabilities from the parallel SREF are generally similar in location and orientation to products from the operational SREF.  Again, there is a tendency for the probability maxima from the parallel SREF to have lower values relative to the product from the operational SREF.  We expect this result to diminish with time as the “moving” calibration tables gradually become better tuned to the 21 member SREF performance after implementation.  **Overall, we expect a smooth transition once the parallel SREF is implemented.  *

## NCEP Service Center Evaluations - SPC

- Recommendation:* *
*Given the improved statistical performance of the parallel SREF coupled with our recent experience evaluating both SREF formulations during four active tornado outbreak episodes in November, the SPC supports the implementation of the 21 member SREF system.*
- Implement as proposed  __X__

## Summary

**Deterministic  and Probabilistic verifications are  generally positive:**
***Moderate reduction of errors and incr. spread ***
***Improved core, physics and IC diversity***
**Strongest impact on lower level variables**
***Additional scenarios captured***
**Both systems show poorer performance for West **
**WRF sub-sets yield best accuracy and spread statistics compared to other sub-sets**
**WRF individual member 2 m temp bias errors are significantly larger than Eta or RSM biases (Physics groups are addressing this)**

## Recommendations

**Implement 6 additional WRF members**
**Create ensemble products using all 21 members**

## Future Issues

**Dissemination of experimental products for improved feedback**
**Additional work to keep up to date with continual WRF Infrastructure changes**
**NCAR Infra-structure upgrades should include operational WRF customizations**
***SI, Binary I/O, optimizations***
**Lack of WRF ARW in-house expertise**

## SREF Planned Upgrades( By FY07)

_**System**_
**Run SREF 4 times per day (03, 09, 15 and 21 UTC) **
**Increase resolution to  ~25 km**
**Use Higher resolution GFS w/ MREF anomolies for SREF Lateral Boundary Conditions**
**Test Ensemble Transform initial condition perturbations**
**Upgrade WRF member codes to final NAM WRF V2.1**
**Replace Eta-BMJ members with e.g. WRF-ARW using BMJ/NCEP physics **
**Replace Eta-KF members with e.g. WRF-NMM using KF/NCAR physics**
**Replace RSM-SAS members with e.g. WRF-ARW using SAS/GFS physics**
**Replace RSM-RAS members with e.g. WRF-NMM using RAS/GFS physics**
_**Products**_
**Improved and new products**
**Mean, spread, prob. products in AWIPS OB7**
_**Post Processing**_
**Implement Grid Based Bias Correction ******** test Du regime dependent system**
**Develop Confidence Factors for forecasts**
**Improve Probabilistic NCEP Forecast Verification System (FVS)**

## BACKUPS

## Extra-Ordinary Challenges during Implementation

**Additional work while trying to finish FY05 SREF deliverables (Alaska, Hawaii products, 87 hr extension,  Bufr soundings (aviation request),  **
**Learning  two new modeling systems in a short amount of time (Du, Xiaxaoue Xue-NCO)**
**Major code modifications to accomodate machine binary in both the SI/REAL/ WRF codes (Jovic and Pyle)**
**Interfacing Breeding system into WRF infra-structure and debugging this code into WRF SI and REAL (Du, Jovic)**** **
**Additional work to initialize WRF LSM using NDAS (Gayno, Jovic)**
**Modifications to the common WRF post (Chuang, Zhou) to output standard fields needed to produce WRF products for AWIPS/ WWE**
**Uncovering additional bugs in both cores.  Bugs in ARW core did derail us for a while due to our lack of familiarity with this code : eg: bug in vertical damping, bug in LSM for V2.03.1 (fixed in later versions but we didn't have the resources to upgrade the infra-structure to V2.1)**
**Numerous experiments (learning exercises) to produce stable runs for both cores while meeting the required compute resources. (All)**
**Late Request  to incorporate a common ensemble product generator to produce required AWIPS outputs for OB7. (Zhou)**

## Extra-Ordinary Challenges during Implementation

**System became semi-stable in late October:**
***Still some timing issues to fit in Operational window***
**Little time left for detailed evaluation and development of a WRF retrospective capability.**
**Decided to focus on careful evaluation:**
***Deterministic and probabilistic statistical web pages developed for evaluator use:***
_**[http://](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[wwwt](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[emc](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[ncep](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[noaa](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[gov](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[/](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[mmb](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[/SREF-Docs/](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[verif](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[/](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**__**[para](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/verif/para)**_
_**[http://](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[wwwt](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[emc](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[ncep](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[noaa](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[.](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[gov](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[/](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[mmb](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**__**[/SREF-Docs/srm32verifprob.html](http://wwwt.emc.ncep.noaa.gov/mmb/SREF-Docs/srm32verifprob.html)**_
***Real-time comparison graphics set up :***
**NCO: prod vs para pages**
**EMC: http://wwwt.emc.ncep.noaa.gov/mmb/SREF/VERIFICATION_32km_mbr/**

## Dissemination	

**Mean, spread, probability files on NCEP FTP site**
**NCEP/EMC web graphics **
***Mean, spread, probs, Individual members, profiles, ***
**NCEP/SPC Convective probabilistic products **
**Mean, spread plots are being added to NCEP Operational web page**
**WFO AWIPS: Scheduled for Build 7 (April 2005)**

## Corrections to Improve Initial System Performance

**Run reduced physics-diversity system & evaluate Modified SREF system:**
**Develop and test scaled IC breeding code**
***breeding perturbation using WRF ***_***scaled perturbation system.***_***  Used average 850 mb T standard deviation (0.5 C)  to scale IC perturbations.  ***
***IC perturbation scale = 0.5/ *********
***Where ************=F******neg******-F******pos ****** of the 12 hour domain avg 850 mb T forecast****** ***

## SREF WRF Member Errors  CONUS Errors by Forecast hr (Nov. 2005)

**SREF WRF Member Errors ***** CONUS Errors by Forecast hr (Nov. 2005)***

**RMSE**

**BIAS**

**24h Precip**

**2 m Temp**

**Notes:**
Large Improvement for 2 m Temperature ~ 0.8 C at 36 hours (21z) daytime max temperature.
This would mean a lot for energy communtiy and load distribution/forecasting
Improvement over the whole conus.
We also see an improvement in RH, that drives THI index used to estimate loads.

## SREF WRF Member Errors  CONUS Errors by height at 48 hrs (Nov. 2005)

**SREF WRF Member Errors ***** CONUS Errors by height at 48 hrs (Nov. 2005)***

**RMSE**

**BIAS**

**UPA Temp**

**UPA Winds**

**Notes:**
Large Improvement for 2 m Temperature ~ 0.8 C at 36 hours (21z) daytime max temperature.
This would mean a lot for energy communtiy and load distribution/forecasting
Improvement over the whole conus.
We also see an improvement in RH, that drives THI index used to estimate loads.