# Presentation: 709344

-          **SREF PROB CONSISTENCY **
**                   ****CHECKING TOOL**
**                 **
-          Binbin Zhou, Jun Du, Jeff McQueen, Geoff DiMego
-                                       EMC/NCEP/NOAA
-                                Presented on 3/8/2005 EMC Predictability Meeting

## Background & Motivation

    - SREF Web displays images in time sequence 
   - Users can only see probability’s time evolution for ONE cycle 
   - One cycle run has less confidence some time
   - Many users strongly suggested us to provide MULTIPLE cycles
   - We added a tool to check previous 4 or 5 cycle runs, call Consistency Check   

-     - SREF Web displays images in time sequence
-    - Users can only see probability’s time evolution for **ONE** cycle
-    - One cycle run has less confidence some time
-    - Many users strongly suggested us to provide **MULTIPLE** cycles
-    - We added a tool to check previous 4 or 5 cycle runs, call _[Consistency Check](http://wwwt.emc.ncep.noaa.gov/mmb/SREF/FCST/COM/web_js/html/prob_snow06h.html)__[   ](http://wwwt.emc.ncep.noaa.gov/mmb/SREF/FCST/COM/web_js/html/prob_snow06h.html)_

- 57 hr

- 45hr

- 33hr

- 21hr

- 9hr

- Verified time

## Benefits
    - Users can focus on a fixed location to see Prob from various historical cycle runs
     - Improve the confidence to the PROB forecast since different cycles use
             different initial conditions
     - Different initial fcst time/initial conditions -> more diverse
Configuration
     Dec15                        Dec16                           Dec17                                Dec18
21z 00  03 06  09z 12  15  18 21z 00  03  06 09z 12  15   18  21z  00   03   06   09z  12   15   18  21z   00   03   06  09z  12  15     
 21z Run   09  12  15  18  21  24  27  30  33   36  39  42   45   48   51   54   57   60   63 
         09z Run: 00   03  06  09 12  15  18   21  24   27  30  33  36   39   42   45   48   51   54    57  60    63   
                                21z Run   00  03  06  09  12   15  18   21  24   27  30   33   36    39   42   45   48    51   54   57   60   63 
                                                        09z Run   00   03  06   09 12   15   18   21   24    27   30   33   36   39    42   45   48   51    54
                                                                                 21z Run   00   03   06   09   12   15   18    21   24   27    30   33   36   39   42
                                                                                                              09z Run  00   03   06   09   12   15    18   21    24   27   30

                                                                 Verified-Time:   Dec17  00z 03z 06z  09z 12z 15z 18z  21z  00z  03z  06z  09z 12z 15z

-     - Users can focus on a fixed location to see Prob from various historical cycle runs
-      - Improve the confidence to the PROB forecast since different cycles use
-              different initial conditions
-      - Different initial fcst time/initial conditions -> more diverse
**Configuration**
-      ***Dec15                        Dec16                           Dec17                                Dec18***
***21z**** 00  03 06  ****09z**** 12  15  18 ****21z**** 00  03  06 ****09z ****12  15   18  ****21z****  00   03   06   ****09z****  12   15   18  ****21z****   00   03   06  ****09z****  12  15 *
-  21z Run   09  12  15  18  21  24  27  30  33   36  39  42   45   48   **51   54   57   60****   63**
-          09z Run: 00   03  06  09 12  15  18   21  24   27  30  33  36   **39****   ****42****   ****45   48**   **51   54    57  60    63**
-                                 21z Run   00  03  06  09  12   15  18   21  24   **27****  ****30   33****   ****36**    **39   42   45   48    51  **** ****54   57   60   63**
-                                                         09z Run   00   03  06   09 12   **15****   ****18   21   24    ****27****   ****30   33   36   39****    ****42   45   48   51**   ** 54**
-                                                                                  21z Run   00   **03****   ****06   09   12   ****15   18    21   24   27****    ****30   33   36   39**   **42**
-                                                                                                               09z Run  00   **03****   ****06   09   12   15****    ****18   21    24   27**   **30**
-                                                                  **Verified-Time: **  **Dec17  00z 03z 06z  09z 12z 15z 18z  21z  00z  03z  06z  09z** **12z 15z**

## SREF web interface for consistency checking

- Left above: Probability page.  Right above: Consistency checking page
- Usage: Click “Click here” button on left page, the right page will pop up automatically,
-  then select a threshold and verified time from right page

## General Behavior of probability

      - Case 1:Closer to verified time, more and more members capture an event

        Case2: Closer to verified time, less and less members capture an event

        Case3: As close to verified time, probability is not steadily  increase or decrease
                    But oscillate with the time

      - In case 1, forecasters have more and more confidence to this forecast

        In case 2, forecasters have less confidence or have confidence of not occurring

        In case 3, the confidence depends on overall probability gain or loss

**    **
-       - Case 1:Closer to verified time, more and more members capture an event
-         Case2: Closer to verified time, less and less members capture an event
-         Case3: As close to verified time, probability is not steadily  increase or decrease
-                     But oscillate with the time
-       - In case 1, forecasters have more and more confidence to this forecast
-         In case 2, forecasters have less confidence or have confidence of not occurring
-         In case 3, the confidence depends on overall probability gain or loss

## Verification

-     Verification with  Low Level Wind Shear (LLWS)
-       LLWS: Wind vector change between surface and 2000feet
-       Severe LLWS is defined as LLWS > 20 knots/2000feet
-       Hazardous to airplane take off/landing
-     SREF LLWS product has
-         - Mean and spread
-         - Prob of LLWS > 20 knots/2000feet
-      Verified Data: EDAS Analysis data
-      GRID-to-GRID (We are developing G2G verification package)
-      Verification time: Nov.10 ~ Dec. 21, 2004, severe LLWS freq ~ 14.7% on average
-      5 cycle forecasts are verified at a specific time:
-       51hr                                                                                 P1
-                         39hr                                                               P2
-                                        27hr                                                P3
-                                                        15hr                                P4
-                                                                              3hr            P5

## Left: Conditional probabilities of different cycles before severe LLWS event happenedRight: Conditional probabilities of different cycles when severe LLWS event not happenCycle1: 51 hr, Cycle2: 39hr, Cycle3: 27hr, Cycle4: 15hr, Cycle5: 03hre.g. For severe LLWS event happen case,  if cycle1’s 51hr fcst prob has 30%(pink), the later cycle’s fcst will increase as closer to the event. Cycle5 (most recent cycle)’s 3hr fcst prob will reach to 50%

## Probability gain and loss of cycles

-   Def: PROB Gain: if  *dP=P2 - P1 >0*
-                      Loss  if  *dP=P2 - P1 < 0*
-   P1 is prob in previous cycle, P2 is prob in current cycle
-   Total Gain or Loss over all previous cycles
-   If Total is gain (>0), confident, otherwise less confident
-   Advantage: rule out the forecast probability Bias by subtraction (is it true?)

## Slide 9

## Prob from different cycles’ forecast shown above: snow prob in DC increase steadily

- But NBC TV Morning Forecast  for Feb 3 (Thu) for DC saying that:
- Feb 1(Tue): light snow on 3th,   Feb 2(Wed):  light snow on 3th,  Feb 3(Thu) morning:  clear sky today
- Feb 2(Wed):  both Operational ETA and GFS forecasts: no snow tomorrow
- Feb 3(Thu) actual weather: Snow from noon to  3:00 pm, obvious accumulation on the ground,

## Conclusion

   Probability of multi-cycles are put together 

      -  Review PROB from different cycles at a specific area on fixed 
                verified time
      -  Improve the forecast confidence from probability consistency, which can
               be quantified with PROB gain /loss magnitude
      -  PROB gain and loss over different cycles are bias free? (need to discuss)
      -  Has been built into a tool and added to current SREF web page

-    Probability of multi-cycles are put together
-       -  Review PROB from different cycles at a specific area on fixed
-                 verified time
-       -  Improve the forecast confidence from probability consistency, which can
-                be quantified with PROB gain /loss magnitude
-       -  PROB gain and loss over different cycles are bias free? (need to discuss)
-       -  Has been built into a tool and added to current SREF web page