# Presentation: 160425

## Development of Fast and Accurate Neural Network Emulations of Long Wave Radiation for the NCEP Climate Forecast System Model V. Krasnopolsky, M. Fox-Rabinovitz, S. Lord, Y.-T. Hou, and A. Belochitski, 

**Acknowledgments: **Drs. H.-L. Pan, S. Saha, S. Moorthi, and M. Iredell
- for their useful consultations and discussions. The research is supported by the NOAA CPO CDEP CTB grant NA06OAR4310047.

***NOAA 32nd Annual*** ***Climate Diagnostics and Prediction Workshop, Oct. 22-26, 2007, Tallahassee, FL***

## OUTLINE

**CTB project: Transferring the developed Neural Network (NN) methodology applied to physics of NCEP CFS**
**Goals: (a) improving computational efficiency of radiation of the CFS model, (b) providing a practical opportunity for using new more sophisticated and time-consuming radiation and other physics components for the CFS model **
**NN applications to the CFS model:** Development of NN emulations for LW and SW radiation of the CFS model (SWR – work in progress)
**Background information on the NN approach**
**Development of NN emulations of LWR** (Long-Wave Radiation) for the CFS model and evaluation of their ***accuracy*** vs. the original LWR
**Validation of NN emulations** of LWR through the CFS model run using the LWR NN emulations vs. the control run using the original LWR
- Conclusions and plans

## Background

- Any parameterization of model physics is a relationship or **MAPPING** (continuous or almost continuous) between two vectors: a vector of input variables, *X*, and a vector of output variables, *Y*,
- NN is a generic approximation for **any** continuous or almost continuous mapping given by a set of its input/output records:
- SET = {*X**i*, *Y**i*}*i* *= 1, ...,N*

## Neural Network

***Y = F******NN******(X)***

**Continuous Input to Output Mapping**

**Neuron**

## Major Advantages of NNs:

- NNs are *generic, *very* **accurate and convenient* mathematical (statistical) models which are able to emulate numerical model components, which are complicated nonlinear input/output relationships (continuous or almost continuous mappings ).
- NNs are ***robust*** with respect to random noise and fault- tolerant.
- NNs are *analytically differentiable* (training, error and sensitivity analyses): almost free Jacobian!
- NNs are well-suited for parallel and vector processing
- NNs emulations are **accurate and fast but there is **
_**NO FREE LUNCH**_**!**
- Training is a complicated and time consuming nonlinear optimization procedure; _*however, training should be done only once for a particular application!*_

## NN Emulations of Model Physics Parameterizations Learning from Data

**GCM**

***X***

***Y***

**Parameterization**

***F***

***X***

***Y***

**NN Emulation**

***F******NN***

***Training***
***Set***

***...******, {******X******i******, ******Y******i******}, ...  ************X******i************ D******phys***

**NN Emulation**

***F******NN***

## CFS Model: LWR NN emulation 

-         ***NN dimensionality and other parameters***:
- 591 inputs: 12 variables (pressure, T, moisture, cloudiness parameters, surface emissivity, gases (ozone, CO2)
- 69 outputs: 6 variables (heating rates, fluxes)
- Dimensionality of the NN training space: 50,000 to 100,000
- Training and testing data sets are produced by saving inputs and outputs of LWR during 2-year T126L64 CFS simulations; half of the data is used for training and another half for validation or NN accuracy estimation vs. the original LWR
- Number of neurons for NN versions: 50 to 150

## NN Approximation Accuracy for Heating Rates (on independent data set) vs. Original Parameterization (all in K/day), Mean HR = -1.88, HR = 2.28

| Parameterization | NN | Bias
(K/day)
 | PRMSE(K/day)
 |
| --- | --- | --- | --- |
| LWR | NN75 | 1.5 10-3 | 0.37 |
|  | NN95 | 4. 10-3 | 0.28 |
|  | NN200 | 0.3 10-3 | 0.21 |

_**NN Computational Performance:**_** **LWR NN emulations are *two orders of magnitude faster* than the original LWR** **
_**Overall**__** CFS model computational**__ __**performance:**_** ***~30% faster* when using LWR NN emulations vs. the original LWR

## Profiles of RMSE for NNs, in K/day

## Individual HR Profiles for NN75

## Global and time mean Seasonal & Daily T-850 Temperatures and their Differences

**Global and time mean Seasonal & Daily T-850 Temperatures and their Differences**

**Max Difference = 0.06 K**

**Max Difference = 0.1 K**

## Top of Atmosphere Upward LWR Flux Differences

**Season 2: **
**0-5 W/m2, max 10-20 W/m2**
**Season 4: **
**0-5 W/m2, max 10-20 W/m2**

## Differences between the seasonal (1- 4) CFS runs  with NN LWR emulation and with the original LWR

|                 Field |   Mean  |   Max  |
| --- | --- | --- |
| Top of Atm. Upward LWR Flux (in W/m2)  |  0 – 0.5  |  10 – 20  |
| Surface Downward LWR Flux (in W/m2)  |  0 – 0.5  |  10 – 20  |
| Zonal mean T (in K) |  0 – 0.5 | 1.5 – 2.5 |
| Zonal mean U (in m/s) |   0 – 1 |      2 |
| Zonal mean V (in m/s) |  0 – 0.1 | 0.2 – 0.4 |
| T-500 (in K) |   0 – 1 |   2 - 3 |
| Relative Humidity (in %) |   0 – 2 |   4 - 6 |

**NOTE-1: For seasons 1-4 the mean and maximum differences are about the same, i.e. the differences**** are not increasing**** during seasonal CFS model integrations.**
**NOTE-2: The differences are ****within the uncertainty**** of observations or reanalysis**

## Day-2: Differences between CFS runs  with NN LWR emulation and with the original LWR 

|                   Field  |   Mean |   Max |
| --- | --- | --- |
| Upward Top of Atmosphere LWR Flux (in W/m**2) |   0 - 2  |  10 - 20  |
| Surface Upward LWR Flux (in W/m**2) |   0 - 2  |  10 - 20 |
| T-850 (in K)  |  0 – 0.2  | 0.5 – 1.5  |
| U-850 (in m/s) |  0 - 0.1  | 0.5 – 1  |

**Day Seven: T- 850 Differences (in K): 0 – 1, max 2 - 3**

**Multi-year Mean Upward Top of Atmosphere LWR Flux (in W/m2) ****Differences: 0-5, max 10-20**

## Recent Journal and Conference Papers

_***Journal Papers:***_** **
**V.M. Krasnopolsky, M.S. Fox-Rabinovitz, and A. Beloshitski, 2007, “Compound Parameterization for a Quality Control of Outliers and Larger Errors in NN Emulations of Model Physics", *****Neural Networks*****, submitted.**
**V.M. Krasnopolsky, M.S. Fox-Rabinovitz, and A. Beloshitski, 2007, “Decadal climate simulations using accurate and fast neural network emulation of full, long- and short wave, radiation. *****Mon. Wea. Rev.,***** accepted.**
**V.M. Krasnopolsky, 2007, “Neural Network Emulations for Complex Multidimensional Geophysical Mappings: Applications of Neural Network Techniques to Atmospheric and Oceanic Satellite Retrievals and Numerical Modeling”, *****Reviews of Geophysics*****, in press **
**V.M. Krasnopolsky, 2007: “Reducing Uncertainties in Neural Network Jacobians and Improving Accuracy of Neural Network Emulations with NN Ensemble Approaches”, *****Neural Networks*****, Neural Networks, 20, pp. 454-46**
**V.M. Krasnopolsky and M.S. Fox-Rabinovitz, 2006: "Complex Hybrid Models Combining Deterministic and Machine Learning Components for Numerical Climate Modeling and Weather Prediction", *****Neural Networks*****, 19, 122-134 **
**V.M. Krasnopolsky and M.S. Fox-Rabinovitz, 2006: "A New Synergetic Paradigm in Environmental Numerical Modeling: Hybrid Models Combining Deterministic and Machine Learning Components", *****Ecological Modelling*****, v. 191, 5-18**
_***Conference Papers;***_
**V.M. Krasnopolsky, M. S. Fox-Rabinovitz, Y.-T. Hou, S. J. Lord, and A. A. Belochitski, 2007,** “**Development of Fast and Accurate Neural Network Emulations of Long Wave Radiation for the NCEP Climate Forecast System Model”, submitted to the NOAA 32nd Annual Climate Diagnostics and Prediction Workshop**
**V.M. Krasnopolsky, M. S. Fox-Rabinovitz, Y.-T. Hou, S. J. Lord, and A. A. Belochitski, 2007, “Accurate and Fast Neural Network Emulations of Long Wave Radiation for the NCEP Climate Forecast System Model”, submitted to 20th Conference on Climate Variability and Change, New Orleans, January 2008**
**M. S. Fox-Rabinovitz, V. Krasnopolsky, and A. Belochitski, 2006: “Ensemble of Neural Network Emulations for Climate Model Physics: The Impact on Climate Simulations”, *****Proc.,***** *****2006 International Joint Conference on Neural Networks, *****Vancouver, BC, Canada, July 16-21, 2006, pp. 9321-9326, CD-ROM**

## Conclusions 

- Developed NN emulations of LWR for the CFS model show _***high accuracy and computational efficiency***_. Due to a near zero bias, NN errors are not accumulating in time during model integrations.
- Validation of NN emulations for LWR through CFS model runs using the NN emulations vs. the CFS model run with the original LWR show a _***close similarity***_ of the runs, namely the differences are _***within the***__** **__***uncertainty of observational data and/or reanalysis for seasonal predictions, climate simulations, and short- to medium-range forecasts***_***. ***
- Obtaining small and not accumulating in time differences between the NN and control runs is crucially important for the CFS model as a _***complex nonlinear system***_
- Near-term plans: Development of SWR NN emulations is in progress. Using SWR NN emulations will increase the _***o***__**verall CFS model computational performance by ~ **__***60 - 70%***_** **
- Opportunity for using new more sophisticated and time-consuming radiation and other physics components
- Future developments: Other model physics components
- Potential applications of the NN approach to GFS and DAS

## Additional Plots

## NN - Continuous Input to Output Mapping

_***Multilayer Perceptron:***_** Feed Forward, Fully Connected**

**Input **
**Layer**

**Output **
**Layer**

**Hidden **
**Layer**

***Y = F******NN******(X)***
***Jacobian !***

***Neuron***

***t******j***

- V. Krasnopolsky & M. Fox-Rabinovitz, Neural Networks for Model Physics

- 5/31/2007;  GFDL

## Top of Atmosphere Upward LWR Flux Global Seasonal and Daily Differences

## U Zonal Mean Differences

**Season 1: 0 – 1 m/s, max 3 - 4 m/s **
**Season 2: 0 – 1 m/s, max 2 m/s**
**Season 3: 0 – 1 m/s,  max 2 m/s**
**Season 4: 0 -  1m/s,  max 2 m/s**

## T Zonal Mean Differences

**Season 1: 0 – 0.5 K, max 1.5-2 K **
**Season 2: 0 – 0.5 K, max 1.5-2.5 K**
**Season 3: 0 – 0.5 K,  max 1.5-2 K**
**Season 4: 0 – 0.5 K,  max 2-3 K**

## V Zonal Mean Differences

**Season 1: 0 – 0.1 m/s, max 0.2 - 0.4 m/s **
**Season 2: 0 – 0.1 m/s, max 0.2 - 0.3 m/s**
**Season 3: 0 – 0.1 m/s,  max 0.2 - 0.4 m/s**
**Season 4: 0 -  0.1m/s,  max 0 .2 - 0.4 m/s**

## Day Two: Upward Top of Atmosphere LWR Flux) Differences near 0 - 2 W/m2, a few minor max of 10 - 20 W/m2

- Orig. - NN

## Day Two: T-850 Differences near 0 – 0.2 K, max 0.5 – 1.5 K

## Day Two: U-850Differences 0 - 0.1 m/s, max 0.5 – 1 m/s 

## Day Seven: T-850Differences near 0. – 1.0 K, max 2 - 3 K

## 2-Year Mean OLR (Upward Top of Atmosphere LWR Flux), in W/m2 Differences 0-5 W/m2, max 10-20 W/m2

**2-Year Mean OLR (Upward Top of Atmosphere LWR Flux), in W/m2**** ****Differences 0-5 W/m2, max 10-20 W/m2**

## Near term (FY07 and Year-2 of the project) science plans 

- Completing work on LWR NN emulation
- Generating more representative data sets
- Continuing training and validation of LWR NN emulations for the CFS model
- Continuing experimentation and validation of seasonal climate predictions with LWR NN emulations
- Refining the NN methodology for emulating model physics
- Work on an NN ***ensemble*** approach aimed at improve accuracy of NN emulations
- Develop a compound parameterization for ***quality control*** (QC) and for dynamical adjustment of the NN emulations
- Refining experimentation and validation framework
- Continuation of development of NN emulations for the CFS model radiation block
- Analysis of CFS ***SWR***, generating initial training data sets, and developing initial SWR NN emulations
- Initial experiments with ***SWR*** NN emulation for the CFS model
- Initial development of the project ***web site*** and its link to a relevant NCEP and/or CTB web sites

## Future (FY08 and Year-3 of the project) science plans 

- Completing work on ***SWR*** NN emulation for the CFS model
-      - Training and validation of SWR NN emulations
- -  Performing seasonal climate predictions with SWR NN emulation
- Performing extensive CFS seasonal climate predictions with developed NN emulations for the CFS ***full radiation block*** (LWR & SWR), and validating their overall impact computational efficiency
- Completing the ***transition*** of the developed NN radiation products into the NCEP operational CFS
- Completing development of the ***project web site*** and using it for interactions with potential NCEP users and other users in educational and research communities
- Preparation for ***future*** developments: other CFS model physics components, and potential applications to other NCEP systems like GFS and climate predictions