NOAA Technical Report OAR CPO-X

Climate Model 

Development Task Force 

Final Report

 https://doi.org/10.7289/V5/TR-OAR-CPO-6
NOAA Technical Report OAR CPO-X

Climate Model Development Task Force Final Report

https://doi.org/10.7289/V5/TR-OAR-CPO-6

Climate Program Office

Silver Spring, Maryland

August 2017

Report Team Authors

James L. Kinter1, Suranjana Saha2, Stephen M. Griffies3, with contributions and review by CMDTF participants1

Production Assitance

Alison Stevens4

Affiliations

1

George Mason University, Fairfax, Virginia

2

Environmental Modeling Center, NOAA/NWS/NCEP, College Park, Maryland

3

NOAA/Geophysical Fluid Dynamics Laboratory, Princeton, New Jersey

4

NOAA/OAR/Climate Program Office, Silver Spring, Maryland

United States

Department of Commerce

Wilbur Ross

Secretary

National Oceanic and

Atmospheric Administration

Benjamin Friedman

Acting Undersecretary for Oceans 

and Atmospheres

Office of Oceanic and

Atmospheric Research

Craig McLean

Assistant Administrator

Notice from NOAA

Mention of a commercial company or product does not constitute an endorsement by NOAA/OAR. Use of information from this 

publication concerning proprietary products or the tests of such products for publicity or advertising purposes is not authorized. 

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not 

necessarily reflect the views of the National Oceanic and Atmospheric Administration.

1 See Appendix B for list of participants.

Climate Model Development Task Force Final Report
3

Executive Summary

The Climate Model Development Task Force 

(CMDTF; 2014-2017) was formed to bring together 

representatives from National Oceanic and Atmospheric 

Administration (NOAA) laboratories, the NOAA National 

Centers for Environmental Prediction (NCEP), and 

Climate Process Teams research centers and other 

projects funded by the NOAA Modeling, Analysis, 

Prediction and Projections (MAPP) program, with an 

interest in furthering the coupled modeling capability 

for sub-seasonal to seasonal and longer time scale 

prediction. The CMDTF provided a platform to discuss 

model development efforts and best practices. The 

CMDTF made contributions to the planning process 

including defining requirements for the transition 

from operations to research (O2R) and for future 

model configurations and output, and defining metrics 

and a test harness for evaluation of climate models 

and climate predictions. A number of the projects 

that were engaged in the CMDTF contributed to 

designing and developing subgrid-scale and land 

surface process representations, component models, 

coupling infrastructure, and multi-model combination 

strategies. The CMDTF served the functions of both 

keeping the diverse projects in these categories 

informed of progress and findings as well as stimulating 

collaboration among these projects.

Among several contributions from the CMDTF, special 

emphases were placed on better representing: 

atmospheric shallow and deep convection, including its 

initiation, its evolution and magnitude, and its interaction 

with turbulent and cloud processes; the interactions 

between clouds, aerosols and radiation; the structure 

and evolution of ozone concentration in the atmosphere 

and its interaction with other chemically active species 

and stratospheric processes in general; soil-hydrologyvegetation interactions, including more accurately 

initializing soil moisture and better representing the 

roles of lakes in coupled system prediction; and ocean 

circulation and the interaction of the upper ocean with 

the lower atmosphere. The CMDTF also contributed 

in the areas of software infrastructure for coupling 

model components, diagnosing and evaluating coupled 

models and coupled model predictions, and optimally 

composing multi-model ensembles.

The vigorous CMDTF discussions exposed several 

critical scientific and organizational challenges 

that must be addressed as the National Weather 

Service evolves its long-range weather and climate 

outlook capabilities. Among the scientific challenges 

are correcting the relative lack of sensitivity of the 

atmospheric component to variations in the surface 

conditions, particularly over continental areas; 

incorporating atmospheric composition (notably 

aerosol concentration) into the new atmospheric 

dynamical core; diagnosing the behavior of the 

complex coupled Earth system; and addressing several 

issues associated with increasing spatial resolution 

of the models, especially, the problems of scaleaware parameterization, trade-offs among resolution 

choices in different component models, and resolution 

implications for coupling strategies. Organizationally, 

the CMDTF recognized that resources to support 

the transition from research to operations (R2O) are 

insufficient, including base funding and project grant 

funds for collaborative research, and high-performance 

computing resources. There are also important 

challenges in providing to researchers the appropriate 

access to model codes and development tools, and 

ensuring that collaborative groups employ best modern 

software engineering practices. A chronic issue 

has been the difficulty in fostering and maintaining 

communication between rank-and-file NCEP model 

developers and researchers outside NCEP or NOAA. 

Finally, the CMDTF clearly indicated the need for a 

detailed development plan – including a timeline with 

benchmarks and milestones, and pathways for external 

partners to contribute – for the next generation coupled 

Climate Forecast System that goes beyond the planning 

already in place for the atmospheric component of the 

Next Generation Global Forecast System (NGGPS).

3
4

1. Overview

The Climate Model Development Task Force (CMDTF) 

was created in 2014 by the Modeling, Analysis, 

Predictions and Projections (MAPP) program of the 

Climate Program Office (CPO) in the Office of Oceanic 

and Atmospheric Research (OAR) of the National 

Oceanic and Atmospheric Administration (NOAA), 

based on discussions with Bill Lapenta. Dr. Lapenta 

was at the time Director of the Environmental Modeling 

Center (EMC) of the National Centers for Environmental 

Prediction (NCEP), National Weather Service (NWS), 

NOAA, and he sought to have the CMDTF help EMC 

make progress on Climate Forecast System (CFS) 

coupled model development. Much of the work of the 

CMDTF was focused on version 2 of CFS (CFSv2), 

which became operational in 2011, or its component 

models. 

The CMDTF mission was to coordinate MAPPfunded modeling projects and work in synergy with 

climate model development efforts in NOAA labs/

centers with an emphasis on research contributing 

to the advancement of the NCEP CFS via activities 

associated with the NCEP Climate Test Bed (CTB). The 

MAPP-funded projects included Climate Process Teams 

(CPT), the Center for Ocean-Land-Atmosphere Studies 

(COLA) and several individual principal investigator 

(PI) led activities. The CPTs are small multi-institutional 

groups of observationalists, theoreticians, small-scale 

process modelers, and scientists at modeling centers 

working closely together to improve parameterizations 

of a particular process in coupled climate or Earth 

system models. 

The Task Force also coordinated these activities with 

the NWS Office of Science and Technology Integration 

(STI) Next-Generation Global Prediction System 

(NGGPS) project, with some of the projects being 

co-funded with NGGPS. The NGGPS was organized 

by NWS in 2015 as a means of accelerating progress 

toward a new global prediction system, in response to 

recommendations from the University Corporation for 

Atmospheric Research (UCAR) Community Advisory 

Committee for NCEP (UCACN). The UCACN had called 

for a unified modeling strategy initially aimed at the 

global prediction problem and eventually encompassing 

all environmental prediction within the purview of NCEP. 

Toward that end, NWS funded NGGPS sub-projects 

at NCEP, the Geophysical Fluid Dynamics Laboratory 

(GFDL), the Earth System Research Laboratory 

(ESRL) and several entities external to NOAA through a 

competitive grants program.

In the course of its three-year existence, the CMDTF 

conducted over 30 monthly conference calls, and 

developed several initiatives for helping to foster 

the interaction between the model developers within 

EMC and the external community. The full Task Force 

charge is given in Appendix A. A list of projects and PIs 

participating in the Task Force is given in Appendix B. 

The sections below provide details of the Task Force 

activities, contributions to planning and development, 

opportunities, and challenges.

2. CMDTF Activities

The CMDTF activities included co-leads (see Appendix 

B) conference calls to discuss the mission and scope of 

CMDTF and conference call topics and CMDTF monthly 

conference calls. The monthly conference calls, some 

with presentations, covered the following topics:

1. Inventories of MAPP funded projects and 

 modeling efforts in other modeling centers, 

 including MAPP-CTB funded projects, 

 MAPP-funded modeling infrastructure projects, 

 and both component development and coupling 

 efforts in other centers.

Modeling centers:

• NOAA/OAR Geophysical Fluid 

Dynamics Laboratory (GFDL)

• National Center for Atmospheric 

Research (NCAR)

• United Kingdom Meteorological 

Climate Model Development Task Force Final Report
5

Office (UKMO)

• European Centre for Medium-range 

Weather Forecasts (ECMWF)

• NOAA/OAR Global Test Bed (GTB)

Dates: Several

2. NCEP updates on CFSv3 development strategy/ 

 plan/timeline.

 Dates: 4/22/14, 5/27/14, 6/24/14, 8/26/14, 

 9/23/14, 2/15/15, 8/25/15, and 2/23/16

3. PIs’ inputs to NCEP CFS planning process 

 including O2R requirements

 Dates: 10/14/14 (see Appendix C) and 

 9/22/15 

4. Metrics/protocols for evaluating NCEP climate 

 models and forecasts 

 Dates: 5/27/14 and 8/26/14

5. Town Hall Meeting: NWS Model Development 

 Forum 

 Date: 1/5/15 (95th American Meteorological 

 Society Annual Meeting)

6. Modeling infrastructure/framework

 Dates: 6/24/14, 2/15/15, 3/17/15, 3/29/16, 

 5/31/16, and 6/24/16

7. Updates on the UCACN Modeling Advisory 

 Committee (UMAC) and EMC plans

 Date: 8/29/16

8. Plans for GFDL contributions to the global 

 prediction capability at NCEP, including the 

 atmospheric dynamical core (FV3) and the ocean 

 model (MOM6)

 Dates: 10/18/16 and 11/29/16

Task Force activities also included a face-to-face 

Climate Process Team (CPT) meeting with NCEP and 

a breakout session on modeling and data assimilation 

in the Climate Test Bed Science Meeting on 11/10/15. 

The agenda is available at http://www.nws.noaa.gov/

ost/CTB/mts-ctb15-d2.htm. Topics discussed in the 

breakout session were the CFSv3 plan, performance 

metrics and diagnostics, the NOAA Environmental 

Modeling System (NEMS), and ways to facilitate 

external community contributions to CFSv3. In the faceto-face meeting, the leads of individual CMDTF projects 

discussed with NCEP collaborators issues relating to 

atmospheric modeling, land modeling, and R2O/O2R. 

3. Planning Contributions 

The CMDTF included representatives from NCEP, 

GFDL, several CPTs, COLA, and other projects with an 

interest in furthering the coupled modeling capability 

for sub-seasonal to seasonal and longer time scale 

prediction. The CMDTF provided a platform for funded 

PIs and modeling centers (e.g., NCEP, GFDL) to 

discuss model development efforts and best practices. 

The CMDTF contributions to the planning process 

included:

• Defining requirements for operationsto-research (O2R) support of researchers

• Defining user (e.g. Climate 

Prediction Center) requirements for 

future model configurations and output

• Defining metrics for climate model evaluation

• Defining a test harness for evaluating development 

milestones in the evolution of a new coupled 

prediction system (see section 4.10). 

Several individuals involved in the CMDTF were also 

involved in activities of great relevance to the Task 

Force, notably:

• Developing subgrid-scale physical 

process representations 

• Developing component models 

at GFDL and other labs

• Developing a coupled modeling capability with 
6

the NOAA Environmental Modeling System 

(NEMS), based on the National Unified Operational 

Prediction Capability (NUOPC) and the Earth 

System Modeling Framework (ESMF)

• Designing a system architecture suitable for 

both operational prediction on sub-seasonal 

to seasonal (S2S) time scales and original 

research on prediction and predictability

• Developing a new multi-model combination 

strategy to produce an optimized single 

forecast from ensemble forecasts

• Applying the new strategy to the multi-model 

ensemble forecasts provided by the international 

Sub-seasonal to Seasonal (S2S) Prediction 

Project and the North American Multi-Model 

Ensemble (NMME) Project to improve S2S 

forecast skill of North American precipitation 

and surface air temperature. (Note: A separate 

project called SubX was solicited and initiated 

during the period of the CMDTF activities, but was 

largely outside the purview of the Task Force.) 

• Examining specifically how land surface 

factors contribute to the forecast skill of 

precipitation and air temperature 

• Improving the quality of the land surface initial 

states and evaluating the impact of land surface 

initialization on the S2S forecast skill of North 

American precipitation and surface air temperature

The CMDTF served the functions of both keeping 

the diverse projects in these categories informed 

of progress and findings as well as stimulating 

collaboration among these projects.

4. Key Model Development Contributions

CMDTF projects made several key contributions 

to EMC model development. These included 

improvements in the physical parameterization 

of processes associated to clouds and boundary 

layer, aerosols, land surface, and lake effects, and 

the development of the NEMS modeling software 

infrastructure. Although originally focused on 

advancing EMC’s climate modeling capabilities, 

these projects ultimately contributed to the overall 

advancement of the EMC modeling suite, given the 

current unified modeling approach.

4.1 Soil Moisture Bias 

In several sub-projects relating to soil moisture 

(COLA: Predictability and Prediction of Climate from 

Days to Decades, J. Kinter), an examination of the 

behavior of the water cycle in CFSv2 reforecasts and 

reanalysis revealed that biases in soil moisture grow 

throughout the duration of forecasts due largely to 

systematic errors in precipitation (Dirmeyer 2013). 

The Noah land surface scheme showed the necessary 

relationships between evaporation and soil moisture 

for land-driven climate predictability, but CFSv2 cannot 

maintain the link between precipitation and antecedent 

soil moisture as strongly as in the real atmosphere, 

hampering prediction skill (Dirmeyer 2013, Dirmeyer 

and Halder 2017a). Because the precipitation errors 

vary or even change sign with lead-time and season, 

the utility of reforecast programs for bias correction is 

evident for operational forecast systems.

The National Aeronautics and Space Administration 

(NASA) Goddard Space Flight Center (GSFC), and 

NCAR global models in uncoupled (land model 

only), coupled and reanalysis configurations were 

compared to observations of daily soil moisture from 

19 networks over the United States (Dirmeyer et al. 

2016). Model soil moistures were evaluated using 

three metrics: standard deviation in time, temporal 

correlation (memory), and spatial correlation (length 

scale). Models do relatively well in capturing largescale variability of metrics across climate regimes, but 

they poorly reproduce observed patterns at scales of 

hundreds of kilometers and smaller. Reanalyses do 

not outperform free-running models, showing model 

parameterization is more important than initialization 

Climate Model Development Task Force Final Report
7

for determining land-atmosphere interaction in models.

Land surface initialization was further evaluated in 

retrospective forecasts from CFSv2 on sub-seasonal 

to seasonal time scales (Dirmeyer and Halder 

2017a,b). Soil moisture memory is the most broadly 

important element for significant improvement of 

realistic land initialization on forecast skill, extending 

skill of daily forecasts by 1-4 days in most locations, 

and 5-day means by 1-2 pentads. Importantly, errors 

in land surface initialization and shortcomings in the 

parameterization of atmospheric processes sensitive 

to surface fluxes may have greater consequences than 

previously recognized, the latter exemplified by the 

lack of impact on precipitation forecasts even though 

the simulation of boundary layer development is shown 

to be greatly improved with realistic soil moisture 

initialization.

4.2 Representation of Soil-Hydrology-Vegetation 

Interactions

The overarching goal of this collaborative effort among 

NCAR (F. Chen and M. Barlage), NCEP (M. Ek, R. 

Yang, and J. Meng), and the University of Texas at 

Austin (Z-L Yang), was to improve the CFS forecast 

skill by enhancing the representation of soil-hydrologyvegetation interactions through the use of the new 

community Noah2 with Multiple-Parameterizations 

(Noah-MP) land surface model (LSM). Soil moisture 

memory was identified as a key factor in determining 

seasonal predictability in climate forecast systems. 

This project leveraged on the previous work of the 

NCEP/EMC land team on testing Noah-MPv1 in 

CFSv2, and further evaluated and improved the 

newly released community Noah-MPv2. Numerous 

enhancements in Noah-MPv2 were released in April 

2016, and this new version has been successfully 

implemented in CFSv2. The land-cover and soil2 http://www.emc.ncep.noaa.gov/mmb/gcp/noahlsm/Noah_LSM_

USERGUIDE_2.7.1.htm

texture datasets in CFSv2 were replaced by the new

global International Geosphere-Biosphere Programme 

(IGBP) vegetation-classification data (derived from 

the Moderate Resolution Imaging Spectroradiometer 

- MODIS) and global soil type (State Soil Geographic 

- STATSGO) datasets, which is consistent with recent 

community efforts to improve the specification of 

surface characteristics.

Various CFS sensitivity experiments were conducted 

with the climatology green vegetation fraction (GVF) 

replaced by near real-time satellite observations, 

and with the Noah-MP dynamic vegetation model to 

generate inter-annual GVF and leaf area index (LAI) 

variability. To examine these effects on CFS prediction 

skill, T126 CFS reforecast experiments were carried 

out for 11 selected years (1982, 1987, 1996, 1988, 

2000, 2007, 1986, 1991,1999, 2011, 2012) with 4 

ensemble members (00z of May 1 to May 4). The 11 

years are composed of 3 years with cold El Niño and 

the Southern Oscillation (ENSO) states, 3 ENSOwarm, and 5 neutral years. The experimental CFS with 

the Noah-MP dynamic vegetation model improves the 

summer seasonal precipitation anomaly-correlation 

scores over the western states with sparse vegetation 

and over the central Great Plains where the soil 

moisture memory and land-atmospheric coupling 

strength showed significant impacts on seasonal 

precipitation prediction. The CFS with satellite GVF 

showed improvement over most of the U.S., indicating 

the important role of vegetation characteristics in 

Noah. Compared to the CFS operational setting 

with the Noah LSM, the hindcast with Noah-MP 

significantly improved the precipitation prediction skill 

over the Pacific Northwest and the Gulf states, and the 

CFS with Noah-MP dynamic vegetation had the best 

performance over the central Great Plains. Similarly, 

the CFS coupled to Noah-MP performed better in 

predicting 2m temperature anomalies.
8

4.3 Lake Process Modeling

This project (J. Jin, Utah State University; M. Ek and 

Y. Wu, EMC/NCEP) was to incorporate a numerically 

efficient, physically based lake model into NCEP’s 

operational CFSv2 in order to advance climate 

prediction at intraseasonal to interannual (ISI) time 

scales in North America. North America has the largest 

total lake volume and surface area of any continent 

on Earth. Some lakes (e.g., the Great Lakes) also 

trigger severe storms during early winter and spring. 

However, in CFSv2, running at approximately 100-

km resolution, lake processes and their interactions 

with the atmosphere are largely neglected, degrading 

climate forecasting skill.

An existing physically based lake model was 

coupled to CFSv2 in order to dynamically predict 

lake processes and their effects on climate in North 

America at ISI time scales. The lake model selected 

was the freshwater lake (FLake) model, which is 

a one-dimensional, two-layer energy and mass 

balance model. It includes parameterizations of 

lake thermocline, lake ice and snow, and surface 

momentum, water, and heat fluxes. FLake has been 

implemented in several operational and research 

climate models across the world, resulting in improved 

predictions of lake-atmosphere interactions and 

thermal conditions for different-sized lakes at hourly 

to interannual time scales. However, as shown in 

Fig. 1, the original offline FLake model is unable 

to accurately reproduce the seasonal cycle of lake 

surface temperature (blue line). Essentially, the errors 

result from the oversimplified model structure of FLake 

with only two model layers. For this project, FLake 

was modified by adding one additional model layer to 

better capture the vertical lake temperature profile. 

The three-layer FLake model significantly improved 

lake surface temperature simulations (red line) when 

compared to buoy station observations (solid dark line) 

and the MODIS data (dashed dark line).

The original and improved FLake models was also 

applied to simulating lake ice in the Great Lakes 

(Fig. 2). The observations are for ice fraction, but the 

simulations are for ice thickness. Although these two 

variables are not directly comparable, one can examine 

their spatial distribution. Figure 2 shows that the original 

FLake greatly underestimates the lake ice. With the 

modified Flake model, the spatial distribution of the 

simulated lake ice is significantly improved. Thus, it was 

concluded that the change in the FLake model structure 

contributed to better predictions of lake processes in 

the climate system. 

Currently, retrospective forecasts with the coupled CFSFLake model are being performed for historical periods 

on a high-performance computing platform and will be 

quantitatively evaluated using standard NCEP metrics 

for model evaluation with a focus on lake-related 

processes. The coupling work will provide a framework 

for the next CFS (version 3). The biggest challenge 

for this project is to generate sufficient ensemble 

members for a long-term period to objectively evaluate 

the performance of the coupled model and accurately 

identify the role of the lakes in North America in the 

climate system.

4.4 Ocean Circulation Model Codes at NCEP

Ocean circulation modeling at NCEP has traditionally 

made use of two codes: MOM from NOAA/GFDL and 

the Hybrid Coordinate Ocean Model (HYCOM) from 

the US Navy. These models serve complementary 

roles, with MOM in use for seasonal predictions, and 

HYCOM being used for shorter range coastal and 

weather forecasting. With the release of MOM6, this 

split between two ocean models at NCEP is being 

reconsidered, since MOM6 incorporates many of the 

numerical features formerly unique to HYCOM. The 

MOM6 project is led by GFDL scientists Alistair Adcroft, 

Robert Hallberg, and Stephen Griffies, in collaboration 

with other GFDL scientists and engineers, as well as 

a growing national and international community of 

developers.

Climate Model Development Task Force Final Report
9

Recent discussions involving GFDL, NCEP, Navy, and 

university PIs have led some to consider merging key 

features of HYCOM into MOM6. These discussions 

originated from a meeting at NCEP on 3-4 October 

2016, where a number of key U.S. ocean model 

developers discussed the pros and cons of various 

numerical methods for ocean modeling. As a result of 

that meeting, it became clear that MOM6 offered the 

optimal baseline to serve the many needs of operations 

and research.

MOM6 has been chosen by the National Science 

Foundation (NSF) sponsored Community Earth System 

Model (CESM) leads to serve as the ocean code for 

their new climate model development for CESM version 

3 - to occur after the Coupled Model Inter-comparison 

Project, phase 6 (CMIP6) is completed. Consequently, 

there are a number of distinct U.S. modeling threads 

that are aligning with the development and use of 

MOM6.

In addition to the above developments of ocean code, 

GFDL and NCEP/EMC are participating in a two-year 

(Aug 2016 to July 2018) MAPP-funded project to 

incorporate WAVEWATCH III into MOM6 as part of the 

development of new weather and seasonal prediction 

models at NCEP. The PIs for this project are Stephen 

Griffies, Alistair Adcroft and Robert Hallberg at GFDL; 

Arun Chawla, Suranjana Saha, and Hendrik Tolman at 

EMC; and Stephen Penny at University of Maryland. 

This project offers a further strong link between GFDL 

and NCEP, thus complementing the atmospheric 

modeling links facilitated by the use GFDL FV3 

atmospheric dynamical core for the new NCEP weather 

forecasting model.

4.5 Triggering Convection

The conditions necessary for convection to be initiated 

in the atmospheric component of a coupled model are 

referred to as the triggering conditions. In this project 

(Kinter, COLA/GMU), a hypothesis was formulated 

that the triggering conditions in various versions of 

the NCEP global models (CFSv2, the Global Forecast 

System - GFS and the Global Ensemble Forecast 

System - GEFS) were responsible for some of the 

bias in precipitation, notably, the intensity of rainfall. 

A series of tests were made with alternative versions 

of the triggering conditions by COLA (Bombardi et al. 

2015; 2016). In particular, the Heated Condensation 

Framework (HCF; Tawfik and Dirmeyer 2014; 2015a; 

2015b) was inserted in CFSv2 and several re-forecast 

experiments were run. In one set of experiments 

(Bombardi et al. 2015), the HCF trigger was used in 

addition to the trigger applied in the original convection 

scheme of CFSv2 (Simplified Arakawa-Schubert, SAS; 

Arakawa and Schubert, 1974; Pan and Wu 1995; 

Han and Pan 2011), which has the effect of allowing 

convection to occur more frequently, which improves 

the rainfall amounts in monsoon regimes such as the 

Indian summer monsoon (Fig. 3).

In another set of experiments (Bombardi et al. 2016), 

the HCF trigger was applied to the updated SAS 

scheme (new-SAS) as a means of restricting the 

initiation of convection. By causing the convection to 

occur less frequently, the atmospheric column has 

the opportunity to build up instability, so that when 

convection occurs, the rainfall intensity is increased. 

This is an improvement in tropical regimes (Fig. 4), but 

it can exacerbate the error in extratropical regimes. It 

was determined that the HCF trigger could be tuned 

together with the representation of shallow convection 

to reduce rainfall bias, suggesting a way forward in 

the further development of global models that must 

parameterize cumulus parameterization.

4.6 Improving Turbulence and Cloud Processes

One of the CPTs (Lead PI: Steven Krueger, University 

of Utah), which is now in its last year, was intended 

to (1) unify the representation of sub-grid scale 

(SGS) deep convective precipitation and grid-scale 

precipitation as the horizontal resolution decreases 

by implementing (installing) the Arakawa-Wu unified 

parameterization (UP) in the GFS, (2) unify the 

representation of turbulence and SGS cloud processes 

by implementing (installing) SHOC (Simplified Higher-
10

Order Closure) into the GFS, and (3) improve the 

representation of cloud, radiation, and microphysics 

interactions.

The Arakawa-Wu UP grows individual clouds when/

where the resolution is high, parameterizes convection 

when/where resolution is low, with continuous scaling 

in between, one set of equations, one code. The 

version implemented (installed) into and tested in 

the GFS is based on the Chikira-Sugiyama cumulus 

parameterization, which allows multiple cloud types 

and predicts the updraft vertical velocity for each cloud 

type. Remaining tasks include coupling with SHOC and 

radiation.

SHOC is now incorporated into NEMS as well as 

operational versions of the GFS. In these models, 

SHOC replaces the boundary layer turbulence and 

shallow convection parameterizations, calculates the 

cloud fraction, and adds a new prognostic scalar, subgrid-scale (SGS) turbulent kinetic energy. The radiative 

transfer scheme currently uses cloud fraction and 

condensate computed by SHOC. The current challenge 

is to allow condensate detrained from parameterized 

deep convection to affect the SGS variability of total 

water. To do this, prognostic variance of total water 

that includes a source due to detrainment from deep 

convection was added.

There is current work to couple both “stratiform” (gridscale) and parameterized deep convective clouds to 

radiation through McICA (Monte Carlo Independent 

Column Approximation) sampling. Grid-scale clouds are 

coupled by sampling the SGS distributions of liquid or 

ice concentration at each level obtained from SHOC to 

represent SGS structure in cloud properties. The initial 

focus is on SHOC clouds first. An important challenge is 

representing the clouds without having to compute too 

many expensive random numbers. The SHOC PDF is 

able to be reproduced using 4.5 random numbers per 

cell (compared to 1 for current GFS).

4.7 Cloud and Boundary Layer Processes

A MAPP-funded CPT (C. Bretherton, University 

of Washington; J. Teixeira, NASA Jet Propulsion 

Laboratory; H. Pan, NOAA EMC; C. Golaz and M. Zhao, 

NOAA GFDL) worked to improve cloud and boundary 

layer processes in NCEP’s Global Forecast System 

(GFS) and Climate Forecast System (CFS) models, 

as a sequel to a MAPPfunded CPT on the subtropical 

stratocumulus to cumulus transition. The primary goal 

was to simultaneously improve the cloud climatology, 

energy budget, and operational forecast skill of the 

GFS and the nextgeneration CFS. A secondary goal 

was to identify weather regimes where clouds are either 

forecast much better or much worse by GFDL global 

climate models vs. GFS, as a step toward improving 

cloudrelated parameterizations in both models.

The earlier work found that both the operational 

GFS and CFS severely underpredict cloud amount, 

water content, and cloud radiative impact over most 

of the globe, producing unacceptably large global 

and regional biases in the net topofatmosphere and 

surface energy budgets. Reducing these biases would 

provide a strong foundation for reducing systematic 

errors in extendedrange and seasonal forecasts. This 

project developed a portable singlecolumn version of 

the operational GFS, which was used to improve the 

boundarylayer and shallow cumulus parameterizations, 

modestly improving global cloud distributions. A new 

cloud fraction parameterization was also developed that 

somewhat increases GFSsimulated global cloudiness, 

and a new eddydiffusivity massflux scheme for GFS 

that combines the simulation of turbulence and shallow 

cumulus convection. The project sought to advance 

these first steps into a GFS version that has clouds 

whose radiative properties are simulated as skillfully 

as in leading climate models, while at the same time 

maintaining or improving conventional measures 

of weather forecast skill. The strategy involved 

careful testing and improvement of the microphysics 

and precipitation parameterizations, singlecolumn 

Climate Model Development Task Force Final Report
11

and global analysis of the fidelity of parameterized 

interactions among clouds, turbulence and precipitation 

in the revised GFS, and detailed comparisons of cloud 

simulations in hindcasts by GFS and two GFDL models, 

AM3 and HiRAM.

4.8 Representation of Aerosol Processes

In another CPT project (S. Lu, SUNY-Albany; Y-T Hou, 

NCEP/EMC; A. da Silva, NASA/GMAO), improving 

the representation of aerosol processes, cloud 

microphysics, and aerosol-cloud-radiation interaction 

in NCEP global models was targeted. The project 

advanced the physical parameterization suite in the 

NEMS-based Global Spectral Model (GSM) by:

• Implementing a multimodal and doublemoment modal aerosol module

• Implementing a double-moment cloud 

microphysics scheme (developed in 

a separate project – S. Moorthi)

• Coupling among cloud micro- and macrophysics, radiation, aerosol physicochemical 

properties, and cloud properties

• Updating the aerosol optical 

properties look-up-tables

• Adding the option to determine aerosol 

optical properties and cloud condensation 

nuclei and ice nuclei (CCN/IN) activation 

from prescribed aerosol distributions

• Enabling NEMS GSM to output additional 

cloud diagnostic fields, including liquid cloud 

water path, ice cloud water path, cloud 

optical depth, and cloud emissivity

Tests of the physics upgrades were conducted 

individually (uncoupled) initially. Experiments to test 

these upgrades interactively (coupled) are in progress 

to investigate aerosol direct and indirect effects in 

NEMS GFS. Sensitivity experiments have been 

conducted for selected cases, e.g., the 2016 Louisiana 

flooding and Hurricane Matthew, and the results were 

compared with in situ observations, satellite retrievals, 

and reanalysis. The new implementation results in a 

general model improvement in cloud fields (e.g., cloud 

fraction, cloud optical depth). However, the impact of 

the physics upgrade on precipitation is insignificant. 

Additional tuning and adjustments will be necessary 

to ensure the physics upgrades indeed lead to better 

weather forecast and climate prediction. The need for 

model refinement in turn calls for the need to enhance 

the model evaluation/verification package (see Section 

5).

4.9 Software Modeling Infrastructure 

A MAPP-funded infrastructure project (Lead PIs: C. 

DeLuca, U. of Colorado; J. Kinter, COLA; V. Balaji, 

GFDL) collaborated with NCEP EMC, NCAR, and other 

partners to design, implement, and test a prototype 

coupled model (the Unified Global Coupled Model; 

UGCS) that targets seasonal time scales. Constituent 

science components of the prototype include the NCEP/

EMC GSM, the Los Alamos sea ice model (CICE), and 

MOM5. This coupled model, possibly with different 

science components, is being evaluated as a successor 

to CFSv2.

A principal task was to develop a flux coupler based 

on the ESMF and NUOPC community standards 

(Theurich et al. 2016), and ensure that field exchanges 

through the coupler were operating correctly. This 

activity evolved and extended the NEMS infrastructure 

initiated by NCEP/EMC developers several years 

before. The MAPP-funded collaborators created 

infrastructure suitable for a NEMS-based unified 

modeling system, spanning multiple time scales, 

spatial scales, and predictive targets. An important 

new construct was the “application” or “app,” a set 

of coupled components associated with a specific 

predictive target. Different apps (e.g. seasonal, 

regional nest, etc.) can share components and 

infrastructure in a controlled manner. Also added 

to NEMS were “component sets” or “compsets”, a 

labeling system used by the CESM project for many
12

years to simplify the description of coupled runs.

Several milestone releases of UGCS-Seasonal 

were delivered. The most recent, UGCS-Seasonal 

0.4 (in Feb. 2017), is a three-way coupled system 

with a full set of field exchanges, that shows Earthlike behavior and runs on multiple NOAA computing 

platforms including Theia and Gaea. Test runs were 

initialized using a cold start run sequence using Climate 

Forecast System Reanalysis (CFSR) data. An endto-end workflow for testing this model is now being 

implemented by NCEP/EMC.

Other work under this MAPP-funded infrastructure 

project was to establish the Earth System CoG, a wiki 

for hosting projects and accessing distributed data, 

as the new user interface to the Earth System Grid 

Federation (ESGF), in support of CMIP6. This process 

is complete. A questionnaire for collecting CMIP6 

metadata was also developed but has not yet been 

deployed.

4.10 The CFS Test Harness

A MAPP-funded project (Saha, NOAA EMC) was 

undertaken to define, conduct, and publically 

release a test harness for coupled climate runs and 

possibly, conduct next-generation coupled prediction 

experiments as part of a CFS test harness. The NCEP 

CFSv2 became operational in March 2011. There have 

been many developments that can be utilized to make a 

significant advance from CFSv2 and there is an urgent 

need to quickly develop, test and implement a new and 

improved CFSv3 into operations. NCEP/EMC put a 

team together to develop and test the next generation 

of CFS. In anticipation of a working version of the 

coupled system available in NEMS, the development 

of a test harness and associated validation for coupled 

model runs was initiated using existing output from the 

archive of CFSv2 retrospective forecasts (spanning 

the period from 1982-present). As soon as working 

versions of CFSv3 become available, experimental 

runs will be made to ensure that all process-oriented

validation procedures work satisfactorily. The MAPP 

support to this project aimed at speeding up EMC 

CFSv3 development and coordinating this development 

with MAPP’s Climate Model Development and Model 

Diagnostic Task Forces as well as with the Climate Test 

Bed (CTB) activities.

4.11 Prognostic Ozone Parameterization

A MAPP-funded project (G. Compo, CU/CIRES 

and NOAA ESRL; J. Whitaker ,NOAA ESRL; P. 

Sardeshmukh ,CU/CIRES and NOAA ESRL; C. 

Long, NOAA CPC; S. Moorthi, NOAA EMC; S. 

Lu, NOAA EMC; J. McCormack, NRL) aimed to 

improve the representation of stratospheric ozone 

(O3) and water vapor in NOAA’s climate reanalyses 

in order to improve NOAA’s simulation, analysis, 

and forecasting of weather and climate variability, 

including forecasts of UV radiation to protect public 

health. A complete treatment of O3 photochemistry 

is too computationally intensive for current models. 

Therefore, a parameterization is included in the 

current NCEP GFS atmosphere/land model used 

in the 20th Century Reanalysis and operational 

forecasts, and also used in the CFS Reanalysis and 

operational CFSv2. The GFS parameterization for 

the time tendency of O3 is based on parts of NRL’s 

CHEM2D Ozone Photochemistry Parameterization 

(CHEM2DOPP). It includes terms representing net 

production and loss and a dependency on the ozone 

mixing ratio itself. It is based on gasphase chemistry 

of the late20th century, which includes the depletion 

of ozone by chlorofluorocarbons (CFCs). For climate 

reanalyses and climate modeling extending back 

to the early 20th century or earlier, before large 

quantities of CFCs began to be released into the 

atmosphere, a new version of this parameterization 

was needed to represent preCFC stratospheric 

O3 chemistry. To understand, analyze, and predict 

atmospheric variability in the 21st century, the 

parameterization should utilize additional interactions 

included in CHEM2DOPP that affect stratospheric 

Climate Model Development Task Force Final Report
13

O3. Stratospheric water vapor is also an important 

radiative constituent. Its representation in the GFS 

was also improved, paving the way for improved 

assimilation of satellite radiances and for interactive 

chemistry.

A more advanced O3 parameterization using the 

full CHEM2DOPP and an improved treatment of 

stratospheric water vapor was implemented for 

use in new versions of the GFS, CFS, and next 

generation NOAA climate reanalysis systems. The 

O3 parameterization included the effect of changes 

in temperature, changes in the vertical distribution 

of O3, and the timevariation of CFCs. As a first step, 

the parameterization was tested with two modes, 

one for times before CFCs and one for times after 

CFCs began to be released in large quantities. The 

team also implemented a new stratospheric H2O 

climatology as a necessary first step toward future 

implementation of parameterized H2O photochemistry. 

The upgraded parameterization and new climatology 

were tested in climate reanalyses and weather and 

climate simulations. The impact of the new O3 and 

water vapor treatments on reanalysis, GFS mediumrange forecast skill, and CFS climate simulations was 

evaluated using comparisons with both historical and 

modern O3 and temperature observations throughout 

the troposphere and stratosphere as well as with UV 

radiation observations.

4.12 Advanced Ocean Data Assimilation

A MAPP-funded project (J. Carton and E. Kalnay, 

University of Maryland, D. Behringer and Hendrik 

Tolman, NOAA/NCEP) has been upgrading NCEP’s 

Global Ocean Data Assimilation System (GODAS) from 

the current 3DVar system implemented in 2003 to the 

ensemble Local Ensemble Transform Kalman Filter 

(LETKF). GODAS serves as the ocean component 

of the integrated atmosphereocean analysis system, 

in turn providing the initial conditions for NCEP’s

atmosphere/ocean Climate Forecast System, version 

2, (CFSv2). The project also has been combining the 

3DVar and LETKF systems to form a hybrid version of 

GODAS for exploring the effectiveness of this hybrid 

system to represent time-evolving local correlations 

due, for example to fronts or currents, while at the 

same time maintaining largescale correlations. The 

computational efficiency of the hybrid filter relative to its 

3DVar and LETKF alternatives has been examined.

The proposal brought together researchers from the 

University of Maryland experienced in the development 

and use of LETKF with NCEP researchers who have 

overseen development of 3DVarGODAS and the first 

two versions of the CFS. The rationale for proposed 

work is to: 1) Provide an upgrade of the ocean analysis 

system to one that will be analogous to NCEP’s 3DVarhybrid Gridpoint Statistical Interpolation system (GSI) 

used for atmospheric analysis. This upgrade will allow 

the next generation CFS to gain the benefit of a more 

integrated atmosphereoceanland analysis system in 

which both ocean and atmosphere components use 

coupled ensemblebased estimates of flux error at the 

interface. 2) The implementation of LETKF in GODAS 

provides NCEP with a more flexible ocean analysis 

system, for example simplifying the inclusion of new 

observational data sets like sea surface salinity and 

providing an error estimate for the ocean state. This 

flexibility is important to allow NCEP to implement 

assimilation upgrades to both GODAS (using a MOMbased model) and the eddyresolving RealTime Ocean 

Forecast System (RTOFS) (using a HYCOMbased 

ocean model). The work was carried out in NCEP’s 

computing environment thus facilitating the integration 

of the resulting system into operations.

4.13 Stratospheric Processes

The primary purpose of the reanalysis effort is to 

advance climate studies by eliminating fictitious trends 

caused by model and data assimilation changes that 

occurred in real time. Reanalyses are to represent the 

observations as closely as possible and could be used 

as surrogates where observations are not available. 
14

Each generation of reanalyses has improved upon 

its predecessor in many ways by: reduction of errors, 

increased spatial and vertical resolution, and addition 

of more variables. The current generation of reanalyses 

provides more information about the stratosphere than 

previous versions. This is important for monitoring 

the impacts of climate change and ozone depletion 

on the stratospheric circulation and the stratospheric 

interactions with the troposphere. Assessments of the 

stratosphere in the latest generation of reanalyses 

revealed several issues that may hinder the full use 

of these reanalyses for climate studies. This was 

particularly true for the NOAA Climate Forecast System 

Reanalysis (CFSR). This reanalysis contains jumps 

in data records during stream transitions, warming 

trends in the upper stratosphere between streams, poor 

representation of the Quasibiennial Oscillation (QBO) 

winds, ozone observations not being assimilated in the 

upper stratosphere, and poor representation of water 

vapor above the tropopause. It is important to rectify 

these issues before the next NOAA reanalysis effort.

This MAPP-funded project (C. Long, NOAA CPC; 

J. Perlwitz CU/CIRES and NOAA ESRL; F. Sassi, 

NRL) addressed the climate objectives outlined in 

the NOAA Next Generation Strategic Plan (NGSP) 

and a major CPO/Modeling, Analysis, Predictions, 

and Projections (MAPP) Program priority: Research 

to Advance Climate Reanalysis, particularly “issues 

with the quality of reanalyses in the stratosphere” by 

improving the characterization of the stratosphere 

in reanalysis by building upon research conducted 

following the CFSR. We propose to: reduce the impacts 

of data inhomogenuity on temperature and ozone, to 

improve the thermal structure of the upper stratosphere, 

improve the representation of the QBO winds and 

residual circulation in the tropics, and improve the 

depiction of ozone and water vapor in the stratosphere. 

Success in providing these improvements will lead to 

a better characterization of the stratosphere. A well 

characterized stratosphere may enable better weather 

and climate research and services by: 1) providing a 

more accurate depiction of past weather and climate 

conditions, 2) improving the monitoring of current 

climate conditions, and 3) enabling the attribution of 

climate variations and change by comparison with past 

conditions.

4.14 Next Generation Reanalysis

The fidelity of new reanalysis datasets (MERRA, 20CR, 

CFSR, ERAInterim) at representing climate variability 

of the 20th century has enabled significant advances in 

climate research. In a MAPP-funded project (A. Kumar, 

NOAA CPC; G. Compo, CU/CIRES and NOAA ESRL; 

J. Whittaker, NOAA ESRL; P. Sardeshmukh, CU/CIRES 

and NOAA ESRL; R. Vose, NOAA National Climatic 

Data Center, now National Environmental Information), 

investigated known shortcomings of these datasets, 

while developing a framework for a new NOAA Climate 

Reanalysis (NCR) system to ameliorate them. The 

NCR will eventually have four “streams” to meet the 

various user needs for reanalysis information:

• Stream 0: Boundaryforced, 1850present 

“AMIP” simulation with large ensemble 

• Stream 1: Historical, 1850present 

using only surface data

• Stream 2: Modern, 1946present using only 

surface and conventional upper air data 

• Stream 3: Satellite, 1973present using qualitycontrolled satellites, Global Positioning 

System Radio Occultation, and surface 

and conventional upper air data. 

One of the foci of this research has been to use 

observing system experiments. In these, the 2000-

-2010 observing system is reduced to that of selected 

historical periods to investigate the impact on the 

timevarying quality and density of the observing 

system and determine ways to reduce this impact. 

The project employed innovative methods to 

assess the relative importance and impact of model 

errors and observational errors on the quality and 

Climate Model Development Task Force Final Report
15

homogeneity of the reanalysis fields, with particular 

attention to reducing or eliminating spurious jumps 

and trends. The framework for the NCR system 

leveraged recent advances in operational data 

assimilation for global weather prediction, as well 

as newly digitized observational datasets and global 

model improvements. While initially focusing on the 

atmosphere to develop the NCR framework, this 

project could serve as the basis for further NCR 

efforts, incorporating advancements generated by 

other projects supported by MAPP, such as integration 

of ocean, chemistry, and land components and 

the treatment of observational and model biases. 

International coordination and data sharing with 

NOAA’s reanalysis partners at NASA, ACRE, ECWMF, 

and JMA and synergies from the NOAA Reanalysis 

Task Force were important parts of the project.

4.15 Surface Fluxes in Global Energy and Water 

Budgets

This MAPP-funded project (L. Yu, Woods Hole 

Oceanographic Institution; Y. Xue, NOAA CPC) aimed 

to provide a comprehensive assessment of the partially 

coupled Climate Forecast System Reanalysis (CFSR) 

in representing airsea heat, freshwater, and momentum 

fluxes in the context of the global energy and water 

budgets. The CFSR is the first and only reanalysis 

that incorporates a coupled atmosphereocean-land 

climate system with an interactive seaice component, 

and has the finest spatial resolution (~0.5°). There 

are clear advantages of the finerresolution coupled 

CFSR reanalysis in characterizing air-sea fluxes at 

regional and global scales, but biases/errors in the 

CFSR flux components at various temporal scales 

have also been found. The biases/errors appear to 

have significant impact on the estimates of the energy 

and water budgets over the global oceans. Currently, 

the CFSR produces a global energy imbalance of 15 

Wm2, which is about 10 Wm2 higher than estimates 

from earlier NCEP reanalyses. Balancing the global 

energy/water budgets has long been a challenge, with 

global energy budgets differing considerably, from 2 

to 30 Wm2, when computed using reanalyzed, ship-

, and satellitebased flux products. However, the global 

energy/water budgets are central to the understanding 

of climate variability and climate changes produced 

by the reanalyses. A good knowledge of the impact of 

biases/errors in surface flux components on the global 

budget estimates will be highly beneficial to not only 

the users of CFSR products but also the developers for 

the nextgeneration Earth System reanalysis. This study 

analyzed the biases/errors in the CFSR surface fluxes 

in the context of the global energy/water budget and 

also compared the CFSR with other reanalyses.

The approaches included: (i) in situ validation, in which 

a database consisting of more than 130 flux buoys was 

used as ground truth for identifying and quantifying 

biases/errors in flux products; (ii) spectral analysis, in 

which ship and satellitebased global flux analyses were 

used as reference to evaluate and characterize the 

regional and global spectral structures of flux products, 

and (iii) dynamical diagnosis, in which dynamic 

constraints (such as energy and freshwater budgets 

in an enclosed volume) were used to test the physical 

consistency of flux products with ocean state variables 

(temperature and salinity).

The primary objectives of the proposed research were 

to (i) identify the strength and weakness of the CFSR 

surface flux components by comparison with in situ 

flux measurement, satellitebased analyses and other 

reanalyses and tounderstand the sources of biases, 

(ii) examine the effect of spatial resolution in improving 

the accuracy and spatial structure of CFSR fluxes on 

regional and global scales, and (iii) investigate the 

use of physical constraints together with ocean state 

variables to diagnose and understand the uncertainties 

in CFSR airsea fluxes.
16

4.16 Multi-Model Ensembles

A new multi-model combination strategy has been 

developed to produce an optimized single forecast from 

multi-model ensemble forecast and the forecast has 

been evaluated against the observations. Preliminary 

results suggest that the new methodology outperforms 

individual models and can increase the one-month lead 

forecast skill of surface air temperature by 50% over the 

simple multi-model average across much of the area of 

focus.

5. Opportunities and Critical Challenges

The discussions of the CMDTF have exposed a 

number of critical challenges for future collaborative 

development of operational prediction systems, such 

as are envisioned by NCEP and the Next Generation 

Global Prediction System (NGGPS) project. Addressing 

these critical challenges is a necessary, but not 

sufficient, condition for making progress toward 

the community modeling paradigm currently being 

discussed in strategic planning at NCEP. These 

challenges are both scientific and organizational.

Scientific challenges:

• Although there is clear responsiveness of surface 

heat fluxes, near surface temperature, humidity 

and daytime boundary layer development to 

variations in soil moisture over much of the globe, 

precipitation in CFSv2 is unresponsive. Failure to 

realize potential predictability from land surface 

states is undoubtedly connected to piecewise 

model development practices, where land and 

atmosphere models are not developed and tested 

in coupled configurations (Dirmeyer et al. 2015). 

Consideration of surface-atmosphere interactions 

are critical for realistic simulation of boundarylayer development and convective initialization 

(Tawfik and Dirmeyer 2014; Tawfik et al. 2015a,b).

• NCEP is now evolving toward a unified modeling 

framework with the FV3 dynamic core. An 

important challenge is incorporating prognostic

aerosols into that unified modeling system. 

Specifically, NCEP will need to determine 

how atmospheric composition (aerosols) 

should be integrated into the unified model 

architecture. This is an active area of discussion 

among the NGGPS system architecture, 

infrastructure, and physics working groups.

• There is a need for an observation-based diagnosis 

package, with well-defined metrics, to examine 

whether a given model change makes a significant 

improvement. A more process-oriented diagnostic 

approach is needed to evaluate physically-based 

parameterization schemes. For example, an 

observation-based diagnosis package is needed 

to determine whether the model with an improved 

aerosol-cloud package can better represent 

aerosol/cloud properties and the processes 

relevant to aerosol-cloud-radiation interaction.

• As spatial resolution in component models 

increases, there is a need for evaluating the 

impact on long-standing issues such as bias, 

variability and coupling (e.g. ocean eddies – see 

bullet below). Any discretization or filtering scheme 

either explicitly or implicitly discriminates between 

resolved and unresolved scales such that features 

with a size of 5-10 ΔX and larger are considered 

“resolved”. The rectified effects of processes 

occurring on unresolved scales – which can be 

first-order in the resolved scales due to increasing 

entropy, irreversible processes and dynamical 

nonlinearity – must be parameterized. In order to 

preserve numerical stability, parameterizations are 

generally local, diffusive and equilibrium-restoring; 

hence, they do not adequately represent processes 

that are non-local, latent, up-gradient, or locally 

unstable. A related issue concerns the utility of 

parameterizations that are aware of the resolved 

scales. Ideally, such scale-aware schemes (e.g., 

the Smagorinsky viscosity scheme) require minimal 

retuning as model resolution is altered. Scale-aware 

methods are challenging to develop, but they offer 

Climate Model Development Task Force Final Report
17

important payoffs that warrant continued research.

• In addition to increasing computational cost 

and exerting pressure on operational prediction 

schedules, the trend toward higher spatial 

resolution in coupled models requires an 

evidence-based assessment of the tradeoffs 

involved in increasing resolution in individual 

component models. For example, how closely 

does the resolution in the atmospheric and oceanic 

component models have to match to avoid errors 

in the representation of the effects of small-scale 

features in, say, the atmospheric component 

on the response in the oceanic component? 

Similarly, there are scientific issues associated 

with mismatched horizontal and vertical resolution 

in the atmosphere and the extent of the domain 

(e.g. well-reserved stratosphere or inclusion of 

deep atmospheric effects on troposphere).

• As prediction models refine their ocean resolution, 

transient mesoscale eddies appear and can have 

an impact on the atmospheric variability that goes 

beyond that seen with coarser resolution ocean 

models. Recent research suggests that these 

fine scale air-sea interactions have particular 

importance for storm tracks and tropical cyclones. 

A major challenge in prediction models will be 

to incorporate realistic boundary layer models, 

using refined vertical and horizontal resolutions, 

in both the ocean and atmosphere to ensure 

accurate representation of the physical processes. 

Stimulated by the work done by CMDTF, many 

of these topics were discussed during the MAPP 

Webinar Series on 13 March 2017: High-resolution 

modeling: working toward improved process 

representation and simulations of the Earth system.

• The traditional paradigm for coupling sea ice 

models to ocean models has its limitations, 

particularly when moving to finer resolution models 

where numerical instabilities manifest due to 

mismatches between fluxes transferred across 

the model components. These limitations are 

overcome when embedding the sea ice dynamics 

into the ocean dynamics, which is a project that 

is ongoing at GFDL based on the MOM6 ocean 

and SIS2 sea ice models (R. Hallberg, personal 

communication). This work is also critical as 

climate models consider the additional complexities 

associated with coupling with interactive ice shelf 

models, with such coupling a necessary aspect 

of unraveling questions about sea level rise.

Organizational challenges:

• There is currently insufficient operations-toresearch support for model developers, both inside 

and outside NCEP, which has greatly slowed 

the progress of MAPP-funded projects whose 

goals include improving operational models.

• The access to high-performance computing 

(HPC) has been slow or limited for some 

projects and individuals due to protracted 

authorization procedures and security issues.

• PI access to model codes and development 

tools (e.g., the hierarchy of model components, 

run scripts, diagnostic codes, documentation 

etc.) for experimentation is a critical 

requirement that is currently inadequate. 

• Modern software engineering practices, including 

version control to manage code, are still lacking.

• Despite the contribution of the CMDTF, 

communication between NCEP developers and 

externally-funded PIs is still an issue and needs 

to be facilitated even more strenuously, including 

the clear definition of NCEP staff involvement.

• There has not been a clear EMC-CFS 

development plan, with a timeline, definition 

of benchmarks and milestones, and pathways 

for external partners to contribute. 
18

6. Conclusion

The Climate Model Development Task Force 3-year 

term ended in January 2017. During its lifetime, the 

Task Force organized by the MAPP program has 

engaged with NCEP staff to provide strategic and 

technical input to its planning activities. Task Force 

contributions to NCEP model development include 

the testing of new process parameterizations for 

convection, aerosols and land via the Climate Test Bed, 

as well as progress in data assimilation and in model 

infrastructure as part of NEMS development. Activities 

have been carried out in coordination with the NGGPS 

program. A number of mainly organizational challenges 

were identified and are being addressed by EMC in 

coordination with the MAPP and NGGPS programs. 

In order to advise on how to facilitate community 

engagement in model development, the Task Force 

developed the following suggestions:

• Communication. The information exchange 

afforded by monthly conference calls of the 

CMDTF were effective at keeping a diverse set of 

relevant researchers informed about each other’s 

progress. There was a general feeling, however, 

that these calls were not sufficient to provide the 

sort of tight coordination that would be needed to 

accelerate progress. To its credit, building on the 

work of the CMDTF, NOAA’s National Weather 

Service and other line offices have developed a 

Strategic Implementation Planning (SIP) process, 

in conjunction with the Next Generation Global 

Prediction System (NGGPS) project that includes 

the formation of several working groups that have 

been very effective at fostering communication 

among model development groups and individuals.

• Observations. The concept of the CPTs – to 

bring together experts in relevant processes 

with experts in model development – was 

successful within the limited scope in which it 

was applied. We thus suggest that a more robust 

program involving process-oriented metrics 

be routinely applied to models in development 

in order to scientifically confront models with 

observations in a scientific way is suggested.

• Support. Several participants expressed the view 

that a more robust level of support for developers 

and users of operational models would go a long 

way to accelerating progress. Such support would 

include documentation, HPC resources, automation 

of routine tasks such as model setup, testing and 

diagnosis, and the formation of short-term expert 

teams (“tiger teams”) to address specific issues.

• Testing. While NCEP operations has a number 

of standard tests with standard metrics and 

benchmarks, largely driven by Government 

Performance and Results Act (GPRA) strictures, 

there are many other tests and metrics that 

could be used to advance climate model 

development. A more comprehensive system 

for gathering and implementing these tests 

is needed that can be accessible to both the 

EMC and external developers and analysts.

• Planning and best practices. The CMDTF spent 

considerable time discussing potential pathways 

for development without the benefit of a concrete 

modeling plan. It is critically important, as 

emphasized in several UCACN and UMAC reports, 

that a strategic plan be developed by EMC to guide 

implementation and collaborative development

Climate Model Development Task Force Final Report
19

Selected CMDTF Publications

Bombardi, R. J., E. K. Schneider, L. Marx, S. Halder, B. 

Singh, A. B. Tawfik, P. a. Dirmeyer, and J. L. Kinter (2015), 

Improvements in the representation of the Indian summer 

monsoon in the NCEP climate forecast system version 2, 

Climate Dyn., 45(9-10), 2485–2498, doi:10.1007/s00382-015-

2484-6.

Bombardi R. J., Tawfik A. B., Manganello J. V., Marx L., Shin 

C.-S., Halder S., Schneider E. K., Dirmeyer P. A., Kinter J. L. 

(2016) The heated condensation framework as a convective 

trigger in the NCEP climate forecast system version 2. J. Adv. 

Model. Earth Sys., DOI: 10.1002/2016MS000668

Broennimann, 2015: Upper-air observations from the German 

Atlantic Expedition (1925-27) and comparison with the 

Twentieth Century and ERA-20C reanalyses. Met. Zeit., 525-

544. doi:10.1127/metz/2015/0683.

Brugnara, Y., and Coauthors, 2015: A collection of subdaily pressure and temperature observations for the early 

instrumental period with a focus on the “year without a 

summer” 1816, Climate Past, 11, 1027-1047, doi:10.5194/cp11-1027-2015.

Chaney, N. W., J. D. Herman, M. B. Ek, and E. F. Wood, 

2016: Deriving global parameter estimates for the 

Noah land surface model using FLUXNET and machine 

learning, J. Geophys. Res. Atmos., 121, 13,218–13,235, 

doi:10.1002/2016JD024821.

Coccia, G., A. Siemann, M. Pan, and E.F. Wood, 2015: 

Creating consistent datasets combining remotely-sensed 

data and Land Surface Model estimates through a Bayesian 

uncertainty post-processing: the case of Land Surface 

Temperature from HIRS. Remote Sens. Envir., 170, 290-305, 

doi:10.1016/j.rse.2015.09.010.

Compo, G., J. Carton, X. Dong, A. Kumar, S. Saha, J. S. 

Woollen, L. Yu, and H. M. Archambault, 2016: Report from the 

NOAA Climate Reanalysis Task Force Technical Workshop. 

NOAA Technical Report OAR CPO-4, Silver Spring, MD. doi: 

10.7289/V53J39ZZ.

Cram, T.A., and Coauthors, 2015: The International Surface 

Pressure Databank version 2. Geosci. Data J., 2, 31-46. 

doi:10.1002/gdj3.25.

DelSole, T., L. Trenary, M.K. Tippett, and K. Pegion, 2017: 

Predictability of week-3–4 average temperature and 

precipitation over the contiguous United States. J. Climate, 

30, 3499–3512, doi: 10.1175/JCLI-D-16-0567.1. 

DelSole, T., and M. K. Tippett (2014), Comparing forecast 

skill. Mon. Wea. Rev., 142(12), 4658–4678, doi:10.1175/

MWR-D-14-00045.1.

Dirmeyer, P. A., C. Peters-Lidard, and G. Balsamo, 2015: 

Land-Atmosphere Interactions and the Water Cycle. [Chapter 

8 in: Seamless Prediction of the Earth System: from Minutes 

to Months (G Brunet, S Jones, PM Ruti Eds.)], World 

Meteorological Organization (WMO-No. 1156), Geneva, (also 

available in French and Spanish). 

Dirmeyer, P. A., J. Wu, H. E. Norton, W. A. Dorigo, S. M. 

Quiring, T. W. Ford, J. A. Santanello Jr., M. G. Bosilovich, M. 

B. Ek, R. D. Koster, G. Balsamo, and D. M. Lawrence, 2016: 

Confronting weather and climate models with observational 

data from soil moisture networks over the United States. J. 

Hydrometeor., 17, 1049-1067, doi: 10.1175/JHM-D-15-0196.1.

Dirmeyer, P. A., and S. Halder, 2017: Application of the landatmosphere coupling paradigm to the operational Coupled 

Forecast System (CFSv2). J. Hydrometeor., 18, 85-108, doi: 

10.1175/JHM-D-16-0064.1.

Dirmeyer, P. A., and S. Halder, 2017: Sensitivity of surface 

fluxes and atmospheric boundary layer properties to initial 

soil moisture variations in CFSv2. Wea. Forecasting, 32, doi: 

10.1175/WAF-D-16-0049.1 (early release).

Han, J., M. Witek, J. Teixeira, R. Sun, H.-L. Pan, J. K. 

Fletcher, and C. S. Bretherton, 2015: Implementation in the 

NCEP GFS of a hybrid eddy-diffusivity mass-flux (EDMF) 

boundary layer parameterization with dissipative heating and 

modified stable boundary layer mixing. Wea. Forecasting, 31, 

341-352. doi:10.1175/WAFD-15-0053.1.

Haughton, N., G. Abramowitz, A. J. Pitman, D. Or, M. J. 

Best, H. R. Johnson, G. Balsamo, A. Boone, M. Cuntz, B. 

Decharme, P. A. Dirmeyer, J. Dong, M. Ek, Z. Guo, V. Haverd, 

B. J. van den Hurk, G. S. Nearing, B. Pak, C. Peters-Lidard, J. 

A. Santanello Jr., L. Stevens and N. Vuichard, 2016: 
20

The plumbing of land surface models: why are models

performing so poorly? J. Hydrometeor., 17, 1705–1723, doi: 

10:1075/JHM-D-15-0171.1.

Jones, C. R., C. S. Bretherton, and M. S. Pritchard, 2015: 

Mean state acceleration of cloud resolving models and large 

eddy simulations. J. Adv. Model. Earth Sys., 7, 1643-1660. 

doi:10.1002/2015MS000488.

Liang, X. and L. Yu, 2016: Variations of the global net air-sea 

heat flux during the “hiatus” period (2001-2010). J. Climate, 

29, 3647–3660. doi: 10.1175/JCLI-D-15-0626.1. 

Liu, X., F. Chen, M. Barlage, G. Zhou, and D. Niyogi, 2016: 

Noah-MP-Crop: Introducing dynamic crop growth in the 

Noah-MP land surface model, J. Geophys. Res. Atmos., 121, 

13,953–13,972, doi:10.1002/2016JD025597.

Penny, S.G., 2014: The Hybrid Local Ensemble Transform 

Kalman Filter, Mon. Wea. Rev., 142, 2139-2149.

Penny, S.G., D. Behringer, J. Carton, E. Kalnay, 2015: A 

Hybrid Global Ocean Data Assimilation System at NCEP, 

Mon. Wea. Rev., 143, 4660-4677.

Sardeshmukh, P. D., G. P .Compo, and C. Penland, 2015: 

Need for caution in interpreting extreme weather statistics. J. 

Climate, 28, 9166-9187, doi:10.1175/JCLI-D-15-0020.1.

Siemann, A.L., G. Coccia, M. Pan, and E.F. Wood, 2016: 

Development and Analysis of a long-term, global, terrestrial 

land surface temperature dataset based on HIRS satellite 

retrievals. J. Climate, 29, 3589–3606, doi: 10.1175/

JCLI-D-15-0378.1. 

Stickler, A., S. Storz, R. Wartenburger, H. Hersbach,G. P. 

Compo, P. Poli, D. Dee, and S. Theurich, G., C. DeLuca, 

T. Campbell, F. Liu, K. Saint, M. Vertenstein, J. Chen, R. 

Oehmke, J. Doyle, T. Whitcomb, A. Wallcraft, M. Iredell, T. 

Black, A.M. Da Silva, T. Clune, R. Ferraro, P. Li, M. Kelley, 

I. Aleinov, V. Balaji, N. Zadeh, R. Jacob, B. Kirtman, F. 

Giraldo, D. McCarren, S. Sandgathe, S. Peckham, and R. 

Dunlap, 2016: The Earth system prediction suite: Toward a 

coordinated U.S. modeling capability. Bull. Amer. Meteor. 

Soc., 97, 1229–1247, doi: 10.1175/BAMS-D-14-00164.1.

Tawfik, A. B., P. A. Dirmeyer, and J. A. Santanello, 2015: 

The heated condensation framework. Part I: Description 

and Southern Great Plains case study. J. Hydrometeor., 16, 

1929–1945, doi: 10.1175/JHM-D-14-0117.1.

Tawfik, A. B., P. A. Dirmeyer, and J. A. Santanello, 2015: 

The heated condensation framework. Part II: Climatological 

behavior of convective initiation and land-atmosphere 

coupling over the continental United States. J. Hydrometeor., 

16, 1946–1961, doi: 10.1175/JHM-D-14-0118.1.

Theurich, G., C. DeLuca, T. Campbell, F. Liu, K. Saint, M. 

Vertenstein, J. Chen, R. Oehmke, J. Doyle, T. Whitcomb, 

A. Wallcraft, M. Iredell, T. Black, A.M. da Silva, T. Clune, R. 

Ferraro, P. Li, M. Kelley, I. Aleinov, V. Balaji, N. Zadeh, R. 

Jacob, B. Kirtman, F. Giraldo, D. McCarren, S. Sandgathe, 

S. Peckham, and R. Dunlap, IV, 2016: The Earth System 

Prediction Suite: Toward a coordinated U.S. modeling 

capability. Bull. Amer. Meteor. Soc., 97, no. 7, 1229-1247, 

doi:10.1175/BAMS-D-14-00164.1.

Turney, C. S. M., R. T. Jones, D. Lister, P. Jones, A. N. 

Williams, A. Hogg, Z. A. Thomas, G. P. Compo, X. Yin, C. J. 

Fogwill, 2016: Anomalous mid-twentieth century atmospheric 

circulation change over the South Atlantic compared to the 

last 6000 years. Env. Res. Lett.,11, 1-14. doi:10.1088/1748-

9326/11/6/064009.

Wen, C., Y. Xue, A. Kumar, D. Behringer, and L. Yu, 2017: 

How do uncertainties in NCEP R2 and CFSR surface fluxes 

impact tropical ocean simulations? J. Climate, in press. doi: 

10.1007/s00382-016-3516-6.

Yu, L., X. Jin, S.A. Josey, T. Lee, A. Kumar, C. Wen, and 

Y. Xue, 2017:The global ocean water cycle in atmospheric 

reanalysis, satellite, and ocean salinity. J. Climate, 30, 

3829–3852, doi: 10.1175/JCLI-D-16-0479.1. 

References

Arakawa, A., and W. H. Schubert (1974), Interaction of a 

cumulus cloud ensemble with the large-scale environment, 

Part I, J. Atmos. Sci., 31(3), 674–701, doi:10.1175/1520-

0469(1974)031<0674:IOACCE>2.0.CO;2.

Dirmeyer, P. A., 2013: Characteristics of the water cycle 

and land-atmosphere interactions from a comprehensive 

reforecast and reanalysis data set: CFSv2. Climate Dyn., 41, 

1083-1097, doi: 10.1007/s00382-013-1866-x.

Climate Model Development Task Force Final Report
21

Han, J., and H.-L. Pan 2011: Revision of convection and 

vertical diffusion schemes in the NCEP Global Forecast 

System, Wea. Forecasting, 26(4), 520–533, doi:10.1175/

WAF-D-10-05038.1.

Pan, H.-L., and W.-S. Wu 1995: Implementing a mass flux 

convective parameterization package for the NMC medium 

range forecast model, NMC Off. Note 409, 40 pp. [Available at 

http://www.emc.ncep.no.]

Tawfik, A. B., and P. A. Dirmeyer (2014), A process-based 

framework for quantifying the atmospheric preconditioning 

of surface-triggered convection, Geophys. Res. Lett., 41, 

173–178, doi:10.1002/2013GL057984.

Tawfik, A. B., and P. A. Dirmeyer, 2014: A process-based 

framework for quantifying the atmospheric preconditioning of 

surface triggered convection. Geophys. Res. Lett., 41, 173-

178, doi: 10.1002/2013GL057984.
22

Appendix A: Charge to Climate Model Development Task Force

The following is the charge to the Climate Model Development Task Force from the Modeling, Analysis, Prediction 

and Projections (MAPP) program of the Climate Program Office (CPO) of the Office of Oceanic and Atmospheric 

Research (OAR) of the National Oceanic and Atmospheric Administration (NOAA), issued in December 2013 for a 

three-year Task Force period of performance commencing on 1 January 2014:

The Climate Model Development Task Force is an initiative of NOAA’s MAPP Program in partnership with the 

Climate Test Bed to achieve significant advances in NOAA’s climate models in support of improved predictions. 

The Task Force brings together scientists from universities, research laboratories, and NOAA’s centers and labs 

and builds-on MAPP funded research activities. It is envisioned that the Task Force will work in synergy with 

internal climate model development efforts at NOAA’s research laboratories and centers and will emphasize 

research contributing to the development of the next-generation operational climate forecasting system via Climate 

Test Bed activities. Research objectives of the Task Force will contribute to advance NOAA’s next-generation 

climate modeling capability in the broader weather-to-climate modeling context addressing improvements in 

the representation of physical processes, coupling of model components and software infrastructure. Research 

activities will build on MAPP projects that include improving the representation of atmospheric convection, land 

and hydrology, ocean processes, coupling of various components and modeling infrastructure. The Climate Model 

Development Task Force will coordinate with the Climate Reanalysis Task Force on aspects relating to model 

improvement and initialization. Research will be done in coordination with other relevant national and international 

climate modeling activities in the context of USGCRP and WCRP. 

Climate Model Development Task Force Final Report
23

Appendix B: Projects and Participants in the Climate Model Development Task Force

CMDTF Projects

Model Development

• A CPT for Improving Turbulence and Cloud Processes in the NCEP Global Models 

PI: Krueger (MAPP-CTB) 

• A CPT to improve cloud and boundary layer processes in GFS/CFS 

PI: Bretherton (MAPP-CTB) 

• Improving the Prognostic Ozone Parameterization in the NCEP GFS and 

CFS for Climate Reanalysis and Operational Forecasts 

PI: Compo 

• Improving the NCEP Climate Forecast System (CFS) through Enhancing 

the Representation of Soil-Hydrology-Vegetation Interactions

PIs: Chen and Ek (MAPP-CTB; co-supported by NGGPS)

• Advances in Lake-Effect Process Prediction within NOAA’s Climate Forecast System for North America

PIs: Jin, Chen and Ek (MAPP-CTB; co-supported by NGGPS)

• Improving cloud microphysics and their interactions with aerosols in the NCEP Global Models

PI: Lu (MAPP-CTB; co-supported by NGGPS)

Software Modeling Infrastructure

• Software modeling infrastructure: Couple NCEP EMC NEMS to MOM5/

ice for more capable CFSv3 and single framework for EMC/CPC 

PI: DeLuca

Data Assimilation

• Exploration of advanced ocean data assimilation schemes at NCEP 

PI: Carton 

• Strategies to Improve Stratospheric Processes in Climate Reanalysis 

PI: Long 

• Research towards the next generation of NOAA Climate Reanalyses 

PIs: Kumar and Compo 

• Improving the land surface components of Climate Forecast System Reanalysis (CFSR) 

PI: Ek 

Model/Analyses Evaluation

• COLA: Predictability and Prediction of Climate from Days to Decades 

PI: Kinter 

• Evaluating CFSR Air-Sea Heat, Freshwater, and Momentum Fluxes in 
24

the context of the Global Energy and Freshwater Budgets 

PIs: Yu and Xue

CMDTF Participants

Co-Leads

• Stephen Griffies, Geophysical Fluid Dynamics Laboratory (NOAA)

• James Kinter, Center for Ocean-Land-Atmosphere Studies, George Mason University

• Suranjana Saha, Environmental Modeling Center, National Centers for Environmental Prediction (NOAA)

All Participants3

Donifan Barahona NASA Goddard

Dave Behringer EMC/NCEP/NOAA

Partha Bhattacharjee EMC/NCEP/NOAA

Peter Bogenschutz NCAR

Chris Bretherton University of Washington

Fei Chen NCAR

Dan Collins CPC/NCEP/NOAA

Tony Craig NEMS/NOAA

Anton Darmenov NASA Goddard

Arlindo da Silva NASA Goddard

Cecelia DeLuca CIRES / NESII/ESRL/NOAA

Paul Dirmeyer COLA/George Mason University

Michael Ek EMC/NCEP/NOAA

Chris Golaz GFDL/NOAA

Stephen Griffies (Co-Lead) GFDL/NOAA

Jongil Han CPC/NCEP/NOAA

Yu-Tai Hou NWC/EMC/NOAA

Jin Huang NCEP/NOAA, Climate Test Bed

Bohua Huang COLA/George Mason University

Jiming Jin Utah State University

Jim Kinter (Lead) COLA / George Mason University

Daryl Kleist EMC/NCEP/NOAA

Steven Krueger University of Utah

Arun Kumar CPC/NCEP/NOAA

Craig Long CPC/NCEP/NOAA

Sarah Lu State University of New York at Albany

Jesse Meng EMC/NCEP/NOAA

Qilong Min State University of New York at Albany

3 Note: At the request of EMC management, the Task Force also included several NCEP scientists not directly funded by the MAPP program.

Climate Model Development Task Force Final Report
25

Kingtse Mo CPC/NCEP/NOAA

Shrinivas Moorthi EMC/NCEP/NOAA

Judith Perlwitz PSD/ESRL/NOAA

Robert Pincus University of Colorado at Boulder

David Randall Colorado State University

Suranjana Saha (Co-Lead) EMC/NCEP/NOAA

Fabrizio Sassi NSEGST/SSD/NRL

Ed Schneider COLA/George Mason University

Cristiana Stan COLA/George Mason University

Ruiyu Sun EMC/NCEP/NOAA

Shan Sun ESRL/NOAA

Joao Teixeira JPL/NASA

Gerhard Theurich CIRES/NESII/ESRL/NOAA

Hendrik Tolman EMC/NCEP/NOAA

Jun Wang EMC/NCEP/NOAA

Wanqiu Wang CPC/NCEP/NOAA

Jack Woollen EMC/NCEP/NOAA
26

Appendix C: User Requirements

In October 2014, members of the CMDTF engaged with model developers and users to generate two lists of 

requirements. One is a set of operational requirements articulated by the Climate Prediction Center. The other is a 

list of characteristics that emerged from the workshop held at that time.

CPC Operational Requirements/Metrics

A) Climate Forecasts

1. Essential

1.1. Intra-seasonal

1.1.1 A daily higher resolution coupled ensemble forecast system, and ensemble size similar to 

 GEFS, is needed for intra-seasonal forecasts out to at least 32 days. (perhaps even longer 

 for multi-day lead forecasts)

1.1.2 Intra-seasonal forecast system needs to be upgraded more frequently (every two years or less) than 

 the seasonal forecast system.

1.1.3 Reforecasts for the intra-seasonal forecast system needs to have at least 20 years of 

 reforecasts for bias correction and calibration and needs to be performed every time there is 

 an upgrade.

1.1.4 Reforecasts should include at least 6 members.

1.1.5 Consistent ensemble generation methods for both the reforecasts and the realtime forecasts.

1.1.6 Realtime forecasts generated at least two times per week.

1.2 Seasonal 

1.2.1 A seasonal forecast system is needed for forecasts out to 15 months.

1.2.2 Forecast frequency of at least once every 5 days.

1.2.3 Reforecasts for the seasonal forecast system need to have at least 30 years (1981- present) 

 of reforecasts for bias correction and calibration. 

1.2.4 Skill for key variables (sea ice, ENSO, MJO, and surface temperature and precipitation over 

 the US) should be better than the current system.

1.2.5 A lower resolution version of the seasonal forecast model should be available for sensitivity and 

 attribution studies 

1.2.6 Proper analysis/coupling of the ocean mixed layer, air-sea fluxes, sea ice, and ocean 

 currents.

1.2.7 Realistic simulation of the Atlantic Meridional Overturning Circulation.

1.3 All time-scales

1.3.1 The forecast model used to generate the initial conditions must be the same for hindcasts 

 and real-time forecasts. 

1.3.2 Temporal consistency in the initialization of ocean, soil moisture, sea-ice across hindcast and 

 real-time forecasts (to avoid issues with discontinuity and bias correction).

1.3.3 Coupling of atmosphere with ocean, land, sea ice models is preferred but left to the 

 discretion of EMC.

1.3.4 Adequate spin up for land surface and ocean conditions. 

Climate Model Development Task Force Final Report

Appendix B: Projects 

and Participants in 

the Climate Model 

Development Task Force
27

1.3.5 A realistic QBO produced by an improved gravity wave parameterization. 

1.4 Output Format: Hourly (at least 3-hrly) output of surface flux fields including 

 precipitation variables.

2. Metrics:

2.1 Relationships between CPC & EMC need to be promoted to interact on development and validation of CFS 

 forecasts and reanalysis.

2.2 Operational: to be evaluated on the entire hindcast history (instead of one month at a time) or all 

 relevant spatial and temporal scales:

2.2.1 Precipitation

2.2.2 2-meter max/min air temperature

2.2.3 SST (ENSO)

2.2.4 MJO

2.2.5 Sea ice

2.2.6 Streamflow

2.2.7 QBO

2.2.8 Radiation balance

2.3 Developmental: to be evaluated on specifically designed experiments while conducting experimental 

 research on process-oriented physics, etc. Climate “scorecard” of basic indices such as ENSO, MJO, NAO, 

 etc. to be developed by the NMME group in CPC.

2.4 Test “harness” for quicker evaluation during development.

B) Climate Reanalysis

1. Essential 

1.1 Sufficient (i.e. multi-year) spin up to create the reanalysis. 

1.2 Resolution in the horizontal and vertical is comparable to the Climate Forecast System.

1.3 Forecast model must stay consistent with that used to generate hindcasts.

1.4 Minimize jumps during transitions in observations.

2. Ideal 

2.1 One stream

2.2 A low-resolution pilot run of the entire reanalysis to check for trends and discontinuities. 

2.3 Make this run available for users to evaluate.

2.4 Replacement for R1 (i.e. a reanalysis back to 1948) 

3. Metrics

3.1 Validation against non-assimilated data sets AND other available reanalyses

3.2 Mechanism to determine unwanted trends in reanalysis.

Characteristics of Next Generation Coupled Climate Forecast System

1. CFSv3 Planning and Development Process

1.1 Need to provide the basis (scientific facts, drivers and boundary conditions) and process for 

 making the choices of the model physics, dynamics, and resolutions.

1.2 Need to survey what’s in place in other NOAA programs (e.g. NGGPS) and other operational 

 modeling centers, and the state of the science.
28

1.3 Need to clarify meaning of ‘requirements”, “predictands”, “metrics”, and “constraints” so that the Task Force 

 members will know how to provide inputs to the requirements and metrics documents.

1.4 Need to better define the governance of the CFSv3 development, such as, engagement process, 

 decision-making authority and process, the role of CPC in CFSv3 development.

1.5 The above processes should be transparent to everyone, including the external community.

1.6 The CFS development team should be responsible for implementing the CFSv3 plan and be the POC within 

 NCEP and with the external community.

2. Technical comments

2.1 Coupling of six model components should be addressed in a scientific context. 

2.2 Although NEMS cannot replace basic research, NEMS is a necessary infrastructure to ensure the 

 system components are compatible and structured efficiently.

2.3 Development and cal/val should be done for both component models and 3-D coupled system in 

 NEMS.

2.4 Additional wish list for CFSv3 includes energy conservation, avoiding spin-up and initialization 

 shock problems, improving stratosphere-troposphere coupling and ozone, assimilating satellite 

 data, higher resolution for intra-seasonal forecasts etc.

2.5 Need to better coordinate and prioritize the planning between atmospheric physics and aerosols.

2.6 The data assimilation people and MAPP Reanalysis Task Force should provide feedback to 

 EMC’s Reanalysis plan.

3. User interface and O2R support to the community

3.1 User requirements from the external community include: NCEP’s CFSv3 development plan, 

 timelines and metrics; model documentation; a script to run the model, post-processing, and diagnostics for 

 CFS development; model (and possibly data assimilation) codes (the latest GFS; coupled models, 

 single column model); experimental model outputs.

3.2 Collaborative model development efforts between EMC and external users, such as Climate 

 Process Teams (CPTs) need to get functional version of codes and scripts used as a ‘test harness’ 

 to develop and evaluate CFSv2 as soon as possible, so that they can optimize their activities 

 toward useful skill improvements for CFSv3. This can and should be done before a full NEMS 

 interface and support for a new candidate ocean model are available.

3.3 The CFSv3 plan needs to describe the user interface and the mechanisms to meet the above 

 user requirements from the community, e.g., how to make the model codes available to users. 

 NOAA Climate Test Bed (CTB) can be a mechanism to provide infrastructure and Operation-to- 

 Research (O2R) support to the external community in CFS development

Climate Model Development Task Force Final Report
29

Appendix D: Acronyms

3DVar Three-Dimensional Variational Analysis (Data Assimilation)

ACRE Atmospheric Circulation Reconstructions over the Earth

AM3 Atmospheric Model, version 3 (GFDL)

AMIP Atmospheric Model Intercomparison Project

BCL Buoyant Condensation Level 

CESM Community Earth System Model 

CFS Climate Forecast System

CFSR CFS Reanalysis

CIN Convective Inhibition

CIRES Cooperative Institute for Research in Environmental Sciences

CMDTF Climate Model Development Task Force

CMIP Coupled Model Inter-comparison Project

COLA Center for Ocean-Land-Atmosphere Studies

CPO Climate Program Office

CPT Climate Process Teams

CTB Climate Test Bed

CTRL Control

CU University of Colorado

CWF Cloud Work Function

ECMWF European Centre for Medium-range Weather Forecasts

EMC Environmental Modeling Center

ENSO El Niño and the Southern Oscillation

ERA ECMWF Re-Analysis

ESGF Earth System Grid Federation

ESMF Earth System Modeling Framework

ESRL Earth System Research Laboratory

FLake Freshwater Lake

FV3 Finite Volume Cubed-Sphere Dynamical Core 

GEFS Global Ensemble Forecast System

GFDL Geophysical Fluid Dynamics Laboratory

GFS Global Forecast System

GODAS Global Ocean Data Assimilation System

GPRA Government Performance and Results Act

GSFC Goddard Space Flight Center

GSI Gridpoint Statistical Interpolation

GSM Global Spectral Model

GTB Global Test Bed

GVF Green Vegetation Fraction

HCF Heated Condensation Framework
30

HiRAM High-resolution Atmospheric Model (GFDL)

HPC High-Performance Computing

HYCOM Hybrid Coordinate Ocean Model

IGBP International Geosphere-Biosphere Programme

ISI Intra-Seasonal to Interannual

JMA Japan Meteorological Agency

LAI Leaf Area Index

LETKF Local Ensemble Transform Kalman Filter

LFC Level of Free Convection

LSM Land Surface Model

MAPP Modeling, Analysis, Predictions and Projections

McICA Monte Carlo Independent Column Approximation

MERRA Modern Era Retrospective-analysis for Research and Applications

MODIS Moderate Resolution Imaging Spectroradiometer

MOM (GFDL) Modular Ocean Model

NASA National Aeronautics and Space Administration

NCAR National Center for Atmospheric Research

NCEP National Centers for Environmental Prediction

NCR NOAA Climate Reanalysis

NEMS NOAA Environmental Modeling System

NGGPS Next-Generation Global Prediction System

NMME North American Multi-Model Ensemble

NOAA National Oceanic and Atmospheric Administration

NOAH NCEP – OAR – Air Force –Hydrologic Research Laboratory LSM

NOAH-MP Noah with Multiple-Parameterizations

NSF National Science Foundation 

NUOPC National Unified Operational Prediction Capability

NWS National Weather Service

O2R Operations-to-Research

OAR Office of Oceanic and Atmospheric Research

PI Principal Investigator

PBL Planetary Boundary Layer

PDF Probability Density Function

QBO Quasi-Biennial Oscillation

R2O Research-to-Operations

RTOFS RealTime Ocean Forecast System

S2S Sub-seasonal to Seasonal

SAS Simplified Arakawa-Schubert

SGS Sub-Grid-Scale

SHOC Simplified Higher-Order Closure

SIP Strategic and Implementation Plan

STATSGO State Soil Geographic

Climate Model Development Task Force Final Report
31

STI NWS Office of Science and Technology Integration

UCACN UCAR Community Advisory Committee for NCEP

UCAR University Corporation for Atmospheric Research

UGCS Unified Global Coupled System

UKMO United Kingdom Meteorological Office

UMAC UCACN Modeling Advisory Committee

UP Unified Parameterization

USGCRP United States Global Change Research Program

WCRP World Climate Research Programme
32

FIGURES

 

 

 

Figure 1: Seasonal lake surface temperature observations and simulations averaged over 2000-2010 for the 

Great Lakes. Solid dark line is buoy station observations; dashed dark line is MODIS satellite observations; 

dashed blue line is the original FLake simulations (FLake-old with the RMSE with the MODIS data), and red solid 

line is modified FLake simulations (FLake-new with the RMSE with the MODIS data). In the title at the top of 

each figure, the first number is the buoy station ID, and the second number after the colon is the lake depth at the 

station.

Climate Model Development Task Force Final Report
33

Figure 2: Lake ice observations (ice fraction; top figure) and simulations (ice thickness; two bottom figures) for 

the Great Lakes averaged over 1984-2002.
34

Figure 3: Frequency distribution of daily precipitation (mm/day) for TRMM, HCF, and CTRL over Central India 

[16.5oN-26.5oN,74.5oE-86.5oE]. The HCF and CTRL distributions are different at 95% confidence level according 

to a maximum likelihood ratio test.

Climate Model Development Task Force Final Report
35

Figure 4: Comparison between the HCFv2 trigger and the original triggering (CTRL) mechanisms shown by (a) 

the Heidke Skill Score; (b) the Equitable Threat Score; (c) the Bias; and (d) the triggering frequency for each 

sounding release location.
36

NOAA Technical Report OAR CPO-X

Climate Model 

Development Task Force 

Final Report 

 https://doi.org/10.7289/V5/TR-OAR-CPO-6