# Presentation: 079969

## Subsetting, Visualization and Analysis of Cloud Resolving Model Data

- Karen Schuchardt
- Pacific Northwest National Laboratory

- Thanks to Charlotte DeMott (CSU) for permission to re-use slides.

## Outline

- CRM Project Overviews and Data
- Motivating Example
- Visualization Tools/Needs
- Collaboration Thoughts

## Projects

- Design and Testing of a Global Cloud Resolving Model (GCRM) (Scidac SA, Randall)
- Center for Multi-scale Modeling of Atmospheric Processes (CMMAP) (NSF STC, Randall)
- Community Access to Global Cloud Resolving Model Data and Analyses (Scidac, Schuchardt)

## Why Cloud Resolving Models

- [cirrus cloud] representation in Numerical Weather Prediction (NWP) models and General Circulation Models (GCMs) has been identified as one of the greatest uncertainties in weather and climate research.
- CRMs may be increasingly used in GCMs to replace the cumulus and stratiform cloud parameterizations (e.g., Khairoutdinov and Randall 2001). Thus, evaluating the representation of cirrus clouds in a CRM will soon be considered part of evaluating GCMs.

- Yali Luo, Steven K. Krueger, and Gerald G. Mace, Kuan-Man Xu

## Global Cloud Resolving Model(GCRM)

- prototype of **future-generation** cloud-resolving global atmospheric models for use in both **weather forecasting and climate** simulation
**global atmospheric circulation** model with a grid-cell spacing of approximately 2-4 km, capable of **simulating the circulations associated with large convective clouds**
- Applications of the model will include development of improved versions of more conventional, less expensive models.

**Dave Randall (PI) (CSU), Akio Arakawa (UCLA) (CO-PI)**
- CSU Researchers: Heikes, Konor, DeMott, Dazlich...

## GCRM

- Computationally **MOST** expensive to run
- Data intensive
- Conservatively, 1 TB / hourly averages
- In 5 year time frame:
- 1 month runs aimed at numerical weather predictions
- Run >= two annual-cycle simulations
- Targeting primarily ORNL and NERSC facilities
- Production use: 20 year time frame

## GCRM Data

- Geodesic grid (to 2-4 km resolution)
- Time averaged global maps
- 2d, 3d
- Time averaged region (more frequent data in some regions)
- Point data for pre-selected points (much more frequent data for some points)
- Restart data

## Geodesic Grids

- Regular Icosahedron
-   Inscribed in a unit sphere
-   20 triangular faces
- 12 vertices (grid points)

## Slide 9

**Notes:**
NCAR Tutorial May 1999.  Ringler, Heikes, Randall

## CMMAP

- $19 Million NSF Science and Technology Center (STC) to build climate models that will more accurately depict cloud processes, improving climate and precipitation forecasting.
- 100+ individuals at 29 organizations
- Roles:
- Model developers
- Model validators
- Data Management
- Meteorology
- Students
- Education outreach
- Computer Company Outreach

**Activities**
- _[Research](http://kiwi.atmos.colostate.edu:16080/cmmap/contact-research.html)__ _
- _[Education, Outreach and Diversity](http://kiwi.atmos.colostate.edu:16080/cmmap/contact-eod.html)__ _
- _[Knowledge Transfer](http://kiwi.atmos.colostate.edu:16080/cmmap/contact-kt.html)__ _

## CMMAP

- Apply cloud resolving model to existing modeling frameworks (such as MMF) where the cloud model runs within each cell.
- Less computationally challenging and data intensive than GCRM

**Notes:**
Grid within a grid

## CMMAP

**Notes:**
Randomly distributed “self adapting” creates challenges for data analysis.

## CMMAP

- Computationally, more expensive than GCM but less expensive than GCRM
- Research:
- Heavy emphasis on validation against observed data
- ARM surface data
- CloudSat and Calypso satellite data
- Performance optimization
- Visualization
- Data Management and Distribution of large data sets.

## Petascale Subsetting,Analysis, and Visualization of Atmospheric Global Environment

- Recall: data set sizes of ~ 1-10 TB/hour
- .8 PB/month 8 PB/annual cycle
- IO Benchmarking to determine what is feasible
- Data subsetting and delivery
- Provide server-side analysis and visualization
- Bring the tools to the data

## Architecture

- Request Handler Cluster

- Parallel File System

- Queue System

- Network Port

- Requested Data

## Software Architecture

- Portal

- Specify
- region
- Time
- Temporal Strides
- Spatial Strides
- Averages,
- Joint histograms
- Satellite simulations

- netCDF,
- Images,animations,
- Applets, KLM

- request

- View
- Data summaries
- visualizations

- Averaging Binning

- subsetting

- post
- processing

- Browse
- Simulation sets
- Staged data

## Motivation: GEWEX Cloud System Study (GCSS)

- Can models reproduce the main properties of the diurnal cycle over subtropical oceans?
- Can models and satellites help characterize the humidity structure of the upper troposphere?

- http://www.gewex.org/gcss.html

**Notes:**
Global Energy and Water Cycle Experiment (GEWEX)

## Questions Relevant to MMF

- Can MMF meet the challenges posed by GCSS
- Can the MMF produce boundary layer clouds and the transition to deep tropical convection?
- Does MMF really produce stratocumulus clouds?
- Do the MMF stratus clouds behave as they do in nature?
- Does the MMF produce a transition zone between Sc and Cu clouds?

## Slide 19

## Slide 20

## Slide 21

## Example for CRM on MMF

- Data Output
- Lat x lon x level x variables (4D)
- Lon x lat x height x 32 CRM grid columns
- ~20 such arrays per hour
- Data Use
- Extract a few GCM grid points
- Concatenate into a time series
- For “short” time series, all fields and times into one files
- For “longer” time series, must write one file per field

## Data Issues - Schematically

## Bottlenecks – Lots of Data

- “The current process for moving output files to local storage where data can be extracted and subsequently analyzed by primarily serial tools breaks down”
- “...researchers expect to download one or more NetCDF files to local storage for analysis.  The files typically contain more parameters than of interest for a particular analysis, thus creating unnecessary network traffic and increased processing time.”

## Bottlenecks – Lots of Data

- In many cases, subsetting of data may still produce data sets that are too large; **server side analysis and visualization** capabilities are needed
- Animations of a cell or region, or globe
- Parallel support (for subsetted local data) will be useful to take advantage of the multi processor desktop chips.

## Current Visualization Tools

- Model Developers: Everybody has their own favorite
- MatLab
- IDL
- NCAR Tools
- CMMAP Community
- Ditto except on a wider scale
- Platforms (GCRM): Linux, Mac

## Future Capabilities

- Server side subsetting and analysis
- Predefined subsetting, analysis, visualization
- Plug in mechanism to allow users to add custom features
- Examples with abstractions so users don’t have to know all the complexity of parallel tools

## Capabilities (Continued)

- Effective use of tools such as GoogleEarth, Google maps, NASA’s World Wind
- Course detail, zoom to fine detail
- Typical visualizations
- Lines (as in GCSS)
- Blocks
- Region of time
- Moving regions
- Orographic maps
- “The skies the limit”

## Collaboration Opportunities

- Model vs  Observed comparisons
- Analysis / viz tools we can plug into back end services
- Fly throughs
- Direct support for geodesic grids