# Presentation: 200380

## Parallel MM5 Simulation & ResultsforCentral America

## Objectives

- Parallel Programming Concepts
- Set up MM5 for long simulations using parallel architecture
- Initialize MM5 using GCM output
- Overcoming challenges in running long simulations
- Discussion of results
- Use Web based tool to order data (sub divide data into regions)

## What is Parallel Computing?

***parallel computing*** is the simultaneous use of multiple compute resources to solve a computational problem
- The compute resources can include:
-   -  A single computer with multiple processors;
-   - An arbitrary number of computers connected by a network;
-   - A combination of both.

## Why use Parallel Computing ?

- Save time - wall clock time
- Solve larger problems
- Taking advantage of non-local resources
- Cost savings
- Overcoming memory constraints

**Notes:**
There are two primary reasons for using parallel computing: 
Save time - wall clock time 
Solve larger problems 
Other reasons might include: 
Taking advantage of non-local resources - using available compute resources on a wide area network, or even the Internet when local compute resources are scarce. 
Cost savings - using multiple "cheap" computing resources instead of paying for time on a supercomputer. 
Overcoming memory constraints - single computers have very finite memory resources. For large problems, using the memories of multiple computers may overcome this obstacle. 

## General Parallel Terminology

- What is a task ?
-  -A logically discrete section of computational work
- What is a Parallel task ?
-  -A task that can be executed by multiple processors safely (yields correct results)

**Notes:**
Like everything else, parallel computing has its own "jargon". Some of the more commonly used terms associated with parallel computing are listed below. 
Task 
A logically discrete section of computational work. A task is typically a program or program-like set of instructions that is executed by a processor. 
Parallel Task 
A task that can be executed by multiple processors safely (yields correct results) 
Serial Execution 
Execution of a program sequentially, one statement at a time. In the simplest sense, this is what happens on a one processor machine. However, virtually all parallel tasks will have sections of a parallel program that must be executed serially. 
Parallel Execution 
Execution of a program by more than one task, with each task being able to execute the same or different statement at the same moment in time.

## General Parallel Terminology Continued

- Communications
-  -Parallel tasks need to exchange data through shared memory bus or network
- Synchronization
-  -Coordination of Parallel tasks associated with communications in real time

**Notes:**
Communications 
Parallel tasks typically need to exchange data. There are several ways this can be accomplished, such as through a shared memory bus or over a network, however the actual event of data exchange is commonly referred to as communications regardless of the method employed. 
Synchronization 
The coordination of parallel tasks in real time, very often associated with communications. Often implemented by establishing a synchronization point within an application where a task may not proceed further until another task(s) reaches the same or logically equivalent point. Synchronization usually involves waiting by at least one task, and can therefore cause a parallel application's wall clock execution time to increase. 

## Parallel Overhead

- Task start-up time
- Synchronizations
- Data communications
- Software overhead imposed by parallel compilers, libraries, tools, operating system, etc.
- Task termination time

**Notes:**
The amount of time required to coordinate parallel tasks, as opposed to doing useful work. Parallel overhead can include factors such as: 

## Parallel Computer Memory Architecture

- Shared Memory

**Notes:**
Shared memory parallel computers vary widely, but generally have in common the ability for all processors to access all memory as global address space. 
Multiple processors can operate independently but share the same memory resources. 
Changes in a memory location effected by one processor are visible to all other processors. 
Shared memory machines can be divided into two main classes based upon memory access times: UMA and NUMA.

## Shared Memory

- Advantages

- Fast and Easy

- Disadvantages

- Scalability and Cost

**Notes:**
Advantages: 
Global address space provides a user-friendly programming perspective to memory 
Data sharing between tasks is both fast and uniform due to the proximity of memory to CPUs 
  Disadvantages: 
Primary disadvantage is the lack of scalability between memory and CPUs. Adding more CPUs can geometrically increases traffic on the shared memory-CPU path, and for cache coherent systems, geometrically increase traffic associated with cache/memory management. 
Programmer responsibility for synchronization constructs that insure "correct" access of global memory. 
Expense: it becomes increasingly difficult and expensive to design and produce shared memory machines with ever increasing numbers of processors. 

## Parallel Computer Memory Architecture

- Distributed Memory

**Notes:**
Like shared memory systems, distributed memory systems vary widely but share a common characteristic. Distributed memory systems require a communication network to connect inter-processor memory. 

Processors have their own local memory. Memory addresses in one processor do not map to another processor, so there is no concept of global address space across all processors. 
Because each processor has its own local memory, it operates independently. Changes it makes to its local memory have no effect on the memory of other processors. Hence, the concept of cache coherency does not apply. 
When a processor needs access to data in another processor, it is usually the task of the programmer to explicitly define how and when data is communicated. Synchronization between tasks is likewise the programmer's responsibility. 
The network "fabric" used for data transfer varies widely, though it can can be as simple as Ethernet.

## Parallel Computer Memory Architecture

- Advantages
-   -Memory is Scalable with number of processors
-   -Cost Effectiveness
- Disadvantages
-    -Responsibility for data communication between processors
-    -Difficult to convert programs based on global memory to this memory organization

**Notes:**
Advantages: 
Memory is scalable with number of processors. Increase the number of processors and the size of memory increases proportionately. 
Each processor can rapidly access its own memory without interference and without the overhead incurred with trying to maintain cache coherency. 
Cost effectiveness: can use commodity, off-the-shelf processors and networking. 
  Disadvantages: 
The programmer is responsible for many of the details associated with data communication between processors. 
It may be difficult to map existing data structures, based on global memory, to this memory organization. 
Non-uniform memory access (NUMA) times 

## 	Parallel Computer Memory Architecture

- Hybrid Distributed-Shared Memory

**Notes:**
Comparison of Shared and Distributed Memory Architectures
ArchitectureCC-UMA
CC-NUMA
Distributed
ExamplesSMPs Sun Vexx DEC/Compaq SGI Challenge IBM POWER3
SGI Origin Sequent HP Exemplar DEC/Compaq IBM POWER4
Cray T3E Maspar IBM SP2
CommunicationsMPI Threads OpenMP shmem
MPI Threads OpenMP shmem
MPI 
Scalabilityto 10s of processors
to 100s of processors
to 1000s of processors 
Draw BacksMemory-CPU bandwidth
Memory-CPU bandwidthNon-uniform access times
System administration Programming is hard to develop and maintain
Software Availabilitymany 1000s ISVs
many 1000s ISVs
100s ISVs 
The largest and fastest computers in the world today employ both shared and distributed memory architectures
The shared memory component is usually a cache coherent SMP machine. Processors on a given SMP can address that machine's memory as global. 
The distributed memory component is the networking of multiple SMPs. SMPs know only about their own memory - not the memory on another SMP. Therefore, network communications are required to move data from one SMP to another. 
Current trends seem to indicate that this type of memory architecture will continue to prevail and increase at the high end of computing for the foreseeable future. 
Advantages and Disadvantages: whatever is common to both shared and distributed memory architectures. 

## Parallel Programming Models

- In the threads model of parallel programming, a single process can have multiple, concurrent execution paths

**Notes:**
Perhaps the most simple analogy that can be used to describe threads is the concept of a single program that includes a number of subroutines: 
The main program a.out is scheduled to run by the native operating system. a.out loads and acquires all of the necessary system and user resources to run. 
a.out performs some serial work, and then creates a number of tasks (threads) that can be scheduled and run by the operating system concurrently. 
Each thread has local data, but also, shares the entire resources of a.out. This saves the overhead associated with replicating a program's resources for each thread. Each thread also benefits from a global memory view because it shares the memory space of a.out. 
A thread's work may best be described as a subroutine within the main program. Any thread can execute any subroutine at the same time as other threads. 
Threads communicate with each other through global memory (updating address locations). This requires synchronization constructs to insure that more than one thread is not updating the same global address at any time. 
Threads can come and go, but a.out remains present to provide the necessary shared resources until the application has completed. 
Threads are commonly associated with shared memory architectures and operating systems.

## Parallel Programming Model

- Message Passing Model Implementation
-  -Subroutines imbedded in the source code
-  -MPI – Message Passing Interface

**Notes:**
From a programming perspective, message passing implementations commonly comprise a library of subroutines that are imbedded in source code. The programmer is responsible for determining all parallelism. 
Historically, a variety of message passing libraries have been available since the 1980s. These implementations differed substantially from each other making it difficult for programmers to develop portable applications. 
In 1992, the MPI Forum was formed with the primary goal of establishing a standard interface for message passing implementations. 
Part 1 of the Message Passing Interface (MPI) was released in 1994. Part 2 (MPI-2) was released in 1996. Both MPI specifications are available on the web at www.mcs.anl.gov/Projects/mpi/standard.html. 
MPI is now the "de facto" industry standard for message passing, replacing virtually all other message passing implementations used for production work. Most, if not all of the popular parallel computing platforms offer at least one implementation of MPI. Very few have a full implementation of MPI-2. 
For shared memory architectures, MPI implementations usually don't use a network for task communications. Instead, they use shared memory (memory copies) for performance reasons.

## Parallel Programming Models

- Thread Model Implementation
- -- Subroutines called from within Parallel source code
- -- Compiler directives in either serial or parallel code
- --OpenMP or POSIX Threads

**Notes:**
The main program a.out is scheduled to run by the native operating system. a.out loads and acquires all of the necessary system and user resources to run. 
a.out performs some serial work, and then creates a number of tasks (threads) that can be scheduled and run by the operating system concurrently. 
Each thread has local data, but also, shares the entire resources of a.out. This saves the overhead associated with replicating a program's resources for each thread. Each thread also benefits from a global memory view because it shares the memory space of a.out. 
A thread's work may best be described as a subroutine within the main program. Any thread can execute any subroutine at the same time as other threads. 
Threads communicate with each other through global memory (updating address locations). This requires synchronization constructs to insure that more than one thread is not updating the same global address at any time. 
Threads can come and go, but a.out remains present to provide the necessary shared resources until the application has completed. 
Threads are commonly associated with shared memory architectures and operating systems.

## Parallel Programming Models

- Message Passing Model

**Notes:**
A set of tasks that use their own local memory during computation. Multiple tasks can reside on the same physical machine as well across an arbitrary number of machines. 
Tasks exchange data through communications by sending and receiving messages. 
Data transfer usually requires cooperative operations to be performed by each process. For example, a send operation must have a matching receive operation.

## Cluster Components

- Parallel Applications
- Linux or Windows
- Fast/GigE   Infiniband   Myrinet
- Intel/AMD Processors

## Cluster Software

- Message Passing Interface
- Easy to use interface for installation and Management
- Batch/queue Software (PBS)

**Notes:**
MPICH is a freely available, portable implementation of MPI (Message Passing Interface) the standard for message-passing libraries.  MPICH is maintained by Argonne Natgional Lab (ANL) and Mississipi State Univ.  Different versions of the MPICH may be necessary to support different compilers, and different versions of compiler may be necessary for compatability with other parts of the clustering environment.

## Beowulf Cluster

- Massively Parallel Computer
- Runs a free operating system
- Connected by high speed interconnect
- Compute nodes are dedicated
- Everything in a Beowulf is open-source and open standard- easier to manage/upgrade

## Linux Distributions

- CentOS  - RHEL compatible
- Fedora   -  Red Hat’s free version
- Owl        - Openwall GNU/Linux (security                  enhancements)
- Redhat    - Supported(RHEL)
- SuSe        - Novell
- Lindows   - Runs Linux and MS/Windows software

## Beowulf Interconnect

- The most important component
- Factors to consider
- Bandwidth
- Latency
- -  Price
- Software Support
- Ethernet (Inexpensive,reasonably fast)
- Myrinet (low latency and high bandwidth-popular)

## Cheetah at ORNL

## Cheetah’s Configuration

- IBM SP4 Supercomputer
- Each of 24 nodes has 32 processors
- Provides 4 Teraflops of computing power
- More than a terabyte of Memory
- 40 Terabytes of disk space
- Connected to the Mass Storage System by HPSS

**Notes:**
Jaguar-cray XT3
The current Jaguar installation has 3782 nodes, each with a 2.4-GHz AMD Opteron processor and 2 GB of memory. 3724 nodes are available in the compute partition, and the remainder provide I/O and login services. 
Each node is connected to a Cray Seastar router through Hypertransport, and the Seastars are all interconnected in a 3D-torus topology. The resulting interconnect has very high bandwidth, low latency, and extreme scalability. 
The operating system is UNICOS/lc, which is a combination of Linux on the service nodes and the Catamount microkernel on the compute nodes. Catamount is designed to minimize system overhead, thus allowing scalable low-latency global communication. 
By July of 2005, Jaguar will be upgraded to 5294 nodes, with 5212 available as application nodes. 
CRAY X1 (Phoenix)
Phoenix has 512 multi-streaming vector processors (MSPs), where each MSP has 2 MB of cache and a peak computation rate of 12.8 GF. Four MSPs form a node with 16 GB of shared memory. Memory bandwidth is very high, roughly half the cache bandwidth. The interconnect functions as an extension of the memory system, offering each node direct access to memory on other nodes at high bandwidth and low latency. 
Over the summer of 2005, Phoenix will be upgraded to an X1E. The total memory and interconnect will remain the same, but each X1 node will be replaced by two X1E nodes. After the full upgrade, Phoenix will have 1024 MSPs, each with a peak computation rate of 18 GF. Each node will have 4 MSPs sharing 8 GB of memory. 

## Parallel MM5

- Runtime System Library (RSL)
-   -Provides inter-domain communication
-   -Irregular domain decomposition
-   -Distributed I/O
-   -Dynamic load balancing

**Notes:**
It provides high-level stencil and inter-domain communication, irregualr domain decomposition, automatic local/global index translation, distributed I/O, and dynamic load balancing. 
Parallelization of multiple nested domains requires addtional services form a parallelization library for the conveyance of forcing and feedback data between a parent and nested domain, both decomposed over processors. Finally, the efficient implementaion of these models requires attention to load imbalances. 
RSL[1] is a parallel library developed at Argonne National Laboratory
Parallelization of multiple nested domains requires addtional services form a parallelization library for the conveyance of forcing and feedback data between a parent and nested domain, both decomposed over processors. Finally, the efficient implementaion of these models requires attention to load imbalances. 
RSL[1] is a parallel library developed at Argonne National Laboratory

## Parallelism in MM5

- Runs on shared-memory architecture
- Runs on distributed memory architecture

**Notes:**
Distrinuted shared Memory SGI origin 2000
Pure Distributed Memory : IBM SP, Beowulf cluster
Uniprocessor (non-parallel) architecture

## Building MM5 on Parallel Architecture

- Download:
- _[ftp://ftp.ucar.edu/mesouser/MM5V3/MM5.TAR.gz](ftp://ftp.ucar.edu/mesouser/MM5V3/MM5.TAR.gz)_
- _[ftp://ftp.ucar.edu/mesouser/MM5V3/MPP.TAR.gz](ftp://ftp.ucar.edu/mesouser/MM5V3/MPP.TAR.gz)_
- Unzip and untar :
- gzip –d –c MM5.TAR.gz | tar xf –
- cd MM5
- gzip –d –c MPP.TAR.gz | tar xf -

## Building MM5 on Parallel Architecture

- Edit configure.user file for computer and configuration
- Find the subsection in section 7 to define your computer architecture
- Adjust PROCMIN_NS and PROCMIN_EW settings at top of section 7 for memory scaling
- Build the model : make mpp
- Executable : Run/mm5.mpp

## Building MM5 on Parallel Architecture

- To remake the code in different configuration:
-      make mpclean
- To reinstall the code in different location:
-     make uninstall

## MM5 Input/Output

- Overcoming file size limits of 2GB on LINUX
- 	-If you anticipate that your MM5 model output will be larger than 2GB (the file size limit on LINUX machines), add the following to the compiler loading flags:
- 	LDFLAGS = -L/usr/pgi/linux86/liblf

## NCAR CCSM3.0

- IPCC runs made at T85 resolution (about 140 km in latitude and longitude)
- Output saved 4X daily
- Fortran program to convert to pregrid output format

## CCSM3 Soil Levels

- Variable name “levsoi” in (m)
-  levsoi=10
- Soil levels at
-    (0.007100635, 0.027925, 0.06225858, 0.1188651, 0.2121934, 0.3660658, 0.6197585, 1.038027, 1.727635, 2.864607)
- Soil Temperature and Soil water written at  4,6,8,9 to match mm5 soil moisture levels 10,40,100,20

## CCSM3 Output

- Atmospheric hybrid sigma pressure coordinate; “lev=26”
-      -lev =
- 	 3.54463800000001, 7.38881350000001, 13.967214, 23.944625, 37.2302900000001, 53.1146050000002, 70.0591500000003, 85.4391150000003, 100.514695, 118.250335, 139.115395, 163.66207, 192.539935, 226.513265, 266.481155, 313.501265000001, 368.817980000002, 433.895225000001, 510.455255000002, 600.524200000003, 696.796290000003, 787.702060000003, 867.160760000001, 929.648875000002, 970.554830000001, 992.5561

## CCSM3 Output

- atm/  cpl/  ice/  lnd/  ocn/
- 411582540 *6 = 2.4 Gb (approx.) (atm)
-      b30.042e.cam2.h3.2005-06-05-21600.nc
-     	b30.042e.cam2.h3.2005-06-10-21600.nc
-      b30.042e.cam2.h3.2005-06-15-21600.nc
-      b30.042e.cam2.h3.2005-06-20-21600.nc
-      b30.042e.cam2.h3.2005-06-25-21600.nc
-      b30.042e.cam2.h3.2005-06-30-21600.nc
- 25169936  (lnd)
- 	b30.042e.clm2.h0.2005-06.nc

## CCSM3 Output

-         'Skin Temperature (TS) in K'
-         'Source model terrain height m'
-         '2-m Temperature (TREFHT) in K'
-         'Sea Level Pressure (PSL)'
-         'Surface zonal wind (usfc)'
-         'Surface meridional wind (vsfc)'
-         'Surface relative humidity (%)'
-         'Total Precip (mm/day)'

## CCSM3 Output

- 'Water equivalent snow depth (mm)'
- 'Upper Air Temperature (T)‘
- 'Upper Air Zonal Wind (U)'
- 'Upper Air Meridional Wind (V)'
- 'Upper Air Temperature (T) '
- 'Geopotential Height'
- 'volumetric soil water (mm3/mm3)‘
- 'Soil temperature (K)'

## MM5 Input Files

- Four years of CCSM3 input chosen (2005,2010,2015,2025)
- REGRID and INTERPF Programs run to create initial and lower boundary files

## MM5 Runtime Options

- FDDAGD=0    ; no 4DDA
- FDDAOB=0    ;no obs 4DDA
- MAXNES=2   ; no. of domains
- MIX=180        ; no. of grid points in I dir
- MJX=255       ; no. of grid points in J dir.
- MKX= 23        ;no. of grid points in K dir

## MM5 Runtime Options

- Atmospheric Radiation scheme option
-     IFRAD=2  ; Cloud radiation scheme
-     ISOIL=2 ; Noah land-surface scheme
-     ISHALLO=0 ;no shallow convective scheme

## MM5 Runtime Options

- Physics Option
- IMPHYS = 4 ; Simple Dudhia
- MPHYSTBL = 0 ;not using look-up table
- ICUPA = 3 ; Grell cumulus parameterization
- IBLTYP =  5 ; MRF PBL

## Mmlif Options

- FORECAST TIME AND TIME STEP  **TIMAX** = 43200.,
-     forecast length in minutes**TISTEP** = 60.,
-   coarse domain DT in model,   use 3*DX

## Mmlif Options

- ; Default soil layers expected as input for ISOIL 2 & 3; These values reflect the BOTTOM of the soil layer availableISTLYR = 10,40,100,200,ISMLYR = 10,40,100,200,

## Mmlif options

- ;**LEVIDN** =  0,   1,   2,  1,  1,  1,  1,  1,  1,  1,  ; level of nest for each domain**NUMNC** =  1,   1,   2,  1,  1,  1,  1,  1,  1,  1,  ; ID of mother domain for each nest**NESTIX** =165,  178,  136,  181,  211,  221 ;domain size i**NESTJX** =, 250, 253,  181,  196,  211,  221  ; domain size j**NESTI**  =  1,   56,   28,   35,   45,  50  ; start location i**NESTJ**  =  1,   86,   25,   65,   55,  50;start location i**XSTNES** = 0.,  0., 900.,  0., 0.,  0.,  0.,  0.,  0.,  0., ; domain initiation**XENNES** =99999,99999,99999.,720.,720.,720.,720.,720.,720.,720.; domain termination**IOVERW** =    2, 2, 0, 0, 0, 0, 0, 0, 0, 0, ; overwrite nest input                 ; 0=interpolate from coarse mesh (for nest domains);                 ; 1=read in domain initial conditions                 ; 2=read in nest terrain fileIACTIV = 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, ;; in case of restart: is this domain active?

## MM5 Input/Output

| Size | File Name |
| --- | --- |
| 568802216 | BDYOUT_DOMAIN1
 |
| 590686376 | BDYOUT_DOMAIN2
 |
| 82048388 | LOWBDY_DOMAIN1
 |
| 89555844 | LOWBDY_DOMAIN2
 |
| 4708148132 | MMINPUT_DOMAIN1
 |
| 5139826852 | MMINPUT_DOMAIN2 |

## MM5 Input/Output

| Size  | File Name |
| --- | --- |
| 11019512 | TERRAIN_DOMAIN1 |
| 12018488 | TERRAIN_DOMAIN2
 |
| 10098567060 | MMOUT_DOMAIN1 |
| 11024707988 | MMOUT_DOMAIN2 |

## MM5 FAQ

- Can I get CCSM3.0 output to initialize MM5 ?
-    Only By Permission

## MM5 FAQ

- Where can I get a copy of the most recent version of MM5 source code
- -- NCAR anonymous ftp (_[ftp.ucar.edu](ftp://ftp.ucar.edu/)__)_
-    /mesouser/MM5V3/TERRAIN.TAR.gz
-    /mesouser/MM5V3/REGRID.TAR.gz
-    /mesouser/MM5V3/INTERPF.TAR.gz
-    /mesouser/MM5V3/MM5.TAR.gz
-    /mesouser/MM5V3/MPP.TAR.gz

## MM5 FAQ

- Is it possible to run all MM5 modeling system programs on a Linux Machine ?
-  Yes, it is possible. Compiler options provided by Portland Group Inc. is provided.

**Notes:**
PGI supports cray pointers which are used in MM5; it has f90 compiler

## MM5 FAQ

- How long does it take to run a simulation with nested domain at 12 km and 36 km ?
- It takes about 36 hours of wall clock time [on one node with 32 processors] on an IBM sp2.

## MM5 FAQ

- What are the fields available in the MM5 output file on the Web ?
-   Output for the runs are stored on the archival system at Oak Ridge National Laboratory.  Few fields are extracted and gif images of those fields are available on the SERVIR web page.

## MM5 FAQ

- What is the URL of the Web site for viewing a few extracted variables from the simulation ?
- _[http://](http://servir.nsstc.nasa.gov/climate_change/scenarios.html)__[servir.nsstc.nasa.gov/climate_change/scenarios.html](http://servir.nsstc.nasa.gov/climate_change/scenarios.html)_

## MM5

- How do I visualize the output of MM5?
-    -There is a utility program called mm5tograds (MM5toGrADS.tar.gz)
-    -There is also a program which will convert the output to vis5d (tovis5d.tar.gz) available from MM5 home

**Notes:**
Vis5D development has ceased at SSEC, but continues as the Vis5d+ Project on SourceForge. This was initiated by Steven Johnson of MIT and now includes Jim Edwards, the NCAR visualization team and others. We will continue to serve our older versions of Vis5D but recommend Vis5d+.

## MM5 Output fields [2d]

**ps       pstar (Pa)**
**tg       ground temp (K)**
**rc       accum conv pcn (cm)**
**rn       accum non-c pcn (cm)**
**ter      ter elevation (m)**
**xmf      cross map factor**
**dmf      dot map factor**
**cor      coriolis (s-1)**
**tr       reservoir temp (K)**
**xlat     cross lat (degree)**
**xlon     cross lon (degree)**
**lu       land use**
**sc       snow cover**
**pblh     pbl height (m)**
**pblr     pbl regime**
**shf      sen heat flux (W/m2)**

## MM5 Output fields [2d]

**lhf      lat heat flux (W/m2)**
**ust      friction vel (m/s)**
**swd      down sw rad (W/m2)**
**lwd      down lw rad (W/m2)**
**swo      out sw rad (W/m2)**
**lwo      out lw rad (W/m2)**
**sst      sea sfc temp**
**st1      soil temp 1 (K)**
**st2      soil temp 2 (K)**
**st3      soil temp 3 (K)**
**st4      soil temp 4 (K)**
**sm1      soil moisture 1 (m3/**
**sm2      soil moisture 2 (m3/**
**sm3      soil moisture 3 (m3/**
**sm4      soil moisture 4 (m3/**
**can      canopy moisture (m)**
**ssi      seaice**

## MM5 Output Fields [2d]

**sro      surface runoff (mm)**
**uro      underground runoff (**
**t2m      2 m temperature (K)**
**q2m      2 m mixing ratio (kg**
**u10      10 m u wind (m/sec)**
**v10      10 m v wind (m/sec)**
**pslv     sea level prs (mb)**
**cref     composite refl (dbz)**
**iclw     integrat clw (cm)**
**irnw     integrat rnw (cm)**
**pwat     precipit water (cm)**
**clfrlo   low cloud fraction**
**clfrmi   mid cloud fraction**
**clfrhi   high cloud fraction**

## MM5 Output fields [3d]

**u         u wind (m/s)**
**v         v wind (m/s)**
**w         vertical vel (m/s)**
**pp        prs pert (Pa)**
**t         temperature (C)**
**q         mixing ratio (kg/kg)**
**clw       cloud water (kg/kg)**
**rnw       rain water (kg/kg)**
**rtnd      rad tend (K/day)**
**z         height AGL (m)**
**h         geopot height (m)**
**td        dewpoint temp (C)**

## MM5 Output fields [3d]

**rh        rel humidity (%)**
**th        potential temp (K)**
**the       theta-e (K)**
**prs       pressure (Pa)**
**vor       vorticity (s-1)**
**pv        potential vort (pvu)**
**dbz       reflectivity (dbz)**
**div       divergence (s-1)**
**dir       dir (degrees)**
**tadv      temp advection (K/s)**

## DEMO

- Look at some extracted variables from the runs
- Vis5D demo
-    Vis5D version 5.2 is available at _[ftp://ftp.ssec.wisc.edu/pub/vis5d-5.2](ftp://ftp.ssec.wisc.edu/pub/vis5d-5.2)__._

**Notes:**
Vis5D is a system for interactive visualization of large 5-D gridded data sets such as those produced by numerical weather models. One can make isosurfaces, contour line slices, colored slices, volume renderings, etc of data in a 3-D grid, then rotate and animate the images in real time. There's also a feature for wind trajectory tracing, a way to make text anotations for publications, support for interactive data analysis, etc.