# Presentation: 861217

## Massively Parallel Imagery Assimilation Using the 3D Multiscale Multicomponent Modeling Framework (MMMF) 

**K. Palaniappan**
**Dept of CS**
**U of Missouri-Columbia**

**Bo-Wen Shen**
**U of Maryland CP &**
**NASA-Goddard**

**C. Kambhamettu**
**Dept. of CIS**
**U of Delaware**

- AISR Meeting May 5 -7, 2008

## Project Objectives

- Process satellite imagery using new image analysis algorithms implemented on (cheap/novel) parallel architectures GPU, multicore or Cell processors
- Fuse imagery from multiple EOS satellites
- Provide dense atmospheric parameter estimates of multi-level winds, cloud-top surfaces, microphysics, etc. from multispectral satellite imagery.
-  Multiscale Multicomponent Modeling Framework : Assimilate fields using 4D observations into Cloud Resolving Models (GCE) embedded in a global circulation model (fvGCM) - Columbia supercomputer
- Demonstrate benefits including improved forecasting (tracks, intensities, structures) of severe storms and hurricanes

- NOAA

- Target 48hr forecast with 106 nmi accuracy

- NOAA

- NOAA

## Parallel Image Structure & Motion Analysis

- General motion
- module

- Global-local
- framework

- Global motion
- analysis module

- Local motion
- analysis module

- Extended
- Superquadrics

- Shape-based
- application

- Nonshape-based
- Application

- A general scheme for
-  structure and nonrigid
- motion tracking
-  from 2D images

## Proposed Parallel Image Structure & Motion Analysis Overview

**Local Nonrigid**
**Motion Tracking**

**Structure**
** ****Nonrigid motion **
**3D correspon-**
**dences**

**Global**
**Regulari-**
**zation**

**Local Nonrigid**
**Motion Tracking**

**Local Nonrigid**
**Motion Tracking**

**Global **
**Constraints**

**2D Image**
**Sequence**

**Segmentation**

- Local motion
- analysis module

- Global motion
- analysis module

**Proposed Parallel Image Structure & Motion Analysis Overview**

**Notes:**
This is the system achitecture of the new SMAS system.

First the input image sequences are evenly segmented into many small areas. For each area, local nonrigid motion analysis is performed to tracking local details. Then some global constraints which are defined accoring to the fluid dynamics are used to regularize the locally tracked results.

The output of the systrem is ...

## Initial Experiments for Hurricane Luis

**Initial Experiments for Hurricane Luis**

**Notes:**
Ex.....

This one recovered structure of hurricane Luis. And this is a sample of recover fluid motion. You can see a lot of fluidity recovered locally. Let's look at the movie.

## Initial Experiments for Hurricane Luis

**Initial Experiments for Hurricane Luis**

- 	Although the initial mean errors are very large, they decrease very quickly after the global fluid constraints are applied. Stable results are achieved at the end of the iterations.

**Notes:**
Here, I show some intermediate results during the iterations.
The left one is the recovered z velocity field during iterations. You can see that the recovered z velocity is very bad and has a lot of noises  initially. After the global fluid constraints are applied , the results are getting better and better. The final recovered z velocity field has larger value in the hurricane eye and smaller value in the hurricane body. And it is regular phenomena for mature hurricanes.

The right one is the mean errors of our recovered hurricane structure when compared with disparities. 
This line is..
...

Al....
Also, the postprocessing procedure decrease the error in the hurricane edge dramatically because there are a lot of density variations in the hurricane edge.

However, the post... is not very effective for the hurricane eye. A possible reason is that the motion within the hurricane eye area is so complex and dynamic and it can't be tracked by local affine model. More complex models for the eye area will be investigated in the future.

## <footer>

- <date/time>

## Multiview Local Constraints Using Multiple EOS Satellite Observations

## Proposed Multi-satellite Multi-spectral Approach

- Image Sequence 1

- Motion Flow

- Motion  Flow

- Motion  Flow

- 3D Affine Model

- Stereo Constraints

- Regularization Constraints

- 3D Scene Flow
- 3D Correspondences
- Dense Scene Structure

- Image Sequence 2

- Image Sequence N

- Satellite 1

- Satellite 2

- Satellite N

## Proposed Fusing Multispectral Imagery

- Moving Object Detection - foreground mask FGM identifying moving blobs, using the flux tensor method from IR and WV
- Object Segmentation - Partition object clusters, FGM, into individual objects using level set based geodesic active contours that combines information from visible, IR, WV. Refined foreground mask FGR
- Multiple Object Tracking - resolve frame-to-frame correspondences between moving blobs in FGR. Output moving object statistics and trajectories.
- Cluster Trajectory Analysis - combine segments and resolve incorrect trajectory merges from under-segmentation of neighboring objects or partial and full occlusions

- Improve performance of cloud tracking, through fusion of
- information from both visible, infrared, and water vapor

## Motion Detection 3D Spatiotemporal Volume

- Spatio-temporal volume of hall monitor sequence: (a) Left entire volume, (b) Middle: cut taken at vertical position y0, (c) Right: Cut taken at vertical Position y1.

- Gerald Kuhne, “Motion-based segmentation and classification of video objects”
- PhD Dissertation, Univ. of Mannheim, 2002

**Motion Detection ****3D Spatiotemporal Volume**

## Flux Tensor Matrix

- Advantage: Elements of flux tensor incorporate information about temporal gradient changes
- 	efficient discrimination between stationary and moving image features
- 	Trace of flux tensor matrix can be directly used to classify moving and non-moving regions
- 	Equivalent cost to computing structure tensor matrix without eigen-decompositon

- Each pixel in an IR or WV image frame IIR (x,t) is classified as moving or
- stationary by thresholding trace of associated flux tensor matrix *trace(J**F**)*
- and a motion blob mask FGM is obtained.

## Flux Tensor Cloud Motion Extraction

## Proposed Multispectral Edges

- 	Edge detection in multi-channel images (i.e. color edge strength) is not straight forward to generalize since gradients in different channels can have inconsistent orientations.
- 	Ruzon and Tomasi classify color edge detection algorithms into three categories based on when the individual channel responses are fused
- Output Fusion Methods: gray-scale edge detection is carried out in each channel independently then combined using methods such as weighted sum.
- Multi-dimensional Gradient Methods: estimate a single estimate of the orientation and strength of an edge at a point.
- Vector Methods: color information is treated as a vector through the whole process of edge detection such as in the case of edge detection based on vector order statistics.

- Edge, junction, and corner detection using color distributions _[Ruzon](http://ieeexplore.ieee.org/search/searchresult.jsp?disp=cit&queryText=%28ruzon%20%20m.%20a.%3CIN%3Eau%29&valnm=Ruzon%2C+M.A.&reqloc%20=others&history=yes)__[, M.A.](http://ieeexplore.ieee.org/search/searchresult.jsp?disp=cit&queryText=%28ruzon%20%20m.%20a.%3CIN%3Eau%29&valnm=Ruzon%2C+M.A.&reqloc%20=others&history=yes)_   _[Tomasi](http://ieeexplore.ieee.org/search/searchresult.jsp?disp=cit&queryText=%28%20tomasi%20%20c.%3CIN%3Eau%29&valnm=+Tomasi%2C+C.&reqloc%20=others&history=yes)__[, C.](http://ieeexplore.ieee.org/search/searchresult.jsp?disp=cit&queryText=%28%20tomasi%20%20c.%3CIN%3Eau%29&valnm=+Tomasi%2C+C.&reqloc%20=others&history=yes)_, PAMI Nov. 2001.

## Multi-dimensional Gradient

- 	Main issue: Combination of the individual channel gradients into a final multi-dimensional gradient.
- 	Simple methods: use operations such as sum, weighted sum, max, min etc. to produce the final multi-dimensional/color gradients.
-  	_Problem:_ summation of the individual channel gradients discard the correlation between the channels and may result in cancellation effects.
- 	Tensor methods: Pioneering work on how to combine the gradients of each channel is done by DiZenzo [DiZenzo:CVGIP86] who considered the multi-channel image as a vector field and computed the tensor gradient.

## Edge Indicator Functions for Multi-Channel Images

## Luis Assimilation & Modeling Study

- <date/time>

**Luis Assimilation & Modeling Study**

**MM5 modeling of Luis**
**Simulation of mesovortex dynamics using multiple nested grids**
**Assimilation of satellite-derived rainfall and automatic wind estimates**
**Impact of satellite information on intensity and track prediction**
**Improvements to automatic estimation of multispectral stereo heights and winds**

## Horizontal

-  Potential
- Vorticity
- (6 pv units)

- Outflow Layer Winds (16 km)

- Inflow Layer Winds (500m)
- (max 70 m/s)

- MM5 Luis Model Results

- 10.8 km Grid

- 1200 UTC 6 Sept ‘95:

- Surface Rainfall (max 50mm/hr)

## Limitations of GCMs for Hurricane Modeling 

-  General Circulation Models (GCMs) have insufficient resolution to accurately simulate hurricane near-eye structure and intensity.
-  GCE physics packages (e.g., cumulus parameterizations) are also known limiting factors in simulating hurricanes.

## Limitations of GCMs for Hurricane Modeling 

**MMMF**
- Handles data redistribution
- Responsible for i/o (optional)

**fvGCM**
- Provides large-scale forcing
- Parallel implementation using MPI and OpenMP in y and z directions, respectively

**MS-GCEM**
- Manages GCEMs on the meta grid
- Handles i/o on meta grid points
- 2-D parallel implementation using MPI

- GCE

- GCE

- GCE

## Goddard Multiscale Modeling Framework 

-  Simulated precipitation at CRM grids in MMF, showing heavy precipitation along the ITCZ.

## Forecasts of Track, Intensity, Structures 

- Hugo, 1989, $17B

- Andrew, 1992, $40B

- Katrina, 2005, $100B

- Luis, 1995, $3B

## Forecasts of Katrina’s  Track, Intensity, Structures 

-  Six 5-day simulations of Katrina at both 0.25o and 0.125o show comparable track forecasts, but the higher-resolution (0.125o) runs provide much better intensity forecasts, producing the center pressure with errors of only +- 12 hPa.
-  Realistic near-eye wind distribution and vertical structure are also obtained as cumulus parameterizations are disabled.

## Forecasts of Katrina’s  Track, Intensity, Structures 

**GFS Analysis (~35km) valid at 08/29/12z**

**96 h Simulations with no CPS**

_**High-resolution runs simulate realistic intensity, RMW (radius of max wind) and warm core (shaded)**_

- Landfall errors:
-   e32 (1/4o): 50km, g48(1/8o): 14km, g48ncps (1/8o w/o CPs): 30km

- Six 5-day simulations of Katrina higher-resolution (0.125o) runs provide much better intensity forecasts, producing the center pressure with errors of only +- 12 hPa.
- Realistic near-eye wind distribution and vertical structure

## Forecasts of Katrina’s  Track, Intensity, Structures 

**Near-eye Wind Distributions in a 2****o****x2****o**** box (a) AOML/HRD high-resolution surface wind analysis, (b) the 0.25****o**** 99h simulations, (c) the 0.125****o **** 99h simulations, (d) the 0.125****o**** 99h simulations without convection parameterizations (CPs).**

- Six 5-day simulations of Katrina higher-resolution (0.125o) runs provide much better intensity forecasts, producing the center pressure with errors of only +- 12 hPa.
- Realistic near-eye wind distribution and vertical structure

## Forecasts of Katrina’s  Track, Intensity, Structures 

## Forecasts of Katrina’s  Track, Intensity, Structures 

## Global Mesoscale Modeling on the NASA Columbia Supercomputer

**5-day forecasts of total precipitable water**
** ****initialized at 0000 UTC 1 September, 2004 with the 1/12****o ****fvGCM**

*Frances*

*Sonda*

**Global Mesoscale Modeling on the NASA Columbia Supercomputer**

- A

- B

- C

- D

- A: Atlantic Hurricanes

- B: Catalina Eddy

- C: Hawaiian Lee Wakes

- D: Asian Mei-Yu Front

- E

- E: Indian Twin Tropical Cyclones

## WRF Cases (high-resolution runs, 1 - 3 km grid size)

***GPM C3VP (2007)***

***India - Monsoon***
***(May-June 2005 &2006)***

***Katrina (2005)***

***Forest Fire (2007)***

***TC4 2007***

***IHOP***

***Orgraphic Precipitation***
***(2007)***

## Project Objectives

- Process satellite imagery using new image analysis algorithms implemented on (cheap/novel) parallel architectures GPU, multicore or Cell processors
- Fuse imagery from multiple EOS satellites
- Provide dense estimates of multi-level winds, cloud-top surface structures, microphysics and other parameters from multispectral satellite imagery.
- Multiscale Multicomponent Modeling Framework : Assimilate fields using 4D observation-nudging into Cloud Resolving Models (GCE) embedded in a global circulation model (fvGCM)
- Demonstrate benefits including improved forecasting (tracks, intensities, structures) of severe storms and hurricanes