A Parallel Lattice Boltzma

Algorithm for Multicompo

Fluid Flow in Complex Geom

John Hagedorn

Delphine Goujon

Nicos Martys

Judith Devaney

National Institute of Standards and Techn
Purpose

To model the flow of multiple fluids thrporous media such as rock, concrete, anApplications:

• oil recovery• service life of materials

• spread of hazardous waste
Approach

A lattice Boltzmann (LB) method is use

At each cell the fluid mass and velocity 

information is represented.At each iteration, mass is redistributed tneighboring cells and velocities change on: • interactions between fluids • interactions between fluids and so • an external force (gravity)
Example media: sandstone

7.5% porosity 22% poro
Boundary Conditions

X & Y : mirror boundary conditionZ : periodic boundary condition
Problem

 The LB method is extremely memor compute intensive.Solution Restructure memory representation t memory usage Parallelize
Memory Usage

The initial version of the algorithm storfloats for each fluid at each site. Furthesecond data volume was used to hold 

intermediate results.So the total memory required is: N

3 · 2 · (# fluids) · 19 · sizeof(float)

For a 256

3 system and two fluid compo

we require approximately 4.8 gigabytes
Memory Optimization

• Store data only at active sites: yields l

 savings based on the porosity of the  medium.• Use a 3 plane circular buffer for interm results: eliminates need for an entire  data volume.
Memory Optimization (continued

Each active cell points to a structure in an aEach inactive cell points to NULL.

NULL

. . 

Active cells (pore)Inactive cells (solid)

array of cell da
Memory Optimization (continued

Now the memory requirement is: (N

3+3N

2

) · (sizeof(pointer) + 

ρ·(19·#fluids)

where 

ρ is the porosity of the medium.

For a 256

3 system with two fluid compo

10% porosity, this requires approximate

megabytes. This is a savings of about 93%.
Memory Optimization (continued

Additional large memory savings can bachieved by making simplifying assumpthe LB equations.These assumptions enable us to reduce 

at each cell to 3+(#fluids) floats.For a two fluid model, this is a savings 86% of the memory devoted to the activ
Parallelization

Needed to improve performanceThe nearest−neighbor dependency sugg

that the algorithm would be well suited parallelization.We used:

• MPI• SPMD programming model
Parallelization (continued)

Processor 0Processor 1Processor 2Processor 3

The data volume idivided into blockthe Z axis. Each bassigned to a procAt the end of each iteration, partial refrom the bordering are exchanged betadjacent blocks.
Numerical Tests

The algorithm was run on several types "ideal" media such as parallel plates, anoverlapping spheres: Flow through parallel plates. Flow through spher

on cubic lat
Comparison with 

Experimental Data Measured and modeled permeabilites of Fontainebleau sandst
Performance Results

We have run performance tests on an IBM SP

SGI Onyx. The performance data closely agrvery simple model:

T = P / N + S

Where:

T is the total time for one iterationP is the time for the parallelizable compu

 N is the number of processors

S is the time for the non−parallelizable co
Performance Results (continued)

For a two−component, 128

x128

x256 system

measured performance closely matched: T = 24.36 / 

N + .237 seconds on the SG

 and T = 59.92 / 

N + 2.09 seconds on the IBM 
Performance Results (continued)

SGI Onyx IBM SP
Future Work

 • More tests on real−world media • Additional performance studies • Tests on very large systems • Investigation of impact of data lay

performance

 • Refinement of the model
Conclusion

 • This LB model has proven accurat

effective.

 • Through some simple restructuring 

we have obtained large memory 

reductions.

 • The model parallelized very well, 

us to run system sizes that were pr

not feasible.

We have an effective tool for investigatmulti−fluid flow through complex poro