# Presentation: 281039

## Using Kepler to Perform Parameter Studies in Subsurface Sciences

***Jared Chase***
***Scientific Data Management CET All Hands Meeting 11/28/2007***

**http://subsurface.pnl.gov**

## Project Descriptions/Goals

- CHIPPS (Tim Scheibe, Environmental Technology)
- _Project Goal:_  To develop an integrated multiscale modeling framework with the capability of directly linking different process models at continuum, pore, and sub-pore scales.
- SALSSA (Karen Schuchardt, Applied Computer Science)
- _Project Goal:_  To develop a process integration framework that combines and extends leading edge technologies for process automation, data and metadata management, and large-scale data visualization.
- GWACCAMOLE (Bruce Palmer, High Performance Computing)
- _Project Goal:_  To apply a component-based framework to the development of a new hybrid model for performing subsurface simulations that will combine different physical models into a single coherent simulation.

## Different Scale Models

- Continuum

- Pore

- Sub-Pore

## Calcite Precipitation Problem

- Interested in understanding (and ultimately controlling) the distribution of solid minerals that form from reaction of two dissolved chemicals (solutes).
- This study will allow us to gain understanding of the impact of either high- or low-permeability inclusions along a mixing pathway on the effectiveness of mixing.
- The results of modeling studies such as this will be used to design mesoscale laboratory experiments to validate our conclusions, which will in turn be used to design field-scale pilot and full-scale implementation strategies.

## Hybrid Multiscale ModelingBenchmark Problem

**Hybrid Multiscale Modeling****Benchmark Problem**

## Project SALSSA’s Goals and Requirements

- Create a System that ...
- Automates and integrates research processes.
- Provides records for verifiability.
- Shares and documents:  data, results, tools, and hopefully processes.
- Can be used by all types of users; model developers on down to experimentalists.
- Has longevity so scientists can modify the system to suit their needs.

## Continuum Workflow

- Configuration

- Fixed1

- Uncertain1

- Initial Grid Generation2

- Data Preparation
- And Management12

- Grid Parameter Specification3

- Mathematical
- Model Definition11

- Run Numerical Model4

**Continuum Workflow**

- Grid Refinement9

- Output
- Visualization5

- Output
- Analysis6

- Comparative Analysis
- (with results of previous runs
- and/or observational data)

- Done?

- Stop

- Yes

- No – Refine Grid

- Parameter
- Modification10

- No – Modify Parameter(s)

- No – Modify Model

- Qualitative /
- Quantitative
- Comparisons7

- Horizontal
- Flow
- Simulation8

- SummaryGraphics14

- Simulation Data
- Management
- (I/O Documentation and Storage)13

**Notes:**
Everything is connected to the Simulation Data Management.  Trying to keep track of the entire workflow process.  There is a lot of human workflow.  We want to capture everything.
All processes go into data management.
This is different from the traditional idea of workflow because there is a human interaction component involved.

## Calcite Precipitation Use Case

**Calcite Precipitation Use Case**

**Create stomp study**
- First run a single job with both the porosity and permeability the same to serve as a base case.
- Next run a set of jobs where the fine sand (material 2) becomes progressively less permeable (decrease value by 10, 100, 1000).  keep porosity the same as case #1.
- Starting with settings from #1, increase permeability by 10, 100,1000.  hold porosity the same.
- Starting with settings from #1,  keep permeability the same but decrease porosity by .05 for a couple of iterations.  Again this applies to the find sand.
- 5.       Take result where we decreased permeability by 10 and use it to create a new study.  Its not clear to me why you would start a new study.  Maybe its just an artificial case of notion of making a new study?  We could also use the case of switching to a finer grid as the cause for a new study if you think its less artificial.

**Stomp Run:**	[Permeability (init) = Porosity (init)]

**Stomp Run:****	**[Permeability = Permeability (init) * 0.1]

**Stomp Run:****	**[Permeability = Permeability (init) * 0.01]

**Stomp Run:****	**[Permeability = Permeability (init) * 0.001]

**Stomp Run:****	**[Porosity = Porosity (init) * 0.05]

**Stomp Run:****	**[Porosity = Porosity (init) * 0.10]

**Stomp Run:****	**[Porosity = Porosity (init) * 0.15]

**Stomp Run:****	**[Permeability = Permeability (init) * 10]

**Stomp Run:****	**[Permeability = Permeability (init) * 100]

**Stomp Run:****	**[Permeability = Permeability (init) * 1000]

**Stomp Study**

- 1.

- 2.

- 3.

- 4.

- 5. Create New Stomp Study

- ?

## SALSSA Components and Architecture

## Applying Kepler to Subsurface Research WorkflowUsing Kepler as an End User Tool

- Approach
- End users are able to add components and tools.
- End users can manage their own processes using Kepler.
- End users would create their own workflows using pre-made higher level actor abstractions.
- ConclusionKepler/Virgil is NOT suitable for end users.  Most of this pertains to “workflow designers” as well.
- Complex Types
- Type Checking
- Recording of Provenance
- Animation
- Creating Actors
- Managing technology
- Multiple Instances of Kepler
- Robustness

## Applying Kepler to Subsurface Research Workflow  Using Kepler for Job Execution

- Execute Parameter Studies and Sensitivity Studies
- Launch and monitor multiple jobs using various queuing systems:  SGE, LSF (mpp2), fork.
- Monitor each job within the workflow.
- Notify other tools of job state.
- Move input/output files.
- Workflow Provenance Capture
- Working to define an API specific for provenance capture.

## Issues To Address In Kepler

- Performance
- You can only have one instance of Kepler running on the client machine at one time.
- Kepler takes up a lot of memory.  Possibly there could be a mechanism for packaging just the parts you use.
- Kepler take a long time to startup.
- Building Workflows
- No simple plug in model (ala spring). A mechanism to reuse/extend existing code instead of writing new custom classes (i.e. a framework for connecting existing components instead of framework to develop components).
- Better documentation for actors so that the end user does is not required to read code to understand components and know which you can hook up.
- Components at too low of a level.  There is a need for high level components for job launching/monitoring/file movement.
- Support for parameter studies including a component for load balancing across machines.
- A system built for extensibility to complex and semantic data types.
- A set of actors should be built for easy iteration and parameter studies.
- More control is needed within execution domains.  (i.e. Using PN Directors inside composite actors when a PN Director is used in the parent workflow. http://www.mail-archive.com/ptolemy-hackers@doppler.eecs.berkeley.edu/msg00381.html)

## Organizer

## Stomp Input Wizard

**Stomp Input Wizard**

## Demo

- Workflow Parameters
- numInstances:  the number of jobs that the workflow will execute.
- InputData:  Input data for each of the jobs.

## Next Generation

- Setup
- Stomp

- User works within a “Study” where a Study can be represented as a graph of processes and data inputs/outputs.  Some processes are triggered by the user,  others appear as by-products of user actions.

- 1. Baseline computation

- 3. Vary other parameters...

## Future SALSSA Work

- Deploy current tools to INEL to support experimental work on calcite precipitation problem
- Apply current parameter study workflow to the CCA-based SPH code under development by Bruce Palmer
- Integrate SciDAC Visualization and Analysis capabilities
- Work with SDM center to developer higher level Kepler components
- Job launching, file movement, realtime monitoring
- A workflow environment that combines interactive and automated workflow into one environment with appropriate user abstractions
- Connect all steps into a meta workflow through provenance
- User control over details of view
- Different views of data lineage, processing steps
- Extend Stomp UI wrapper to support more input options
- Support Hybrid model and the additional processing required for setup, execution, and analysis

## Acknowledgment

- Funding for this research is provided by the    U. S. Department of Energy through the following programs:
- Office of Science, Biological and Environmental Research and Advanced Scientific Computing Research, Scientific Discovery through Advanced Computing (SciDAC) program.
- Office of Science, Biological and Environmental Research, Environmental Remediation Sciences Program (ERSP).