# Presentation: 497215

## WIGLAF - An API for Genetic Algorithm Research on Cluster Computer SystemsA GA practitioner’s view on what is needed for efficient use of a 1000 CPU cluster

- Gerhard Klimeck
- Tom Cwik, Ed Vinyard, Frank Villegas, Carlos Salazar-Lazaro, Ricardo Zebulum, Adrian Stoica, Didier Keymeulen
- gekco@jpl.nasa.gov
- (818) 354 2182
- http://hpc.jpl.nasa.gov/PEP/gekco
- PS.: Wiglaf is a character in the Beowulf saga - it is also an acronym** **

## Presentation Outline

**A practitioner’s experience with genetic algorithms**
**Parallel GA development for efficient use of a large cluster**
**GA application management on a large cluster **
- web-based interaction with a remote cluster -> WIGLAF

- Background:
- Development of a an interface to a parallel genetic algorithm
- Integration of SPICE circuit simulation (=>evolvable hardware)
- Software delivered to DARPA
- Integration of electromagnetic filter design

- Integration of NEMO (Nanoelectronic Modeling) => primary research interest
- 		GENES: Genetically Engineered NanoElectronic Structures
**What is needed for GA research on a large cluster?**
-   3 examples: 1sec/individual, 30minutes/individual, Nirvana

## Mapping of Atomic Orbitals to Bulk Bandstructure 

- Structure

- Atomic Orbitals
- size: 0.2nm

**Bulk Semiconductors are described by:**
- Conduction and valence bands, bandgaps (direct, indirect), effective masses
- 10-30 physically measurable quantities

**Mapping of Atomic Orbitals to Bulk Bandstructure**** **

- First Application Example

## Semiconductor Compounds:cation: In, Ga, Alanion: Sb, As, P

- Match experimental data in various electron transport areas of the Brillouin zone:
- Effective masses of electrons at , X and L
- Effective masses of holes at 
- Bandedges at , X and L

- Each individual material poses a 15 dimensional fitting problem.

**P**

**As**

**Sb**

**In**

**Ga**

**Al**

- First Application Example

## Computational Cost

**P**

**As**

**Sb**

**In**

**Ga**

**Al**

**Each material:**
**15 parameters**
**1500 individuals in pool**
**5000 generations with 20% replacement**
**~1.5 Million Individuals**
**1second compute time / individual**
**417 CPU hours ****	****serial**
**32 hours ****	****	****	****32CPUs **
**9 materials **
**150 serial CPU days**
**4.7 wall clock days on 32CPUs****- no counting the failed attempts!!****- but:****=> successfully established a new   ****	****basis set**

- First Application Example

## Heterostructure Applications:Transitions & Functionality Controlled by Design

- Second Application Example

## Genetically EngineeredNanoelectronic Structures (GENES)

**Objectives:**
- Automate nanoelectronic device synthesis, analysis, and optimization using genetic algorithms (GA).
**Approach:**
- Augment parallel genetic algorithm (PGApack).
- Combine PGApack with NEMO.
- Develop graphical user interface for GA.
**How do you know what you have built?**

**PGAPACK**

- BestStructure

**Genes**

**Gene**
**Fitness**

**Desired**
**Data**

**NEMO**

**Simul.**
**Data**

**Fitness**

**Software Package with****Graphical User Interface**
**Input: **
**• ****	****Simulation Targets**
**• ****	****GA parameters**
**• ****	****Platform parameters**
**• ****	****Interaction between workstation/PC and supercomputer**
**Output:**
**• ****	****Evolution development**
**• ****	****Evolution results**
**• ****	****Evolution statistics**

**Architecture**

**Results:**
- Nanoelectronic
- Device
- Structural
- analysis

- GA analyzed atomic  monolayer  structure and doping profile of RTD device
- Black: structure specs, Blue: Best fit

- Second Application Example

## Genetically EngineeredNanoelectronic Structures (GENES)

**Computational cost:**
- 30 minutes compute time / individual
- 2000 individuals evaluated
- 1000 serial compute hours
- 31 hours compute time on 32 CPUs (in 1998 on neptune.cacr.caltech.edu)

**PGAPACK**

- BestStructure

**Genes**

**Gene**
**Fitness**

**Desired**
**Data**

**NEMO**

**Simul.**
**Data**

**Fitness**

**Software Package with****Graphical User Interface**
**Input: **
**• ****	****Simulation Targets**
**• ****	****GA parameters**
**• ****	****Platform parameters**
**• ****	****Interaction between workstation/PC and supercomputer**
**Output:**
**• ****	****Evolution development**
**• ****	****Evolution results**
**• ****	****Evolution statistics**

**Architecture**

**Results:**
- Nanoelectronic
- Device
- Structural
- analysis

- GA analyzed atomic  monolayer  structure and doping profile of RTD device
- Black: structure specs, Blue: Best fit

- Second Application Example

## Bottom-up, Atomistic 3-D Quantum Dot Simulation forRevolutionary Computing and Sensing 

- Structuremillions of atoms

- Atomic Orbitals
- size: 0.2nm

- DesignedOptical
- Transitions
- Sensors

- Quantum Dot
- Arrays
- Computing

**Opportunity:**
- Nanoscale electronic structures can be built!=> Artificial Atoms / Molecules
**Problem:**
- The design space is huge: choice of materials, compositions, doping, size, shape.
**Approach:**
- Deliver a 3-D atomistic simulation tool
- Enable analysis of arbitrary crystal structures, atom compositions and bond/structure configurations.

**Bottom-up, Atomistic 3-D Quantum Dot Simulation for****Revolutionary Computing and Sensing**** **

- The Nirvana Example

## Bottom-up, Atomistic 3-D Quantum Dot Simulation forRevolutionary Computing and Sensing 

- Structure

- Atomic Orbitals
- size: 0.2nm

- DesignedOptical
- Transitions
- Sensors

- Quantum Dot
- Arrays
- Computing

**Computational Cost:**
**One structure / individual:  1-10 hours on 20-40 CPUs running in parallel**
**Cannot even think about running GA based evolution right now!!!!**
**Have no implemented architecture **
**Need a parallel GA running parallel applications underneath**

**Bottom-up, Atomistic 3-D Quantum Dot Simulation for****Revolutionary Computing and Sensing**** **

- The Nirvana Example

## Can a Dedicated 1000CPU cluster help?Look at 3 classes of problems

**CPU time/individual ~ 1-2 seconds**
**Interesting new things for parameter sets >100-200 ****=> need population sizes >10,000, need generations >10,000****=> 20Million individuals => 5,555 CPU hours**
**GA embarrassingly parallel - but:**
- 100 real parameters * 8 bytes => 800bytes/message/1second
- 63 CPUs contact master:  0.8% of 100Mbps (without latency cost)
- 999 CPUs contact master:  12.8% of 100Mbps (without latency cost)=> starting to push the 100Mbps network to its limits there is no 1000 node switch => staggered switches => more overhead

- Need ~1000 generations for the first improvements
- 8.6 hours into the computation (64 CPUs) => greater than any public queue => need long queues=> need storage of population => database approach

- First Application Example

## Can a Dedicated 1000CPU cluster help?Look at 3 classes of problems

**CPU time/individual ~ 30 minutes**
**Interesting new things for parameter sets >10-30 ****=> need population sizes >1,000, need generations >1,000****=> 200,000 individuals (20%replacement) => 100,000 CPU hours**
**This problem will scale perfectly on a 200 CPU cluster (20% replacement / generation)**
**But it is still 21 days on 200 CPUs**

- Second Application Example

## Can a Dedicated 1000CPU cluster help?Look at 3 classes of problems

**CPU time/individual ~ 1 hour on 40 CPUs**
**1 day on a 1000CPUs will evaluate****600 individuals****=> can think more about statistical analysis in ****     nano structures**

**Conclusion for GA aplications:**
- The utilization of a 1000 node cluster for one experiment at a time requires a significant software effort.
- The first utilization will rely on individual runs with ~100CPUs at a time with long queue times.=> 10 experiments / users at the same time => application management

**Problems with cluster applications:**
- There is no easy access / management / execution environment on clusters!!!
- Need user interfaces to the applications
- => WIGLAF

- The Nirvana Example

## WIGLAF Goals and Approach

**Web-based GUI framework for application developers and users**
**Access to parallel / remote machines from web browser**
**Independent of applications and remote machines**
**Java: browser applets and networking**
**XML: data portability and tools - generation of application input**

**Notes:**
Explain: environmental knowledge includes parallel programming, queuing.

## WIGLAF:  Integration Levels

**Simple integration:**
**No modification needed to the application**
**text-based input to application, generated by XML**
**integration of an application within a week**
**Nirvana level integration:**
**Application reads and returns XML based data**

**Notes:**
Explain: environmental knowledge includes parallel programming, queuing.

## WIGLAF Status

**XML DTD for application description**
**Client and server implementations**
**Pre-existing application integrated *****without***** *****changes!***

**Hierarchical**
**input**

**Simple**
**Graphics**

**Text**
**Outputs**

## WIGLAF Needed Tasks

**Full web integration **(WIGLAF is a stand-alone application now)
**More sophisticated data visualization (3-D)**
**Data management**
- GA’s generate a tremendous set of data
- Possibly use general database
- Enable statistical analysis
- Enable reuse of individuals that may be expensive to generate
**Application control**
- Right now WIGLAF launches application and looks for output
- Need at least start/stop
- Possibly simulation steering
**Interaction with queue management (expl.: PBS)**
**Security: Access to remote clusters**

## Can a Dedicated 1000CPU cluster help?YES!!!

**Problems with cluster applications:**
- There is no easy access / management / execution environment on clusters!!!
- Need user interfaces to the applications
- => WIGLAF
**Maintenance and efficient use of such a machine is non-trivial !!!!**

## Basic Genetic Algorithm

**Basic Genetic Algorithm**

**Genetic algorithm parameter optimization is based on:**
**Survival**** of good parameter sets**
**Evolution**** of new parameter sets**
**Survival of a diverse population**
**Optimization can be performed globally, rather than locally.**

## Basic Evolution Operations

**Basic Evolution Operations**

**Creation**** of new gene values.**

**Each set (Si) consists of several parameters (Pj)**
**The parameters Pj can be of different kinds: real, integers, symbols, ....**

- Mutation operation

**Gross Exploration**

**Fine Tuning**