# Presentation: 641708

## SciDAC Accelerator Modeling ProjectKwok Ko and Robert D. RyneSciDAC PI meetingCharleston, South CarolinaMarch 23, 2004

## Outline

**Project overview**
**Applications**
**Collaborations in Applied Math and Computer Science**
**Future Plans**

## Outline

**Project overview**
**Applications**
**Collaborations in Applied Math and Computer Science**
**Future Plans**

## DOE’s Facilities for the Future of Science, 20yr Outlookis a testament to the importance of DOE/SC and the importance of particle accelerators

**Of the 28 priorities on the list,****nearly **_**1/2**_** are accelerator facilities**

## Accelerator projects on 20yr list

**LCLS**
**RIA**
**CEBAF upgrade**
**BTeV**
**Linear Collider**
**SNS upgrade**
**RHIC II**
**NSLS upgrade**
**Super Neutrino Beam**
**ALS upgrade**
**APS upgrade**
**eRHIC**
**IBX**

## SciDAC Accelerator Modeling Project

- Goal: Create a comprehensive simulation environment, capable of modeling a broad range of physical effects, to solve the most challenging problems in 21st century accelerator science and technology
- Sponsored by: DOE/SC Office of High Energy Physics (formerly HENP) in collaboration w/ Office of Advanced Scientific Computing Research

## SciDAC codes are having a major impact on existing accelerators and future projects

**PEP-II interaction region heating analysis (Omega3P,Tau3P,T3P)**
**Simulation of beam-beam effects in Tevatron, PEP-II, RHIC, and LHC (BeamBeam3D)**
**Discovery that self-ionization can lead to meter-long high density plasma sources for plasma accelerators**
**NLC acc. structure design (Omega3P) & wakefield computation (Omega3P, S3P, Tau3P)**
**Beam loss studies at FNAL booster (Synergia)**
**Study of e-cloud instability in LHC (QuickPIC)**
**NLC peak surface fields and dark current simulations (Tau3P, Track3P)**
**Gas jet modeling (Chombo/EB)**
**RIA RFQ cavity modeling (Omega3P)**

## The SciDAC Accelerator Modeling Project team:A multidisciplinary, multi-institutional team producing comprehensive terascale accelerator design tools

*UCLA, USC, Tech-X, U. Colorado*

*Stanford, LBNL (CRD)*
**Parallel Linear Solvers,**
**Eigensolvers, PDE Solvers, AMR**

*LBNL (AFRD)*
**Beam-Beam; Space Charge in linacs & rings; parallel Poisson solvers**

**Plasma-Based Accelerator Modeling; Parallel PIC framworks (UPIC)**

- The SciDAC Accelerator Modeling Project team:A multidisciplinary, multi-institutional team producing comprehensive terascale accelerator design tools

*FNAL*
**Space-charge in rings; software integration; Booster expts**

*UC Davis*
**Particle & Mesh Visualization**

*SLAC*
**Large-Scale Electromagnetic Modeling**

*SNL*
**Mesh**
**Generation**

*U. Maryland*
** ****Lie Methods in Accelerator Physics**

*LANL*
** ****High Intensity Linacs, Computer Model Evaluation**

*M=e**:f**2**:** e**:f**3**:** e**:f**4**:**...*
*N=A**-1 **M A*

*BNL*
**Space-charge in rings; wakefield effects; Booster expts**

## Code Development

**Electromagnetics**
**Omega3P, Tau3P,T3P, S3P, Track3P**
**Beam Dynamics**
**BeamBeam3D, IMPACT, MaryLie/IMPACT, Synergia, Langevin3D**
**Advanced Accelerators**
**OSIRIS, VORPAL, QuickPIC, UPIC**

## IMPACT code suite User-Map

- SLAC
- LBNL
- LANL
- TX corp
- FNAL
- ORNL
- MSU
- BNL
- JLab

- RAL
- PSI
- GSI
- KEK

## Collaborations with Applied Math and Computer Science

**SciDAC ISICs (TOPS, APDEC, TSTT), SAPP**
**Eigensolvers and linear solvers**
**Poisson solvers**
**AMR**
**Meshing & Discretization**
**Parallel PIC methods**
**Partitioning**
**Visualization**
**Stat methods**

## Outline

**Project overview**
**Applications**
**Collaborations in Applied Math and Computer Science**
**Future Plans**

## Modeling the PEP-II Interaction Region

**Beam heating in the beamline complex near the IR limited the PEP-II from operating at high currents. Omega3P analysis helped in redesigning the IR for the upgrade. **

**FULL-SCALE OMEGA3P MODEL FROM CROTCH TO CROTCH **

- Courtesy K. Ko et al., SLAC

## Modeling the PEP-II Interaction Region

## Tevatron Modeling

- Large computing requirement: each point requires 12 hrs x 1024 procs
- Recent result: good agreement for pbar lifetime vs proton intensity

- Courtesy Fermilab and LBNL

## Beam-Beam Studies of PEP-II

**Collaborative study/comparison of beam-beam codes**
**Predicted luminosity sensitive to # of slices used in simulation **

## Modeling a Plasma Wakefield Accelerator with added realism in full 3D models (OSIRIS, VORPAL)

- Full EM PIC simulation of drive beam **ionizing **Lithium in a gas cell. Courtesy W. Mori et al, UCLA

## Full Scale modeling of 30-cell Structure 

- Distributed model on a mesh of half million hexahedral elements
- Study RF damage at high power X-Band operation using Tau3P & Ptrack3D

- Courtesy K. Ko et al., SLAC

## NLC Accelerating Structure Design

## QuickPIC calculations have resulted in up to 500x increase in performance over fully EM PIC

**Wake produced by an electron beam propagating through a plasma cell**

## Modeling beam loss in the Fermilab Booster using Synergia

- Booster simulation and experimental results. (P. Spentzouris and J. Amundson, FNAL)

## Outline

**Project overview**
**Applications**
**Collaborations in Applied Math and Computer Science**
**Future Plans**

## Collaboration w/ SciDAC ISICs 

**TOPS: linear algebra libraries, preconditioners, eigensolvers for better convergence & accuracy**
**APDEC: solvers based on block-structured AMR, and methods for AMR/PIC**
**TSTT: gridding and meshing tools**

## Collaboration with TOPS:Eigensolvers and Linear Solvers

## Collaboration w/ TOPS: Partitioning

## Collaboration with APDEC 

- AMR for particle-in-cell.
- Goal:  Develop a flexible suite of fast solvers for PIC codes, based on ADPEC’s Chombo framework for block-structured adaptive mesh refinement (AMR).
- Block-structured adaptive mesh solvers.
- Fast infinite-domain boundary conditions.
- Flexible specification of interaction  between grid and particle data.
- Accurate representation of complex geometries.

**Notes:**
Some of the solvers are from TOPS (e.g., Hyper).

## Collaboration with APDEC:Benefits from Heavy Ion Fusion program  

**AMR modeling of an HIF source and triode region in (r,z) geometry**

**In this example, we obtain a ~ 4x savings in computational cost for ~ the same answer**

- Courtesy of A. Friedman, P. Colella et al., LBNL

## Collaboration with APDEC:Embedded boundary methods for gas jet modeling

## Collaboration with TSTT: Meshing & Discretization

## Collaboration with TSTT: AMR on Unstructured Grids

## SciDAC Accelerator Modeling Project provides challenging visualization problems

- Courtesy K.-L. Ma et al., UC Davis

## 	Courtesy Andreas Adelmann (PSI) and Cristina Siegerist (NERSC viz group)

- Courtesy Andreas Adelmann and PSI viz group

**Simulating high intensity beams & beam halos**

## Parallel Performance and Parallel Implementation Issues

**Example:BeamBeam3D**

| PEs | time (sec) |
| --- | --- |
| 128 | 1612 |
| 256 | 858 |
| 512 | 477 |
| 1024 | 303 |
| 2048 | 212 |

- Scaling using weak-strong option

- Performance of different parallelization
- techniques in strong-strong case

- Milestone: First-ever million particle, million turn, strong-strong simulation performed for LHC

## High Aspect Ratio solver based on Integrated Green Function (IGF):New algorithm provides < 1% accuracy using 64x64 grid (black curve). 

_**High Aspect Ratio**_** **solver based on Integrated Green Function (IGF):New algorithm provides < 1% accuracy using 64x64 grid (black curve).** **

- 64x1024

- 64x2048

- 64x4096

- 64x8192

**64x16384**

**IGF**
**64x64**

## Comparisons with Experiments

**LANL proton radiography (single-particle optics)**
**LANL LEDA beam halo experiment**
**J-PARC front end test (collab w/ KEK/JAERI)**
**FNAL booster**
**BNL booster**
**CERN PS (collab w/ CERN, GSI)**

## Statistical Methods for Calibration and Forecasting

- Simulation of a high intensity proton beam through a series of quadrupole magnets.  Statistical techniques were used to combine 1D profile monitor data with simulations to infer the 4D beam distribution. The figure shows the 90% intervals for the predicted profile at scanner #6 (shaded regions), and, for comparison, the observed data (black line). Only data from the odd numbered scanners were used to make the prediction.

**Determining initial phase space distribution from 1D wire scan data.**
**Courtesy D. Higdon (LANL) et al.**

## Outline

**Project overview**
**Applications**
**Collaborations in Applied Math and Computer Science**
**Future Plans**

## NLC Accelerating Structure Design

## NLC Accelerating Structure Design

## 3D First-Principles Fokker-Planck Modeling

- Requires analog of 1000s of space-charge calculations/step
- “...it would be completely impractical (in terms of # of particles, computation time, and statistical fluctuations) to actually compute [the Rosenbluth potentials] as multiple integrals” J.Math.Phys. 138 (1997).

- Self-Consistent Diffusion Coefficients

- Spitzer approximation

- Previous approximate calculations performed
- w/out parallel computation
- were not self-consistent

***FALSE. Feasibility demonstrated on parallel machines at NERSC and ACL***

- Courtesy J. Qiang (LBNL) and S. Habib (LANL)

## Optimization

**Accelerator system design including space charge**
**Shape optimization**
**Plasma afterburner**

## Slide 42