**Electromagnetic Systems Simulation -- Prototyping through
Computation**

N. Folwell^1^, L. Ge^1^, A. Guetz^1^, V. Ivanov^1^, K. Ko^1^ (**PI**),
M. Kowalski^1^, R. Lee^1^, Z. Li^1^, C. Ng^1^, Y. Sun^1^, R.
Uplenchwar^1^, M. Wolf^1^ **Affliliated Researchers:** E. Ng^3^, W.
Gao^3^, P. Husbands^3^, X. Li^3^, A. Pinar^3^, C. Yang^3^ (SAPP/TOPS),
G. Golub^2^, Y. Liu^2^, K. Ma^4^, G. Schussman^4^ (SAPP), D. Brown^5^,
K. Chand^5^, K. Devine^6^, L. Diachin^5^, W. Henshaw^5^, P. Knupp^6^, Y.
Luo^7^, M. Shephard^7^, T. Tautges^6^, D. White^5^ (TSTT)

^1^Stanford Linear Accelerator Center, ^2^Stanford University,
^3^Lawrence Berkeley National Lab., ^4^UC Davis, ^5^Lawrence Livermore
National Lab., ^6^Sandia National Lab., ^7^Rensselear Polytechnic
Institute

***The Office of Science's accelerator facilities have started to reap
benefits from DOE's investment in Terascale Computing as an increasing
number of challenging design and analysis problems are being solved
through large-scale simulations. Central to this major advancement is
the SciDAC Accelerator Simulation Project which supports the development
of a suite of parallel electromagnetic codes significantly strengthened
through computer science and applied mathematics (CS/AM) collaborations
that improve their accuracy and maximize their performance on Terascale
Computers.***

**1. Introduction**

The DOE's SciDAC project, "Advanced Computing for 21^st^ Century
Accelerator Science and Technology", has supported a well-integrated,
multi-institutional, multi-disciplinary team to focus on the large-scale
simulations necessary for the design and optimization of electromagnetic
systems essential to accelerator facilities throughout the Office of
Science complex. Significant progress has been made in the development
of a set of parallel electromagnetic codes that are further improved
through SAPP and ISICs collaborations, and in the application of this
comprehensive terascale simulation capability to challenging problems
facing major DOE accelerator projects such as the Positron-Electron
Project (PEP)-II, Next Linear Collider (NLC), and the Rare Isotope
Accelerator (RIA).

**2. Parallel Code Development and CS/AM Collaborations**

Two new field solvers, *S3P* that calculates the scattering matrix of
open structures and *T3P*, a finite element time-domain solver, have
been added in the past year to the suite of 3D parallel electromagnetic
codes that already consists of *Omega3P* (eigensolver), *Tau3P*
(transient solver), and *Track3P* (particle-tracker). These codes
comprise a comprehensive toolset for modeling near- and long-term
projects. SAPP and ISIC collaborations in CS/AM to further advance this
new software capability include:

a). *[New algorithms]{.underline}* such as AV formulation and a
multilevel hierarchical preconditioner to accelerate convergence in
*Omega3P*,

b). A *[linear solver framework]{.underline}* that enables the use of
either direct solvers (e.g., WSMP, SuperLU) or iterative methods
(Iterative Template Library) in large sparse systems,

c). *[Adaptive mesh refinement]{.underline}* in *Omega3P* and *S3P* to
improve solution accuracy at much reduced computational cost,

d). *[Better partitioning schemes]{.underline}* for modeling with
*Tau3P* to obtain better scalability (e.g., RCB1D in place of ParMETIS),

e). *[Advanced visualization techniques]{.underline}* to extract useful
features efficiently from large, multi-stream, unstructured data sets.

These efforts have contributed to the success of many simulations with
10X improvement in speed, accuracy, and convergence rates in some cases.
Further gains can be expected as these technologies continue to mature.

**3. Electromagnetic Systems Simulations**

a). **[PEP-II]{.underline}**, a High Energy Physics (HEP) facility,
currently operates at 2X design luminosity and is aiming for another
twofold increase. Beam heating in the Interaction Region (IR) could
limit the accelerator from meeting that goal. *Tau3P* has been used to
calculate the heat load in the present design (Fig. 1). Higher currents
and shorter bunches will require modifications to the IR to reduce the
increased heating. *T3P* will be able to model the entire IR geometry
(from crotch to crotch) and to simulate the actual curved beam paths in
any new IR design.

![](media/image1.png){width="3.0in" height="0.32708333333333334in"}

*Figure. 1 Tau3P simulation of beam heating in PEP-II IR.*

b). **[RIA]{.underline}**, a proposed Nuclear Physics project ranking
3^rd^ in SC's 20-year Science Facility Plan (SFP), is designing a
variety of RFQ cavities (Fig. 2) for its low-frequency linacs. Due to
lack of accurate predictions, tuners are designed to cover frequency
deviations of about 1%. With *Omega3P,* frequency accuracy of 0.1% can
be reached which will significantly reduce the number of tuners and
their tuning range. Parallel computing will play an important role in
prototyping RIA's linacs and help reduce their costs.

![](media/image2.png){width="1.0833333333333333in"
height="1.0659722222222223in"}

![](media/image3.png){width="1.1666666666666667in"
height="0.9027777777777778in"}

*Figure. 2 (Left) A model of RIA's hybrid RFQ and (Right) an enlarged
portion of the corresponding Omega3P mesh.*

c). **[NLC]{.underline}**, a TeV scale accelerator for HEP research and
of high priority on the 20-year SFP, adopts the Damped Detuned
Structure![](media/image4.png){width="3.25in"
height="0.4423611111111111in"} (DDS) as its main linac design. The DDS
uses damping through external manifolds and detuning by cell-to-cell
variation for suppressing long-range dipole wakefields to control the
emittance growth of long bunch trains. Because of the complex geometry,
wakefield analysis of the DDS has been limited to equivalent circuit
models. Using *Tau3P*, the DDS has been simulated with a transit beam so
that the complete wakefield was found directly for the first time (Fig.
3). *Omega3P* and *S3P* analyses in the frequency domain are under way
to verify the result. Ongoing DDS simulations also include the study of
surface field increase due to the finite RF drive pulse (*Tau3P)*, and
dark current generation resulting from surface emissions under high
fields (*Track3P)*.

![](media/image4.png){width="3.25in" height="0.44305555555555554in"}

![](media/image5.png){width="3.0in" height="0.2076388888888889in"}

![](media/image6.png){width="3.1666666666666665in"
height="0.5715277777777777in"}

*Figure 3 (Top) 55-cell DDS model; (Middle) Transient electric field in
the structure after beam transit; (Bottom) Dipole wakefields in DDS
(blue), detuning only (Pink).*

**4. Concluding Remarks**

High-resolution, system-scale simulation utilizing terascale computers
such as the IBM/SP at NERSC, has been made possible by SciDAC-supported
code development efforts and the CS/AM collaborations with SAPP and the
ISICs. This team-oriented approach to computing is beginning to make a
*[qualitative]{.underline}* difference in the R&D of major DOE
accelerators, existing or planned.

**For further information on this subject contact:**

Dr. Kwok Ko

Advanced Computations Department

Stanford Linear Accelerator Center

2575 Sand Hill Road, MS27

Menlo Park, Calif. 94025

Phone: 650-926-2349

E-mail: <Kwok@slac.stanford.edu>