# Presentation: 526687

## Simulating the Cosmic History of BaryonsDiscoveries Using Advanced ComputingMichael L. Norman, Physics Dept., UC San Diego

- San Diego Supercomputer Center, UCSD

- Keck Observatory, HI

- Simulating the Cosmic History of Baryons*Discoveries Using Advanced Computing*Michael L. Norman, Physics Dept., UC San Diego

- validation

## Cosmic History of Baryons 

- linear perturbation theory

- nonlinear simulations

- phase transitions

- gravitational instability

**Structure Formation**:
- 50 Myr < t < 14 Gyr    *synthesis era*

## We are here

## Cosmological N-body Simulation

- A. Evrard and the Virgo Consortium

## Slide 5

## Multiscale Challenge

## Grand Challenges in Computational Hydrodynamic Cosmology

- Formation and evolution of stellar systems on all scales and epochs
- Chemical enrichment and reionization of intergalactic medium
- Formation of massive black holes and nature of the quasar phenomenon
- Cosmological constraints on nature of dark matter and dark energy

## Outline

- Cosmology’s Standard Model
- Universe in a Box
- History of Baryons: *Discoveries using Advanced Computing*
- Exciting Opportunities Ahead
*Cosmological limits on dark matter mass*
*Measuring dark energy equation of state*

## Concordance model
H0=72+/-7 km/s/Mpc
expansion rate accelerating (q0<0)
flat universe (k=0)
dominated by dark matter and dark energy
baryons minor constituent

*Concordance model*
- H0=72+/-7 km/s/Mpc
- expansion rate *accelerating* (q0<0)
- flat universe (k=0)
- dominated by *dark matter* and *dark energy*
- baryons minor constituent

- Cosmology’s Standard Model

- Perlmutter (2003), Physics Today

## Evidence for an Accelerating Universe

- S. Perlmutter, Physics Today (2003)

## Cosmic Microwave BackgroundTemperature Fluctuations 380,000 yr ABB 

- T/T ~  ~ 10-4

- NASA WMAP

## CMB Angular Power Spectrum

## Mass-Energy Budget of the Universe (WMAP+SNe+XRC)

- 

## Universe in a Box

## The Universe is an IVP suitable for computation

*Globally*, the universe evolves according to the Friedmann equation

- Hubble parameter

- mass-energy
- density

- spacetime
- curvature

- scale factor a(t)

- cosmological constant

## The Universe is an IVP...

*Locally**, its contents obey:
- Newton’s laws of gravitational N-body dynamics for stars and *cold dark matter*
- Euler or MHD equations for baryonic gas/plasma
- Atomic and molecular processes important for radiative cooling of gas and condensation to form stars and galaxies
- Radiative transfer equation for photons

- (*scales << horizon scale ~ ct)

-  Numerical astrophysics on a cosmic scale

## radiation

- background

- galaxies

- IGM

- photo-ionization
- photo-heating

- absorption

- feedback
- (energy, metals)

*SF-recipe*

- self-shielding

- photo-evaporation

- infall

- ionizing
- flux

*multi-species*
*hydrodynamics*

*radiative transfer*

*N-body dynamics*

- cosmic expansion

- self-gravity

- dark matter
- dynamics

- baryonic universe

**Notes:**
Three actors in the evolution of the baryonic universe:
galaxies, IGM, and radiation backrounds
the baryons are animated by the gravity of the dark matter whose clustering has been well studied using N-body simulations 
over the past 10 years, my group has been building codes and code modules to handle the complexities of the baryonic/photonic component
each actor is coupled to the others through two-way interfaces (describe)

## Cold Dark Matter

- Dominant mass constituent: cdm~0.23
- Only interacts gravitationally with ordinary matter (baryons)
- Candidates: WIMPs or axions
- Collisionless dynamics governed by Vlasov-Poisson equation
- Solved numerically using fast N-body methods

## Gridding the Universe

- Transformation to *comoving coordinates* x=r/a(t)

- a(t1)

- a(t2)

- a(t3)

- Triply-periodic boundary conditions

- But what about initial conditions?

## Matter Power Spectrum P(k)

- http://www.hep.upenn.edu/~max

- Concordance
- model

## Gravitational Instability: Origin of Cosmic Structure

- A

- B

- C

- A

- B

- C

- x

- x

- 

- 

- 

- 

*very small fluctuations*

*gravity amplifies fluctuations*

## Formation of the Cosmic Web:

- Sky Dome Rendering for DomeFest 2005
- Michael Norman, Brian O’Shea, UCSD
- Donna Cox, Robert Patterson, Stuart Levy, UIUC
- Steve Cutchin, Amit Chourasia, SDSC

## Technical Details

- Simulation (*Enzo*)
- Dark matter, gravity, multispecies gas dynamics, photo-ionization and, radiative cooling
- 1 billion cells, 1 billion particles
- 512 cpu, NCSA TeraGrid cluster
- Data
- 512x512x512 arrays of density
- 2000 timesteps
- 1 Terabyte of data
- Volume rendering
- SDSC IBM DataStar

## Slide 24

## Structured Adaptive Mesh Refinement (Berger and Colella 1989)

## Cosmological Adaptive Mesh Refinement(Bryan & Norman 1997)

- Cosmological Adaptive Mesh Refinement(Bryan & Norman 1997)

- Spatial dynamic range unlimited *in principle*
*Today:*
- L/ = 104 in statistical volumes
- L/ =1010 single objects of interest
*Petascale:*
- L/ =106 in statistical volumes

- _[http://cosmos.ucsd.edu/enzo](http://cosmos.ucsd.edu/enzo)_

## Enzo Implementation Details 

- Enzo Implementation Details

- Multi-scale in space and time
- Arbitrary # levels of refinement
- Arbitrary # grids per level
- Portable, MPI-parallel, C++/C/F77 hybrid
- Nonlocal dynamic load balancing
- Ported to IA64, SGI Altix, IBM SP, BG/L, your mother’s Linux cluster, .....

## Slide 29

## Technical details
2563 base grid
>32,000 grid patches @ 8 levels of refinement
110,000 cpu-hrs on 128 cpu Origin2000
0.5 TB of data
Run at NCSA in 1999

- Technical details
- 2563 base grid
- >32,000 grid patches @ 8 levels of refinement
- 110,000 cpu-hrs on 128 cpu Origin2000
- 0.5 TB of data
- Run at NCSA in 1999

**Image credit: D. Cox et al.**

**Science credit: M. Norman, G. Bryan, B. O’Shea**

- Galaxy Formation and Large Scale Structure

## Computational Discoveriesusing Advanced Computing 

**First baryonic**
**condensations**

## “Bottom-Up” Galaxy Formation

- “Bottom-Up” Galaxy Formation

-  large galaxies form from mergers of smaller galaxies
-  where does this begin?
- What are the first objects to form?

- Lacey & Cole (1993)

## First objects: a well-posed problem

- Initial conditions specified: i, P(k)
- Macroscopic dynamics understood
- Microphysics of primordial gas known
- Have 3D solution-adaptive algorithms
- Have adequate computer power

- February 2003

## Formation of First StarsAdaptive Mesh Refinement SimulationAbel, Bryan & Norman (2001)

- Formation of First StarsAdaptive Mesh Refinement SimulationAbel, Bryan & Norman (2001)

- 1 x

- 10 x

- 100 x

- 1000 x

- 104 x

- 105 x

- 106 x

- 107 x

*Cosmic scales*

*Solar system scales*

## Birth and Death of the First Star in the Universe

**Movie credit: R. Kaehler & T. Abel**

**Science credit: T. Abel, G. Bryan, M. Norman**

## Impact of the first stars

-  the first stars in the universe began forming around *50 million years after the big bang*
-  they were *exceptionally massive and bright*, bringing an *earlier end to the cosmic “dark ages”* than previously thought
-  when they exploded as supernovae they *seeded the universe with heavy elements* essential for planets and life
-  they *kick-started the cosmogonic sequence* which eventually formed galaxies, clusters and superclusters

## Computational Discoveriesusing Advanced Computing 

**structure of**
**intergalactic**
**medium **

## The Intergalactic Medium

- Source: M. Murphy

## Structure of the IGM

- Baryon Overdensity, z=3

## Matter Power Spectrum P(k)

- http://www.hep.upenn.edu/~max

- CDM

## Computational Discoveriesusing Advanced Computing 

**whereabouts**
**of missing**
**baryons**

## Missing Baryons at z=0

- Galaxies in local universe account for only 10% of baryons we know exist due to three independent measurements, which all agree to 2
- Big bang nucleosynthesis
- CMB anisotropies
- IGM absorption at high redshift
- Where are the baryons now?

## Whereabouts of the missing baryons: Warm-Hot intergalactic gas 

***Cen & Ostriker (1998)***

**warm-hot gas**

**“****galaxies”**

- N=5123

## Exciting Opportunities Ahead(require Terascale and beyond)

- Predicting properties of first galaxies
- Understanding quasar-galaxy connection
- Self-consistent simulation of the reionization era
*Cosmological limits on dark matter mass*
*Measuring the dark energy equation of state*

## Effect of DM particle mass on first objects: critical threshold

- 25 keV

- 10 keV

**O’Shea & Norman (2005)**

## Measuring Dark Energy EOS

- Principal goal of NASA/DOE JDEM mission
- _Approach_: precision measurements of expansion history of the universe using Type Ia SN *standardizable candles*
- _Complimentary approach_: redshift distribution of galaxy clusters

## Lightcone Simulation(A. Evrard et al. 2003)

- Evrard et al.
- Single, 10243 P3M
- L/=104
- Dark matter only

- Our plan
- Multiple, 5123 AMR
- Optimal tiling of lightcone
- L/=105
- Dark matter + gas

**ct (Gyr)**

- _Cosmic Simulator_
- A *software facility for physical cosmology*
- A new collaboration between LLNL and UCSD
- Scientific data management focus
- Simulations: LLNL Thunder, BG/L
- Data management: SDSC SRB
- Public archive @ UCSD
- Science driver:
- LSST (Large Synoptic Survey Telescope)