Package: 21cmFAST
A semi-numerical cosmological simulation code for the radio 21-cm signal.This is the official repository for21cmFAST: a semi-numerical code that is able to
produce 3D cosmological realisations of many physical fields in the early Universe.
It is super-fast, combining the excursion set formalism with perturbation theory to
efficiently generate density, velocity, halo, ionization, spin temperature, 21-cm, and
even ionizing flux fields (see the above lightcones!).
It has been tested extensively against numerical simulations, with excellent agreement
at the relevant scales.21cmFASThas been widely used, for example, by the Murchison Widefield Array (MWA),
LOw-Frequency ARray (LOFAR) and Hydrogen Epoch of Reionization Array (HERA), to model the
large-scale cosmological 21-cm signal. In particular, the speed of21cmFASTis important
to produce simulations that are large enough (several Gpc across) to represent modern
low-frequency observations.As ofv3.0.0,21cmFASTis conveniently wrapped in Python to enable more dynamic code.New Features in 3.0.0+Robust on-disk caching/writing both for efficiency and simplified reading of
previously processed data (using HDF5).Convenient data objects which simplify access to and processing of the various density
and ionization fields.De-coupled functions mean that arbitrary functionality can be injected into the process.Improved exception handling and debuggingComprehensive documentationComprehensive test suite.Strictsemantic versioning.InstallationWe support Linux and MacOS (please let us know if you are successful in installing on
Windows!). On these systems, the simplest way to get21cmFASTis by usingconda:conda install -c conda-forge 21cmFAST21cmFASTis also available on PyPI, so thatpip install 21cmFASTalso works. However,
it depends on some external (non-python) libraries that may not be present, and so this
method is discouraged unless absolutely necessary. If usingpipto install21cmFAST(especially on MacOS), we thoroughly recommend reading the detailedinstallation instructions.Basic Usage21cmFASTcan be run both interactively and from the command line (CLI).InteractiveThe most basic example of running a (very small) coeval simulation at a given redshift,
and plotting an image of a slice through it:>>> import py21cmfast as p21c
>>> coeval = p21c.run_coeval(
>>>     redshift=8.0,
>>>     user_params={'HII_DIM': 50, "USE_INTERPOLATION_TABLES": False}
>>> )
>>> p21c.plotting.coeval_sliceplot(coeval, kind='brightness_temp')The coeval object here has much more than just thebrightness_tempfield in it. You
can plot thedensityfield,velocityfield or a number of other fields.
To simulate a full lightcone:>>> lc = p21c.run_lightcone(
>>>     redshift=8.0,
>>>     max_redshift=15.0,
>>>     init_box = coeval.init_struct,
>>> )
>>> p21c.plotting.lightcone_sliceplot(lc)Here, we used the already-computed initial density field fromcoeval, which sets
the size and parameters of the run, but also means we don’t have to compute that
(relatively expensive step again). Explore the full range of functionality in theAPI Docs,
or read morein-depth tutorialsfor further guidance.CLIThe CLI can be used to generate boxes on-disk directly from a configuration file or
command-line parameters. You can run specific steps of the simulation independently,
or an entire simulation at once. For example, to run just the initial density field,
you can do:$ 21cmfast init --HII_DIM=100The (quite small) simulation box produced is automatically saved into the cache
(by default, at~/21cmFAST-cache).
You can list all the files in your cache (and the parameters used in each of the simulations)
with:$ 21cmfast queryTo run an entire coeval cube, use the following as an example:$ 21cmfast coeval 8.0 --out=output/coeval.h5 --HII_DIM=100In this case all the intermediate steps are cached in the standard cache directory, and
the finalCoevalbox is saved tooutput/coeval.h5. If no--outis specified,
the coeval box itself is not written, but don’t worry – all of its parts are cached, and
so it can be rebuilt extremely quickly. Every input parameter to any of theinput classes(there are a lot of parameters) can be specified at the end of the call with prefixes of--(likeHII_DIMhere). Alternatively, you can point to a config YAML file, eg.:$ 21cmfast lightcone 8.0 --max-z=15.0 --out=. --config=~/.21cmfast/runconfig_example.ymlThere is an example configuration fileherethat you
can build from. All input parameters aredocumented here.DocumentationFull documentation (with examples, installation instructions and full API reference)
found athttps://21cmfast.readthedocs.org.AcknowledgingIf you use21cmFAST v3+in your research please cite both of:Murray et al., (2020). 21cmFAST v3: A Python-integrated C code for generating 3D
realizations of the cosmic 21cm signal. Journal of Open Source Software, 5(54),
2582,https://doi.org/10.21105/joss.02582Andrei Mesinger, Steven Furlanetto and Renyue Cen, “21CMFAST: a fast, seminumerical
simulation of the high-redshift 21-cm signal”, Monthly Notices of the Royal
Astronomical Society, Volume 411, Issue 2, pp. 955-972 (2011),https://ui.adsabs.harvard.edu/link_gateway/2011MNRAS.411..955M/doi:10.1111/j.1365-2966.2010.17731.xIn addition, the following papers introduce various features into21cmFAST. If you use
these features, please cite the relevant papers.Mini-halos:Muñoz, J.B., Qin, Y., Mesinger, A., Murray, S., Greig, B., and Mason, C.,
“The Impact of the First Galaxies on Cosmic Dawn and Reionization”https://arxiv.org/abs/2110.13919(for DM-baryon relative velocities)Qin, Y., Mesinger, A., Park, J., Greig, B., and Muñoz, J. B.,
“A tale of two sites - I. Inferring the properties of minihalo-hosted galaxies from
current observations”, Monthly Notices of the Royal Astronomical Society, vol. 495,
no. 1, pp. 123–140, 2020.https://doi.org/10.1093/mnras/staa1131.
(for Lyman-Werner and first implementation)Mass-dependent ionizing efficiency:Park, J., Mesinger, A., Greig, B., and Gillet, N.,
“Inferring the astrophysics of reionization and cosmic dawn from galaxy luminosity
functions and the 21-cm signal”, Monthly Notices of the Royal Astronomical Society,
vol. 484, no. 1, pp. 933–949, 2019.https://doi.org/10.1093/mnras/stz032.Changelogdev-versionv3.3.1 [24 May 2023]FixedCompilation of C code for some compilers (#330)v3.3.0 [17 May 2023]InternalsRefactored setting up of inputs to high-level functions so that there is less code
repetition.FixedRunning withR_BUBBLE_MAXtoo large auto-fixes it to beBOX_LEN(#112)Bug in callingclear_cache.Inconsistency in the way that the very highest redshift of an evolution is handled
between low-level code (eg.spin_temperature()) and high-level code (eg.run_coeval()).AddedNewvalidate_all_inputsfunction that cross-references the four main input structs
and ensures all the parameters make sense together. Mostly for internal use.Ability to save/read directly from an open HDF5 File (#170)An implementation of cloud-in-cell to more accurately redistribute the perturbed mass
across all neighbouring cells instead of the previous nearest cell approachChanged PhotonConsEndCalibz from z = 5 -> z = 3.5 to handle later reionisation
scenarios in line with current observations (#305)Add in an initialisation check for the photon conservation to address some issues
arising for early EOR histories (#311)AddedNON_CUBIC_FACTORtoUserParamsto allow for non-cubic coeval boxes (#289)v3.2.1 [13 Sep 2022]ChangedIncluded log10_mturnovers(_mini) in lightcone class. Only useful when USE_MINI_HALOSv3.2.0 [11 Jul 2022]ChangedFloats are now represented to a specific number of significant digits in the hash of
an output object. This fixes problems with very close redshifts not being read from
cache (#80). Note that this means that very close astro/cosmo params will now be read
from cache. This could cause issues when creating large databases with many random
parameters. The behaviour can modified in the configuration by setting thecache_param_sigfigsandcache_redshift_sigfigsparameters (these are 6 and
4 by default, respectively).NOTE: updating to this version will cause your previous cached files to become
unusable. Remove them before updating.FixedAdded a missing C-based error to the known errors in Python.v3.1.5 [27 Apr 2022]v3.1.4 [10 Feb 2022]Fixederror in FFT normalization in FindHaloesdocs not compiling on RTD due to missingscipy.integratemock moduleUpdated matplotlib removed support for setting vmin/vmax and norm. Now passes vmin/vmax
to the norm() constructor.v3.1.3 [27 Oct 2021]FixedFAST_FCOLL_TABLESso it only affects MCGs and not ACGs. Added tests of this
flag for high and low z separately.v3.1.2 [14 Jul 2021]InternalsMINIMIZE_MEMORYflag significantly reduces memory without affecting performance much,
by changing the way some arrays are allocated and accessed in C. (#224)ChangeUpdatedUSE_INTERPOLATION_TABLESto be default True. This makes much more sense as
a default value. Until v4, a warning will be raised if it is not set explicitly.v3.1.1 [13 Jun 2021]FixedBug in deployment to PyPI.v3.1.0 [13 Jun 2021]AddedAbility to access all evolutionary Coeval components, either from the end Coeval
class, or the Lightcone.Ability to gather all evolutionary antecedents from a Coeval/Lightcone into the one
file.FAST_FCOLL_TABLESinUserParamswhich improves speeds quite significantly for
~<10% accuracy decrease.Fast and low-memory generation of relative-velocity (vcb) initial conditions. Eliminated hi-res vcb boxes, as they are never needed.Also output the mean free path (i.e. MFP_box in IonizedBox).Added the effect of DM-baryon relative velocities on PopIII-forming minihaloes. This now provides the correct background evolution jointly with LW feedback. It gives rise to velocity-induced acoustic oscillations (VAOs) from the relative-velocity fluctuations. We also follow a more flexible parametrization for LW feedback in minihaloes, following new simulation results, and add a new index ALPHA_STAR_MINI for minihaloes, now independent of regular ACGs.Newhookskeyword to high-level functions, that are run on the completion of each computational step, and can
be used to more generically write parts of the data to file.Ability to pass a function towrite=to write more specific aspects of the data (internally, this will be put into thehooksdictionary).run_lightconeandrun_coevaluse significantly less memory by offloading initial conditions and perturb_field instances to disk if possible.FixedBug in 2LPT whenUSE_RELATIVE_VELOCITIES=True[Issue #191, PR #192]Error raised when redshifts are not in ascending order [Issue #176, PR #177]Errors whenUSE_FFTW_WISDOMis used on some systems [Issue #174, PR #199]Bug in ComputeIonizedBox causing negative recombination rate and ring structure inGamma12_box[Issue #194, PR #210]Error in determining the wisdom file name [Issue #209, PR#210]Bug in which cached C-based memory would be read in and free’d twice.InternalsAddeddft.c, which makes doing all the cubic FFTs a lot easier and more consistent. [PR #199]More generic way of keeping track of arrays to be passed between C and Python, and their shape in Python, using_get_box_structures.
This also means that the various boxes can be queried before they are initialized and computed.More stringent integration tests that test each array, not just the final brightness temperature.Ability to plot the integration test data to more easily identify where things have gone wrong (use--plotsin thepytestinvocation).Nicer CLI interface forproduce_integration_test_data.py. New options tocleanthetest_data/directory,
and also test data is saved by user-defined key rather than massive string of variables.Nicer debug statements before calls to C, for easily comparing between versions.Much nicer methods of keeping track of array state (in memory, on disk, c-controlled, etc.)Ability to free C-based pointers in a more granular way.v3.0.3Addedcoeval_callbackandcoeval_callback_redshiftsflags to therun_lightcone.
Gives the ability to run arbitrary code onCoevalboxes.JOSS paper!get_fieldsclassmethod on all output classes, so that one can easily figure out
what fields are computed (and available) for that class.FixedOnly raise error on non-availableexternal_table_pathwhen actually going to use it.v3.0.2FixedAdded prototype functions to enable compilation for some standard compilers on MacOS.v3.0.1Modifications to the internal code structure of 21cmFASTAddedRefactor FFTW wisdom creation to be a python callable functionv3.0.0Complete overhaul of 21cmFAST, including a robust python-wrapper and interface,
caching mechanisms, and public repository with continuous integration. Changes
and equations for minihalo features in this version are found inhttps://arxiv.org/abs/2003.04442All functionality of the original 21cmFAST v2 C-code has been implemented in this
version, includingUSE_HALO_FIELDand performing full integration instead of using
the interpolation tables (which are faster).AddedUpdated the radiation source model: (i) all radiation fields including X-rays, UV
ionizing, Lyman Werner and Lyman alpha are considered from two seperated population
namely atomic-cooling (ACGs) and minihalo-hosted molecular-cooling galaxies (MCGs);
(ii) the turn-over masses of ACGs and MCGs are estimated with cooling efficiency and
feedback from reionization and lyman werner suppression (Qin et al. 2020). This can
be switched on using newflag_optionsUSE_MINI_HALOS.Updated kinetic temperature of the IGM with fully ionized cells following equation 6
of McQuinn (2015) and partially ionized cells having the volume-weightied temperature
between the ionized (volume: 1-xHI; temperature T_RE ) and neutral components (volume:
xHI; temperature: temperature of HI). This is stored in IonizedBox as
temp_kinetic_all_gas. Note that Tk in TsBox remains to be the kinetic temperature of HI.Tests: many unit tests, and also some regression tests.CLI: run 21cmFAST boxes from the command line, query the cache database, and produce
plots for standard comparison runs.Documentation: Jupyter notebook demos and tutorials, FAQs, installation instructions.Plotting routines: a number of general plotting routines designed to plot coeval
and lightcone slices.New power spectrum option (POWER_SPECTRUM=5) that uses a CLASS-based transfer
function. WARNING: If POWER_SPECTRUM==5 the cosmo parameters cannot be altered, they
are set to the Planck2018 best-fit values for now (until CLASS is added):
(omegab=0.02237, omegac= 0.120, hubble=0.6736 (the rest are irrelevant for the
transfer functions, but in case:  A_s=2.100e-9, n_s=0.9649, z_reio = 11.357)Newuser_paramsoptionUSE_RELATIVE_VELOCITIES, which produces initial relative
velocity cubes (option implemented, but not the actual computation yet).Configuration management.global params now has a context manager for changing parameters temporarily.Vastly improved error handling: exceptions can be caught in C code and propagated to
Python to inform the user of what’s going wrong.Ability to write high-level data (CoevalandLightconeobjects) directly to
file in a simple portable format.ChangedPOWER_SPECTRUMoption moved fromglobal_paramstouser_params.Default cosmology updated to Planck18.v2.0.0All changes and equations for this version are found inhttps://arxiv.org/abs/1809.08995.ChangedUpdated the ionizing source model: (i) the star formation rates and ionizing escape
fraction are scaled with the masses of dark matter halos and (ii) the abundance of
active star forming galaxies is exponentially suppressed below the turn-over halo
mass, M_{turn}, according to a duty cycle of exp(−M_{turn}/M_{h}), where M_{h} is a
halo mass.Removed the mean free path parameter, R_{mfp}. Instead, directly computes
inhomogeneous, sub-grid recombinations in the intergalactic medium following the
approach of Sobacchi & Mesinger (2014)v1.2.0AddedSupport for a halo mass dependent ionizing efficiency: zeta = zeta_0 (M/Mmin)^alpha,
where zeta_0 corresponds to  HII_EFF_FACTOR, Mmin –> ION_M_MIN,
alpha –> EFF_FACTOR_PL_INDEX in ANAL_PARAMS.Hv1.12.0AddedCode ‘redshift_interpolate_boxes.c’ to interpolate between comoving cubes,
creating comoving light cone boxes.Enabled openMP threading  for SMP machines.  You can specify the number of threads
(for best performace, do not exceed the number of processors) in INIT_PARAMS.H. You do
not need to have an SMP machine to run the code. NOTE: YOU SHOULD RE-INSTALL FFTW to
use openMP (see INSTALL file)Included a threaded driver file ‘drive_zscroll_reion_param.c’ set-up to perform
astrophysical parameter studies of reionizationIncluded explicit support for WDM cosmologies; see COSMOLOGY.H.  The prescription is
similar to that discussed in Barkana+2001; Mesinger+2005, madifying the (i) transfer
function (according to the Bode+2001 formula; and (ii) including the effective
pressure term of WDM using a Jeans mass analogy.  (ii) is approximated with a sharp
cuttoff in the EPS barrier, using 60* M_J found in Barkana+2001 (the 60 is an
adjustment factor found by fitting to the WDM collapsed fraction).A Gaussian filtering step of the PT fields to perturb_field.c, in addition to the
implicit boxcar smoothing.  This avoids having”empty” density cells, i.e. delta=-1,
with some small loss in resolution.  Although for most uses delta=-1 is ok, some Lya
forest statistics do not like it.Added treatment of the risidual electron fraction from X-ray heating when computing
the ionization field.  Relatedly, modified Ts.c to output all intermediate evolution
boxes, Tk and x_e.Added a missing factor of Omega_b in Ts.c corresponding to eq. 18 in MFC11.  Users who
used a previous version should note that their results just effecively correspond to a
higher effective X-ray efficiency, scaled by 1/Omega_baryon.Normalization optimization to Ts.c, increasing performace on arge resolution boxesFixedGSL interpolation error in kappa_elec_pH for GSL versions > 1.15Typo in macro definition, which impacted the Lya background calculation in v1.11 (not applicable to earlier releases)Outdated filename sytax when calling gen_size_distr in drive_xHIscrollRedshift scrolling so that drive_logZscroll_Ts.c and Ts.c are in sync.ChangedOutput format to avoid FFT padding for all boxesFilename conventions to be more explicit.Small changes to organization and structurev1.1.0AddedWrapper functions mod_fwrite() and mod_fread() in Cosmo_c_progs/misc.c, which
should fix problems with the library fwrite() and fread() for large files (>4GB) on
certain operating systems.Included print_power_spectrum_ICs.c program which reads in high resolution initial
conditions and prints out an ASCII file with the associated power spectrum.Parameter in Ts.c for the maximum allowed kinetic temperature, which increases
stability of the code when the redshift step size and the X-ray efficiencies are large.FixedOversight adding support for a Gaussian filter for the lower resolution field.