Package: correlcalc
A Python package to calculate 2-point correlation function(2pCF) from
galaxy redshift surveys for any generic model of Cosmology or geometry.Summarycorrelcalc calculates two-point correlation function (2pCF) of
galaxies/quasars using redshift surveys. It can be used for any assumed
geometry or Cosmology model. Using BallTree algorithms to reduce the
computational effort for large datasets, it is faster than brute-force
methods. It takes redshift (z), Right Ascension (RA) and Declination
(DEC) data of galaxies and random catalogs given by redshift survey as
inputs. If random catalog is not provided, it generates one of desired
size based on the input redshift distribution and a mangle polygon file
in .ply format describing the survey geometry. It also calculates
anisotropic 2pCF. Optionally it makes healpix maps of the survey
providing visualization.InstallationTo install this package type “pip install correlcalc” in your
terminal. If this method doesn’t workTo install the package Download this git repositry and in terminal enter
the folder that contains setup.py and type “pip install .” or
“python setup.py install”If you do not have root permission, you can install by adding
“--user” at the end of above commandsIf you have an older version installed already you can upgrade by
“pip install correlcalc--upgrade” commandA note on Dependencies:All the required dependencies such as sklearn, cython, scipy, numpy etc.
should get automatically installed if installed through pip. In case, if
some of the dependencies do not automatically get installed. The list of
dependencies can be seen in the setup.py file to manually install them.
In case of any problems feel free to raise an issue. “healpix_util”
package fromhttp://github.com/esheldon/healpix_utilis not available
on pip. So it needs to be manually installed following the commands to
install from git repositry in the above sectionTheoryThe algorithm and formulae used are presented in the paper entitledA
`Generic’ Recipe for Quick Computation of Two-point Correlation
functionIt is available on arXiv:1710.01723 at https://arxiv.org/abs/1710.01723.Please cite the same if you use this package or the ‘recipe’ presented
hereinUsageCalculation of 2pCFUsage of the package is given in jupyter notebook “Using correlcalc
example.nb” and inmain.pyAll the methods in correlcalc can be imported using the following
commandfrom correlcalc import *We first need to define bins (in\(c/H_0\)units) to calculate 2pCF.
For e.g. to calculate correlation between 0-180Mpc in steps of 6Mpc, we
saybins=np.arange(0.002,0.06,0.002)To calculate 2pCF using input data file (both ascii and fits files are
supported), usetpcfmethod as followscorrel,poserr=tpcf('/path/to/datfile.dat',bins,randfile='/path/to/randomfile.dat',weights='eq')If random file is not available or not provided, we can generate random
catalog by providing the mangle mask file in.plyformat along with
specifying the size of the catalog in multiples of size of data catalog
(default 2x size). To do thiscorrel,poserr=tpcf('/path/to/datfile.dat',bins,maskfile='/path/to/maskfile.ply',weights=True, randfact=3)This returnscorrelandposerrnumpyarrays corresponding to
Two-point correlation and Poisson errorKeyword ArgumentsThe following keyword arguments can be included as neededData file (Mandatory)Data file of galaxy/quasar redshift survey must be passed as the first
argument to bothtpcfandatpcfmethods.Supported filetypes: ascii text files with columns, csv files or
fits files are all supported. Most files provided by SDSS Value added
catalogs should be directly usable.To contain: Any type of file provided must at least have columns
namedZ(redshift),RA(Right Ascension),DEC(Declination).
These column names can be in any case.If one intends to useweights=Trueoption (must to obtain accurate
results) the data file must also contain radial weights with column
titleradial_weightorWEIGHT_SYSTOTbins (Mandatory)A numpy array with ascending values in\(c/H_0\)units must be
provided as the second argument to bothtpcfandatpcfmethods.
In case ofatpcfit automatically creates 2D bins asbins2d=(bins,bins)from provided 1Dbinsrandfile=Path to random file (semi-Optional)If not provided,maskfile=argument must be given.plyfile.Supported filetypes: ascii text files with columns, csv files or
fits files are all supported. Most files provided by SDSS Value added
catalogs should be directly usable.To contain: Any type of file provided must at least have columns
namedZ(redshift),RA(Right Ascension),DEC(Declination).
These column names can be in any case.If one intends to useweights=Trueoption (must to obtain accurate
results) the data file must also contain radial weights with column
titleradial_weightorWEIGHT_SYSTOTBeta Testing:Beta support for other column titles for weights is
added.Also added is calculation of weights from n(z) during random catalog
generation.mask=Path to mangle polygon file (semi-Optional)If not provided,randfile=argument must be provided.Supported filetypes:.plyfile containing Mangle polygons
describing survey geometry in the standard format. Most files provided
by SDSS Value added catalogs should be directly usable.randfact=(Optional)Size of the random catalog in integer multiples of size of data catalog
if random catalog file is not provided. Default value is2weights=(Optional)It is highly recommended to use weights argument by providingweights=Trueorweights='eq'to obtain accurate two-point
correlation calculations. This picks up radial weights in the prescribed
format (with column titleradial_weightorWEIGHT_SYSTOT)
from the data and random files provided.weights=eq’sets equal weights and hence adds *+1* - This implementation is parallelized and is faster thanweights=False`
implementation on most machinesIfweights=False, by default+1will be added for each
galaxy/random pair found within the bin instead of adding total weight.
For more details on weights and references, seehttp://www.sdss3.org/dr9/tutorials/lss_galaxy.phpgeometry='flat'(Optional)Available options:'flat'(default) - for flat geometry of the Universe'open'- for Open Universe models like Milne'close'- for Closed UniverseCustomizationFormulae for calculation of distances between two points (Z1, RA1, DEC1)
and (Z2, RA2, DEC2) is taken fromT. Matsubara, Correlation function in
deep redshift space as a cosmological probe, The Astrophysical Journal
615 (2) (2004) 573. Using the formulae in this paper, distances squares
(to reduce additional computational time distance squares are calculated
to avoid using expensivesqrtfunction every time) are computed in
themetrics.pyxfile for all the above mentioned geometries.Cythonis chosen for implementation to obtain faster results in
buildingBallTrees calculatingcdistand to reducequerytime.One can customize metric definitions as per one’s need by editing this
file. AlsoK(curvature parameter) in the formulae given in this
reference need to be manually changed in themetrics.pyxfor closed
and open cases as per the model. After changing this compile it usingpython metricsetup.py build_ext--inplacecosmology='lcdm'(Optional)Used to calculate co-moving distances from redshifts.Available options:'lcdm'(default)- for Lambda CDM model'lc'- for\(R_h=ct\)and linear coasting modelsTo add:wcdmand other popular cosmology models soonestimator=(Optional)Available options:'dp'- Davis - Peebles estimator (default - fastest)'ls'- Landy - Szalay estimator'ph'- Peebles- Hauser estimator'hew'- Hewitt estimator'h'- Hamilton estimatorFor more details on estimator formulae seehttps://arxiv.org/pdf/1211.6211.pdfCalculation of Anisotropic (3D) 2pCFUsage of the package is given in jupyter notebook “Using correlcalc
example-anisotropic.nb” and inmain.pyAll the methods in correlcalc can be imported using the following
commandfrom correlcalc import *We first need to define bins (in\(c/H_0\)units) to calculate 2pCF.
For e.g. to calculate correlation between 0-180Mpc in steps of 6Mpc, we
saybins=np.arange(0.002,0.06,0.002)To calculate anisotropic 2pCF using input data file (both ascii and fits
files are supported), useatpcfmethod as followscorrel3d,poserr=atpcf('/path/to/datfile.dat',binspar,binsper,randfile='/path/to/randomfile.dat',vtype='sigpi',weights=True)If random file is not available or not provided, we can generate random
catalog by providing the mangle mask file in.plyformat along with
specifying the size of the catalog in multiples of size of data catalog
(default 2x size). To do thiscorrel3d,poserr=atpcf('/path/to/datfile.dat',binspar, binsper,maskfile='/path/to/maskfile.ply',vtype='smu',weights='eq',randfact=3)This returnscorrel3dandposerrnumpyarrays corresponding
to anisotropic Two-point correlation and Poisson errorKeyword ArgumentsThe following keyword arguments can be included as neededData file (Mandatory)Data file of galaxy/quasar redshift survey must be passed as the first
argument to bothtpcfandatpcfmethods.Supported filetypes: ascii text files with columns, csv files or
fits files are all supported. Most files provided by SDSS Value added
catalogs should be directly usable.To contain: Any type of file provided must at least have columns
namedZ(redshift),RA(Right Ascension),DEC(Declination).
These column names can be in any case.If one intends to useweights=Trueoption (must to obtain accurate
results) the data file must also contain radial weights with column
titleradial_weightorWEIGHT_SYSTOTbinspar (Mandatory)A numpy array with ascending values in\(c/H_0\)units (for
distances) or\(\delta z\)as per choice of'vtype'must be
provided as the second argument toatpcfmethod.binsper (Mandatory)A numpy array with ascending values in\(c/H_0\)units (for
distances),\(z\delta \theta\)or\(\mu = \cos \alpha\)must be
provided as the third argument toatpcfmethod.randfile=Path to random file (semi-Optional)If not provided,maskfile=argument must be given.plyfile.Supported filetypes: ascii text files with columns, csv files or
fits files are all supported. Most files provided by SDSS Value added
catalogs should be directly usable.To contain: Any type of file provided must at least have columns
namedZ(redshift),RA(Right Ascension),DEC(Declination).
These column names can be in any case.If one intends to useweights=Trueoption the data file must also
contain radial weights with column titleradial_weightorWEIGHT_SYSTOTBeta Testing:Beta support for other column titles for weights is
added.Also added is calculation of weights from n(z) during random catalog
generation.mask=Path to mangle polygon file (semi-Optional)If not provided,randfile=argument must be provided.Supported filetypes:.plyfile containing Mangle polygons
describing survey geometry in the standard format. Most files provided
by SDSS Value added catalogs should be directly usable.randfact=(Optional)Size of the random catalog in integer multiples of size of data catalog
if random catalog file is not provided. Default value is2weights=(Optional)It is highly recommended to use weights argument by providingweights=Trueorweights='eq'to obtain accurate two-point
correlation calculations. This picks up radial weights in the prescribed
format (with column titleradial_weightorWEIGHT_SYSTOT)
from the data and random files provided.weights=eq’sets equal weights and hence adds *+1* - This implementation is parallelized and is faster thanweights=False`
implementation on most machinesIfweights=False, by default+1will be added for each
galaxy/random pair found within the bin instead of adding total weight.
For more details on weights and references, seehttp://www.sdss3.org/dr9/tutorials/lss_galaxy.phpMetrics in parallel and perpendicular directionsCalculates anisotropic 2pCF for the following cases.vtype=Valuation methodAvailable options:'smu'(default)- Calculates 2pCF in s - mu'sigpi'- Calculates 2pCF using parallel and perpendicular distances'ap'calculates 2pCF for small\(\Delta \theta\)and\(z \Delta\theta\). But results can be converted to any cosmology
model of choice (ref:https://arxiv.org/pdf/1312.0003.pdf)CustomizationFormulae for calculation of distances in parallel and perpendicular
directions is taken fromhttps://arxiv.org/pdf/1312.0003.pdf. Using the
formulae in this paper,\(\Delta z\)and\(z \Delta \theta\)are
computed in themetrics.pyxfile for the above mentioned.Cythonis chosen for implementation to obtain faster results in buildingBallTrees calculatingcdistand to reducequerytime.One can customize metric definitions as per one’s need by editing themetrics.pyxfile. After changing this compile it usingpython metricsetup.py build_ext--inplaceTo add:Direct calculation of distances in LOS and perpendicular to the LOS to
be added to support standard model Cosmology and other popular models.
For now, one needs to manually convert the angular bins to physical
distances to get the approximate resultscosmology='lcdm'(Optional)Used to calculate co-moving distances from redshifts.Available options:'lcdm'(default)- for Lambda CDM model'lc'- for\(R_h=ct\)and linear coasting modelsTo add:wcdmand other popular cosmology models soongeometry='flat'(Optional)Used to calculate co-moving distances between a pair of objectsAvailable options:'flat'(default)- for Lambda CDM model'open''close'estimator=(Optional)Available options:'dp'- Davis - Peebles estimator (default - fastest)'ls'- Landy - Szalay estimator'ph'- Peebles- Hauser estimator'hew'- Hewitt estimator'h'- Hamilton estimatorFor more details on estimator formulae seehttps://arxiv.org/pdf/1211.6211.pdf