Package: wtlike
Making FermiLAT gamma-ray light curves with wtlikeQuickly create a light curve for any 4FGL source, on any time scale, with optional Bayesian Block analysisgithub links:this document,the repositoryIntroductionwtlike(Perhaps pronounced "DUB-Tee-like"), is a library optimized for interactive exploration in aJupyter notebookwith access to all Fermi-LAT data, and to analyze the time dependence of any source in the
4FGL catalog on any time scale, with the option of performing aBayesian Blockpartition to select optimal time intervals. The source can be identified by the 4FGL name, or any equivalent common name.Here is a minimal demo:fromwtlikeimport*config=Config()valid=config.validifvalid:withTimer()ast:wtl=WtLike('3C 273',week_range=(9,3*52+9))print(t)SourceData:  4FGL J1229.0+0202: Restoring from cache with key "P88Y3157_weeks_9-165"
SourceData: Source 3C 273 with:
	 data:        33,210 photons from 2008-08-04 to 2011-08-03
	 exposure:   713,320 intervals,  average effective area 2783 cm^2 for 21.3 Ms
	 rates:  source 2.02e-07/s, background 3.58e-07/s,
CellData.rebin: Bin photon data into 156 1-week bins from 54683.0 to 55775.0
LightCurve: select 156 cells for fitting with e>35 & n>2
elapsed time: 0.7s (0.0 min)This created aWtLikeobject, loading the first 3 years of data, by specifying weeks from #9, the first data-taking week.
The reason to specify only the first three years here is to avoid the 10 min or so that extracting the furl 13+ years would take for this demo. If that had already been done, then using an includedcache system, it only has to be done once.Now ask it to make a plot:ifconfig.valid:wtl.plot(UTC=1);This assumes that the name for the source, in this case the historically famous firstquasarto be discovered, can be associated with a 4FGL catalog source. The plot shows, as a function of UTC (or MJD if desired) time, weekly measurements of deviations of the flux relative to the average of the 12-year interval used to define the 4FGL-DR3 catalog.OverviewThis package has code that was developed with thenbdevcode/tests/documentation environment from thegithub package lat-timingto generate light curves of Fermi-LAT sources.It is based on apaperby Matthew Kerr, which derives theweighted likelihood formalismused here, specifically with
theBayesian Blockto detect and characterize variability of a gamma-ray source.There are several innovative design features that significantly improve the speed and portability.Condensed photon and spacecraft data.Weight tablesA cache to improve interactivityLikelihood functions fit to a modified Poisson functionUnbinned likelihoodA simple user interfaceHow it worksHistorically, gamma-ray source measurements have used two methods:For a fixed time interval, where the energy and
position are used, with a model including all potential sources and a model of the detector response to
define the likelihood. This has been the only way to study weak sources. A light curve must apply this to
each time interval.Or, for very bright flares, for example GRBs or AGN flares, one can simply count the number of photons within a
circular region, that is, aperture photometry.Matthew Kerrintroduceda third method, basically counting photons but using information from a static
likelihood analysis to assign a "weight" to each photon, the probability for being from the source in question, then optimizing this likelihood. This assumes that the only thing changing is the flux of
the source.  He calls it "retrospective", since the analysis for the full time is then applied back to the individual photons. Another way of looking at it is to make the assumption that the time dependence of a source's photon flux factorizes according to the energy and spatitial dependences.Likelihood evaluationIn a significant modification from Kerr's implemetation as described in that paper, we evaluate weights for each photon by a table lookup.Assumptions:Source photons are completely contained in the dataset boundaries (the "ROI").The instrument response is constant with time.The background pattern is constant. (Clearly violated if a neighboring source varies!)The likelihood evaluation is implemented in the moduleloglike.Photon DataFermidata are retrieved from the Fermi-LAT weekly data files extracted from theGSFC FTP server,
with subfolders for the photon data,photonand spacecraft data,spacecraft. It isdescribed here. The files are organized into individual weeks, starting UTC midnight each Thursday. Files for the most recent week are updated daily.We convert each pair of photon and spacecraft files to two DataFrame tables with the minimum information needed to evaluate the likelihood, as compressed as possible. Particularly, the energy and position are binned. Details can be seen in the moduledata_man.The entire data set (SOURCE class, energy>100 MeV) in this format occupies ~2 GB.Select Data for a sourceAll further processing uses a subset of the photons within a cone, currently $4^\circ$, about the selected source, and
evaluates the exposure during 30-s intervals for this direction.  In the classSourceData, implemented in the modulesource_dataweExtract photonsEvaluate exposure, using the effective area tables and a spectral assumption.Determine the weight for each photon, using the table for the source. See the moduleweightsfor details.The result is a photon DataFrame, containing for each photon, the time $t$ in MJD units, $w$.This class is a superclass of the user interface classWtLikeintroduced above.Partition into cellsA "cell", the terminology used by Kerr, the set of photons in a specific time interval. The classCellDatain the modulecell_data, a subclass of SourceData manages this.This class is instantiated with a tuple to define the binning in time. Denoted by(a, b, c), the elements are:a--start timeb--stop timec-- bin size in days, but 0 means orbit-based, intervals are contiguous eposure.For the start and stop, values > 50000 are interpreted as MJD. Otherwise they are relative to start if positive
or stop if negative for the full dataset, both rounded to a full day. Zero means actual start foraand stop forb.
The default binning is simply(0, 0, 7)for weekly bins with the full dataset. Hourly for the most recent week would be(-7, 0, 1/24).A DataFrame table of the cells is created as a data membercells, with contentt-- MJD time at cell centertw-- cell width in dayse-- cell exposure, for referencen-- the number of photonsw-- a list ofnweightsS-- expected number source photons, the nominal source rate times the total exposure.B-- expected number of background photons (unused)ViewsCellDataimplements a methodview, which accepts a binning tuple as an argument, returns acopyof the current object, which can be a subclass, assigning to it the binning. Thus the view has all the attributes of its parent, but
with a different set of cells.So the following creates a new WtLike object that we generated above, rebins a copy with 25-day bins in the first 100 days, generates a list of the cells, then removes it since it wasn't assigned a reference variable.ifconfig.valid:print(wtl.view(0,100,25).cells)CellData.rebin: Bin photon data into 4 25-day bins from 54683.0 to 54783.0
LightCurve: select 4 cells for fitting with e>125 & n>2
         t    tw            e     n  \
0  54695.5  25.0  1488.182373   553   
1  54720.5  25.0  1868.273926  1438   
2  54745.5  25.0  1450.828979  1183   
3  54770.5  25.0  1929.570801  1175   

                                                   w           S           B  
0  [4.8901367e-01, 0.8066406, 0.11303711, 0.19165...  300.871887  532.046936  
1  [0.4345703, 0.6064453, 0.0690918, 0.056274414,...  377.716553  667.935242  
2  [0.33911133, 0.3100586, 0.7089844, 0.06994629,...  293.320007  518.692566  
3  [0.09112549, 0.58251953, 0.07537842, 0.3457031...  390.109192  689.849792Evaluate Likelihoods and make light curve plotsThe classLightCurve, implemented in the modulelightcurveis a subclass ofSourceData.
An instance invokes its superclass to generate the set of cells, then evaluates the likelihoods.Poisson-like LikelihoodWe fit the likelihood for each cell, using only a few evaluations, to a3-parameter Poisson-like formula. Two advantages of this are:efficiency -- for large numbers of photons, this is much fasterconvenience -- thePoissonobject implements functions that return the TS, 95% CL, maximum, and uncertainties, using theincomplete gamma function.This results in a DataFramefitscontainingt, tw, n, from the cell, andfit, thePoissonobjectLight curve plotsThe functionLikelihood.plotactually invokesflux_plotApply Bayesian BlocksThe classWtLike,
implememted in the modulemain, adds an implementation of the Bayesian Block (BB) algorithm, from the modulebayesianusing likelihood instead of counts.
There we have two subclasses ofastropy.stats.bayesian_blocks,CountFitnessand the defaultLikelihoodFitness.This code creates partitions between boundaries of a set of cells. Usage is via a special view,bb_view`ifconfig.valid:bb=wtl.bb_view()bb.plot(UTC=1);Bayesian Blocks: partitioning 156 cells using LikelihoodFitness with penalty 5%
	found 43 / 156 blocks.
LightCurve: Loaded 43 / 43 cells for fittingAs you see, this made 94 blocks from the 656 weeks, fit each, and overplotted in on the weekly light curve.Code details ...# collapse-hide@ipynb_docdefpgram(wtl):"""### Kerr PeriodogramOne can analyze the frequency spectrum by generating a[Kerr periodogram](https://arxiv.org/pdf/1910.00140.pdf).A `WtLike` object has a `periodogram` function that returns a `TimeSeries` object.{out}{fig}"""withcapture_hide('Setup output')asout:p=wtl.periodogram()p.power_plot(pmax=100);fig=plt.gcf()returnlocals()ifconfig.valid:pgram(wtl)Kerr PeriodogramOne can analyze the frequency spectrum by generating aKerr periodogram.
AWtLikeobject has aperiodogramfunction that returns aTimeSeriesobject.Setup outputCellData.rebin: Bin photon data into 26256 1-hour bins from 54683.0 to 55777.0Code details ...# collapse-hide@ipynb_docdefaitdemo(week,nside,name='3C 454.3',size=40):"""## Aitoff and ZEA projections of a HEALPix mapChoose week {week}, with a large 3C 454.3 flare.The all-sky Aitoff image of the photon count map, counts per nside={nside} pixel.{fig1}And zoom to small region, with a ZEA projection, a ${size}^\circ$ square.{fig2}"""fromwtlike.skymapsimportSquareWCSfromwtlike.sourcesimportfindsourcefromutilitiesimporthealpixashpxcntmap=DataView(FermiInterval(7)[week-1],nside=nside).count_map()kw=dict(vmin=1,vmax=100,)fig1=hpx.HPmap(cntmap,f'week{week}',unit='counts/pixel',sigma=0.5).ait_plot(**kw);wcs=SquareWCS(findsource(name,gal=True),size=size,)fig2=wcs.plot(cntmap,title=name,cb_kw=dict(shrink=0.8),**kw)returnlocals()week=79nside=64ifvalid:aitdemo(week,nside,fignum=1)Aitoff and ZEA projections of a HEALPix mapChoose week 79, with a large 3C 454.3 flare.The all-sky Aitoff image of the photon count map, counts per nside=64 pixel.And zoom to small region, with a ZEA projection, a $40^\circ$ square.SimulationFinally, a simulation option is available. See thetutorialfor an exampleInstallationNote that this is in beta mode.
It depends on the packages:  matplotlib, pandas, scipy, astropy, scipy and healpy, which pip will install if needed.To install from pyPI:pip install wtlikeor to upgrade:pip install -U wtlikeInput dataThere are three data sources whichwtlikeneeds to function:The photon/spacecraft data, one file per weekA table of weights for each sourceAn effective area table, a grid in energy/angle for "front" and "back" eventsThese must be found in a folder, which by default is~/wtlike-data. In that folder there must be (perhaps links to) three folders nameddata_files,weight_files,aeff_files, and a zip file,weight_file.zip.  A current copy can be found in myGoogle Drive.Google colab setupAn easy way to test, or even use, this system is withGoggle's colab. To use it, you need a Google account.
All the data files are in my shared folderwtlike-data.
It includes asetup notebookwhich demonstrates setting up a free Google-hosted VM to run wtlike code and makes a few demo plots.