Package: pidcalib2
PIDCalib2A set of software tools for estimating LHCb PID efficiencies.The package includes several user-callable modules:make_eff_histscreates histograms that can be used to estimate the PID efficiency of a user's sampleref_calibcalculates the LHCb PID efficiency of a user reference samplemerge_treesmerges two ROOT files with compatibleTTreesplot_calib_distributionsallows you to plot distributions of variables in the calibration datasetspklhisto2rootconvertsPickledboost-histograms to ROOT histogramsThe term "reference dataset/sample" refers to the user's dataset to which they want to assign PID efficiencies. The "calibration datasets/samples" are the special, centrally managed samples used internally by PIDCalib for PID efficiency estimation. The--sampleargument always concerns these calibration samples.Slides with additional information, example output, and plots are available onIndico.SetupWhen working on a computer where the LHCb software stack is available (LXPLUS, university cluster, etc.), one can setup PIDCalib2 by runninglb-condapidcalibbashAfter this, the following commands will be availablepidcalib2.make_eff_hists
pidcalib2.ref_calib
pidcalib2.merge_trees
pidcalib2.plot_calib_distributions
pidcalib2.pklhisto2rootYou can skip the bash invocation and join the setup and run phases into a single commandlb-condapidcalibpidcalib2.make_eff_histsTo runmake_eff_hists, you will need access to CERN EOS. You don't need to do anything special on LXPLUS. On other machines, you will usually need to obtain a Kerberos ticket by runningkinit[username]@CERN.CHInstalling from PyPIThe PIDCalib2 package is available onPyPI. It can be installed on any computer viapipsimply by running (preferably in a virtual environment; seevenv)pipinstallpidcalib2Note that this will install thexrootdPython bindings. One also has to install XRootD itself for the bindings to work. Seethis pagefor XRootD releases and instructions.make_eff_histsThis module creates histograms that can be used to estimate the PID efficiency of a user's sample.Reading all the relevant calibration files can take a long time. When running a configuration for the first time, we recommend using the--max-files 1option. This will limit PIDCalib2 to reading just a single calibration file. Such a test will reveal any problems with, e.g., missing variables quickly. Keep in mind that you might get a warning about empty bins in the total histogram as you are reading a small subset of the calibration data. For the purposes of a quick test, this warning can be safely ignored.OptionsTo get a usage message listing all the options, their descriptions, and default values, typepidcalib2.make_eff_hists --helpThe calibration files to be processed are determined by thesample,magnet, andparticleoptions. All the valid combinations can be listed by runningpidcalib2.make_eff_hists--listconfigsAliases for standard variables are defined to simplify the commands. We recommend users use the aliases when specifying variables. When you use a name that isn't an alias, a warning message like the following will show up in the log.'probe_PIDK' is not a known PID variable alias, using raw variableAll aliases can be listed by runningpidcalib2.make_eff_hists--listaliasesNote that there are many more variables than there are aliases. If you want to find a variable for which no alias exists, you can check one of the calibration files yourself. The paths to the calibration files are printed when the--verboseoption is specified. Alternatively, you can simply guess the name - if it doesn't exist, PIDCalib2 will let you know and might provide a list of similar names that do exist.A file with alternative binnings can be specified using--binning-file. The file must contain valid JSON specifying bin edges. For example, two-bin binnings for particlePi, variablesPandPTcan be defined as{"Pi":{"P":[10000,15000,30000],"PT":[6000,10000,20000]}}An arbitrary number of binnings can be defined in a single file.Complex cut expressions can be created by chaining simpler expressions using&(logical and) and|(logical or). One can also use standard mathematical symbols, like*,/,+,-,(,). Whitespace does not matter.ExamplesCreate a single 3D efficiency histogram for a single PID cutpidcalib2.make_eff_hists--sampleTurbo18--magnetup--particlePi--pid-cut"DLLK > 4"--bin-varP--bin-varETA--bin-varnSPDhits--output-dirpidcalib_outputCreate multiple histograms in one run (most of the time is spent reading
in data, so specifying multiple cuts is much faster than running
make_eff_hists sequentially)pidcalib2.make_eff_hists--sampleTurbo16--magnetup--particlePi--pid-cut"DLLK > 0"--pid-cut"DLLK > 4"--pid-cut"DLLK > 6"--bin-varP--bin-varETA--bin-varnSPDhits--output-dirpidcalib_outputCreate a single efficiency histogram for complex cuts using only negatively charged trackspidcalib2.make_eff_hists--sampleTurbo18--magnetup--particlePi--pid-cut"MC15TuneV1_ProbNNp*(1-MC15TuneV1_ProbNNpi)*(1-MC15TuneV1_ProbNNk) < 0.5 & DLLK < 3"--cut"IsMuon==0 & Brunel_PT>250 & trackcharge==-1"--bin-varP--bin-varETA--bin-varnSPDhits--output-dirpidcalib_outputCaveatsNot all datasets have all the variables, and in some cases, the same variable is named differently (e.g.,probe_Brunel_IPCHI2is namedprobe_Brunel_MINIPCHI2in certain electron samples). The aliases correspond to the most common names, but you might need to check the calibration files if PIDCalib2 can't find the variable you need.ref_calibThis module uses the histograms created bymake_eff_histsto assign efficiency to events in a reference sample supplied by the user. Adding efficiency to the user-supplied file requires PyROOT and is optional.The module works in two steps:Calculate the efficiency and save it as a TTree in a separate file.Optionally copy the efficiency TTree to the reference file and make it a friend of the user's TTree. The user must request the step by specifying--mergeon the command line.Be aware that--mergewill modify your file. Use with caution.OptionsThesampleandmagnetoptions are used solely to select the correct PID efficiency histograms. They should therefore mirror the options used when runningmake_eff_hists.bin-varsmust be a dictionary that relates the binning variables (or aliases) used to make the efficiency histograms with the variables in the reference sample. We assume that the reference sample branch names have the format[ParticleName]_[VariableName]. E.g.,D0_K_calcETA, corresponds to a particle namedD0_Kand variablecalcETA. If the user wants to estimate PID efficiency of their sample using 1D binning, wherecalcETAcorresponds to theETAbinning variable alias of the calibration sample, they should specify--bin-vars '{"ETA": "calcETA"}'.ref-fileis the user's reference file to which they want to assign PID efficiencies. The parameter can be a local file or a remote file, e.g., on EOS (--ref-file root://eoslhcb.cern.ch//eos/lhcb/user/a/anonymous/tuple.root).ref-parsmust be a dictionary of particles from the reference sample to apply cuts to. The keys represent the particle branch name prefix (D0_Kin the previous example), and the values passed are a list containing particle type and PID cut, e.g.'{"D0_K" : ["K", "DLLK > 4"], "D0_Pi" : ["Pi", "DLLK < 4"]}'.The--mergeoption will copy the PID efficiency tree to your input file and make the PID efficiency tree a "Friend" of your input tree. Then you can treat your input tree as if it had the PID efficiency branches itself. E.g.,input_tree->Draw("PIDCalibEff")should work. ROOT's "Friend" mechanism is an efficient way to add branches from one tree to another. Take a lookhereif you would like to know more.ExamplesEvaluate efficiency of a single PID cut and save it touser_ntuple_PID_eff.rootwithout adding it touser_ntuple.rootpidcalib2.ref_calib--sampleTurbo18--magnetup--ref-filedata/user_ntuple.root--histo-dirpidcalib_output--bin-vars'{"P": "mom", "ETA": "Eta", "nSPDHits": "nSPDhits"}'--ref-pars'{"Bach": ["K", "DLLK > 4"]}'--output-fileuser_ntuple_PID_eff.rootEvaluate efficiency of a single PID cut and add it to the reference fileuser_ntuple.rootpidcalib2.ref_calib--sampleTurbo18--magnetup--ref-filedata/user_ntuple.root--histo-dirpidcalib_output--bin-vars'{"P": "mom", "ETA": "Eta", "nSPDHits": "nSPDhits"}'--ref-pars'{"Bach": ["K", "DLLK > 4"]}'--output-fileuser_ntuple_PID_eff.root--mergeEvaluate efficiency of multiple PID cuts and add them to the reference filepidcalib2.ref_calib--sampleTurbo18--magnetup--ref-filedata/user_ntuple.root--histo-dirpidcalib_output--bin-vars'{"P": "P", "ETA": "ETA", "nSPDHits": "nSPDHits"}'--ref-pars'{"Bach": ["K", "DLLK > 4"], "SPi": ["Pi", "DLLK < 0"]}'--output-fileuser_ntuple_PID_eff.root--mergeCaveatsYou might notice that some of the events in your reference sample are assignedPIDCalibEff,PIDCalibErr, or both of -999.PIDCalibEffis -999 when for at least one trackThe event is out of binning rangeThe relevant bin in the efficiency histogram has no events whatsoeverThe efficiency is negativePIDCalibErris -999 when for at least one trackThe event is out of binning rangeThe relevant bin in the efficiency histogram has no events whatsoeverThe relevant bin in the efficiency histogram has no events passing PID cutsThe efficiency is negativeBecause ofdouble→floatconversion in the original PIDCalib, tiny discrepancies (<1e−3 relative difference) in the efficiencies and/or uncertainties are to be expected.A bug in the original PIDCalib caused the electron calibration datasets to be read twice, resulting in incorrect efficiency map uncertainties.The original PIDCalib didn't apply the correct cuts to Omega samples (K_OmegaandK_DD), leading to non-sensical efficiency maps.plot_calib_distributionsThis tool allows you to plot distributions of variables in the calibration datasets. You can supply the same cuts and custom binnings that you would use formake_eff_hists. If you wish to plot a variable for which no binning exists, a uniform binning with 50 bins will be used. You can change the number of bins using--binsand force a uniform binning even if another binning is defined via--force-uniform.A plot for every requested variable will be created in the--output-dirdirectory. The format of the plots can be controlled by--format. Furthermore,plot_calib_distributions.pklwill be saved in the same directory, containing all the histograms, should the user want to make the plots manually.ExamplesCreate plots of the variables DLLK and P using 1 calibration filepidcalib2.plot_calib_distributions--sampleTurbo18--magnetup--particlePi--bin-varDLLK--bin-varP--output-dirpidcalib_output--max-files1Create PDF plots of variable P with 95 uniform binspidcalib2.plot_calib_distributions--sampleTurbo18--magnetup--particlePi--bin-varP--output-dirpidcalib_output--max-files1--formatpdf--force-uniform--bins95Create plots of variable P using custom binningpidcalib2.plot_calib_distributions--sampleTurbo18--magnetup--particlePi--bin-varP--output-dirpidcalib_output--max-files1--formatpng--binning-filemy_binning.jsonpklhisto2rootThis tool converts pickled PIDCalib2 histograms toTH*Dand saves them in a ROOT file. It can be used on histograms produced bymake_eff_histsorplot_calib_distributions. Note that ROOT supports only 1-, 2-, and 3-dimensional histograms; attempting to convert higher-dimensional histograms will fail.ExampleConvert pickled boost_histograms frommake_eff_histsto ROOTpidcalib2.pklhisto2root"pidcalib_output/effhists-Turbo18-up-Pi-DLLK>4-P.ETA.nSPDhits.pkl"This will translate the histograms and save them topidcalib_output/effhists-Turbo18-up-Pi-DLLK>4-P.ETA.nSPDhits.root.DevelopmentWith lb-condaOn machines wherelb-condais available, you may use thepidcalibenvironment for PIDCalib2 development. This is mainly useful for small modifications and only if you don't need to add any new dependencies.Clone the repository fromGitLabEnter the PIDCalib2 directorycdpidcalib2Start a new BASH shell within thepidcalibenvironmentlb-condapidcalibbashRun yourlocalPIDCalib2 codecdsrc
python-mpidcalib2.make_eff_hists-hWithout lb-condaThis is a more versatile (if convoluted) method. It gives you full control of the dev environment and the ability to use IDEs, etc.Clone the repository fromGitLabEnter the PIDCalib2 directorycdpidcalib2(Optional) Set up a virtual environmentpython3-mvenv.venvsource.venv/bin/activateInstall pinned dependenciespipinstall-rrequirements-dev.txtInstallxrootd(possibly manually; see thisissue)Run the testspytestRun the modulescdsrc
python3-mpidcalib2.make_eff_hists-hTipsCertain tests can be excluded using markers like thispytest-m"not xrootd"See available markers by runningpytest --markers(the list will start with PIDCalib2 custom markers, then it will include all the pytest built-in markers).LinksPIDGen2- a tool to resample MC PID variables based on distributions from data calibration samples