Project: datamanagement
Issue DM-3766: Add Butler access to calibration data in obs_decam
Goals: - Have something to ingest calibration data and create a calibration registry (calibRegistry.sqlite3) - Add mapping class calibration into DecamMapper and make butler able to get bias/flat/fringe. - Include calib in testdata_decam and add to the unit test retrieving calib data. Summary: - A new task {{ingestCalibs.py}} is added to {{pipe_tasks}} for parsing through calibration data and creating a calibration registry. The DECam customized configuration is added in {{DecamCalibsParseTask}} in {{obs_decam}}. {{pipe_tasks/ingest.py}} is modified slightly for more general use while its original behaviors are kept by default. - The calibration data are DECam Community-Pipeline products, including nightly-MasterCal bias/flat downloadable from NOAO Science Archive, and fringe of the latest version 56876 from http://www.ctio.noao.edu/noao/content/decam-calibration-files or also on /lsst8/decam/cal/DECamMasterCal_56876/ Note that files from the two sources have different formats (MEF one HDU for each detector, or one single-HDU fits for each detector). - New Butler dataset types bias/flat/fringe are added along with functions to standardize them to Exposures in {{DecamMapper}}. - Tests of retrieving bias/flat/fringe by Butler are added. - testdata_decam used by the unit tests is at lsst-dev:/lsst8/testdata_decam/ Known caveats for future users - The task could be a bit noisy when ingesting DECam calibration data products. But given the variety of data products on hand I might rather have those reminders than letting all pass silently. - I wouldn't be too surprised if future DECam Community-Pipeline products appear in different formats. Depending on how different they become, {{DecamCalibsParseTask}} might or might not need future modifications.
Story points: 21