Project: datamanagement
Issue DM-3141: Reduce verbosity of astrometry
The astrometry.net solver that runs by default in meas_astrom 10.1 is very verbose. Here's an example running HSC data with an SDSS reference catalog: {code} $ processCcd.py /tigress/HSC/HSC --output /tigress/pprice/lsst --id visit=904020 ccd=49 --clobber-config : Loading config overrride file '/home/pprice/LSST/obs/subaru/config/processCcd.py' WARNING: Unable to use psfex: No module named extensions.psfex.psfexPsfDeterminer hscAstrom is not setup; using LSST's meas_astrom instead Cannot import lsst.meas.multifit: disabling CModel measurements Cannot import lsst.meas.extensions.photometryKron: disabling Kron measurements Cannot enable shapeHSM ('MEAS_EXTENSIONS_SHAPEHSM_DIR'): disabling HSM shape measurements Cannot import lsst.meas.extensions.photometryKron: disabling Kron measurements Cannot enable shapeHSM ('MEAS_EXTENSIONS_SHAPEHSM_DIR'): disabling HSM shape measurements : Loading config overrride file '/home/pprice/LSST/obs/subaru/config/hsc/processCcd.py' : input=/tigress/HSC/HSC : calib=None : output=/tigress/pprice/lsst CameraMapper: Loading registry registry from /tigress/pprice/lsst/_parent/registry.sqlite3 CameraMapper: Loading calibRegistry registry from /tigress/HSC/HSC/CALIB/calibRegistry.sqlite3 processCcd: Processing {'taiObs': '2013-11-02', 'pointing': 671, 'visit': 904020, 'dateObs': '2013-11-02', 'filter': 'HSC-I', 'field': 'STRIPE82L', 'ccd': 49, 'expTime': 30.0} processCcd.isr: Performing ISR on sensor {'taiObs': '2013-11-02', 'pointing': 671, 'visit': 904020, 'dateObs': '2013-11-02', 'filter': 'HSC-I', 'field': 'STRIPE82L', 'ccd': 49, 'expTime': 30.0} processCcd.isr WARNING: Cannot write thumbnail image; hsc.fitsthumb could not be imported. afw.image.MaskedImage WARNING: Expected extension type not found: IMAGE processCcd.isr: Applying linearity corrections to Ccd 49 processCcd.isr.crosstalk: Applying crosstalk correction afw.image.MaskedImage WARNING: Expected extension type not found: IMAGE : Empty WCS extension, using FITS header processCcd.isr: Set 0 BAD pixels to 647.04 processCcd.isr WARNING: There were 6192 unmasked NaNs processCcd.isr WARNING: Cannot write thumbnail image; hsc.fitsthumb could not be imported. processCcd.isr: Flattened sky level: 647.130493 +/- 12.733898 processCcd.isr: Measuring sky levels in 8x16 grids: 648.106765 processCcd.isr: Sky flatness in 8x16 grids - pp: 0.024087 rms: 0.006057 processCcd.calibrate: installInitialPsf fwhm=5.88235294312 pixels; size=15 pixels processCcd.calibrate.repair: Identified 80 cosmic rays. processCcd.calibrate.detection: Detected 303 positive sources to 5 sigma. processCcd.calibrate.detection: Resubtracting the background after object detection processCcd.calibrate.initialMeasurement: Measuring 303 sources (303 parents, 0 children) processCcd.calibrate.astrometry: Applying distortion correction processCcd.calibrate.astrometry: Solving astrometry LoadReferenceObjects: read index files processCcd.calibrate.astrometry.solver: Number of selected sources for astrometry : 258 processCcd.calibrate.astrometry.solver: Got astrometric solution from Astrometry.net LoadReferenceObjects: getting reference objects using center (1023.5, 2084.5) pix = Fk5Coord(320.3431396, 0.5002365, 2000.00) sky and radius 0.00194896 rad LoadReferenceObjects: search for objects at Fk5Coord(320.3431396, 0.5002365, 2000.00) with radius 0.111667372351 deg LoadReferenceObjects: found 495 objects LoadReferenceObjects: trimmed 257 out-of-bbox objects, leaving 238 processCcd.calibrate.astrometry.solver: Fit WCS: use iter 2 because it had less linear scatter than the next iter: 0.307471 vs. 0.320229 pixels processCcd.calibrate.astrometry: 186 astrometric matches processCcd.calibrate.astrometry: Refitting WCS processCcd.calibrate.astrometry: Astrometric scatter: 0.047945 arcsec (with non-linear terms, 174 matches, 12 rejected) processCcd.calibrate.measurePsf: Measuring PSF /tigress/HSC/LSST/stack10_1/Linux64/anaconda/2.1.0-4-g35ca374/lib/python2.7/site-packages/numpy/core/_methods.py:59: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /tigress/HSC/LSST/stack10_1/Linux64/anaconda/2.1.0-4-g35ca374/lib/python2.7/site-packages/numpy/core/_methods.py:71: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) /home/pprice/LSST/meas/algorithms/python/lsst/meas/algorithms/objectSizeStarSelector.py:143: RuntimeWarning: invalid value encountered in less update = dist < minDist processCcd.calibrate.measurePsf: PSF star selector found 163 candidates processCcd.calibrate.measurePsf: PSF determination using 114/163 stars. processCcd.calibrate.repair: Identified 92 cosmic rays. processCcd.calibrate: Fit and subtracted background processCcd.calibrate.measurement: Measuring 303 sources (303 parents, 0 children) processCcd.calibrate.astrometry: Applying distortion correction processCcd.calibrate.astrometry: Solving astrometry processCcd.calibrate.astrometry.solver: Number of selected sources for astrometry : 258 Solver: Arcsec per pix range: 0.153025, 0.18516 Image size: 2054 x 4186 Quad size range: 205.4, 4662.78 Objs: 0, 50 Parity: 0, normal Use_radec? yes, (320.343, 0.500178), radius 1 deg Verify_pix: 1 Code tol: 0.01 Dist from quad bonus: yes Distractor ratio: 0.25 Log tune-up threshold: inf Log bail threshold: -230.259 Log stoplooking threshold: inf Maxquads 0 Maxmatches 0 Set CRPIX? no Tweak? no Indexes: 3 /tigress/HSC/astrometry_net_data/sdss-dr9-fink-v5b/sdss-dr9-fink-v5b_and_263_0.fits /tigress/HSC/astrometry_net_data/sdss-dr9-fink-v5b/sdss-dr9-fink-v5b_and_263_1.fits /tigress/HSC/astrometry_net_data/sdss-dr9-fink-v5b/sdss-dr9-fink-v5b_and_263_2.fits Field: 258 stars Quad scale range: [641.674, 2208.56] pixels object 1 of 50: 0 quads tried, 0 matched. object 2 of 50: 0 quads tried, 0 matched. object 3 of 50: 0 quads tried, 0 matched. object 4 of 50: 0 quads tried, 0 matched. object 5 of 50: 0 quads tried, 0 matched. object 6 of 50: 0 quads tried, 0 matched. Got a new best match: logodds 787.099. log-odds ratio 787.099 (inf), 178 match, 1 conflict, 75 distractors, 220 index. RA,Dec = (320.343,0.500213), pixel scale 0.167612 arcsec/pix. Hit/miss: Hit/miss: ++-+++++-++++++++++++--++-+--+++++-+-+++++++++-+++++++-+++++-+++++++++++++-++++++-++++++-+++++++-+++ Pixel scale: 0.167612 arcsec/pix. Parity: pos. processCcd.calibrate.astrometry.solver: Got astrometric solution from Astrometry.net LoadReferenceObjects: getting reference objects using center (1023.5, 2084.5) pix = Fk5Coord(320.3431396, 0.5002365, 2000.00) sky and radius 0.00194896 rad LoadReferenceObjects: search for objects at Fk5Coord(320.3431396, 0.5002365, 2000.00) with radius 0.111667328272 deg LoadReferenceObjects: found 495 objects LoadReferenceObjects: trimmed 257 out-of-bbox objects, leaving 238 processCcd.calibrate.astrometry.solver: Fit WCS: use iter 2 because it had less linear scatter than the next iter: 0.306732 vs. 0.320115 pixels processCcd.calibrate.astrometry: 186 astrometric matches processCcd.calibrate.astrometry: Refitting WCS processCcd.calibrate.astrometry: Astrometric scatter: 0.048271 arcsec (with non-linear terms, 174 matches, 12 rejected) processCcd.calibrate.photocal: Not applying color terms because config.applyColorTerms is False processCcd.calibrate.photocal: Magnitude zero point: 30.685281 +/- 0.058711 from 173 stars processCcd.calibrate: Photometric zero-point: 30.685281 processCcd.detection: Detected 1194 positive sources to 5 sigma. processCcd.detection: Resubtracting the background after object detection processCcd.deblend: Deblending 1194 sources processCcd.deblend: Deblended: of 1194 sources, 143 were deblended, creating 358 children, total 1552 sources processCcd.measurement: Measuring 1552 sources (1194 parents, 358 children) processCcd WARNING: Persisting background models processCcd: Matching icSource and Source catalogs to propagate flags. processCcd: Matching src to reference catalogue LoadReferenceObjects: getting reference objects using center (1023.5, 2087.5) pix = Fk5Coord(320.3429016, 0.5001781, 2000.00) sky and radius 0.00195667 rad LoadReferenceObjects: search for objects at Fk5Coord(320.3429016, 0.5001781, 2000.00) with radius 0.112109149864 deg LoadReferenceObjects: found 499 objects LoadReferenceObjects: trimmed 261 out-of-bbox objects, leaving 238 processCcd.calibrate.astrometry.solver: Fit WCS: use iter 1 because it had less linear scatter than the next iter: 0.300624 vs. 0.300652 pixels {code} The verbosity of the astrometry module is out of proportion with the rest of the modules, which makes it difficult to follow the processing. This is a pull request for fixes I have made.
Story points: 1