Project: datamanagement
Issue DM-3142: Port HSC optimisations for reading astrometry.net catalog
Some astrometry.net catalogs used in production can be quite large, and currently all of the catalog must be read in order to determine bounds for each component. This can make the loading of the catalog quite slow (e.g., 144 sec out of 177 sec to process an HSC image, using an SDSS DR9 catalog). We have HSC code that caches the required information, making the catalog load much faster. The code is from the following HSC issues: * [HSC-1087: Make astrometry faster|https://hsc-jira.astro.princeton.edu/jira/browse/HSC-1087] * [HSC-1143: Floating point exception in astrometry|https://hsc-jira.astro.princeton.edu/jira/browse/HSC-1143] * [HSC-1178: Faster construction of Astrometry.net catalog|https://hsc-jira.astro.princeton.edu/jira/browse/HSC-1178] * [HSC-1179: Assertion failure in astrometry.net|https://hsc-jira.astro.princeton.edu/jira/browse/HSC-1178] While there have been some changes to the LSST astrometry code that will mean we can't directly cherry-pick the HSC code, yet I think the main structure remains, so the approach can be copied without much effort.
Story points: 3