Project: datamanagement
Issue DM-5819: Incorporate Price suggestions to make `validate_drp` faster
Increase the loading and processing speed of {{validate_drp}} following suggestions by [~price] 1. Don't read in footprints Pass {{flags=lsst.afw.table.SOURCE_IO_NO_FOOTPRINTS}} to {{butler.get}} 2. Work on speed of calculation of RMS and other expensive quantities. Current suggestions: a. {{calcRmsDistances}} b. {{multiMatch}} c. {{matchVisitComputeDistance}} d. Consider boolean indexing {code} objById = {record.get(self.objectKey): record for record in self.reference} to: objById = dict(zip(self.reference[self.objectKey], self.reference)) {code} Note that while this ticket will involve work to reduce the memory footprint of the processing, it will not cover work to re-architect things to enable efficient processing beyond the memory on one node.
Story points: 2