Project: datamanagement
Issue DM-2917: obs_cfht unit tests are broken
obs_cfht has one unit test "testButler" that uses git://git.lsstcorp.org/contrib/price/testdata_cfht. 4 of the tests fail, as shown below. In addition, testdata_cfht is huge, and the tests barely use any of it. It's worth considering making a new test repo that is smaller, or if the amount of data is small enough, move it into afwdata or obs_cfht itself. {code} localhost$ tests/testButler.py CameraMapper: Loading registry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/DATA/registry.sqlite3 CameraMapper: Loading calibRegistry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/CALIB/calibRegistry.sqlite3 ECameraMapper: Loading registry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/DATA/registry.sqlite3 CameraMapper: Loading calibRegistry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/CALIB/calibRegistry.sqlite3 ECameraMapper: Loading registry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/DATA/registry.sqlite3 CameraMapper: Loading calibRegistry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/CALIB/calibRegistry.sqlite3 ECameraMapper: Loading registry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/DATA/registry.sqlite3 CameraMapper: Loading calibRegistry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/CALIB/calibRegistry.sqlite3 .CameraMapper: Loading registry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/DATA/registry.sqlite3 CameraMapper: Loading calibRegistry registry from /Users/rowen/LSST/code/testdata/testdata_cfht/CALIB/calibRegistry.sqlite3 E. ====================================================================== ERROR: testBias (__main__.GetRawTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/testButler.py", line 122, in testBias self.getDetrend("bias") File "tests/testButler.py", line 110, in getDetrend flat = self.butler.get(detrend, self.dataId, ccd=ccd) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/butler.py", line 218, in get location = self.mapper.map(datasetType, dataId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/mapper.py", line 116, in map return func(self.validate(dataId), write) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/cameraMapper.py", line 287, in mapClosure return mapping.map(mapper, dataId, write) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 118, in map actualId = self.need(self.keyDict.iterkeys(), dataId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 199, in need lookups = self.lookup(newProps, newId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 345, in lookup return Mapping.lookup(self, properties, newId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 168, in lookup where, self.range, values) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/registries.py", line 120, in executeQuery c = self.conn.execute(cmd, values) OperationalError: no such column: extension ====================================================================== ERROR: testFlat (__main__.GetRawTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/testButler.py", line 117, in testFlat self.getDetrend("flat") File "tests/testButler.py", line 110, in getDetrend flat = self.butler.get(detrend, self.dataId, ccd=ccd) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/butler.py", line 218, in get location = self.mapper.map(datasetType, dataId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/mapper.py", line 116, in map return func(self.validate(dataId), write) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/cameraMapper.py", line 287, in mapClosure return mapping.map(mapper, dataId, write) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 118, in map actualId = self.need(self.keyDict.iterkeys(), dataId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 199, in need lookups = self.lookup(newProps, newId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 345, in lookup return Mapping.lookup(self, properties, newId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 168, in lookup where, self.range, values) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/registries.py", line 120, in executeQuery c = self.conn.execute(cmd, values) OperationalError: no such column: extension ====================================================================== ERROR: testFringe (__main__.GetRawTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/testButler.py", line 127, in testFringe self.getDetrend("fringe") File "tests/testButler.py", line 110, in getDetrend flat = self.butler.get(detrend, self.dataId, ccd=ccd) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/butler.py", line 218, in get location = self.mapper.map(datasetType, dataId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/mapper.py", line 116, in map return func(self.validate(dataId), write) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/cameraMapper.py", line 287, in mapClosure return mapping.map(mapper, dataId, write) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 118, in map actualId = self.need(self.keyDict.iterkeys(), dataId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 199, in need lookups = self.lookup(newProps, newId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 345, in lookup return Mapping.lookup(self, properties, newId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/mapping.py", line 168, in lookup where, self.range, values) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_butlerUtils/10.1-3-g302a9ed/python/lsst/daf/butlerUtils/registries.py", line 120, in executeQuery c = self.conn.execute(cmd, values) OperationalError: no such column: extension ====================================================================== ERROR: testRaw (__main__.GetRawTestCase) Test retrieval of raw image ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/testButler.py", line 101, in testRaw raw = self.butler.get("raw", self.dataId, ccd=ccd, immediate=True) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/butler.py", line 244, in get return callback() File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/butler.py", line 242, in innerCallback(), dataId) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/butler.py", line 238, in callback = lambda: self._read(pythonType, location) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/butler.py", line 426, in _read location.getCppType(), storageList, additionalData) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/daf_persistence/10.1-1-g6edbc00+1/python/lsst/daf/persistence/persistenceLib.py", line 1430, in unsafeRetrieve return _persistenceLib.Persistence_unsafeRetrieve(self, *args) FitsError: File "src/fits.cc", line 1064, in lsst::afw::fits::Fits::Fits(const std::string &, const std::string &, int) cfitsio error: could not open the named file (104) : Opening file '/Users/rowen/LSST/code/testdata/testdata_cfht/DATA/raw/08BL05/w2.+2+2/2008-11-01/i2/1038843o.fits.fz[1]' with mode 'r' {0} lsst::afw::fits::FitsError: 'cfitsio error: could not open the named file (104) : Opening file '/Users/rowen/LSST/code/testdata/testdata_cfht/DATA/raw/08BL05/w2.+2+2/2008-11-01/i2/1038843o.fits.fz[1]' with mode 'r'' ---------------------------------------------------------------------- Ran 6 tests in 3.544s FAILED (errors=4) {code}
Story points: 1