Project: datamanagement
Issue DM-5555: SQuaSH MVP
This is an epic that covers setting up a minimally viable QA environment for executing processing, calculating metrics, storing them and displaying them. This serves a dual purpose: - It results in a limited but still useful production service that developers can take advantage of - It allows us to assess our initial technology stack for suitability for further development. The test case was picked to be a supertask version of the tests described in DM-4730 (informally ci_lauren) used during the merging of the HSC fork. This was meant to also allow us to use and give feedback on the supertask architecture. When it became obvious that we would not take delivery of that infrastructure in time for X16 work, we switched the test case to one of the KPMs calculated in validate_drp. This switch does not affect the engineering aims of this prototype. The MVP based on validate_drp can eventually be extended to service other types of KPM measurement for regression testing and release characterisation. (JH:32, JMP:16, JS:32, MVW:14, AF:20) Outcome: MVP stood up on squash.lsst.codes. Currently collecting AM1, AM2 and PA1 KPMs using validate_drp on validation_data_cfht. After a short period of evaluation of the performance of the toolchains in production we will proceed with more data, more metrics and more features for F16.
Story points: 100