Project: datamanagement
Issue DM-3392: Fix precision related problem in UDFs
SciSQL udfs seem to have a subtle precision problem. The following query that is not relying on scisql returns one row: {quote} select ra, decl, deepSourceId FROM Object o WHERE decl between 0.992 and 0.993 and ra between 19.171 and 19.172; +------------------+-------------------+------------------+ | ra | decl | deepSourceId | +------------------+-------------------+------------------+ | 19.1719166801441 | 0.992087616433663 | 4368217963236477 | +------------------+-------------------+------------------+ 1 row in set (2 min 9.49 sec) {quote} But equivalent scisql-based query: {quote} select ra, decl, deepSourceId FROM Object o WHERE qserv_areaspec_box(0.992, 19.171, 0.993, 19.172); {quote} will fail to find that row. If we relax the search criteria just a little bit, it finds some other row, but still not the one with decl = 0.992087616433663 {quote} select ra, decl, deepSourceId FROM Object o WHERE qserv_areaspec_box(0.99, 19.171, 0.999, 19.172); +-------------------+-----------------+------------------+ | ra | decl | deepSourceId | +-------------------+-----------------+------------------+ | 0.994098536926311 | 19.171425377618 | 4372684729224984 | +-------------------+-----------------+------------------+ {quote}
Story points: 4