Project: datamanagement
Issue DM-630: Non-partitioned table query returns duplicated rows
Running automated test I noticed that a query on non-partitioned table returns multiple copies of the same row, one copy per chunk. Here is example: {code} mysql> SELECT offset, mjdRef, drift FROM LeapSeconds where offset = 10; +--------+--------+-------+ | offset | mjdRef | drift | +--------+--------+-------+ | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | | 10 | 41317 | 0 | +--------+--------+-------+ 13 rows in set (5.62 sec) {code} Czar log file shows that it correctly finds that table is non-chunked but sends query to each chunk anyway: {code} 20140502 16:22:08.745081 0x3172430 INF *** KvInterfaceImplZoo::exist(), key: /DATABASES/LSST/TABLES/LeapSeconds/partitioning 20140502 16:22:08.745735 0x3172430 INF *** LSST.LeapSeconds is NOT chunked. 20140502 16:22:08.745762 0x3172430 INF *** KvInterfaceImplZoo::get2(), key: /DATABASES/LSST/TABLES/LeapSeconds/partitioning/subChunks 20140502 16:22:08.746393 0x3172430 INF *** LSST.LeapSeconds is NOT subchunked. 20140502 16:22:08.746409 0x3172430 INF getChunkLevel returns 0 ..... 20140502 16:22:08.757832 0x3172430 INF Using 85 stripes and 12 substripes. 20140502 16:22:08.775586 0x3172430 INF Using /usr/local/home/salnikov/dm-613/build/dist/etc/emptyChunks.txt as default empty chunks file. 20140502 16:22:08.791559 0x3172430 INF empty_LSST.txt not found while loading empty chunks file. 20140502 16:22:08.791592 0x3172430 ERR Couldn't find empty_LSST.txt, using /usr/local/home/salnikov/dm-613/build/dist/etc/emptyChunks.txt. 20140502 16:22:08.891239 0x7fbb28003660 INF QuerySession::_buildChunkQueries() : Non-subchunked 20140502 16:22:08.891498 0x7fbb28003660 INF Msg cid=6630 with size=153 20140502 16:22:08.891682 0x7fbb28003660 INF Added query id=6630 url=xroot://qsmaster@127.0.0.1:1094//q/LSST/6630 with save /dev/shm/qserv-salnikov-b93a8b7cca1128f50fa5531feb93f8f24a185f162d36c10ee76b8dca/1_6630_0 20140502 16:22:08.891694 0x7fbb28003660 INF Opening xroot://qsmaster@127.0.0.1:1094//q/LSST/6630 20140502 16:22:08.891705 0x7fbb28003660 INF QuerySession::_buildChunkQueries() : Non-subchunked 20140502 16:22:08.891882 0x7fbb28003660 INF Msg cid=6631 with size=153 20140502 16:22:08.892077 0x7fbb28003660 INF Added query id=6631 url=xroot://qsmaster@127.0.0.1:1094//q/LSST/6631 with save /dev/shm/qserv-salnikov-b93a8b7cca1128f50fa5531feb93f8f24a185f162d36c10ee76b8dca/1_6631_0 20140502 16:22:08.892087 0x7fbb28003660 INF Opening xroot://qsmaster@127.0.0.1:1094//q/LSST/6631 20140502 16:22:08.892097 0x7fbb28003660 INF QuerySession::_buildChunkQueries() : Non-subchunked 20140502 16:22:08.892275 0x7fbb28003660 INF Msg cid=6800 with size=153 20140502 16:22:08.892462 0x7fbb28003660 INF Added query id=6800 url=xroot://qsmaster@127.0.0.1:1094//q/LSST/6800 with save /dev/shm/qserv-salnikov-b93a8b7cca1128f50fa5531feb93f8f24a185f162d36c10ee76b8dca/1_6800_0 ... {code} Looking at the code together with Daniel we found that at the Python level (czar/app.py) the code that dispatches query does not check for chunkLevel, this is likely why this happens. The code to look at is in {{InbandQueryAction._applyConstraints()}} method.
Story points: 5