dynamobi/dynamobi — Issue #98: [LDB-225] push down count(*) into LcsRowScanExecStream
Author: dynamobi-build
Created: 2011-12-30 02:13:04+00:00
State: closed
Labels: LCS, LDB

[reporter="jvs", created="Sun, 20 Jun 2010 01:34:42 -0500 (GMT-05:00)"]
I'm working on a somewhat different approach than Rushan in LDB-196.

I'm going to implement something which can handle any count(*), with no other aggs, and no GROUP BY, but with WHERE clause allowed. (And star joins as well if the dimension tables can be completely projected away after bitmap filtering, leaving just a count on the filtered fact table.)

Basically, I'm just doing all the counting inside of the LcsRowScanExecStream, so it only produces one final tuple (the count at the end). But other than that, it executes in exactly the same way as any other scan, including all of the filtering logic for WHERE clause, deleted rows, etc.

This is not as optimal as it could be, since it still has to iterate over every row (instead of just summing up RID's). But it cuts out the expensive unmarshal+marshal required when the counting is done in a separate SortedAggExecStream. In my prototype test counting 16 million BIGINT rows, this reduced the time from 14 seconds to 3.6 seconds on my laptop, a speedup of almost 4x.