dynamobi/dynamobi — Issue #441: [FNL-60] fix all ExecStreams to be capable of restart after early close
Author: dynamobi-build
Created: 2011-12-30 03:13:51+00:00
State: open
Labels: FNL

[reporter="jvs", created="Fri, 5 Jan 2007 15:46:15 -0500 (GMT-05:00)"]
We tried to make all sorters perform early close, but this caused problems with sorters underneath the RHS of cartesian products, since this caused streams below the sorter to be restarted after early close. We fixed some, but not all. Testcase below for one crash, plus ScratchBufferExecStream and others will silently accumulate pinned buffers, since their ScratchSegment never unpins until entire graph is closed. Zelaine has added a flag to control where early close is used, but long-term we'd like to fix this somehow so that we can release resources early for some graph topologies. The change to allow restart after close is questionable; seems like either we need to figure out that a restart is possible (so we don't really want to release resources anyway, since we're just going to reacquire them) and in that case ignore the close. Plus, in non-correlated graphs, the sorter never really needs to restart (it could just reuse its buffered results, assuming it hasn't gotten rid of them). Need dynamic parameter global dependency info to figure that out.

Testcase (requires tweaking sorter to always perform early close)

----

Run farrago/unitsql/optimizer/semijoin.sql first. Then, in the sj schema, do the following:

set schema 'sj';

alter session implementation set jar sys_boot.sys_boot.luciddb_plugin;

create index i_product_color on product(color);

create index i_product_size on product(size);

Then run the following query twice (may crash on first run, or may assert on second):

select a, sid, name, color, size, quantity

from t left outer join

(select * from sales s, product p

where

s.product_id = p.id and p.size = 'S' and p.color = 'black')

on true

order by a, sid;