slamdata/slamengine — pull request #389 by sellout (2014-10-17 23:19:46+00:00)
Title: Eliminate duplicate `_id`s in `PipelineTask`s.

Add a new function, `WorkflowTask.finish` that patches up
tasks. Currently it only finds cases where we can end up with duplicate
ids and eliminates them, which fixes #327.

Had to do a better job of tracking the `_id` field, since it’s so
magical. There is now an extra `id` field on ProjectOp that determines
whether `_id` should be included in the projection. And we keep track of
this when coalescing, etc. Since `_id` was the only use case for
`ExprOp.Exclude` (which had terrible semantics), that has also been
removed.

This is still not perfect. Tests pass, and we generally handle `_id`
better, but we still are a bit over-conservative and have some other
issues (tracked in new tickets). And I added a dependency on Spire (for
`IdHandlingRing`).

This includes a number of other changes:

* add some `WorkflowOp.crush` tests;
* get rid of the `Pipeline` wrapper (`Pipeline` is now just a type alias for
* bump the minimum coverage to 61%;
* Use WorkflowOp’s ExprLabel/Name/Var everwhere and get rid of th
WorkflowBuilder version (since `value` is a special name in MongoDB);
* Add IdLabel/Name/Var to parallel Expr* but for the `_id` field;
* group without “by” now uses `Bson.Null` instead of `
since it’s what the docs recommend;
* tighten up some integration tests that weren’t ignoring `_id` w
they should have been;
* add some pending tests for newly-created issues; and
* fixes the overzealous `UnwindOp`/`UnwindOp` merge that could conflate
two actually-different fields.