Rework history.contents_iter to potentially support multiple types.
Rework history_contents API to allow room for multiple 'type's of history contents.

Only poorly named 'file' (default) is currently implemented, will use new flexibility to add 'collection' type. Room made in create, show, and index methods.
Dataset collections - model layer.

Mapping, model definition, database migration, and model unit tests to support database collections.
Dataset collections - infrastructure glue...

Adds an API and service layer for dataset collections.
Dataset collections - tool parameters.

Utilize dataset collections from tools.
Rework logic related to computing selected datasets.

Allow dataset ids to be used. This will make implementing multi-running of tools possible without expanding values out to hdas. check_params will still be called on each permutation so this shouldn't result in security problems.
Refactor Tool.handle_input result to return list of pairs instead of odict.

This added generality of allowing multiple outputs with the same output name will allow handle_input to execute multiple jobs in subsequent changesets.
Allow running tools on multiple datasets simulatenously (backend+API).

Each data tool parameter can be specified via a similar named parameter but with the suffix |__multirun__. This second parameter variant should be a list of datasets - one job will be created for each such dataset.
Allow running tools on multiple datasets simulatenously (UI).

Update tool form. This commit needs work to ensure this only affects the tool_form.mako.
Add tool meta parameter exception.
Parallel tool execution across dataset collections (backend).
Parallel tool execution across dataset collections (UI).
Implicit creation of analogous dataset collections...

... when running normal tools with normal data inputs across dataset collections in parallel. A previous changeset added the ability to run the tool in parallel - this changeset extends that functionality to create analogous collections from these parallel runs.

For example, if one filters a pair of FASTQ files a pair is created out of the result. Likewise, if one filters a list of FASTQ files - a list collection with same cardinality is built from the results.

There is a lot left TODO here - for one a lot of this logic should be moved into the dataset_collection module. The matching needs to be exact right now - not a problem for pairs (every 'element' has name 'left' or 'right') but for lists with element names - these have to match exactly - but a 'list' like samp1_l, samp2_l, samp3_l should be able to match against samp1_r, samp2_r, samp3_r and create a new list samp1, samp2, and samp3. Even if there is no matching prefixes a new 'unlabeled' list should be able to be created.
Refactor workflow module to allow multiple kinds of inputs.
Add history.active_contents property...

Get all active datasets and collections ordered together by hid.
Handle dataset collections in workflows.

Extend workflow running, extraction, viewing to allow tools with data_collection parameters.

This is brutally ugly because "isinstance( input, DataToolParameter )" which was already not very Pythonic has been updated to be "isinstance( input, DataToolParameter ) or isinstance( input, DataCollectionToolParameter )". This should all be reworked with an "is_data_parameter" on base ToolParameter.
Workflow implicit dataset collection creation and parallelization...

While running a workflow, if a data tool parameter is encountered with a dataset collection as input - run N jobs and create a dataset collection for each of the outputs (similar functionality added for tools in previous changeset). These collections are then used in subsequent steps while connecting steps so the parallelization continues as does the creation of dataset collections.

Much and more remains to be done and tested - chief among these concerns I think is that in order to test this I needed to hand modify a workflow json file. The ability to use this feature should be encorporated into the editor and the interface/controller for extracting a workflow from a history.
WIP: Extracting implicit collections from histories.