Eliminate code dup. in workflow editor for building connectors.
Eliminate code dup. in workflow editor for building input terminals.
Refactor out optional data selection handling.

Reused a few times downstream for dataset collections.
Refactor backbone models to allow different kinds of history contents.

In particular, add a new backbone parent model (HistoryContent) refactored out of HistoryDatasetAssociation. This makes room for a forth coming HistoryDatasetCollectionAssociation model that will be emitted from the history contents API.
Refactor backbone views to allow different kinds of history contents.

This makes room for a forth coming HistoryDatasetCollectionAssociation model that will be emitted from the history contents API.
Dataset collections - models - basics.

Mapping, model definition, database migration, and model unit tests to support database collections.
Dataset collections - models - add history.active_contents.

Get all active datasets and collections ordered together by hid. Used downstream for instance when extracting workflows from histories.
Dataset collections - models - history copying.

Copy history dataset collection associations when histories copied.
Dataset collections - infrastructure glue.

Add an API and service layer for dataset collections.
Dataset collections - tool parameters - basics.

Add parameter, wrapper, etc... to explicitly utilize dataset collections from tools. These require actually modifing tools to take in "data_collection" parameters. Downstream changesets will provide a number of infrastructure enhancements to allow existing tools to "map" and "reduce" dataset collections with existing tool.

TODO: Add tools to test/functional/tools to demonstrate these additions.
Dataset collections - tool parameters - enhanced rerun.

Rerunning tools in same history will work without this, but getting rerun to work on imported histories requires this hack (mirroring the same hack for HDAs in the same file).
Rework logic related to computing selected datasets.

Allow dataset ids to be used. This will make implementing multi-running of tools possible without expanding values out to hdas. check_params will still be called on each permutation so this shouldn't result in security problems.
Refactor Tool.handle_input result to return list of pairs instead of odict.

This added generality of allowing multiple outputs with the same output name will allow handle_input to execute multiple jobs in subsequent changesets.
Allow running tools on multiple datasets simulatenously (backend+API).

Each data tool parameter can be specified via a similar named parameter but with the suffix |__multirun__. This second parameter variant should be a list of datasets - one job will be created for each such dataset.
Allow running tools on multiple datasets simulatenously (UI).

Update tool form. This commit needs work to ensure this only affects the tool_form.mako.
Add tool meta parameter exception.
Dataset collections - "map" with existing tools (backend).

Allow replacing data parameter inputs with collections - this will cause the tool to produce multiple jobs for the submission - one for each combination of input parameters.
Dataset collections - "map" with existing tools (UI).

Add UI components needed for previous commit.
Dataset collections - implicit "map" creates collections.

When running normal tools with normal data inputs across dataset collections in parallel - a collection will be created for each output with a "structure" matching that of the input (for instance pairs will create pairs, lists will create lists with the same identifiers, etc...). A previous changeset added the ability to run the tool in parallel - this changeset extends that functionality to create analogous collections from these parallel runs.

For example, if one filters a pair of FASTQ files a pair is created out of the result. Likewise, if one filters a list of FASTQ files - a list collection with same cardinality is built from the results.

There is a lot left TODO here - for one a lot of this logic should be moved into the dataset_collection module. The matching needs to be exact right now - not a problem for pairs (every 'element' has name 'left' or 'right') but for lists with element names - these have to match exactly - but a 'list' like samp1_l, samp2_l, samp3_l should be able to match against samp1_r, samp2_r, samp3_r and create a new list samp1, samp2, and samp3. Even if there is no matching prefixes a new 'unlabeled' list should be able to be created.
Dataset collections - "reduce" with existing tools.

Allow users to select dataset collections in place of individual datasets for data tool parameters with multiple="true" enabled (if all elements of collection would be valid as input to this parameter).