Continue flailing on environment variable configuraiton of Galaxy.
Expand variables earlier right in app_factory so middleware, etc... pick up defaults/overrrides.

In particular GALAXY_CONFIG_OVERRIDE_DEBUG should now work properly. GALAXY_CONFIG_DEBUG cannot be used because the sample config template overrides debug's default.

Thanks to Bjoern for the bug report.
Functional test fix for config file moves.
Merge next-stable to default.
Use common_startup.sh for buildbot instead of maintaining a duplicate copy of the samples list.
Merge next-stable
Fix run_tests.sh --with_framework_test_tools for 9a05d1e.
Now appends test tools to sample tools instead of to default tools Galaxy would other wise load (config/tool_conf.xml or tool_conf.xml if these exist instead of sample) - so this is different behavior but I am unsure if it is worse. --with_framework_test_tools was added for API tests and these should probably target the distribution and not random tools someone might have configured so perhaps this new behavior is superior.
Improved configuration error messages.
Merge latest next-stable.
ToolForm: Scrolling to error, basic validation, success message, style fixes
Fixes to current dataset indicator and focused icon buttons
Merge default
Dynamically load workflow module descriptions in mako editor.

This again falls under the umbrella of making workflow modules more like true plugins. This should allow one to add new modules downstream without needing to modify templates/webapps/galaxy/workflow/editor.mako. One can imagine a new API endpoint for the new load_module_sections method will be useful as the editor UI shifts to a single page app and actual API clients need to determine what workflow modules are available. Such an API endpoint is not included but would be easy to add now.
Implement workflow requests (API only) and a scheduling 'plugin' framework to evaluate them.

The model changes and WorkQueue class are based on modifications to Kyle Ellrott's Galaxy farm work (https://bitbucket.org/kellrott/galaxy-farm/branch/workflow_migrate). I made heavy modifications to the model to enforce referential integrity on parameter to workflow step mappings and made some cosmetic changes various other details. I further filled out WorkQueue (adding ability to 'complete' requests, implementing tests, etc...). The work context (allowing tool's to be executed outside the context of a web thread), workflow module and API modifications, and the workflow scheduling framework are my own contributions.

I like some of this - I remain skeptical about much of it though. I think maintaining queues of generic requests and distributing them across multiple processes, machines, etc... is both a hard problem and one for which many adequete solutions already exist - some of which we are already using (if only optionally) in other parts of Galaxy - so I don't understand why we need/want to so explicitly replicate this in Galaxy and not just defer to an external message queue.

In internal discussions it has been argued that message queues are appropriate for short requests but long term requests are better tracked in the database. I know that intuitively these operations may feel different - but I am unconvienced they are from a technical standpoint. Delaying an action that would have happened in milliseconds to hours later is many order of magnitude larger than the additional delay when going from the hours to days or weeks.

My own preference would be that both this work and Galaxy farm further investigate using message queues before making such substantial changes to the data model. It certainly may prove that this approach is needed - but I personally would like to understand why maintain the work queue in a message queue would not work first.

However, if the Galaxy project is going to add a WorkRequest object to the database and something that looks like the WorkQueue class in the new galaxy.work module and store and distribute workflows that way for the farm project, then I believe that typical workflow requesting and scheduling should make use of it also instead of developing a competing implementation. (Because my interaction pattern with Galaxy has always been if you cannot beat them join them.)

Testing

One can issue a workflow request to the WorkQueue and scheduled via the scheduling framework by running the following test.

./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_workflow_request

Final Note: The workflow scheduling stuff is a bit rough right now - it is just enough to prove to myself that this approach can be taken and the farm-stuff and workflow scheduling stuff can utilize the same models. I wanted to try to spark a broader conversation on this (re-encounrage use of message queues) before putting more effort into this approach.
Create a log for run_request.py.
Implement pause module for workflows.

Extracts base class ouptut of InputModule for modules that are "simple" - i.e. their configuration state is represented as a dictionary and configuration form is rendered via the generic template.
enable_beta_workflow_modules
Enable workflow recovery over mapping steps of workflows.
Rebase into pause.
Implement explicit workflow invocation cancellation.
Infrastructure for supporting modules cancelling workflows.