Make queue processing more efficient

Instead of processing all of the queues each time any event happens,
first process all of the events (which are really queued up just so
that they can modify the pipeline data safely within the main run loop)
and then run the queue processor (which will then launch or cancel jobs
based on the new state of the queues).

This adds a lock to the run processor which is only needed so that
the test suite can safely inspect the state of the scheduler to
determine whether it is idle.

The result queue is made somewhat more generic in preparation for
handling merge result events.

Change-Id: I6bbb52f77b070df04a110a9d61b426265b1e89cc