Work Queue Cleanup

Major cleanup of work_queue_pool and work_queue_worker.

work_queue_pool now queries the catalog for masters with a given project name (regex), computes the desired number of workers, and submits them with the same project name (regex).

work_queue_worker now clearly queries for the list of matching masters and tries them in a random order.  The worker no longer queries "wq_pool" records from the catalog for configuration.

There are still some problems here that need to be addressed:
- timeouts are used inconsistently, esp regarding when to disconnect after periods of idleness.
- accounting for resources locally doesn't look right, esp dividing by the number of workers.


Oops, that is an accidental commit.
I have pushed a commit to undo it.



On Fri, Jun 27, 2014 at 9:13 AM, Benjamin Tovar <notifications@github.com>
wrote:

> In allpairs/test/TR_allpairs_bitwise.sh:
>
> > @@ -28,9 +28,9 @@ prepare()
> >
> >  run()
> >  {
> > -    ../../work_queue/src/work_queue_worker localhost `cat $PORT_FILE` &
> > -    pid=$!
> > -	echo $pid > $PIDWORKER_FILE
> > +    ../../work_queue/src/work_queue_worker localhost `cat $PORT_FILE` --timeout 2 -d all
> > +#    pid=$!
>
> Is there a reason this is being commented out?
>
> --
> Reply to this email directly or view it on GitHub
> <https://github.com/cooperative-computing-lab/cctools/pull/479/files#r14291631>
> .
>