Add zookeeper job/jobboard impl

Create a base implementation of a
jobboard which serves as a place to post
work to be done, a place to get notified of
new work, and a place which can be used to
atomically acquire that work (so that it
can be worked on) as well as transfer that
work from one entity (say when that entity
fails) to another entity (for further
resumption or other policy/code driven
recovery processes).

Implements: blueprint job-reference-impl

Change-Id: I1de1525df0deee612fb14ca36f0415ea7d2f707c
Fixups for threads_count usage and logging

In the cases where threads_count is not applicable (when a
custom executor is provided) we should not log a message that
says the number of threads being used (since it will likely
not be an accurate message of what the provided executor is
really using).

Change-Id: I092c69716f300b3ac95cc320c96243966eb982ba
Rename uuid to topic

This appears to be the topic and not a uuid.

Change-Id: I191fcef543aa0879913fd965ebef0496b0163336
Merge "Add zookeeper job/jobboard impl"
Updated from global requirements

Change-Id: I9476a3a6b80bb35b2f507d27f77b7e77fabf14a0
Fix try_clean not getting the job_path

When a posting failure happens and an exception
is raised the try_clean method is activated to
remove the posting in progress from the known
jobs, but only if we pass in the path to remove
which we weren't.

Change-Id: Ic697673e1bdee58a299b4cda8669a64a958f9c42
Clean-up several comments in reflection.py

Change-Id: I98756b4eb180f40a9f75511399f9821fd19fac8f
Get rid of openstack.common.py3kcompat

py3kcompat was removed from oslo-incubator because everything it
provided was already in six version we depend on (see oslo-incubator
change I06b90f789ae21f2ef8b8071b4298bfc0406482a6). So we should
use six for urlparse, too.

Change-Id: I0ab7119c985c247c2f701e3bb6b16353cf7399f0
Allow connection string to be just backend name

Having to specify backend connection as URI is somewhat confusing
when other parts of uri (like host or path) are not used. This commit
allows to specify just backend name as connection string.

Partially implements blueprint persistence-uris

Change-Id: Ic38d41d4f24dcd596cbdff33de78d1a137fb2e8f
Add a delay before releasing the lock

To avoid when the lock gets released and the
time to acquire is below the float tolerance
add-on a sleep to ensure that there is always
some variation in timing before the acquisition
by another thread.

If the cause of this bug is a time shift (ntpd?)
then this will not be a fix (a fix for that is
only available in py3.3).

Fixes bug 1288521

Change-Id: I3294e81ce043a2c91730bb614020bb79c867dcdb
Merge "Fixups for threads_count usage and logging"
Merge "Rename uuid to topic"
A few worker-engine cleanups

- Remove usage of % with logs, just let the underlying
  LOG do the interpolation for us if the log level passes.
- Use LOG.exception instead of LOG.warn and then including
  the exception as a interpolated variable. Just let the
  LOG.exception output the exception trace and info.
- Capture the failure object before doing other work using
  a new context manager that can yield back a failure object
  before further code is called (to avoid a context switch
  happening in eventlet or elsewhere that would clear the
  exception state).

Change-Id: Ic1a504ba69c56ba226b1a067b5ade4590c245dd8
Merge "Fix try_clean not getting the job_path"
Merge "Allow connection string to be just backend name"
Worker-based engine clean-ups

* Auto-delete queues, redundant proxy logic removed;
* Updated unit tests;
* Small fixes in comments.

Change-Id: I8d6fdbf06625daad7a886ff87607b31f369062b8
Introduce remote tasks cache for worker-executor

Updated WorkerTaskExecutor to use cache for remote tasks.

Change-Id: I4572052f63647472367cb69fc02911bbec2bd4cc
Correct LOG.warning in persistence utils

Change-Id: Ibfd0bd8ce452810161d25d3ac5441eb9ed1e0e13
Move endpoint subclass finding to reflection util

The endpoint finding uses a utility method that can
be generalized into finding subclasses of a given list
of items, so instead of having this logic be in the worker
just move it to reflection instead.

Change-Id: Id79efd9514f7b2f7713a1426a7e6ed0861ecee70
Merge "Add a delay before releasing the lock"