Add zookeeper job/jobboard impl

Create a base implementation of a
jobboard which serves as a place to post
work to be done, a place to get notified of
new work, and a place which can be used to
atomically acquire that work (so that it
can be worked on) as well as transfer that
work from one entity (say when that entity
fails) to another entity (for further
resumption or other policy/code driven
recovery processes).

Implements: blueprint job-reference-impl

Change-Id: I1de1525df0deee612fb14ca36f0415ea7d2f707c
Unify usage of storage error exception type

- Instead of formatting the exception by creating
  a storage error instance we can just let the already
  included cause make this formatting occur.
- Handle cases in impl_dir which were not catching and
  raising storage errors.
- Unify usage of 'backend internal error' messaging.
- Pass the cause to storage errors where applicable.

Change-Id: Ieb14256b202ccbfa3a96f68fa35db7e40f92114d
Use the stop watch utility instead of custom timing

Instead of retaining a internal expiration and timing
functionality in the remote_task, just use the stop
watch helper class already existing in the misc utility
module.

Change-Id: Ie50c79ee6a5cff16c52a0fae0a65544a0d4202e1
Fixups for threads_count usage and logging

In the cases where threads_count is not applicable (when a
custom executor is provided) we should not log a message that
says the number of threads being used (since it will likely
not be an accurate message of what the provided executor is
really using).

Change-Id: I092c69716f300b3ac95cc320c96243966eb982ba
Rename uuid to topic

This appears to be the topic and not a uuid.

Change-Id: I191fcef543aa0879913fd965ebef0496b0163336
Merge "Use the stop watch utility instead of custom timing"
Merge "Unify usage of storage error exception type"
Merge "Add zookeeper job/jobboard impl"
Updated from global requirements

Change-Id: I9476a3a6b80bb35b2f507d27f77b7e77fabf14a0
Fix try_clean not getting the job_path

When a posting failure happens and an exception
is raised the try_clean method is activated to
remove the posting in progress from the known
jobs, but only if we pass in the path to remove
which we weren't.

Change-Id: Ic697673e1bdee58a299b4cda8669a64a958f9c42
Clean-up several comments in reflection.py

Change-Id: I98756b4eb180f40a9f75511399f9821fd19fac8f
Get rid of openstack.common.py3kcompat

py3kcompat was removed from oslo-incubator because everything it
provided was already in six version we depend on (see oslo-incubator
change I06b90f789ae21f2ef8b8071b4298bfc0406482a6). So we should
use six for urlparse, too.

Change-Id: I0ab7119c985c247c2f701e3bb6b16353cf7399f0
Allow connection string to be just backend name

Having to specify backend connection as URI is somewhat confusing
when other parts of uri (like host or path) are not used. This commit
allows to specify just backend name as connection string.

Partially implements blueprint persistence-uris

Change-Id: Ic38d41d4f24dcd596cbdff33de78d1a137fb2e8f
Add a delay before releasing the lock

To avoid when the lock gets released and the
time to acquire is below the float tolerance
add-on a sleep to ensure that there is always
some variation in timing before the acquisition
by another thread.

If the cause of this bug is a time shift (ntpd?)
then this will not be a fix (a fix for that is
only available in py3.3).

Fixes bug 1288521

Change-Id: I3294e81ce043a2c91730bb614020bb79c867dcdb
Merge "Fixups for threads_count usage and logging"
Merge "Rename uuid to topic"
A few worker-engine cleanups

- Remove usage of % with logs, just let the underlying
  LOG do the interpolation for us if the log level passes.
- Use LOG.exception instead of LOG.warn and then including
  the exception as a interpolated variable. Just let the
  LOG.exception output the exception trace and info.
- Capture the failure object before doing other work using
  a new context manager that can yield back a failure object
  before further code is called (to avoid a context switch
  happening in eventlet or elsewhere that would clear the
  exception state).

Change-Id: Ic1a504ba69c56ba226b1a067b5ade4590c245dd8
Merge "Fix try_clean not getting the job_path"
Merge "Allow connection string to be just backend name"
Worker-based engine clean-ups

* Auto-delete queues, redundant proxy logic removed;
* Updated unit tests;
* Small fixes in comments.

Change-Id: I8d6fdbf06625daad7a886ff87607b31f369062b8