Refactor on-disk knowledge out of object server

Refactor on-disk knowledge out of the object server by pushing the
async update pickle creation to the new DiskFileManager class (name is
not the best, so suggestions welcome), along with the REPLICATOR
method logic. We also move the mount checking and thread pool storage
to the new ondisk.Devices object, which then also becomes the new home
of the audit_location_generator method.

For the object server, a new setup() method is now called at the end
of the controller's construction, and the _diskfile() method has been
renamed to get_diskfile(), to allow implementation specific behavior.

Change-Id: Id945276037dbbeb8a778753e47a7316cd408eac7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Use new ondisk.Devices where possible

We also remove the unused mount_check attribute from the recon
middleware.

Change-Id: Icc593fd548a6e2d9bc5c8178f78424b1d6473f05
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Refactor DiskFile hiding quarantine adding reader

This set of refactorings is proposed as the last step before
officiallly declaring the DiskFile class, and its helpers, to be the
backend API servicing the Object Server REST API implementation code.
The goal is to ensure that the REST API layer in the object server
does not have any access to internal knowledge of how the DiskFile
class is implemented.

We hide the need for the REST API layer to know how and where
quarantining needs to be performed. There are now two places it is
checked internally, on open() where we verify the content-length,
name, and x-timestamp metadata, and in the reader on close where the
etag metadata is checked if the entire file was read.

We add a reader class to allow implementations to isolate the WSGI
handling code for that specific environment (it is used no-where else
in the REST APIs). This simplifies the caller's code to just use a
"with" statement once open to avoid multiple points where close needs
to be called.

For a full historical comparison, including the usage patterns see:
https://gist.github.com/portante/5488238

(as of master 8a255a3 Merge
 "Refactor finalize_put as an    This Commit
 object method")
--------------------------------+------------------------------------
                                 DiskFileManager(conf)

                                   Methods:
                                     .pickle_async_update()
                                     .get_diskfile()
                                     .get_hashes()

                                   Attributes:
                                     .devices
                                     .logger
                                     .disk_chunk_size
                                     .keep_cache_size
                                     .bytes_per_sync

DiskFile(a,c,o,keep_data_fp=)    DiskFile(a,c,o)

  Methods:                         Methods:
   *.__iter__()
    .close(verify_file=)
    .is_deleted()
    .is_expired()
    .quarantine()
    .get_data_file_size()
                                     .open()
                                     .read_metadata()
    .create()                        .create()
                                     .write_metadata()
    .delete()                        .delete()

  Attributes:                      Attributes:
    .quarantined_dir
    .keep_cache
    .metadata
                                *DiskFileReader()

                                   Methods:
                                     .__iter__()
                                     .close()

                                   Attributes:
                                    +.was_quarantined

DiskWriter()                     DiskFileWriter()

  Methods:                         Methods:
    .write()                         .write()
    .put()                           .put()

* Note that the DiskFile class   * Note that the DiskReader() object
  implements all the methods       returned by the
  necessary for a WSGI app         DiskFileOpened.reader() method
  iterator                         implements all the methods
                                   necessary for a WSGI app iterator

                                 + Note that if the auditor is
                                   refactored to not use the DiskFile
                                   class, see
                                   https://review.openstack.org/44787
                                   then we don't need the
                                   was_quarantined attribute

Change-Id: Iefbbe7aada972b1ff2fc9fd5492646ab01fa192c
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Document DiskFile API, provide reference impl

Add sphinx documentation for the DiskFile pluggable backend API. We
also updated references to the other account and container backend
docs.

** We have removed the "upload_size" attribute of the DiskFileWriter,
   changing it to be a return value of the "write()" method instead
   (one step simpler API).

As part of documenting the interface, all of the attributes have had
underscores ("_") appended to define them as internal to the reference
implementation.

A reference "in-memory" object server implementation of a backend
DiskFile class in swift/obj/mem_server.py and
swift/obj/mem_diskfile.py.

One can also reference
https://github.com/portante/gluster-swift/commits/diskfile for the
proposed integration with the gluster-swift code based on these
changes.

Change-Id: I44e153fdb405a5743e9c05349008f94136764916
Signed-off-by: Peter Portante <peter.portante@redhat.com>