object-storage: minor code cleanups

Apply a few minor code cleanups to help reduce future refactoring diffs.

  * Remove extra lines between methods so that all methods are consistently
    spaced by one line

  * Remove unused field meta_file

  * Add missing documentation for uid and gid parameters

Change-Id: I0a38eb06680eb0709edfaa1828d09657823f778e
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4166
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: add missing docs for params

Add missing documentation for constructor parameters.

Change-Id: I3a656cd22f6a2a61e0b4d563c75c2f7fe68d3c54
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4167
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: unused parameter cleanup & handl'n

Unsed parameter cleanup and handling changes. The device and partition
parameters are not used. The account parameter is used in place of the device
parameter. Rather than just making the device parameter reference the value of
the account parameter, we just don't use the device parameter at all and make
references to account. We also remove the device_path field from the class
since it is unused.

Additionally, we assert that we have a logger since the rest of the relies on
that fact.

Change-Id: Iea556319744ab4f729b1ce23d987650a164749db
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4168
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Tested-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
object-storage: rename self.name to self.container

Minor refactoring ahead of further diffs to add clarity to the code.

Rename the DiskDir field from "name" to "container", which is what it is.

Along the way, add a note regarding the one-to-one mapping of account name to
gluster volume name, and fix the get_info() container method documentation to
list all the dictionary keys returned.

Change-Id: I659e1a9a070c16671287f17a170f9d77198534e7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4169
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: fix bad refs to REMOTE_CLUSTER

Change-Id: I07cd997ef9a98645a87792632f9dd612928678b7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4170
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: fix bad refs to check_acct_exists

Fix bad references to check_account_exists: this is a module local method, not
provided by Glusterfs. Not sure how we missed this. More unit tests would
help.

Also added the ability to run the Swift tree unit tests with gluster
artificially enabled to test as many code paths as possible ahead of full
functional tests.

Change-Id: Iaf696d34db8f841c887f03569e65321fca8894d1
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4171
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: DiskAccount subclassing of DiskDir

Changes to make DiskAccount subclass DiskDir complete. This change helps
reduce the diffs ahead of further refactoring. Note that DiskDir initializes
all the same fields as DiskAccount, plus a few more, and ends up reading the
metadata properly, while only adding a couple of fields not used for account
purposes.

This change also passes the account controller's logger for use by the
DiskAccount class.

Along the way we:

    * Remove the redundant update_account() method in the DiskCommon base
      class, since the update_container() method in DiskDir performs the exact
      same functionality

    * Remove the initialization of variables ahead of assignments in two
      cases, and also move the initialization of the list variables closer to
      where they are used

    * Collapse the imports from swift.plugins.utils into one statement

    * Use a temporary directory for the db_file.db when we find ourselves in a
      unit test environment

Change-Id: I4dc59b416a66393adfe0ed32c7a62c79731d70c0
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4172
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Tested-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
object-storage: only update metadata that changes

See BZ 868120 (https://bugzilla.redhat.com/show_bug.cgi?id=868120)

Change-Id: Ia7f5562cff32eee30c5ca18652d5d9a5a2b18bf4
BUG: 868120
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4173
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: remove the device and part params

Change-Id: I60b4b1b976ad8359ba072d5d0ed6a8dd2087a738
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4174
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: fix trailing whitespace

Change-Id: Ib3c671e693c2c332af98a593ca14c42c36f5ca76
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4175
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: swift-diff removal: hide obj only

Towards swift-diff removal: hide object_only behavior inside DiskDir class
objects. See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589)

One of the last few reasons for having the swift diff is the need to update
the account and container information from the file system when the
configuration is not for object-only use (meaning, other updates to the file
system can come from clients other than UFO, which means UFO currently has to
generate the necessary metadata for each request instead of caching it).

This change moves that logic into the DiskDir class methods so that the
container and account server code does not have to track that.

Change-Id: If0970c77c72245facd09c2035083e681d6d80303
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4176
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
object-storage: reduce diffs, bury metadata update

Further reduce the diffs we have to carry by moving the Gluster specific
metadata stored when we create an object to the DiskFile class's put() method.

Change-Id: I8cef5412627d3ddfdadbf78ef947e067db5bf620
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4177
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
object-storage: make proper refs to module vars

Not sure how this worked before this, but mount_ip and mount_path do not
exist, their uppercase versions do.

Change-Id: I53e9b550991d17b5541ed4ada5f1bc9b3cb1b8b1
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4178
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: refactor to use swift devices

Refactor code to use the devices configuration file setting for account,
container, and object servers, dropping mount_path from the fs.conf file, and
constructing new account, container and object server rings.

This removes the next to last set of diffs in the swift diff file. See BZ
870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).

The key to the change is the dropping of the pre-built account, container and
object rings, instead providing a script that will generate them for the user
given the gluster volume name in use. In addition, we override the Swift
check_mount() method and replace it with Gluster's which attempts to
"auto-mount" if it is not already mounted.

The following is an enumeration of the changes contained in this refactoring:

    * The refactoring to override the Swift check_mount() involved condensing
      a lot of code, removing redundancies and simplifying methods across a
      number of modules

    * Drop checking the mount point in the low level DiskDir, DiskAccount and
      DiskFile objects now that Swift's normal mount checking is used, and
      enable it by default in the template configuration file

    * Add missing get_container_timestamp() method for DiskAccount objects

    * Fix the plugin RPM spec file to provide the new ring builders, and while
      we were at it, finally fix the over-writing of the configuration files
      on install

    * Bug fixes

        * The monkey patched version of check_object_creation was not working
          correctly due to a missing import of HTTPBadRequest and a bad
          reference to validate_obj_name_component()

        * Only have the utils module import the plugins constraints module for
          the side effect of monkey patching if gluster is enabled

        * Removed the db_file.db file in the tree since it is created on the
          fly

        * Updated 2011 copyright notices to 2012

Change-Id: I8f4454576b1423021c9bbf3c36176f8db51e62c0
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4179
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
object-storage: final changes to remove swift diff

Final set of changes to remove the diffs carried to make UFO work with
OpenStack Swift. The code is now a complete layering on top of OpenStack Swift
where we either "monkey patch" or subclass as necessary.

See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).

There are a lot of changes here due for the most part to rearranging the
directory hierarchy to have create a proper python module hierarchy under the
"gluster" namespace. Plugin references have been removed. The differences that
used to be in the swift.diff file are now replaced with server implementations
for account, container, object, and proxy that subclass the swift versions.

Additionally, the plugins/conf directory has been moved to the "etc"
directory, and the plugins/bin directory promoted a level.

Unit tests pass.

A new setup.py file is provided so that the install process can use it for
creating all the necessary python install infrastructure (eggs and paste
support).

A new RPM spec file is provided which to properly install the new code, and
the sample configuration files have been modified to reference the new python
egg.

Change-Id: I4316c1b66dca80f847fe9b0d583174689c175599
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4180
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>