cli: use WatchedFileHandler to cooperate with logrotate

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
cli: need to import submodule of logging to use it

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Use the object attribute max_entries, and get rid of the argument.
Merge pull request #8 from clewis/respect-max-entries

Respect max_entries.

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
worker: process all bucket instance log entries at once

Currently if there are more than max_entries in a single bucket
instance's log, only max_entries of those will be processed, and the
bucket instance will not be examined again until it is modified again.

To keep it simple, get the entire log of entries to be updated and
process them all at once. This means one busy shard may block others
from syncing, but multiple instances of radosgw-agent can be run to
circumvent that issue. With only one instance, users can be sure
everything is synced when an incremental sync completes with no
errors.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
worker: convert list to tuple so addition to set works

Without this, an error like: TypeError: unhashable type: 'list' would
occur when metadata was retried.

Fixes: #7526
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
sync: add missing parameter to log statement

Fixes http://tracker.ceph.com/issues/7741
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
lock: fix race in background thread when shard_num changes

shard_num can be unset and set to another value if we don't hold
self.lock while depending on its value. This causes a crash if the
log.error() triggers and tries to format None as an int.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
packaging: add spec file

This was used to build the 1.1 rpms

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
packaging: put requirements on separate lines for rpm

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
packaging: add dependency on python-requests

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
packaging: add init scripts

create log, run, and etc dirs as well

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
packaging: add logrotate config

There's no need to signal running agents since they use
a WatchedFileHandler that detects rotation itself.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
debian: depend on python 2.6 or higher

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
cli: add a default log file

Now that the packages include /var/log/ceph/radosgw-agent, default to
storing logs there, named after the configuration file or
radosgw-agent.log if no config file was used.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
packaging: fix up spec file for non-python files

Correct the installation of the binary, logrotate conf, and init script.
Stop using INSTALLED_FILES. Instead explicitly list files and glob the
python package, as recommended by Fedora's python packaging guide.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
debian: fix init script installation

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
logrotate: don't rotate empty logs

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Add --stats arg, that prints out the current state of replication and exits.
Only print out the region map info when debugging.