add --files-from=FILE to allow transfer of select files only

This solves the change of behavior introduced by processing
excludes/includes during os.walk(), where previously:

s3cmd sync --exclude='*' --include='*.gpg'

would walk the whole tree and transfer only the files named *.gpg.

Since the change to os.walk(), the exclude '*' matches everything, and
nothing is transferred.

This patch introduces --files-from=FILE to match rsync behaviour,
where the list of files to transfer (local to remote) is taken not
from an os.walk(), but from the explicit list in FILE.

The equivalent for remote to local, and remote to remote, is not yet
implemented.
accept --files-from=- to read from stdin

This allows shell syntax:

find . -name \*.gpg | s3cmd sync --files-from=- src dst

to take the list of files to transfer from stdin.

Be careful, as using with a --delete option will cause files on the
remote side not listed in stdin to be deleted too.
add --files-from to manpage
use --files-from only on local source (not remote, not dest)

The restriction to use --files-from on the source and not the
destination list comes from the equivalent behavior in rsync.

The restriction on not using it for remote sources is only because I
haven't figured out the best way to handle that.  That may be added in
the future.
Add option to ignore error on copy when key no longer exists
fixing the order of public URL components
Fix crash when cloudfront operations receive 500 errors.
Retries parameter was not being sent to send_request as a keyword arg.
Apply include/exclude to destinations too, via mdomsch/s3cmd@268e7c3bd37cf60920025d556449fc1b93a1c590 Refs s3tools/s3cmd#109.
Windows fix re / vs os.path.sep

Even on Windows where os.path.sep is backslash, forward-slash is the S3 separator character.
add --max-delete for del and sync commands, fixes bug #155

Fixes https://github.com/s3tools/s3cmd/issues/155

Similar to rsync --max-delete, allow the user to specify a maximum
number of objects to delete for the [del] and [sync] commands.  If
that number would be exceeded, output a warning message and do not
delete anything.  This prevents a simple invocation error
(e.g. launching from the wrong directory) from erasing all the data in
a bucket accidentally.

By default, this is disabled.  A user must explicitly specify
--max-delete=NUM for this protection to be available, just as in
rsync.
sync: ignore --delete-removed if no source files, bug #155

https://github.com/s3tools/s3cmd/issues/155

If --delete-removed is specified, yet no source files were found, all
files at destination are deleted.  This is more likely a user error
than the desired outcome.

This patch adds a check to [sync] for both --delete-removed and no
source files, and refuses to delete on the destination in this case.
The user can specify --force to force deletion if that is really what
is intended.
provide a better exception / error reporting message, bug #158

Trying to cut down on the duplicates to the -bugs mailing list.
Reverted "/usr/bin/python" -> "/usr/bin/env python"

Partially reverted af984ba9
Relaxed timeout for http connection during IAM role lookup
Fix printing error in `report_exception(e)`

Fixes `TypeError: unsupported operand type(s) for %: 'NoneType' and 'str'` arising from using string formatting with the /result/ of sys.stderr.write() instead of the argument to it.
add --xattr option referring to file system extended attribute containing md5sums

Inspired by a patch from Jay McCanta <J.McCanta@F5.com> who suggested
this was possible, 2013-05-14 to the s3tools-general mailing list.

This patch adds the --xattr command line and md5_xattr config file
option, specifying a file system extended attribute in the USER
namespace which contains the md5sum for the file.  If the extended
attribute is not present, the code falls back to performing local file
I/O to calculate the md5sum, as before.  This can be used to speed up
transfers by avoiding the need to calculate md5sums for all files,
assuming this has been done at least once before somehow, with the
results stored in each file's xattr.

In addition to Jay's work, this patch simplifies testing for xattr
module import success, passes the result into the Config object,
and ensures the correct namespace is used.  Also added a test to the test
suite, and the entry on the manpage.
fix conditional import xattr module
push md5_xattr to Config only if xattr module was imported

Otherwise options.md5_xattr does not exist.
Don't emit "cowardly" message unless we have source files & delete files

We were looking at the wrong (post-compare) list lengths, rather than
the original (discovered by walk) list lengths.  And we weren't
checking to see if we had any files to delete, or not, before issuing
the "cowardly" message.  So for a "null" (no files to transfer) sync
run, we would emit the message.  That's wrong.

This patch fixes both problems:
1) it uses the discovered source file count.  If nonzero, we will not
emit the message.

2) it looks at the count of files to delete.  If zero, we will not
emit the message.
Get md5 from remote attribute header in recursive syncs too

We were not getting the md5 value from the attribute header of each
file for the recursive case, only for the non-recursive case.  That
was stupid.  We were certainly _setting_ the md5value into the
attribute header.

This should eliminate the
INFO: disabled md5 check for...
messages on subsequent syncs when s3cmd 1.5.0-alpha+ is used to upload
content in the first place.