Add per policy container count to account stat output

When the account response includes 'X-Storage-Policy-X-Container-Count' the
command helpers will stick a helpful extra line in the output.

If the swift response doesn't include the information the behavior is
unchanged.

Change-Id: Iaaa55d207ebe6a9c16e52adb56eb858c57cfb26a
Fix unit tests failing when OS_ env vars are set

tests/unit/test_shell.py:TestParsing tests can fail if
there are OS_* variables set in the environment. There
is already code in the test setUp to remove ST_* variables,
so we should do the same for OS_* variables.

This patch also changes the mechanism used to remove and
then restore any unwanted variables found in os.environ.

The existing setUp() takes a copy of os.environ and then
deletes any ST_* variables in the original. In tearDown() it
sets os.environ as the copy. However, the environ imported
into shell.py remains pointing to the original os.environ
object. So after the first call to tearDown, subsequent
mocking of os.environ has no effect on shell.environ.
This renders some of the tests ineffective e.g.
test_insufficient_env_vars_v3 is not actually setting any
vars in shell.environ.

The issue can be provoked by repeating a test:
nosetests -w tests/unit/ test_shell.py:TestParsing.test_args_v3 \
 test_shell.py:TestParsing.test_args_v3

The test will pass first time and fail second time.

Change-Id: I5d100f81115e74878d510326acb5777e6a3626c8
Remove a debugging print statement

This patch removes a debugging print statement that slipped into
service.py in patch https://review.openstack.org/#/c/85453/

Change-Id: I80eee49eaa74b52e9c36f00669206aa7b2cdec9b
Merge "Add per policy container count to account stat output"
Merge "Fix unit tests failing when OS_ env vars are set"
Merge "Remove a debugging print statement"
Do not crash with "swift list --lh" for Ceph RadosGW.

If 'x-timestamp' header is not present in 'swift list' result,
date string cannot be correctly generated. Check for correct
exception to catch this case and return a special string value.
This is just a workaround for faulty RadosGW behavior.

Change-Id: I172843a2668e33efac00924906ff0dca3e790050
Closes-Bug: 1349528
Add tests for account listing using --lh switch

Change-Id: I2e68a24212eda463f39a5d0be13f1645ca2a23d8
Use skipTest from testtools instead of inherited Exception

SkipTest is raised if there is no func_test section in the Swift test
config file. However, the currently raised Exception will result in a
failed test, not in a test marked as skipped.

Since we already use testtools we can use the included skipTest easily.

Change-Id: I1bbb1f9dbe31fb0698d774550708d1196b266625
Merge "Use skipTest from testtools instead of inherited Exception"
Allow segment size to be specified in a human readable way.

Instead of always specifying segment size in bytes the user
can now use B,K,M or G as suffixes for the corresponding
size.

Conversion is done with Binary units (1024) rather than SI
units (1000).

e.g

swift upload test_container -S 1073741824 large_file
can now be written
swift upload test_container -S 1G large_file

The change is backwards compatible as it ignores arguments to -S
that don't have a valid suffix.

Updated unit tests and help message.

Change-Id: I6314b4e45cf2fbffde2fe57a02df77a25e911e84
Merge "Allow segment size to be specified in a human readable way."
Fix the info command with --insecure

We forgot to propagate options into http_connection() for the info
aka capabilities command. Fix is to use the proper helper.

Fixes bug: 1381866

Change-Id: I9515203c969a1bba38dd909412355080383905f9
Merge "Fix the info command with --insecure"
Fix race between container create jobs during upload

During a segmented object upload, attempts are made
to create the container for the manifest and the
segment container. These jobs are currently placed
on separate threads and can occur out of order which
causes the unit test failure seen in the bug report.

The container jobs should be ordered because the
segment container job attempts to HEAD the manifest
container to determine correct storage policy.

This patch enforces ordering of the two jobs and
modifies the unit test to assert that both container
PUTs occur in the expected order.

Change-Id: If90bec126867a4be2df34067ccefee660202f59f
Closes-Bug: #1376878
Fix race in shell when testing for errors to raise SysExit

SysExit is raised if the OutputManager has non-zero
error_count. However, error_count is incremented in
a print thread, but tested on the main thread.
Currently error_count is tested before the OutputManager
print threads have exited, which results in a race. This race
means that SysExit is not always raised even when an error
has occured.

This change moves the error_count test after the
OutputManager context manager has exited, which ensures that
the OutputManager threads have exited.

Change-Id: I5ef7d9f10057fe2b41f48ab95066a7265b86a3ac
Closes-Bug: #1379229
Fix KeyError raised from client Connection

Some client.Connection methods will raise a KeyError
if a response_dict argument is passed and an error
occurs during authentication or making the request.

The fix is straightforward: add a test for existence
of a response_dict before attempting to get it from
kwargs.

The bulk of this patch is adding unit tests for the
response_dict feature.

Closes-Bug: 1381304

Change-Id: Ic7e1b3dfae33909533931c52ac97355867a08a07
Replaces Stacktraces with useful error messages.

Changed the message shown when a user doesn't enter project/tenant
id/name to be more informative.

When attempting to stat a container without supplying project/tenant
name or id, an empty response was being returned instead of an error
being raised.

Changed the error raised in swiftclient.client when no tenant or project
is specified to be more specific.

Add tests for basic regression checking.

Closes-Bug: #1372589

Change-Id: I4eb6964d9f1702db119cc0294edc02841b1ecd5f
Make swift post output an error message when failing

swiftclient.shell.st_post was ignoring the result dict
returned from SwiftService.post and therefore giving no
indication when posts failed e.g. due to invalid auth
credentials.

This patch ensures that SwiftService always returns a
result_dict from post() and then checks the result dict.
On failure the shell now outputs the ClientException
message and exits with error code 1.

Also adds unit tests and cleans up some unnecessary mocked
return values in existing tests.

Closes-Bug: 1383243

Change-Id: I111150eb3d026c8d01c2cac13d3613ca7304e5b9
Adds user friendly message when --segment-size is a non-integer

Change-Id: Ic2c062110b2fe2e725b0f4a4517a300cfbf663a8