python3: Introduce py33 to tox.ini

Introduce py33 to tox.ini to make testing with
python3 easier.

Change-Id: Id5c76d58551af4671f30ace85bdbbf5803a58368
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Treat directory objects as not found

Make StaticWeb search for an index file rather than returning a 0-byte
object.

Add new StaticWeb header to identify a directory marker object:
X-Container-Meta-Web-Directory-Type (default to application/directory)
so user can specify the content type for directory marker objects.
This required requesting the container headers earlier in the code and
clearing the value of _error for non-404 errors.

Add unit tests for directory object scenarios.
Fix end tags in an unrelated HTML block.

Bug 1178817

DocImpact

Change-Id: I561f00c099feaa82fd658f5050bd90c76717da24
Add UT for checking special chars in object name

Add special chars could help checking quote/unquote, encode/decode
problems. Those two is one of most common mistakes.

Change-Id: Ife1c0b481f08c1666d62b4fb51b7fdcdfdbf2ba6
Fix ring validate with device prior to rebalance.

A builder file with some devices added but before rebalance() is called
should be invalid (in the sense that validate() raises
RingValidationError instead of TypeError).

Change-Id: I5538239db4b2fde83be2390014504e35ccd7c0d3
Swift tests for unicode headers

Swift functional tests sends ilegal headers (unicode) and expects
these headers to be processed by Swift.

Apache2 seem to remove such headers which is a sound behavior for
illegal headers.

This change restrict the test tpo exist only for swift integral
web front end. For future it is advised to disallow such behaviour.
For example it is suggested to make the decission that in future APIs
illegal headers are removed rather than processed.

Change-Id: Ibf548f474de3f06a2687f8f4812beaeaf81ab216
Fixes: Bug #1189123
Add branch coverage reporting

Except where the python-nose package version does not support the
"--cover-branches" option.

Change-Id: I9ea6452d9834b3ac414e14e628002f593dd4b8c8
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Add "normal", optparse-style options to swift-ring-builder add.

The old format is still present and works just like it did before, so
your existing scripts won't break.

New format pros:
 * it's readable even for Swift newcomers
 * it's easy to extend
 * it's familiar to anyone who's used a Unix command line
 * we don't have to maintain the parser

New format cons:
 * you can't add multiple devices in one go

Old format pros:
 * you can add many devices with one command
 * it's compact

Old format cons:
 * it confuses newcomers
 * "wait, is that zone dash IP colon port slash device, or zone slash
   IP dash port colon meta underscore device?" Just try walking
   someone through adding a device over voice chat.
 * it's annoying to add new fields

Note that this only affects the command "swift-ring-builder
<builderfile> add". Other swift-ring-builder commands are unchanged.

DocImpact

Change-Id: I034b7f79eb6f4d81a5c4da193e1358741441c5b5
Stop getting useless bytes on manifest Range requests.

If you make a range request for a manifest object, then the proxy can
retrieve more data than it actually needs from the object server. It
sends the right bytes to the client, and then spends a bunch of time
reading and discarding bytes it doesn't need.

Example:

   file start  	       	       	       	       	       file end
       |	range start	  range end		  |
       v	     v         	       v       	       	  v
       [--------------------------------------------------]

The proxy would send a request to the object server with "Range:
bytes=${range start}-", and so the object server responds with all the
bytes from range start to file end. This needlessly transfers all the
bytes from range end to file end, as none of those are sent to the
client.

This commit fixes the sent Range header to only request the bytes
needed.

Change-Id: Ie8d4ee2c5e67c08795ac2d39d1df7d11856621a9
Fix formpost with queries without user_agent.

- Fixes bug 1190220.

Change-Id: I3d46cc8f26ce9970dacb82bc5243b3528788c785
Fix 503 on account/container HEAD w/invalid format.

A HEAD request to /v1/a[/c]?format=%FF would result in a 503 since
there was an unhandled UnicodeDecodeError. Now it doesn't.

bug 1190395

Change-Id: I4d8ec677092617391fe154a25f22d919536a72a5
Refactor DiskFile instantiation in object server.

DiskFile() takes a lot of arguments whose value is the same on every
instantiation, so I pulled that stuff out into a utility method. No
functionality change is intended.

Change-Id: I259df5602bed60d6cb48e07d6d9e58e8320ee12a
Improve SAIO deploy document.

improving points:
1. Remove yum install swift in Fedora; Use installing from source for
both Ubuntu and Fedora.
2. Explain you could use all users including root, your own guest. An
d the points developer have to care.

Change-Id: Id6d683441bd790a21734624e29eb7c98bb40de85
Fixes: bug #1126389
Improve swift's keystoneauth ACL support

Below three bug reports talk about one thing.
Current keystoneauth ACL supports as:

tenant_name:user_id         ok
tenant_name:user_name       no
tenant_name:*               no
tenant_id:user_id           ok
tenant_id:user_name         no
tenant_id:*                 no
*:user_id                   ok
*:user_name                 no
*:*                         no

This patch will make all of above work fine.
Applying (tenant/user)name could let user put or get their data in a
more readable way. The tenant_name:* and *:user_name is suitable for
many usage.

note: to keep compatibility here add a new keystone.identity just for
authorize() itself and leave env['keystone.identity'] to other
middlerwares.

Fixes: bug #1020709
Fixes: bug #1075362
Fixes: bug #1155389
Change-Id: I9354dedaad875117f6a9072c67e9ecf69bfca77e
Merge "Add UT for checking special chars in object name"
Merge "Improve SAIO deploy document."
Merge "Fix formpost with queries without user_agent."
Added logging of x-delete-at values

Added logging of the x-delete-at value for object PUTs and POSTs and
made the additional log field generic for other similar use cases.

Logging the x-delete-at is useful because otherwise you might never
know why an object "disappeared" just by looking at saved proxy logs.
You'd see a successful PUT of the object followed some time by a GET
that 404s. If you still have the other logs, you could track down
what happened; but, at least with our clusters, we keep proxy logs
forever but only keep the other logs for a short while.

Change-Id: Ida559abcf8dc5666e2a7eea250a67d1c4b28b1be
Checked user input in XML responses.

Fixes bug 1183884.

* swift/account/utils.py: Escape account name in XML listings.

Change-Id: I2392d012ddeec05a267c3dcf14748112316096f3
Merge "Added logging of x-delete-at values"
Remove Lucid/ppa instructions from SAIO guide

Lucid won't EOL until May of 2014; but I stopped trusting that ppa a long time
ago. Besides with the requires for dnspython and mock where they're at you
almost can't install swift from source on any stock distro and expect tests to
pass with system packages - so we're looking at pypi for depends regardless.

While I'm in there:

* more explanation of <your-user-name> and a helpful find/sed for configs
* group the "setup ~/.bashrc" stuff with the "setup ~/bin" stuff
* some updates/fixes from my experience installing on CentOS
* remove region warnings from remakerings

Change-Id: Ie2e6b06959ab699d853e07e5b7e8cda7036a44fe