jclouds/jclouds — pull request #1391 by shrinandj (2013-03-07 19:54:26+00:00)
Title: Use multi-object delete for AWS-S3.

This commit implements a clearContainer strategy (as well as a clearList strategy) specifically for AWS-S3. It uses the multi-object delete option already exported by the S3 api. This way, 1000 blobs can be deleted with one single http request.

Also added a live test for making sure that clearContainer works.

There can be tricky situation when deleting multiple blobs together. Consider the case where we have a folder and some blobs inside it. Since a folder is nothing but a blob whose name has a trailing slash, we can include the folder in the list of blobs to be deleted. But what if the folder got deleted but some blob inside the folder did not? For e.g. let's say we have a folder called "someDir" and a file inside it called "someDir/file". "someDir/" can get deleted but "someDir/file" can fail for whatever reason.

1. If we simply retry deleting "someDir/file", it should be just fine and the delete can succeed (which is what happens today).

2. If we login to the AWS console when this happens, it shows a folder named "someDir" and "file" inside of it. If we delete "file", "someDir" also goes away. This is expected since the only remaining blob was "someDir/file".