LP-1391914 :: idPrefixWatcher; subtle bug with initial call to Changes()

Failing test to demonstrate bug
Rename test to be more descriptive
Merge pull request #1124 from wallyworld/swift-empty-containername

Abort Remove() if swift container name is empty

Fixes: https://bugs.launchpad.net/bugs/1312217

There have been a couple of goes at this fix. This version does not require any lp.net/goose changes.

From the original PR:

Hi,

I have discovered that through a series of unfortunate events involving a failed destroy-environment run (which I believe was due to a slow Swift API response), and a subsequent "working" (ish) destroy-environment run it is possible for Juju to erroneously send a DELETE HTTP request to the swift account API, which in older versions of swift (and keystone) will result in the account DB for that user in swift being marked as deleted (and the only fix for this is to either a) wait for a cron'd script to clean it up, or b) get someone to manually delete the sqlite3 db for that account hash).

Although I've been unable to find the condition in the environment (e.g. writing/loading the jenv, doing the Remove*(...) calls etc.), but I'm 99.9% recurring that this is the cause for the issue I've had with my Swift account getting into a hard-to-recover state

I've added a simple check to provider/openstack/storage.Remove(...) to ensure the containerName isn't empty (which on a call to delete the container itself, the last step in a destroy env cleanup, will result in a DELETE call to the base URL instead, which swift+keystone will gladly oblige in some OpenStack setups).

I couldn't figure out how to test this correctly, due to the way the Storage instance is created/used in the openstack tests, so any pointers on this would be appreciated.

I feel it's more important to catch this nasty edgecase here above finding it's cause in the env handling purely because it is so destructive in the openstack provider compared to others (e.g. you can't remove the base in S3 so easily).
Merge pull request #1117 from johnweldon/LP-1391914

LP-1391914 :: idPrefixWatcher; subtle bug with initial call to Changes()

Failing test to demonstrate bug + Fix
Merge pull request #1125 from wallyworld/swift-empty-containername-master

Merge pull request #1124 from wallyworld/swift-empty-containername

Abort Remove() if swift container name is empty

Fixes: https://bugs.launchpad.net/bugs/1312217

There have been a couple of goes at this fix. This version does not require any lp.net/goose changes.

From the original PR:

Hi,

I have discovered that through a series of unfortunate events involving a failed destroy-environment run (which I believe was due to a slow Swift API response), and a subsequent "working" (ish) destroy-environment run it is possible for Juju to erroneously send a DELETE HTTP request to the swift account API, which in older versions of swift (and keystone) will result in the account DB for that user in swift being marked as deleted (and the only fix for this is to either a) wait for a cron'd script to clean it up, or b) get someone to manually delete the sqlite3 db for that account hash).

Although I've been unable to find the condition in the environment (e.g. writing/loading the jenv, doing the Remove*(...) calls etc.), but I'm 99.9% recurring that this is the cause for the issue I've had with my Swift account getting into a hard-to-recover state

I've added a simple check to provider/openstack/storage.Remove(...) to ensure the containerName isn't empty (which on a call to delete the container itself, the last step in a destroy env cleanup, will result in a DELETE call to the base URL instead, which swift+keystone will gladly oblige in some OpenStack setups).

I couldn't figure out how to test this correctly, due to the way the Storage instance is created/used in the openstack tests, so any pointers on this would be appreciated.

I feel it's more important to catch this nasty edgecase here above finding it's cause in the env handling purely because it is so destructive in the openstack provider compared to others (e.g. you can't remove the base in S3 so easily).