Do unwatch in watch_notify test so clean-up can remove objects without EBUSY

Signed-off-by: David Zafman <dzafman@redhat.com>
Fix test clean-up code to make sure ioctx is on SNAP_HEAD
Fix remaining test types to save and clean-up test namespace
Fix ceph_test_rados_api_tier so that clean-up of objects works
Do cleanup of base and cache pools after disconnecting tier
Remove snapshots at end of test so cleanup works
Add missing wait_for_latest_osdmap()

Signed-off-by: David Zafman <dzafman@redhat.com>