ceph_test_rados_api_tier: add EvictSnap2 test case

Verify an evict doesn't create a snapdir object.  Reproduces #8629

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: do not clone or preserve snapdir on cache_evict

If we cache_evict a head in a cache pool, we need to prevent
make_writeable() from cloning the head and finish_ctx() from
preserving the snapdir object.

Fixes: #8629
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #2550 from ceph/wip-8629

osd: fix cache_evict vs make_writeable/finish_ctx snapdir bug #8629

Reviewed-by: Samuel Just <sam.just@inktank.com>