librbd: flush cache when set_snap() is called

If there are writes pending, they should be sent while the image
is still writeable. If the image becomes read-only, flushing the
cache will just mark everything dirty again due to -EROFS.

Fixes: #4525
Backport: bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Merge pull request #163 from ceph/wip-rbd-snapset-cache

librbd: flush cache when set_snap() is called

Reviewed-by: Sage Weil <sage@inktank.com>