client, ObjectCacher: drop rx buffers on release

Optionally drop rx buffers (along with clean ones) in teh ObjectCacher
on release.

Update the Client callers to set this flag.  This allows us to make sure
that RX buffers from readahead that we never waited for are discarded
when the final inode reference is destroyed in put_inode().

Note that other ObjectCacher users that do not submit reads they don't
wait ought not to have this problem because they will not try to release
the (rx) buffers in that case.

This fixes a race like so:

 - read_async
    - submit some readahead too
 - read completes
 - file is closed
 - put_inode drops the last ref
   - relase_set returns unclean > 0, and we get a failed assertion
 (due to the RX buffer from the readahead)

Fixes: #7867
Backport: emperor, dumpling
Signed-off-by: Sage Weil <sage@inktank.com>