mds: return ENOSPC on write ops while osds full

Allow removals and read-only ops, prevent others.  This
is a soft policy aimed at reducing the likelihood of a
"full" (mon_osd_full_ratio) becoming physically full
(unable to accept journal writes from MDS even though
it does not respect the FULL flag)

Signed-off-by: John Spray <john.spray@redhat.com>
osdc: add Objecter::op_cancel_writes

This is for use by Client when it encounters
the FULL flag and wants to abort operations
rather than having them pause.

Signed-off-by: John Spray <john.spray@redhat.com>
client: handle ENOSPC in _flush

Previously we would avoid issuing writes
from _write, but the same condition was
not applied to _flush.  This was okay
when we were blocking on ENOSPC, but now
we're cancelling ops on ENOSPC so need
to make sure we don't issue any new ones.

Signed-off-by: John Spray <john.spray@redhat.com>
client: cancel all ops on full flag
osdc/ObjectCacher: invoke flush_set_callback on purge_set

For the benefit of Client, so that it can get its
caps released when using purge_set to deal with
an ENOSPC condition.  Does not affect librbd because
librbd doesn't provide flush_set_callback.

Signed-off-by: John Spray <john.spray@redhat.com>
messages: add osd_epoch_barrier to cap msgs

Extension to client-server protocol to allow clients
to release capabilities conditional on the receiver
having a particular OSD map, and the MDS to issue
caps conditional on the user having a particular
OSD map.

Signed-off-by: John Spray <john.spray@redhat.com>
client: track osd epoch barrier

Signed-off-by: John Spray <john.spray@redhat.com>
mds: implement osd epoch barrier on caps

Signed-off-by: John Spray <john.spray@redhat.com>
mds: add 'osdmap barrier' admin command

This is for use with client eviction.

Signed-off-by: John Spray <john.spray@redhat.com>