mds: make max file recovers configurable

Signed-off-by: Sage Weil <sage@redhat.com>
mds: change mds_max_file_recover from 5 -> 32

These are reasonably cheap operations (stat) and we should be too worried
about queueing up a bunch of them.

Ideally this sort of thing would magically tune to the throughput we can
get from the cluster, but until then, let's choose a default that works for
more users.

Signed-off-by: Sage Weil <sage@redhat.com>
mds/RecoveryQueue: add method to prioritize a file recovery; fix logging

Add a prioritize() method to make file recovery start immediately for the
given inode.  Note that this doesn't respect the max recovery limit: if
someone stats it, they are blocking, and we start the recovery immediately.

Also fix up the dout logging a bit so that everything is prefixed
consistently.

Signed-off-by: Sage Weil <sage@redhat.com>
mds/Locker: prioritize file recovery on rd/wr/x lock

If someone specifically touches the file lock, prioritize recovery for this
file.

Signed-off-by: Sage Weil <sage@redhat.com>
mds/Locker: if a client asks for file caps, prioritize file recovery

This normally happens when a process opens the file.  Prioritize in that
case.

Signed-off-by: Sage Weil <sage@redhat.com>
mds/RecoveryQueue: do not start prioritized items synchronously

When we prioritize an item move it into a second priority list/set, but
do not start immediately, so that we still obey the max.  When we go to
start an item, pull items first off the priority list, then off the regular
list.

Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #2258 from ceph/wip-mds-recover

mds: make max file recovers tunable

Reviewed-by: Yan, Zheng <ukernel@gmail.com>
Reviewed-by: John Spray <john.spray@redhat.com>