osd/: add pg_log_entry_t::reverting_to for LOST_REVERT

Previously, we encoded the version to which we were
reverting in the prior_version field.  Now, we explicitely
encode that version in the reverting_to field.

Using prior_version to encode the reverting_to version
could cause us to revert to the wrong version:

primary osd.1 writes foo 7'6(0'0)
primary osd.1 writes foo 9'9(7'6)
primary osd.0 learns the log up to 9'9 but recovers no objects
primary osd.1 dies
primary osd.0 reverts the foo in version 17'11(7'6) to 7'6
primary osd.1 comes back and starts to recover itself

foo is not missing on osd.1, and so the new log entry
17'11(7'6) causes osd.1's missing set to contain an entry
for foo with need=17'11 and have=7'6.  recover_primary uses
this information to conclude that we can locally recover
the LOST_REVERT event from the local copy which it assumes
is 7'6 but in fact is 9'9.

This bug actually manifested as failing an assert in
populate_obc_watchers since the version on disk didn't
match the prior_version of the log event.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: use remove_object_with_snap_hardlinks for divergent objects

Otherwise, we end up leaving snap hardlinks in the snapshot
index directories.  This eventually results in an EEXIST error
when we attempt to re-link the clone into place during
recovery.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
PG::merge_old_entry: fix case for divergent prior_version

Previously, we asserted that a log entry with a divergent
prior_version must be a clone.  Consider the following
case:

6'11(6'2)  m foo
7'12(6'3) m bar
7'13(7'12) m bar

If this is merged with:

6'11(6'2)  m foo
8'12(6'4) m baz

we will hit the assert.  The correct behavior is simply to remove
the object as in the clone case.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
os/chain_xattr: move chained xattr helpers into separate file/module

* Rename do_* -> chain_*.
* Move #defines to a header.
* Fix *Index users
* Implement both *xattr() and f*xattr() variants for get, set, list,
  remove.

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use f*xattr() methods to test xattr support

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: add _fgetattr(), _fgetattrs() helpers

Introduce new fd-based getattr[s] helpers.

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use _fgetattrs() for GETATTRS op

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use _fgetattrs() for CLONE

We have the src fd.

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use _fgetattrs() for _setattrs()

If we're using omap for spillover, we need to load the xattrs so that
they can be removed if necessary.

Note that we subtly change the failure behavior here to be more
strict: if we can't open the file, we fail, whereas before we did not.

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use _fgetattrs() from _rmattrs()

Note that we subtly change failure handling to be more strict: if we
fail to open the file, we error out; before we ignored errors from
_getattr().

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use _fgetattrs() from collection_getattr()

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: remove unused _getattrs() methods

We are now using the _fgetattrs() variant everywhere instead.

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use _fgetattr() from getattr()

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: use new helpers from collection_{get,set,rm}attr[s]

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: drop _getattr() helpers

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: recast _setattrs() on top of chain_f* methods

This lets us open an fd once and use that.

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: recast _rmattr on top of chain_f*

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: remove lfn_*xattr helpers

No longer used!

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: introduce lfn_close()

Match every lfn_open() with lfn_close() (instead of close(2)).

Signed-off-by: Sage Weil <sage@inktank.com>
os/chain_xattr: fix listxattr buffer size

Intenrally allocate 2x the raw listxattr result; tell the user the same.

Signed-off-by: Sage Weil <sage@inktank.com>