ceph-disk: fix verify_no_in_use check

We only need to verify that partitions aren't in use when we want to
consume the whole device (osd data), not when we want to create an
additional partition for ourselves (osd journal).

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
Revert "Fix for bug #6700"

This reverts commit 673394702b725ff3f26d13b54d909208daa56d89.

This appears to break things when the journal and data disk are *not* the same.
And I can't seem to reproduce the original failure...

Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: fix log syntax error

  File "/usr/sbin/ceph-disk", line 303, in command_check_call
    LOG.info('Running command: %s' % ' '.join(arguments))
TypeError: sequence item 2: expected string, NoneType found

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: use partition type UUIDs, and blkid

Use blkid to give us the GPT partition type.  This lets us distinguish
between dmcrypt and non-dmcrypt partitions.  Fake it if blkid doesn't
give us what we want and try with sgdisk.  This isn't perfect (it can't
tell between dmcrypt and not dmcrypt), but such is life, and we are better
off than before.

Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: move fs mount probe into a helper

Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: display information about dmcrypted data and journal volumes

Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: add get_partition_base() helper

Return the base devices/disk for a partition device.

Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: only fall back to sgdisk for 'list' if blkid seems old

If the blkid doesn't show us any ID_PART_ENTRY_* fields but we know it is
a GPT partition, *then* fallback.  Otherwise, don't bother.

Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: warn about falling back to sgdisk (once)

This way the user knows something funny might be up if dmcrypt is in use.

Signed-off-by: Sage Weil <sage@redhat.com>
shared_cache: use a single lookup for lookup() too

We didn't convert this one to use iterators before.

Signed-off-by: Greg Farnum <greg@inktank.com>
shared_cache: expose prior existence when inserting an element

The LRU now handles you attempting to insert multiple values for the
same key, by telling you that you've done so and returning the
existing value before it manages to muck up existing data.
The param 'existed' is not mandatory, default value is NULL.

Signed-off-by: Greg Farnum <greg@inktank.com>

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
FDCache: implement a basic sharding of the FDCache

This is just a basic sharding. A more sophisticated implementation would
rely on something other than luck for keeping the distribution equitable.
The minimum FDCache shard size is 1.

Signed-off-by: Greg Farnum <greg@inktank.com>

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
FileStore: remove the fdcache_lock

With the changes to the shared_cache, we no longer need the fdcache_lock
to prevent us from inserting a second fd for the same hobject into the cache.

Signed-off-by: Greg Farnum <greg@inktank.com>

Merged conflict fixed.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>

Conflicts:
	src/os/FileStore.cc
shared_cache: pass key (K) by const ref in interface methods

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
FileStore: Index caching is introduced for performance improvement

IndexManager now has a Index caching. Index will only be created if not
found in the cache. Earlier, each op is creating an Index object and other
ops requesting the same index needed to wait till previous op is done.
Also, after finishing lookup, this Index object was destroyed.
Now, a Index cache is been implemented to persists these Indexes since
there is a major performance hit because each op is creating and destroying
these. A RWlock is been introduced in the CollectionIndex class and that is
responsible for sync between lookup and create.
Also, since these Index objects are persistent there is no need to use
smart pointers. So, Index is a wrapper class of CollecIndex* now.
It is the responsibility of the users of Index now to lock explicitely
before using them. Index object is sufficient now for locking and no need
to hold IndexPath for locking. The function interfaces of lfn_open,lfn_find
are changed accordingly.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
FileStore: FDCache lookup is rearranged

In lfn_open() there is no point of building the Index if the
cache lookup is successful and caller is not asking for Index.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
FileStore: No need to hold Index lock during omap calls

The Index lock is held during all the omap calls which is
not necessary.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
FileStore: Introduced a RLock instead of WLock

While calling index->collection_version, there is no need to
hold WLock at the index level. RLock should be sufficient.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
crush: add is_v[23]_rule(ruleid) methods

Add methods to check if a *specific* rule uses v2 or v3 features.  Refactor
the existing checks to use these.

Signed-off-by: Sage Weil <sage@redhat.com>
osd: only require crush features for rules that are actually used

Often there will be a CRUSH rule present for erasure coding that uses the
new CRUSH steps or indep mode.  If these rules are not referenced by any
pool, we do not need clients to support the mapping behavior.  This is true
because the encoding has not changed; only the expected CRUSH output.

Fixes: #8963
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>