run_xfstests.sh: activate more tests that now work

I've gone through the set of xfstests that were previously found to
not work.  Some of those now do work, and with the addition of an
option to pass to "mkfs.xfs" a large number of other tests now
produce expected output as well.

This patch updates the default list of tests to run to reflect
the result of this exercise.  The following 50 additional tests
are now run by default:

    029 074 078 084-087 100 105 117 121 124 126 129-134
    164 165 167 174 181 184 186 187 192 214-216 227 236
    237 241 243 245-249 257-259 261 277 278 280 285 286

Test 127 completed without error, but it took from 1-3 hours so I
kept that out of the list.

Signed-off-by: Alex Elder <elder@inktank.com>
DispatchQueue: lock DispatchQueue when for get_queue_len()

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
PrioritizedQueue: remove internal lock, not used

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
os/FileStore: only try BTRFS_IOC_SUBVOL_CREATE on btrfs

Only try to create a btrfs subvolume if the fs is btrfs.  Otherwise, just
create a directory.  Then we can error out on *any* ioctl error, and not
rely on the ioctl error code to determine if we failed because we are on
a non-btrfs or a real error.

Fixes: #3052
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
build: update for boost_thread library.

There is a difference in naming conventions between debian and
rpm based distributions for this library.  In configure.ac we
check first for boost_thread-mt, then if it's not found check
for boost_thread.  A side effect of the AC_CEHCK_LIB macro is
to add the library to the $LIBS, so the explicit -llibboost_thread
in the Makefile has been removed.
(cherry picked from commit f0c7bb363000037bbf7d58ac6e2d39d0f10200fe)
ObjectCacher: more debugging for BufferHeads

This is useful for checking for lost waiters.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
test: add ObjectCacher stress test that does not use a cluster

Use a fake writeback handler and respond to all requests with -ENOENT.
This tests that all operations will complete, and the cache doesn't
lose waiters or callbacks.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: retry reads when they are incomplete

Skipping these callbacks when there's a racing write or
a gap in the results causes the original reads they represent
to never be completed. If the read falls within the range
of a BufferHead, retry all waiters no matter what.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: don't needlessly increment iterator

This iterator is now reset on each run through the loop,
so there's no point in incrementing it here.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: debug read waiters

Now we can tell which ones will be called.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: assert lock is held everywhere

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: more debugging for read completions

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: add debug function to check BufferHead consistency

This isn't called because it's potentially expensive, but calling it
in various places can help future debugging.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: begin at the right place when iterating over BufferHeads

If the desired offset overlaps a BH, data.lower_bound() will return
the element after it, since it's indexed by the start of a range.

The confusingly similarly named data_lower_bound() method fixes this,
and returns the correct starting element.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: fix off-by-one error in split

This error left a completion that should have been attached
to the right BufferHead on the left BufferHead, which would
result in the completion never being called unless the buffers
were merged before it's original read completed. This would cause
a hang in any higher level waiting for a read to complete.

The existing loop went backwards (using a forward iterator),
but stopped when the iterator reached the beginning of the map,
or when a waiter belonged to the left BufferHead.

If the first list of waiters should have been moved to the right
BufferHead, it was skipped because at that point the iterator
was at the beginning of the map, which was the main condition
of the loop.

Restructure the waiters-moving loop to go forward in the map instead,
so it's harder to make an off-by-one error.

Possibly-fixes: #3286
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: take cache lock when discarding data from cache

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
osd: default journal size to 5GB

Signed-off-by: Sage Weil <sage@inktank.com>
msg/Accepter: only close socket if >= 0

It is possible for rebind() to fail, in which case the OSD will go through
it's shutdown procedure and call stop().  This is simpler than trying to
avoid calling stop() when rebind() fails.

Fixes: #3504
Signed-off-by: Sage Weil <sage@inktank.com>
upstart: set high open file limits

The default 1024 limit is easily hit on larger clusters.

Signed-off-by: Sage Weil <sage@inktank.com>
Merge branch 'wip-oc-hang' into next

Reviewed-by: Sage Weil <sage.weil@inktank.com>