PG: ignore non MISSING pg query in ReplicaActive

1) Replica sends notify
2) Prior to processing notify, primary queues query to replica
3) Primary processes notify and activates sending MOSDPGLog
to replica.
4) Primary does do_notifies at end of process_peering_events
and sends to Query.
5) Replica sees MOSDPGLog and activates
6) Replica sees Query and asserts.

In the above case, the Replica should simply ignore the old
Query.

Fixes: #4050
Backport: bobtail
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
OSD: expand_pg_num after pg removes

Otherwise:
1) expand_pg_num removes a splitting pg entry
2) peering thread grabs pg lock and starts split
3) OSD::consume_map grabs pg lock and starts removal

At step 2), we run afoul of the assert(is_splitting)
check in split_pgs.  This way, the would be splitting
pg is marked as removed prior to the splitting state
being updated.

Backport: bobtail
Fixes: #4449
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
FileJournal: queue_pos \in [get_top(), header.max_size)

If queue_pos == header.max_size when we create the entry
header magic, the entry will be rejected at get_top() on
replay.

Fixes: #4436
Backport: bobtail
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
FileStore: add more debugging for remove and split

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileJournal: add committed_up_to to header

header_t::committed_up_to provides a lower bound for safetly committed
journal entries.  If read_entry fails prior to committed_up_to, we
know we have a corrupt jorunal entry.  Furthermore, if
journal_write_header_frequency is not 0, we will write out the
journal header once every journal_write_header_frequency
journal writes.

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileJournal::wrap_read_bl: convert arguments to explicit in/out arguments

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileJournal,Journal: detect some corrupt journal scenarios

When the checksum or footer are invalid, we will now try to
look at the next entry.  If we find a valid entry, it is likely
that the journal is corrupt.

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileJournal: add testing methods to corrupt entries

Signed-off-by: Samuel Just <sam.just@inktank.com>
test_filejournal: add tests for footer, header, payload corruption

Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge branch 'next'
Merge branch 'wip_4196'

Reviewed-by: Sage Weil <sage@inktank.com>
Fixes: #4196
ceph-fuse: invalidate cache by default

Closes: #2215
Signed-off-by: Sage Weil <sage@inktank.com>
ceph.spec.in:  Additional clean-up on package removal

When removing the last instance of ceph, also remove the files
created by ceph during operation.  These consist of the files
under /var/lib/ceph, /etc/ceph, and /var/log/ceph. Bug #4415.

Signed-off-by: Gary Lowell  <gary.lowell@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
ceph_features: fix CEPH_FEATURE_OSD_SNAPMAPPER definition

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
hobject: fix snprintf args for 32 bit

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
OSD: split temp collection as well

Otherwise, when we eventually remove the temp collection, there might be
objects in the temp collection which were independently pulled into the child
pg collection.  Thus, removing the old stale parent link from its temp
collection also blasts the omap entries and snap mappings for the real child
object.

Backport: bobtail
Fixes: #4452
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
client: Ensure inode/dentries are ref counted

The MetaRequest holds onto inodes and dentries
for retrying unsafe requests, but those objects
might be removed from the cache (unlink for example)
causing the inode/dentry to be freed.  Ensure that
the inode/dentry is never freed while the MetaRequest
holds onto it by putting/getting the refs using
set/get interfaces.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
mds: Use map for CInode pinrefs

Implements pin refs on the inode as a map instead of
a multiset, allowing individual ref counts to act as
real references with values that can be >1.
The pin refs are only used for debugging, but allowing
them to be >1 avoids the need for a separate state field
for things like DIRTYPARENT.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
dencoder:  Add inode_backtrace_t to types

To test the backtrace attributes on objects, we need
to be able to decode the backtrace using ceph-dencoder.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
include/elist: Fix clear() to use pop_front()

elist<T>::clear() is calling remove(), which isn't a
method defined on elist<T> (it was never defined according
to git).  Because elist is templated and no references
to clear() are ever made, the compiler matches remove(T) to the
remove(const char *) system call defined in stdio.h.

Once clear is invoked on an instance of elist<T>, we get the
compile error shown below.

The fix here is to use pop_front() instead of remove().

Compile error is:
In file included from ../../src/mds/CInode.h:22:0,
                 from ../../src/mds/CInode.cc:19:
../../src/include/elist.h: In instantiation of ‘void elist<T>::clear() [with T = cinode_backtrace_info_t*]’:
../../src/mds/CInode.cc:1129:20:   required from here
../../src/include/elist.h:101:7: error: no matching function for call to ‘remove(cinode_backtrace_info_t*)’
../../src/include/elist.h:101:7: note: candidates are:
In file included from ../../src/mds/CInode.cc:17:0:
/usr/include/stdio.h:179:12: note: int remove(const char*)
/usr/include/stdio.h:179:12: note:   no known conversion for argument 1 from ‘cinode_backtrace_info_t*’ to ‘const char*’
In file included from /usr/include/c++/4.7/algorithm:63:0,
                 from /usr/include/c++/4.7/backward/hashtable.h:65,
                 from /usr/include/c++/4.7/ext/hash_map:65,
                 from ../../src/include/encoding.h:292,
                 from ../../src/common/entity_name.h:22,
                 from ../../src/common/config.h:26,
                 from ../../src/mds/CInode.h:20,
                 from ../../src/mds/CInode.cc:19:
/usr/include/c++/4.7/bits/stl_algo.h:1117:5: note: template<class _FIter, class _Tp> _FIter std::remove(_FIter, _FIter, const _Tp&)
/usr/include/c++/4.7/bits/stl_algo.h:1117:5: note:   template argument deduction/substitution failed:
In file included from ../../src/mds/CInode.h:22:0,
                 from ../../src/mds/CInode.cc:19:
../../src/include/elist.h:101:7: note:   candidate expects 3 arguments, 1 provided

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>