Monitor: use a single static accessor for getting CompatSet features off disk

This way we don't need to maintain two working code paths.

Signed-off-by: Greg Farnum <greg@inktank.com>
Monitor: introduce a function that translates quorum features into disk features

After an election, we call apply_quorum_to_compatset_features(). It translates
from the quorum features into monitor disk state features we care about and
adds them to the disk store. This prevents an older daemon from starting up
on a store it doesn't understand.

While strictly speaking we don't need to add the EC feature until we create
an EC pool, all monitors which speak the new OSDMap encoding also support EC
pools, and having more than one feature makes the pattern clearer.

Signed-off-by: Greg Farnum <greg@inktank.com>
OSDMonitor: do not allow creation of EC pools if the quorum can't handle it

Signed-off-by: Greg Farnum <greg@inktank.com>
Monitor: add a function to get the required quorum features from the local compatset

We will use this to avoid deferring to people who can't read our store.

Signed-off-by: Greg Farnum <greg@inktank.com>
Elector: ignore messages from mons without required feature capabilities

We maintain a list of required_features which the other monitor's features
must supply. This starts out at 0 and is initialized from the monitor's
list of features whenever we start electing.
Despite the scary sound of "just ignore it", this is safe: the monitor
will only record features as required once a quorum has formed in which
every monitor supports them. After that happens, monitors which do not
support those features will be unable to read the whole mon store/understand
the pg reports/whatever else, so letting them into the quorum would be buggy
behavior.
So if we ignore a monitor, it will not be able to start nor join
an election round with anybody who was in our quorum -- that is, the
ignored monitor cannot form a separate quorum. By ignoring it here, we
also prevent it from endlessly calling elections against the real
quorum.

Unfortunately there is no way to communicate to old monitors that they
cannot join the quorum -- there are no existing messages for that purpose,
and eg adding a new op to the MMonElection message will just cause it
to crash, which we don't want to do either.

Fixes: #7215

Signed-off-by: Greg Farnum <greg@inktank.com>
Elector: send an OP_NAK MMonElection to old peers who support it

Only new monitors support receiving OP_NAK from a peer without crashing, but
when we add new required features in the future, our monitors can accept
an OP_NAK message which tells them what features they're missing. Then they
will print out an error message and shut down.
(Unfortunately, doing a clean shutdown from here would require a lot of
infrastructure, so we just call exit(0).)

Signed-off-by: Greg Farnum <greg@inktank.com>
mon: do not use CEPH_FEATURES_ALL for things that touch the disk

We want to encode with our quorum_features instead. Remaining uses of
CEPH_FEATURES_ALL are:
1) when the Elector is sharing its supported features
2) in a MonMap function which is used by monmaptool
3) In the Monitor for winning a standalone election, for ephemeral data,
  and for doing mkfs (when we necessarily don't have quorum_features).
4) When doing ceph-mon --inject-monmap (we don't persist the quorum_features
  to disk, so we can't use them here).
5) in MMonElection, for doing the default monmap encoding (which is
  re-encoded later if the final features don't match CEPH_FEATURES_ALL).
6) As the default encode features for the OSDMap (the monitor always
  supplies quorum_features instead).

Signed-off-by: Greg Farnum <greg@inktank.com>
Add a virtual interface init, open, create_and_open to KeyValueDB

This will provide an opportunity to set leveldb_* settings in the next
patch.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Signed-off-by: Sage Weil <sage@inktank.com>
leveldb: add leveldb_* options

The osd_leveldb_* and mon_leveldb_* are now deprecated.  If they are
still used, they override these values.

Signed-off-by: Sage Weil <sage@inktank.com>
Add a new KV wrapper GenericObjectMap

Now we already have DBObjectMap which implement ObjectMap and other
interfaces, and ObjectMap.h implied that ObjectMap is used to encapsulates
the FileStore key value store. There exists limitation in current DBObjectMap
implementation, such as lacking of "coll_t" in "key", complicated prefix
hard-coded and inflexible extending.

So in order to provide a more flexible API and clear implementation to wrap KV
store, I copy the origin DBObjectMap and redesign the partial implementation.
Adding "coll_t" argument to all API and export "prefix" to callers. Prefixes
are divided into two parts "INTERN" and "USER". "INTERN" keys used by self to
manage and "USER" keys are managed by callers. Besides above, misc fixes are
imported such as more clear member function name and extendible header
structure.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
docs: Improve CloudStack documentation.

Add a section about creating a CephX user for CloudStack.
docs: Typo in CloudStack user creation
Add KeyValueStore implementation

KeyValueStore is another ObjectStore implementation with FileStore. It
uses KV store wrapper(StripObjectMap) which inherited GenericObjectMap
to implement ObjectStore APIs.

Each object has a header key in KV backend, which encapsulated the metadata
of object such as size, the status of keys. A complete object data maybe spread
around multi keys. The CRUD operation of object need to access the header key
of object to know the details, then the actual data keys will be get.

Now the actual KV backend of KeyValueStore is only LevelDB, more KV backend
(RocksDB, NVM API) will be introduced in the near future.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Make ObjectStore unit tests shared with KeyValueStore

Via gtest's value-parameterized feature, let the successors of
ObjectStore share the same unit tests.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Merge pull request #1158 from yuyuyu101/keyvaluestore-dev

Keyvaluestore Support

Reviewed-by: Sage Weil <sage@inktank.com>
keyvaluestore: name to keyvaluetore-dev for now

This helps warn the user that the ondisk format may be subject to change.

Signed-off-by: Sage Weil <sage@inktank.com>
Merge remote-tracking branch 'gh/next'
Merge pull request #1148 from ceph/wip-7215-quorum-features

Add monitor quorum features

Reviewed-by: Sage Weil <sage@inktank.com>
ceph_test_object_map: fix build

Signed-off-by: Sage Weil <sage@inktank.com>
doc: Removed extraneous character from path.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>