Install ceph debs and use installed debs

The ceph task installs ceph using the debian
packages now, and all invocations of binaries installed
in {tmpdir}/binary/usr/local/bin/ are replace with
the use of the binaries installed in standard locations
by the debs.

Author:    Sander Pool <sander.pool@inktank.com>
Signed-off-by: Sam Lang <sam.lang@inktank.com>
ceph: fix purge

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: install ceph-mds, ceph-common

Signed-off-by: Sage Weil <sage@inktank.com>
nuke: remove librados, librbd

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: simpilfy package removal

apt-get doesn't have a nice way to tell if the package is not install and
we don't need to purge it.  Well, not one I found in 5 minutes.  Just
do a big purge and assume it works, or failed because there was nothing to
be done.

Signed-off-by: Sage Weil <sage@inktank.com>
nuke: whitespace

Signed-off-by: Sage Weil <sage@inktank.com>
nuke: dpkg --configure -a and apt-get -f install

Installing debs means we are more likely to hit a case where we interrupt
apt/dpkg.  Try to mop up as best we can in nuke.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: put client keyrings in /etc/ceph/ceph.$name.keyring

And make it world readable, for ubuntu's sake.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: simplify apt-key management

Run apt-key as root. No need to initialize ubuntu user's gpg.  Fix
whitespace.

Signed-off-by: Sage Weil <sage@inktank.com>
rbd: remove merge cruft

Signed-off-by: Sage Weil <sage@inktank.com>
install radosgw

Signed-off-by: Sage Weil <sage@inktank.com>
nuke: tolerate failed dpkg --configure -a/apt-get -f install

Signed-off-by: Sage Weil <sage@inktank.com>
rgw: specify keyring location

Otherwise we look at the default /var/lib/ceph/radosgw/ceph-$id/keyring.

Signed-off-by: Sage Weil <sage@inktank.com>
avoid secretfile, except for kclient

Only mount.ceph needs the secret in a standalone file.  Remove other users,
and simplify that one.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: pass package version to apt-get install

This avoids problems when a different or newer version of the package is
already installed.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: don't uninstall librados, librbd

This forces uninstall of kvm too, which is expensive.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: use default data, keyring locations

This required reordering the cluster setup so that we do the ceph-osd
--mkfs --mkkey prior to gathering keys and initializing the monitors.

Also, run daemons as root.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph: store logs in normal location

We need to switch around how these are compressed and pulled, since they
aren't in the regular archive dir anymore.

Signed-off-by: Sage Weil <sage@inktank.com>
cfuse -> ceph-fuse
sudo for admin socket commands

Signed-off-by: Sage Weil <sage@inktank.com>