crush/mapper: fix up the indep tests

Fix indentation.
Simplify+fix the changed vs moved calculation.
Use the new SET_CHOOSE_LEAF_TRIES command.

Signed-off-by: Sage Weil <sage@inktank.com>
crush/mapper: apply chooseleaf_tries to firstn mode too

Parameterize the attempts for the _firstn choose method, and apply the
rule-specified tries count to firstn mode as well.  Note that we have
slightly different behavior here than with indep:

 If the firstn value is not specified for firstn, we pass through the
 normal attempt count.  This maintains compatibility with legacy behavior.
 Note that this is usually *not* actually N^2 work, though, because of the
 descend_once tunable.  However, descend_once is unfortunately *not* the
 same thing as 1 chooseleaf try because it is only checked on a reject but
 not on a collision.  Sigh.

 In contrast, for indep, if tries is not specified we default to 1
 recursive attempt, because that is simply more sane, and we have the
 option to do so.  The descend_once tunable has no effect for indep.

Signed-off-by: Sage Weil <sage@inktank.com>
crush/mapper: add SET_CHOOSE_TRIES rule step

Since we can specify the recursive retries in a rule, we may as well also
specify the non-recursive tries too for completeness.

Signed-off-by: Sage Weil <sage@inktank.com>
crush/CrushWrapper: set chooseleaf_tries to 5 for 'simple' indep rules

When making a generic indep rule, set the recursive retry to 5.  This gives
better overall results.

Signed-off-by: Sage Weil <sage@inktank.com>
crush/CrushCompiler: [de]compile set_choose[leaf]_tries rule step

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: fix feature calculation for CACHEPOOL

We need to include the faeture in the mask.

Signed-off-by: Sage Weil <sage@inktank.com>
crush: CHOOSE_LEAF -> CHOOSELEAF throughout

This aligns the internal identifier names with the user-visible names in
the decompiled crush map language.

Signed-off-by: Sage Weil <sage@inktank.com>
crush: add feature CRUSH_V2 for new indep mode and SET_*_TRIES rule steps

Signed-off-by: Sage Weil <sage@inktank.com>
PendingReleaseNotes: note change of CRUSH indep mode in release notes

Signed-off-by: Sage Weil <sage@inktank.com>
crush/mapper: dump indep partial progression for debugging

...if DEBUG_INDEP is #defined.

Signed-off-by: Sage Weil <sage@inktank.com>