crush: eliminate CRUSH_MAX_SET result size limitation

This is only present to size the temporary scratch arrays that we put on
the stack.  Let the caller allocate them as they wish and remove the
limitation.

Signed-off-by: Sage Weil <sage@inktank.com>
crush: return CRUSH_ITEM_UNDEF for failed placements with indep

For firstn mode, if we fail to make a valid placement choice, we just
continue and return a short result to the caller.  For indep mode, however,
we need to make the position stable, and return an undefined value on
failed placements to avoid shifting later results to the left.

Signed-off-by: Sage Weil <sage@inktank.com>
crush: use breadth-first search for indep mode

Signed-off-by: Sage Weil <sage@inktank.com>
crush/CrushWrapper: add dump_tree()

This is based on the OSDMap::print_tree(), but is a bit simpler because
we do not have up/down information at this level.

Signed-off-by: Sage Weil <sage@inktank.com>
crush/mapper: add note about r in recursive choose

Signed-off-by: Sage Weil <sage@inktank.com>
crush: add a few unit tests for INDEP mode

Signed-off-by: Sage Weil <sage@inktank.com>
crush/mapper: strip firstn conditionals out of crush_choose, rename

Now that indep is handled by crush_choose_indep, rename crush_choose to
crush_choose_firstn and remove all the conditionals.  This ends up
stripping out *lots* of code.

Note that it *also* makes it obvious that the shenanigans we were playing
with r' for uniform buckets were broken for firstn mode.  This appears to
have happened waaaay back in commit dae8bec9 (or earlier)... 2007.

Signed-off-by: Sage Weil <sage@inktank.com>
osd, crush: add 'erasure' pool/pg type

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: do not shift result when removing nonexistent osds

If it is a replicated pool, remove and shift to the left.  For erasure
pools, replace nonexistent items with CRUSH_ITEM_NONE.

Signed-off-by: Sage Weil <sage@inktank.com>
mon, crush: add mode to "osd crush rule create-simple ..."

Add a mode (firstn or indep) to the create-simple command.  Make it
optional and default to firstn (for compatiblity and simplicity).

Note: a "default=..." option for mon commands would be easier.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/osd_types: pg_pool_t: fix /// -> ///< comments

Signed-off-by: Sage Weil <sage@inktank.com>