crush/mapper: add note about r in recursive choose

Signed-off-by: Sage Weil <sage@inktank.com>
crush: add a few unit tests for INDEP mode

Signed-off-by: Sage Weil <sage@inktank.com>
crush/mapper: strip firstn conditionals out of crush_choose, rename

Now that indep is handled by crush_choose_indep, rename crush_choose to
crush_choose_firstn and remove all the conditionals.  This ends up
stripping out *lots* of code.

Note that it *also* makes it obvious that the shenanigans we were playing
with r' for uniform buckets were broken for firstn mode.  This appears to
have happened waaaay back in commit dae8bec9 (or earlier)... 2007.

Signed-off-by: Sage Weil <sage@inktank.com>
osd, crush: add 'erasure' pool/pg type

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: do not shift result when removing nonexistent osds

If it is a replicated pool, remove and shift to the left.  For erasure
pools, replace nonexistent items with CRUSH_ITEM_NONE.

Signed-off-by: Sage Weil <sage@inktank.com>