crush: add dprintk's for crush_calc_straw

These are compiled out by default.

Signed-off-by: Sage Weil <sage@redhat.com>
osdmaptool: --test-map-pgs-dump

Signed-off-by: Sage Weil <sage@redhat.com>
crush: recalculate straw scalers during a reweight

The crushtool --reweight function triggers a fresh calculation of bucket
weights so that they are always the sum of the item weights.  In the
straw bucket case, the weights were updated but the corresponding straw
scalers were not being recalculated.  The result is that there was not
effect on placement in adjusted buckets until the next time a bucket item's
weight was adjusted.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
crushtool/CrushTester: output utilization even with 1 batch

Signed-off-by: Sage Weil <sage@redhat.com>
crush/CrushTester: add new --show-mappings option

This makes --show-utilization and --show-statistics usable.

Signed-off-by: Sage Weil <sage@redhat.com>
crush/CrushWrapper: dump chooseleaf_vary_r with other tunables

Signed-off-by: Sage Weil <sage@redhat.com>
crush: add straw_calc_version tunable

It doesn't do anything, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: default to straw_calc_version 1

Stick with bobtail tunables where it affects compatibility.  Use v1 of
straw_calc, though, since that does not, and we want the best for new
clusters.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: pass crush_map * to various builder methods

In particular, we will need it for crush_calc_straw().

Signed-off-by: Sage Weil <sage@redhat.com>
crush/builder: break out new version 1 of crush_calc_straw

No change, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: fix distortion of straw scalers by 0-weight items

The presence of a 0-weight item in a straw bucket should have no effect
on the placement of other items.  Add a test validating that and fix
crush_calc_straw() to fix the distortion.

Note that this effects the *precalculation* of the straw bucket inputs and
does not effect the actually mapping process given a compiled or encoded
CRUSH map, and only when straw_calc_version == 1 (i.e., the admin opted in
to the new behavior).

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
crush: fix crush_calc_straw() scalers when there are duplicate weights

The straw bucket was originally tested with uniform weights and with a
few more complicated patterns, like a stair step (1,2,3,4,5,6,7,8,9).  And
it worked!

However, it does not behave with a pattern like
 1, 2, 2, 3, 3, 4, 4

Strangely, it does behave with
 1, 1, 2, 2, 3, 3, 4, 4

and more usefully it does behave with
 1, 2, 2.001, 3, 3.001, 4, 4.001

That is, the logic that explicitly copes with weights that are duplicates
is broken.

The fix is to simply remove the special handling for duplicate weights --
it isn't necessary and doesn't work correctly anyway.

Add a test that compares the mapping result of  [1, 2, 2, 3, 3, ...] with
[1, 2, 2.001, 3, 3.001, ...] and verifies that the difference is small.
With the fix, we get .00012, whereas the original implementation gets
.015.

Note that this changes the straw bucket scalar *precalculated* values that
are encoded with the map, and only when the admin opts into the new behavior.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
mon: add 'osd crush {get,set}-tunable <name> [value]' commands

For now, just add the straw_calc_version tunable.

Signed-off-by: Sage Weil <sage@redhat.com>
crush/builder: a note about the original crush_calc_straw()

Signed-off-by: Sage Weil <sage@redhat.com>
crush: set straw_calc_version=1 for default+optimal; do not touch for presets

When using the presets for compatibility (i.e., based on version), do not
touch the straw behavior, as it does not affect mapping or compatibility.
However, make a point of setting it by default and for optimal.

For most users, this means that they will not see any change unless they
explicitly enable the new behavior, or switch to default or optimal
tunables.  The idea is that if they touched it, they shouldn't be
too surprised by the subsequent data movement.

Signed-off-by: Sage Weil <sage@redhat.com>
mon: 'osd crush reweight-all'

This corresponds to the crushtool --reweight command.

Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #3057 from ceph/wip-crush-straw

crush: fix straw bucket scaler bugs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>