crush: add dprintk's for crush_calc_straw

These are compiled out by default.

Signed-off-by: Sage Weil <sage@redhat.com>
osdmaptool: --test-map-pgs-dump

Signed-off-by: Sage Weil <sage@redhat.com>
crush: recalculate straw scalers during a reweight

The crushtool --reweight function triggers a fresh calculation of bucket
weights so that they are always the sum of the item weights.  In the
straw bucket case, the weights were updated but the corresponding straw
scalers were not being recalculated.  The result is that there was not
effect on placement in adjusted buckets until the next time a bucket item's
weight was adjusted.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
crushtool/CrushTester: output utilization even with 1 batch

Signed-off-by: Sage Weil <sage@redhat.com>
crush/CrushTester: add new --show-mappings option

This makes --show-utilization and --show-statistics usable.

Signed-off-by: Sage Weil <sage@redhat.com>
crush/CrushWrapper: dump chooseleaf_vary_r with other tunables

Signed-off-by: Sage Weil <sage@redhat.com>
crush: add straw_calc_version tunable

It doesn't do anything, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: default to straw_calc_version 1

Stick with bobtail tunables where it affects compatibility.  Use v1 of
straw_calc, though, since that does not, and we want the best for new
clusters.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: pass crush_map * to various builder methods

In particular, we will need it for crush_calc_straw().

Signed-off-by: Sage Weil <sage@redhat.com>
crush/builder: break out new version 1 of crush_calc_straw

No change, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: fix distortion of straw scalers by 0-weight items

The presence of a 0-weight item in a straw bucket should have no effect
on the placement of other items.  Add a test validating that and fix
crush_calc_straw() to fix the distortion.

Note that this effects the *precalculation* of the straw bucket inputs and
does not effect the actually mapping process given a compiled or encoded
CRUSH map, and only when straw_calc_version == 1 (i.e., the admin opted in
to the new behavior).

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
crush: fix crush_calc_straw() scalers when there are duplicate weights

The straw bucket was originally tested with uniform weights and with a
few more complicated patterns, like a stair step (1,2,3,4,5,6,7,8,9).  And
it worked!

However, it does not behave with a pattern like
 1, 2, 2, 3, 3, 4, 4

Strangely, it does behave with
 1, 1, 2, 2, 3, 3, 4, 4

and more usefully it does behave with
 1, 2, 2.001, 3, 3.001, 4, 4.001

That is, the logic that explicitly copes with weights that are duplicates
is broken.

The fix is to simply remove the special handling for duplicate weights --
it isn't necessary and doesn't work correctly anyway.

Add a test that compares the mapping result of  [1, 2, 2, 3, 3, ...] with
[1, 2, 2.001, 3, 3.001, ...] and verifies that the difference is small.
With the fix, we get .00012, whereas the original implementation gets
.015.

Note that this changes the straw bucket scalar *precalculated* values that
are encoded with the map, and only when the admin opts into the new behavior.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
Messenger: Add unit tests

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
PipeConnection: Modify "is_connected" semantic

"is_connected" should be used to judge whether have built a successful
connection with peer. Since "pipe == NULL" is useless for whether connected
because pipe can be construct without any potential network problem

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
AsyncConnection: Avoid hungry if mark_down's caller is the eventcenter's owner

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
AsyncConnection: Avoid name conflicting with AsyncMessenger

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
AsyncMessenger: Fix accept connection replacing process

Original there exists bugs that we don't modify existing file event,
it will let remaining alive connection's event won't be called by
EventCenter.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
AsyncConnection: Add loopback connection support

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
WBThrottle: make bytes/ios/inode_wb's perf counter effective

Since sync thread will cause unstable iops and latency performance curve, we
may want make WBThread do more(or moderate?) writeback and avoid sync thread
flush too much which will cause journal io long tail.

Via these counters, we can view how much object or bytes are write backed by
WBThread and how much bytes are flushed . Then we can have a great tuning for
"*bytes_start_flusher", "*ios_start_flusher" and "*inodes_start_flusher".
What we want to see is that in-memory data can writeback into disk with a
moderate rate.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Include common/likely.h in buffer.h

Signed-off-by: Matt Benjamin <matt@cohortfs.com>