Repository: tikv/pd
Pull request #4331 (2021-11-12 09:38:10)

scheduler: low space tikv can not balance  (#4324)
This is an automated cherry-pick of #4324

Signed-off-by: bufferflies <1045931706@qq.com>

<!--
Thank you for working on PD! Please read PD's [CONTRIBUTING](https://github.com/tikv/pd/blob/master/CONTRIBUTING.md) document **BEFORE** filing this PR.
PR Title Format:
1. pkg [, pkg2, pkg3]: what's changed
2. *: what's changed


If you want to open the **Challenge Program** pull request, please use the following template:
https://raw.githubusercontent.com/tikv/.github/master/.github/PULL_REQUEST_TEMPLATE/challenge-program.md
You can use it with query parameters: https://github.com/tikv/pd/compare/master...${you branch}?template=challenge-program.md
-->

### What problem does this PR solve?
close: #4323
<!-- Add the issue link with a summary if it exists. -->

### What is changed and how it works?
available space unit is not same 
### Check List

<!-- Remove the items that are not applicable. -->

Tests

<!-- At least one of them must be included. -->

- Unit test
- Integration test

capacity
  store1,store4: 161gb
  others: 215gb
available ratio and score
![image](https://user-images.githubusercontent.com/23159587/141400684-77a2b94d-d2d5-45ba-9bea-3cda28241305.png)

balance:
![image](https://user-images.githubusercontent.com/23159587/141400916-1049e344-31a6-4fca-bd95-c5b8837d3ffb.png)

there is no balance roll back 




Code changes

Side effects

Related changes

### Release note

<!-- A bugfix or a new feature needs a release note. If there is no need release note, just uncomment the below line. -->
```release-note
None
```


ti-chi-bot: [REVIEW NOTIFICATION]

This pull request has not been approved.


To complete the [pull request process](https://book.prow.tidb.io/#/en/workflows/pr), please ask the reviewers in the [list](https://prow.tidb.io/tichi/repos/tikv/pd/pulls/4331/owners) to review by filling `/cc @reviewer` in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the [list](https://prow.tidb.io/tichi/repos/tikv/pd/pulls/4331/owners) by filling  `/assign @committer` in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found [here](https://prow.tidb.io/command-help?repo=tikv%2Fpd).

<details>

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.
</details>

<!--Review Notification Identifier-->
ti-chi-bot: @ti-chi-bot: This cherry pick PR is for a release branch and has not yet been approved by release team.
Adding the `do-not-merge/cherry-pick-not-approved` label.

To merge this cherry pick, it must first be approved by the collaborators.

**AFTER** it has been approved by collaborators, please ping the release team in a comment to request a cherry pick review.


<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
nolouch: /merge
ti-chi-bot: @nolouch: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests


<details>

Instructions for interacting with me using PR comments are available [here](https://prow.tidb.io/command-help).  If you have questions or suggestions related to my behavior, please file an issue against the [ti-community-infra/tichi](https://github.com/ti-community-infra/tichi/issues/new?title=Prow%20issue:) repository.
</details>
ti-chi-bot: This pull request has been accepted and is ready to merge. <details>Commit hash: d7ac90fedc4b529e79b99725d3df1a37553aa1df</details>
ti-chi-bot: @ti-chi-bot: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.


<details>

Instructions for interacting with me using PR comments are available [here](https://prow.tidb.io/command-help).  If you have questions or suggestions related to my behavior, please file an issue against the [ti-community-infra/tichi](https://github.com/ti-community-infra/tichi/issues/new?title=Prow%20issue:) repository.
</details>
ti-chi-bot: @ti-chi-bot: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.


<details>

Instructions for interacting with me using PR comments are available [here](https://prow.tidb.io/command-help).  If you have questions or suggestions related to my behavior, please file an issue against the [ti-community-infra/tichi](https://github.com/ti-community-infra/tichi/issues/new?title=Prow%20issue:) repository.
</details>

Patch:
diff --git a/server/core/store.go b/server/core/store.go
index dd53902a7..65bdad7f2 100644
--- a/server/core/store.go
+++ b/server/core/store.go
@@ -332,7 +332,7 @@ func (s *StoreInfo) regionScoreV1(highSpaceRatio, lowSpaceRatio float64, delta i
 }
 
 func (s *StoreInfo) regionScoreV2(delta int64, lowSpaceRatio float64) float64 {
-	A := float64(s.GetAvgAvailable())
+	A := float64(s.GetAvgAvailable()) / gb
 	C := float64(s.GetCapacity()) / gb
 	R := float64(s.GetRegionSize() + delta)
 	if R < 0 {


Test patch:
diff --git a/server/core/store_test.go b/server/core/store_test.go
index 8502668e3..7db3c4d59 100644
--- a/server/core/store_test.go
+++ b/server/core/store_test.go
@@ -126,3 +126,47 @@ func (s *testStoreSuite) TestLowSpaceRatio(c *C) {
 	store.rawStats.Available = store.rawStats.Capacity >> 2
 	c.Assert(store.IsLowSpace(0.8), Equals, false)
 }
+
+func (s *testStoreSuite) TestLowSpaceScoreV2(c *C) {
+	testdata := []struct {
+		bigger *StoreInfo
+		small  *StoreInfo
+	}{{
+		// store1 and store2 has same store available ratio and store1 less 50gb
+		bigger: NewStoreInfoWithAvailable(1, 20*gb, 100*gb, 1.4),
+		small:  NewStoreInfoWithAvailable(2, 200*gb, 1000*gb, 1.4),
+	}, {
+		// store1 and store2 has same available space and less than 50gb
+		bigger: NewStoreInfoWithAvailable(1, 10*gb, 1000*gb, 1.4),
+		small:  NewStoreInfoWithAvailable(2, 10*gb, 100*gb, 1.4),
+	}, {
+		// store1 and store2 has same available ratio less than 0.2
+		bigger: NewStoreInfoWithAvailable(1, 10*gb, 1000*gb, 1.4),
+		small:  NewStoreInfoWithAvailable(2, 1*gb, 100*gb, 1.4),
+	}, {
+		// store1 and store2 has same available ratio
+		// but the store1 ratio less than store2 ((50-10)/50=0.8<(200-100)/200=0.5)
+		bigger: NewStoreInfoWithAvailable(1, 10*gb, 100*gb, 1.4),
+		small:  NewStoreInfoWithAvailable(2, 100*gb, 1000*gb, 1.4),
+	}, {
+		// store1 and store2 has same usedSize and capacity
+		// but the bigger's amp is bigger
+		bigger: NewStoreInfoWithAvailable(1, 10*gb, 100*gb, 1.5),
+		small:  NewStoreInfoWithAvailable(2, 10*gb, 100*gb, 1.4),
+	}, {
+		// store1 and store2 has same capacity and regionSize（40g)
+		// but store1 has less available space size
+		bigger: NewStoreInfoWithAvailable(1, 60*gb, 100*gb, 1),
+		small:  NewStoreInfoWithAvailable(2, 80*gb, 100*gb, 2),
+	}, {
+		// store1 and store2 has same capacity and store2 (40g) has twice usedSize than store1 (20g)
+		// but store1 has higher amp, so store1(60g) has more regionSize (40g)
+		bigger: NewStoreInfoWithAvailable(1, 80*gb, 100*gb, 3),
+		small:  NewStoreInfoWithAvailable(2, 60*gb, 100*gb, 1),
+	}}
+	for _, v := range testdata {
+		score1 := v.bigger.regionScoreV2(0, 0.8)
+		score2 := v.small.regionScoreV2(0, 0.8)
+		c.Assert(score1, Greater, score2)
+	}
+}
diff --git a/server/core/test_util.go b/server/core/test_util.go
index f8ad4cd70..99e37b342 100644
--- a/server/core/test_util.go
+++ b/server/core/test_util.go
@@ -83,6 +83,24 @@ func NewTestRegionInfo(start, end []byte) *RegionInfo {
 	}}
 }
 
+// NewStoreInfoWithAvailable is create with available and capacity
+func NewStoreInfoWithAvailable(id, available, capacity uint64, amp float64) *StoreInfo {
+	stats := &pdpb.StoreStats{}
+	stats.Capacity = capacity
+	stats.Available = available
+	usedSize := capacity - available
+	regionSize := (float64(usedSize) * amp) / mb
+	store := NewStoreInfo(
+		&metapb.Store{
+			Id: id,
+		},
+		SetStoreStats(stats),
+		SetRegionCount(int(regionSize/96)),
+		SetRegionSize(int64(regionSize)),
+	)
+	return store
+}
+
 // NewStoreInfoWithLabel is create a store with specified labels.
 func NewStoreInfoWithLabel(id uint64, regionCount int, labels map[string]string) *StoreInfo {
 	storeLabels := make([]*metapb.StoreLabel, 0, len(labels))