Repository: dragonflyoss/Dragonfly2
Pull request #1686 (2022-09-19 10:43:06)

feat: support download empty file
Signed-off-by: Jim Ma <majinjing3@gmail.com>

<!--- Provide a general summary of your changes in the Title above -->

## Description

<!--- Describe your changes in detail -->

## Related Issue

<!--- This project only accepts pull requests related to open issues -->
<!--- If suggesting a new feature or change, please discuss it in an issue first -->
<!--- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!--- Please link to the issue here: -->

## Motivation and Context

<!--- Why is this change required? What problem does it solve? -->

## Screenshots (if appropriate)

## Types of changes

<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Documentation Update (if none of the other choices apply)

## Checklist

<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.


codecov-commenter: # [Codecov](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss) Report
> Merging [#1686](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss) (a20ee0b) into [main](https://codecov.io/gh/dragonflyoss/Dragonfly2/commit/d24a798a1bf6a3c8f36d8b619d77fb3812f36ea3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss) (d24a798) will **decrease** coverage by `0.13%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686/graphs/tree.svg?width=650&height=150&src=pr&token=PUICJYSAT7&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss)](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss)

```diff
@@            Coverage Diff             @@
##             main    #1686      +/-   ##
==========================================
- Coverage   42.01%   41.87%   -0.14%     
==========================================
  Files         125      125              
  Lines       15455    15461       +6     
==========================================
- Hits         6493     6474      -19     
- Misses       8447     8475      +28     
+ Partials      515      512       -3     
```

| Flag | Coverage Δ | |
|---|---|---|
| Object-compatibility-e2etests | `?` | |
| e2etests | `?` | |
| unittests | `41.87% <0.00%> (-0.14%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss) | Coverage Δ | |
|---|---|---|
| [client/daemon/storage/local\_storage.go](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss#diff-Y2xpZW50L2RhZW1vbi9zdG9yYWdlL2xvY2FsX3N0b3JhZ2UuZ28=) | `30.45% <0.00%> (-0.18%)` | :arrow_down: |
| [client/daemon/storage/local\_storage\_subtask.go](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss#diff-Y2xpZW50L2RhZW1vbi9zdG9yYWdlL2xvY2FsX3N0b3JhZ2Vfc3VidGFzay5nbw==) | `28.75% <0.00%> (-0.29%)` | :arrow_down: |
| [client/daemon/peer/peertask\_piecetask\_poller.go](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss#diff-Y2xpZW50L2RhZW1vbi9wZWVyL3BlZXJ0YXNrX3BpZWNldGFza19wb2xsZXIuZ28=) | `34.42% <0.00%> (-15.31%)` | :arrow_down: |
| [client/daemon/peer/peertask\_file.go](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss#diff-Y2xpZW50L2RhZW1vbi9wZWVyL3BlZXJ0YXNrX2ZpbGUuZ28=) | `58.38% <0.00%> (-1.35%)` | :arrow_down: |
| [client/daemon/peer/peertask\_conductor.go](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss#diff-Y2xpZW50L2RhZW1vbi9wZWVyL3BlZXJ0YXNrX2NvbmR1Y3Rvci5nbw==) | `58.22% <0.00%> (+0.60%)` | :arrow_up: |
| [...ent/daemon/peer/peertask\_piecetask\_synchronizer.go](https://codecov.io/gh/dragonflyoss/Dragonfly2/pull/1686/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dragonflyoss#diff-Y2xpZW50L2RhZW1vbi9wZWVyL3BlZXJ0YXNrX3BpZWNldGFza19zeW5jaHJvbml6ZXIuZ28=) | `50.47% <0.00%> (+0.94%)` | :arrow_up: |


Patch:
diff --git a/client/daemon/storage/local_storage.go b/client/daemon/storage/local_storage.go
index f8ebf682..6088ed40 100644
--- a/client/daemon/storage/local_storage.go
+++ b/client/daemon/storage/local_storage.go
@@ -233,6 +233,10 @@ func (t *localTaskStore) UpdateTask(ctx context.Context, req *UpdateTaskRequest)
 func (t *localTaskStore) ValidateDigest(*PeerTaskMetadata) error {
 	t.Lock()
 	defer t.Unlock()
+	if t.ContentLength == 0 {
+		return nil
+	}
+
 	if t.persistentMetadata.PieceMd5Sign == "" {
 		t.invalid.Store(true)
 		return ErrDigestNotSet
diff --git a/client/daemon/storage/local_storage_subtask.go b/client/daemon/storage/local_storage_subtask.go
index 43bf2f54..b4b61064 100644
--- a/client/daemon/storage/local_storage_subtask.go
+++ b/client/daemon/storage/local_storage_subtask.go
@@ -320,6 +320,10 @@ func (t *localSubTaskStore) Store(ctx context.Context, req *StoreRequest) error
 func (t *localSubTaskStore) ValidateDigest(req *PeerTaskMetadata) error {
 	t.Lock()
 	defer t.Unlock()
+	if t.ContentLength == 0 {
+		return nil
+	}
+
 	if t.persistentMetadata.PieceMd5Sign == "" {
 		t.invalid.Store(true)
 		return ErrDigestNotSet


Test patch:
diff --git a/client/daemon/peer/peertask_manager_test.go b/client/daemon/peer/peertask_manager_test.go
index e7ab19f9..e7af5121 100644
--- a/client/daemon/peer/peertask_manager_test.go
+++ b/client/daemon/peer/peertask_manager_test.go
@@ -420,6 +420,17 @@ func TestPeerTaskManager_TaskSuite(t *testing.T) {
 			mockPieceDownloader:  nil,
 			mockHTTPSourceClient: nil,
 		},
+		{
+			name:                 "empty file - p2p",
+			taskData:             []byte{},
+			pieceParallelCount:   4,
+			pieceSize:            1024,
+			peerID:               "empty-file-peer",
+			url:                  "http://localhost/test/data",
+			sizeScope:            commonv1.SizeScope_NORMAL,
+			mockPieceDownloader:  commonPieceDownloader,
+			mockHTTPSourceClient: nil,
+		},
 		{
 			name:                "normal size scope - back source - content length",
 			runTaskTypes:        []int{taskTypeConductor, taskTypeFile, taskTypeStream, taskTypeSeed},
@@ -558,6 +569,30 @@ func TestPeerTaskManager_TaskSuite(t *testing.T) {
 			mockPieceDownloader:  nil,
 			mockHTTPSourceClient: nil,
 		},
+		{
+			name:                "empty file peer - back source - content length",
+			runTaskTypes:        []int{taskTypeConductor, taskTypeFile, taskTypeStream, taskTypeSeed},
+			taskData:            []byte{},
+			pieceParallelCount:  4,
+			pieceSize:           1024,
+			peerID:              "empty-file-peer-back-source",
+			backSource:          true,
+			url:                 "http://localhost/test/data",
+			sizeScope:           commonv1.SizeScope_NORMAL,
+			mockPieceDownloader: nil,
+			mockHTTPSourceClient: func(t *testing.T, ctrl *gomock.Controller, rg *util.Range, taskData []byte, url string) source.ResourceClient {
+				sourceClient := sourcemocks.NewMockResourceClient(ctrl)
+				sourceClient.EXPECT().GetContentLength(source.RequestEq(url)).AnyTimes().DoAndReturn(
+					func(request *source.Request) (int64, error) {
+						return int64(len(taskData)), nil
+					})
+				sourceClient.EXPECT().Download(source.RequestEq(url)).AnyTimes().DoAndReturn(
+					func(request *source.Request) (*source.Response, error) {
+						return source.NewResponse(io.NopCloser(bytes.NewBuffer(taskData))), nil
+					})
+				return sourceClient
+			},
+		},
 	}
 
 	for _, _tc := range testCases {