Repository: pydicom/pynetdicom

Issue:
PixelData truncated when _config.STORE_RECV_CHUNKED_DATASET is enabled
**Describe the bug**
We are running a Storage SCP service using pynetdicom on Windows 10. We are seeing some images stored with truncated PixelData when setting `_config.STORE_RECV_CHUNKED_DATASET = True`. This issue was seen in the field using data sent from a modality and later reproduced with `storescu`. The PixelData is correct when using a Storage SCP with the config setting disabled.

**Expected behavior**
We expect the PixelData to be the correct length.

**Steps To Reproduce**
1. Create a storescp python script in Windows with the following code:  
```
#!/usr/local/bin/python

from pynetdicom import AE, evt, AllStoragePresentationContexts, _config, debug_logger

debug_logger()

_config.STORE_RECV_CHUNKED_DATASET = True

# Implement a handler for evt.EVT_C_STORE
def handle_store(event):
    from pydicom import dcmread

    """Handle a C-STORE request event."""
    with open(event.dataset_path, 'rb') as fh:
      ds = dcmread(fh)

      ds.save_as(ds.SOPInstanceUID, write_like_original = False)

    # Return a 'Success' status
    return 0x0000

handlers = [(evt.EVT_C_STORE, handle_store)]

# Initialise the Application Entity
ae = AE()

# Support presentation contexts for all storage SOP Classes
ae.supported_contexts = AllStoragePresentationContexts

# Start listening for incoming association requests
ae.start_server(('', 104), evt_handlers=handlers)
```
2. Send an example file to the scp:  [example.zip](https://github.com/pydicom/pynetdicom/files/8514505/example.zip)
```
python -m pynetdicom storescu -v 127.0.0.1 104 .\example.dcm
```
3. Note that the resulting output file will have truncated PixelData:
```
(7fe0, 0010) Pixel Data                          OW: Array of 130116 elements
```
Should be:
```
(7fe0, 0010) Pixel Data                          OW: Array of 131072 elements
```

**Storage SCP Debug log**: [debug.txt](https://github.com/pydicom/pynetdicom/files/8514604/debug.txt)

**Your environment**
```
Windows-10-10.0.19041-SP0
Python  3.7.9 (tags/v3.7.9:13c94747c7, Aug 17 2020, 18:58:18) [MSC v.1900 64 bit (AMD64)]
pydicom  2.3.0
pynetdicom  2.0.1
```


Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/pydicom/pynetdicom/pull/766?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom) Report
> Merging [#766](https://codecov.io/gh/pydicom/pynetdicom/pull/766?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom) (27cac23) into [master](https://codecov.io/gh/pydicom/pynetdicom/commit/ddc34625e08f0f9a083a31619aa99aa431ee2bc3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom) (ddc3462) will **decrease** coverage by `0.02%`.
> The diff coverage is `100.00%`.

```diff
@@             Coverage Diff             @@
##            master     #766      +/-   ##
===========================================
- Coverage   100.00%   99.97%   -0.03%     
===========================================
  Files           28       28              
  Lines         8651     8654       +3     
===========================================
+ Hits          8651     8652       +1     
- Misses           0        2       +2     
```

| [Impacted Files](https://codecov.io/gh/pydicom/pynetdicom/pull/766?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom) | Coverage Δ | |
|---|---|---|
| [pynetdicom/dimse\_primitives.py](https://codecov.io/gh/pydicom/pynetdicom/pull/766/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom#diff-cHluZXRkaWNvbS9kaW1zZV9wcmltaXRpdmVzLnB5) | `100.00% <ø> (ø)` | |
| [pynetdicom/dimse.py](https://codecov.io/gh/pydicom/pynetdicom/pull/766/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom#diff-cHluZXRkaWNvbS9kaW1zZS5weQ==) | `100.00% <100.00%> (ø)` | |
| [pynetdicom/dimse\_messages.py](https://codecov.io/gh/pydicom/pynetdicom/pull/766/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom#diff-cHluZXRkaWNvbS9kaW1zZV9tZXNzYWdlcy5weQ==) | `100.00% <100.00%> (ø)` | |
| [pynetdicom/ae.py](https://codecov.io/gh/pydicom/pynetdicom/pull/766/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pydicom#diff-cHluZXRkaWNvbS9hZS5weQ==) | `99.45% <0.00%> (-0.55%)` | :arrow_down: |


Patch:
diff --git a/docs/changelog/v2.1.0.rst b/docs/changelog/v2.1.0.rst
index 07c30038b..2a9df97cf 100644
--- a/docs/changelog/v2.1.0.rst
+++ b/docs/changelog/v2.1.0.rst
@@ -7,9 +7,12 @@ Fixes
 .....
 
 * Fixed reserved A-ASSOCIATE-AC parameters being tested (:issue:`746`)
+* Fixed datasets not transferring correctly when using
+  :attr:`~pynetdicom._config.STORE_RECV_CHUNKED_DATASET` (:issue:`756`)
 
 Enhancements
 ............
 
+
 Changes
 .......
diff --git a/pynetdicom/dimse.py b/pynetdicom/dimse.py
index caf8a8a38..a92d1c97e 100644
--- a/pynetdicom/dimse.py
+++ b/pynetdicom/dimse.py
@@ -324,6 +324,8 @@ class DIMSEServiceProvider:
             #   Reset the DIMSE message, ready for the next one
             self.message.encoded_command_set = BytesIO()
             self.message.data_set = BytesIO()
+            self.message._data_set_file = None
+            self.message._data_set_path = None
             self.message = None
 
     def send_msg(self, primitive: DimsePrimitiveType, context_id: int) -> None:
diff --git a/pynetdicom/dimse_messages.py b/pynetdicom/dimse_messages.py
index a307cfcd4..76eec93c2 100644
--- a/pynetdicom/dimse_messages.py
+++ b/pynetdicom/dimse_messages.py
@@ -418,9 +418,6 @@ class DIMSEMessage:
             # xxxxxx11 - Command information, the last fragment
             control_header_byte = data[0]
 
-            # LOGGER.debug('Control header byte %s', control_header_byte)
-            # print(f'Control header byte {control_header_byte}')
-
             # COMMAND SET
             # P-DATA fragment contains Command Set information
             #   (control_header_byte is xxxxxx01 or xxxxxx11)
@@ -497,6 +494,7 @@ class DIMSEMessage:
                 #   number of P-DATA primitives.
                 if self._data_set_file:
                     self._data_set_file.write(data[1:])
+                    self._data_set_file.file.flush()
                 else:
                     cast(BytesIO, self.data_set).write(data[1:])
 
diff --git a/pynetdicom/dimse_primitives.py b/pynetdicom/dimse_primitives.py
index daa100bb9..0d9b20e28 100644
--- a/pynetdicom/dimse_primitives.py
+++ b/pynetdicom/dimse_primitives.py
@@ -21,11 +21,13 @@ from pynetdicom.utils import set_ae, decode_bytes, set_uid
 
 
 if TYPE_CHECKING:  # pragma: no cover
+    from io import BufferedWriter
     from typing import Protocol  # Python 3.8+
 
     class NTF(Protocol):
         # Protocol for a NamedTemporaryFile
         name: str
+        file: BufferedWriter
 
         def write(self, data: bytes) -> bytes:
             ...


Test patch:
diff --git a/pynetdicom/tests/test_service_storage.py b/pynetdicom/tests/test_service_storage.py
index 3fd41ad55..682ef983c 100644
--- a/pynetdicom/tests/test_service_storage.py
+++ b/pynetdicom/tests/test_service_storage.py
@@ -472,6 +472,7 @@ class TestStorageServiceClass:
         handlers = [(evt.EVT_C_STORE, handle)]
 
         self.ae = ae = AE()
+        ae.maximum_pdu_size = 256
         ae.add_supported_context(CTImageStorage)
         ae.add_requested_context(CTImageStorage)
         scp = ae.start_server(("localhost", 11112), block=False, evt_handlers=handlers)
@@ -492,6 +493,8 @@ class TestStorageServiceClass:
 
         ds = attrs["dataset"]
         assert "CompressedSamples^CT1" == ds.PatientName
+        assert "DataSetTrailingPadding" in ds
+        assert len(ds.DataSetTrailingPadding) == 126
 
         scp.shutdown()