Repository: pydicom/pydicom

Issue:
Write deflated content when called Transfer Syntax is Deflated Explicit VR Little Endian
**Describe the bug**
After using `dcmread` to read a deflated .dcm file created from pydicom's [CT_small.dcm sample](https://github.com/pydicom/pydicom/blob/v1.4.2/pydicom/data/test_files/CT_small.dcm), with the following file meta information
```
(0002, 0000) File Meta Information Group Length  UL: 178
(0002, 0001) File Meta Information Version       OB: b'\x00\x01'
(0002, 0002) Media Storage SOP Class UID         UI: CT Image Storage
(0002, 0003) Media Storage SOP Instance UID      UI: 1.3.6.1.4.1.5962.1.1.1.1.1.20040119072730.12322
(0002, 0010) Transfer Syntax UID                 UI: Deflated Explicit VR Little Endian
(0002, 0012) Implementation Class UID            UI: 1.2.40.0.13.1.1
(0002, 0013) Implementation Version Name         SH: 'dcm4che-2.0'
```

I use `save_as` to save the file. The output file has an unaltered file meta information section, but the group 8 elements and beyond are not written in deflated format, instead appearing to be LEE. In particular, the specific character set element is easily readable from a hex representation of the file, rather than appearing as gobbledygook like one would expect from a deflated stream.

**Expected behavior**
The bulk of the DCM to be written as Deflated Explicit VR Little Endian or the Transfer Syntax UID to be saved with a value that reflects the actual format of the DCM

**Steps To Reproduce**
```python
❯ py
>>> # CT_small_deflated.dcm is CT_small.dcm, deflated using dcm2dcm
>>> ds = pydicom.dcmread("CT_small_deflated.dcm")

>>> ds.save_as("ds_like_orig.dcm", write_like_original=True)
>>> pydicom.dcmread("ds_like_orig.dcm")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\blairyat\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pydicom\filereader.py", line 869, in dcmread
    dataset = read_partial(fp, stop_when, defer_size=defer_size,
  File "C:\Users\blairyat\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pydicom\filereader.py", line 729, in read_partial
    unzipped = zlib.decompress(zipped, -zlib.MAX_WBITS)
zlib.error: Error -3 while decompressing data: invalid stored block lengths

>>> ds.save_as("ds_not_like_orig.dcm", write_like_original=False)
>>> pydicom.dcmread("ds_not_like_orig.dcm")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\blairyat\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pydicom\filereader.py", line 869, in dcmread
    dataset = read_partial(fp, stop_when, defer_size=defer_size,
  File "C:\Users\blairyat\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pydicom\filereader.py", line 729, in read_partial
    unzipped = zlib.decompress(zipped, -zlib.MAX_WBITS)
zlib.error: Error -3 while decompressing data: invalid stored block lengths
```

**Your environment**
Please run the following and paste the output.
```powershell
❯ py -c "import platform; print(platform.platform())"
Windows-10-10.0.18362-SP0

❯ py -c "import sys; print('Python ', sys.version)"
Python  3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 22:39:24) [MSC v.1916 32 bit (Intel)]

❯ py -c "import pydicom; print('pydicom ', pydicom.__version__)"
pydicom  1.4.2
```



Discussion:
Of course, it's entirely possible I'm just not using the library properly.
Regardless, here are my original and generated files, as well as the reproduction steps:
[write_deflated_file.zip](https://github.com/pydicom/pydicom/files/4557981/write_deflated_file.zip)

Thanks for your attention in this. Assuming it's deemed to be a valid bug, I'd be happy to contribute to the development of a solution. (My initial thought is that the Transfer Syntax should be respected and that the file should be written deflated, but I've not investigated deeply.)
Try setting `ds.is_explicit_VR = True` before saving. Changing the Transfer Syntax isn't enough to control the encoding of the written dataset.

We should probably add a warning for when the transfer syntax doesn't match `is_implicit_VR` and `is_little_endian`...

I'm not sure about deflated off hand, I'll need to check
Thanks, @scaramallion. That had no effect. The output is exactly the same.

I fear I may not have been clear. I never did change the Transfer Syntax. It stayed at Deflated Explicit VR Little Endian, just as it was in the source…
We don't have anything out-of-the-box for writing deflated, this should work:
```python
import zlib

from pydicom import dcmread
from pydicom.data import get_testdata_file
from pydicom.filebase import DicomBytesIO
from pydicom.filewriter import write_file_meta_info, write_dataset
from pydicom.uid import DeflatedExplicitVRLittleEndian

ds = dcmread(get_testdata_file("CT_small.dcm"))
ds.file_meta.TransferSyntaxUID = DeflatedExplicitVRLittleEndian

with open('deflated.dcm', 'wb') as f:
    # Write preamble and DICM marker
    f.write(b'\x00' * 128)
    f.write(b'DICM')
    # Write file meta information elements
    write_file_meta_info(f, ds.file_meta)

    # Encode the dataset
    bytesio = DicomBytesIO()
    bytesio.is_little_endian = True
    bytesio.is_implicit_VR = False
    write_dataset(bytesio, ds)

    # Compress the encoded data and write to file
    compressor = zlib.compressobj(wbits=-zlib.MAX_WBITS)
    deflated = compressor.compress(bytesio.parent.getvalue())
    deflated += compressor.flush()
    if len(deflated) %2:
        deflated += b'\x00'

    f.write(deflated)

ds = dcmread('deflated.dcm')
print(ds)
```
Thanks, @scaramallion. Are you looking for a contribution, or are you thinking the design would be would be too complicated for an enthusiastic experienced programmer but first-time pydicomer?
(I'm assuming this would eventually just become automatic behaviour whenever saving something with the matching Transfer Syntax…)
I think this is definitely a contribution a first-timer could make, take a look at `filewriter.dcmwrite()`, if you add a check for the deflated transfer syntax around [line 968](https://github.com/pydicom/pydicom/blob/master/pydicom/filewriter.py#L968) then encode and deflate accordingly.
> this is definitely a contribution a first-timer could make

Then sign me up! Assuming you don't need it by tomorrow. (I won't be super slow. It's just that it's nearly bedtime and I have a day job.)

There's no rush, we (nearly) all have day jobs 

Patch:
diff --git a/pydicom/filereader.py b/pydicom/filereader.py
--- a/pydicom/filereader.py
+++ b/pydicom/filereader.py
@@ -710,7 +710,7 @@ def read_partial(fileobj, stop_when=None, defer_size=None,
         is_implicit_VR = False
         is_little_endian = False
     elif transfer_syntax == pydicom.uid.DeflatedExplicitVRLittleEndian:
-        # See PS3.6-2008 A.5 (p 71)
+        # See PS3.5 section A.5
         # when written, the entire dataset following
         #     the file metadata was prepared the normal way,
         #     then "deflate" compression applied.
diff --git a/pydicom/filewriter.py b/pydicom/filewriter.py
--- a/pydicom/filewriter.py
+++ b/pydicom/filewriter.py
@@ -3,6 +3,7 @@
 
 
 import warnings
+import zlib
 from struct import pack
 
 from pydicom.charset import (
@@ -15,7 +16,8 @@
 from pydicom.multival import MultiValue
 from pydicom.tag import (Tag, ItemTag, ItemDelimiterTag, SequenceDelimiterTag,
                          tag_in_exception)
-from pydicom.uid import UncompressedPixelTransferSyntaxes
+from pydicom.uid import (UncompressedPixelTransferSyntaxes,
+                         DeflatedExplicitVRLittleEndian)
 from pydicom.valuerep import extra_length_VRs
 from pydicom.values import convert_numbers
 
@@ -727,6 +729,42 @@ def write_file_meta_info(fp, file_meta, enforce_standard=True):
     fp.write(buffer.getvalue())
 
 
+def _write_dataset(fp, dataset, write_like_original):
+    """Write the Data Set to a file-like. Assumes the file meta information,
+    if any, has been written.
+    """
+
+    # if we want to write with the same endianess and VR handling as
+    # the read dataset we want to preserve raw data elements for
+    # performance reasons (which is done by get_item);
+    # otherwise we use the default converting item getter
+    if dataset.is_original_encoding:
+        get_item = Dataset.get_item
+    else:
+        get_item = Dataset.__getitem__
+
+    # WRITE DATASET
+    # The transfer syntax used to encode the dataset can't be changed
+    #   within the dataset.
+    # Write any Command Set elements now as elements must be in tag order
+    #   Mixing Command Set with other elements is non-conformant so we
+    #   require `write_like_original` to be True
+    command_set = get_item(dataset, slice(0x00000000, 0x00010000))
+    if command_set and write_like_original:
+        fp.is_implicit_VR = True
+        fp.is_little_endian = True
+        write_dataset(fp, command_set)
+
+    # Set file VR and endianness. MUST BE AFTER writing META INFO (which
+    #   requires Explicit VR Little Endian) and COMMAND SET (which requires
+    #   Implicit VR Little Endian)
+    fp.is_implicit_VR = dataset.is_implicit_VR
+    fp.is_little_endian = dataset.is_little_endian
+
+    # Write non-Command Set elements now
+    write_dataset(fp, get_item(dataset, slice(0x00010000, None)))
+
+
 def dcmwrite(filename, dataset, write_like_original=True):
     """Write `dataset` to the `filename` specified.
 
@@ -938,15 +976,6 @@ def dcmwrite(filename, dataset, write_like_original=True):
     else:
         fp = DicomFileLike(filename)
 
-    # if we want to write with the same endianess and VR handling as
-    # the read dataset we want to preserve raw data elements for
-    # performance reasons (which is done by get_item);
-    # otherwise we use the default converting item getter
-    if dataset.is_original_encoding:
-        get_item = Dataset.get_item
-    else:
-        get_item = Dataset.__getitem__
-
     try:
         # WRITE FILE META INFORMATION
         if preamble:
@@ -954,31 +983,32 @@ def dcmwrite(filename, dataset, write_like_original=True):
             fp.write(preamble)
             fp.write(b'DICM')
 
+        tsyntax = None
         if dataset.file_meta:  # May be an empty Dataset
             # If we want to `write_like_original`, don't enforce_standard
             write_file_meta_info(fp, dataset.file_meta,
                                  enforce_standard=not write_like_original)
+            tsyntax = getattr(dataset.file_meta, "TransferSyntaxUID", None)
+
+        if (tsyntax == DeflatedExplicitVRLittleEndian):
+            # See PS3.5 section A.5
+            # when writing, the entire dataset following
+            #     the file metadata is prepared the normal way,
+            #     then "deflate" compression applied.
+            buffer = DicomBytesIO()
+            _write_dataset(buffer, dataset, write_like_original)
+
+            # Compress the encoded data and write to file
+            compressor = zlib.compressobj(wbits=-zlib.MAX_WBITS)
+            deflated = compressor.compress(buffer.parent.getvalue())
+            deflated += compressor.flush()
+            if len(deflated) % 2:
+                deflated += b'\x00'
+
+            fp.write(deflated)
+        else:
+            _write_dataset(fp, dataset, write_like_original)
 
-        # WRITE DATASET
-        # The transfer syntax used to encode the dataset can't be changed
-        #   within the dataset.
-        # Write any Command Set elements now as elements must be in tag order
-        #   Mixing Command Set with other elements is non-conformant so we
-        #   require `write_like_original` to be True
-        command_set = get_item(dataset, slice(0x00000000, 0x00010000))
-        if command_set and write_like_original:
-            fp.is_implicit_VR = True
-            fp.is_little_endian = True
-            write_dataset(fp, command_set)
-
-        # Set file VR and endianness. MUST BE AFTER writing META INFO (which
-        #   requires Explicit VR Little Endian) and COMMAND SET (which requires
-        #   Implicit VR Little Endian)
-        fp.is_implicit_VR = dataset.is_implicit_VR
-        fp.is_little_endian = dataset.is_little_endian
-
-        # Write non-Command Set elements now
-        write_dataset(fp, get_item(dataset, slice(0x00010000, None)))
     finally:
         if not caller_owns_file:
             fp.close()


Test patch:
diff --git a/pydicom/tests/test_filewriter.py b/pydicom/tests/test_filewriter.py
--- a/pydicom/tests/test_filewriter.py
+++ b/pydicom/tests/test_filewriter.py
@@ -11,6 +11,7 @@
 
 from struct import unpack
 from tempfile import TemporaryFile
+import zlib
 
 import pytest
 
@@ -49,6 +50,7 @@
 
 unicode_name = get_charset_files("chrH31.dcm")[0]
 multiPN_name = get_charset_files("chrFrenMulti.dcm")[0]
+deflate_name = get_testdata_file("image_dfl.dcm")
 
 base_version = '.'.join(str(i) for i in __version_info__)
 
@@ -77,6 +79,18 @@ def bytes_identical(a_bytes, b_bytes):
         return False, pos  # False if not identical, position of 1st diff
 
 
+def as_assertable(dataset):
+    """Copy the elements in a Dataset (including the file_meta, if any)
+       to a set that can be safely compared using pytest's assert.
+       (Datasets can't be so compared because DataElements are not
+       hashable.)"""
+    safe_dict = dict((str(elem.tag) + " " + elem.keyword, elem.value)
+                     for elem in dataset)
+    if hasattr(dataset, "file_meta"):
+        safe_dict.update(as_assertable(dataset.file_meta))
+    return safe_dict
+
+
 class TestWriteFile(object):
     def setup(self):
         self.file_out = TemporaryFile('w+b')
@@ -222,6 +236,41 @@ def test_write_empty_sequence(self):
         ds = read_file(self.file_out)
         assert ds.PerformedProcedureCodeSequence == []
 
+    def test_write_deflated_retains_elements(self):
+        """Read a Deflated Explicit VR Little Endian file, write it,
+           and then read the output, to verify that the written file
+           contains the same data.
+           """
+        original = read_file(deflate_name)
+        original.save_as(self.file_out)
+
+        self.file_out.seek(0)
+        rewritten = read_file(self.file_out)
+
+        assert as_assertable(rewritten) == as_assertable(original)
+
+    def test_write_deflated_deflates_post_file_meta(self):
+        """Read a Deflated Explicit VR Little Endian file, write it,
+           and then check the bytes in the output, to verify that the
+           written file is deflated past the file meta information.
+           """
+        original = read_file(deflate_name)
+        original.save_as(self.file_out)
+
+        first_byte_past_file_meta = 0x14e
+        with open(deflate_name, "rb") as original_file:
+            original_file.seek(first_byte_past_file_meta)
+            original_post_meta_file_bytes = original_file.read()
+        unzipped_original = zlib.decompress(original_post_meta_file_bytes,
+                                            -zlib.MAX_WBITS)
+
+        self.file_out.seek(first_byte_past_file_meta)
+        rewritten_post_meta_file_bytes = self.file_out.read()
+        unzipped_rewritten = zlib.decompress(rewritten_post_meta_file_bytes,
+                                             -zlib.MAX_WBITS)
+
+        assert unzipped_rewritten == unzipped_original
+
 
 class TestScratchWriteDateTime(TestWriteFile):
     """Write and reread simple or multi-value DA/DT/TM data elements"""