Repository: pulp/pulpcore

Issue:
Exports are bottlenecked by gzip compression which cannot be disabled
**Version**
Any

**Describe the bug**

> 
> We identified a severe bottleneck in the way hammer exports work and found the code in upstream Pulp.
> 
> Line 406 of https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/export.py
> 
> "with tarfile.open(tarfile_fp, "w|gz", fileobj=split_process.stdin)"
> 
> The tarfile Python module creates tar files using the gizp Python module.
> 
> Data compression for the gzip Python module is provided by the zlib Python module.
> 
> The zlib Python module calls the zlib library.
> 
> If defaults are used the whole way through this series of events, the result is a single threaded pulp process doing compression of a tarball containing a massive content library. This bottleneck and can make large hammer exports take several days.
> 
> Modifying the lines that tell the tarfile.open function to NOT use compression ( change "w|gz" to "w" ) dramatically speeds up the hammer export. In our testing it reduced the time from days to just hours. The drawback is the file size was significantly larger, but the trade-off is worthwhile given we have tight timeframes and plentiful disk capacity. 
> 
> Can this bottleneck be addressed with multi-threaded gzip compression?
> 
> and/or
> 
> Can a hammer command line option for no compression be implemented?
> 
> Run a hammer export and monitor Pulp processes. One process with run at 100% CPU. Modify the abovementioned Python script to NOT use gzip encryption, and an uncompressed tarball will be created instead much quicker and with multiple Pulp processes. 
> 
> 
> Steps to Reproduce:
> 1. Run a "hammer export". Monitor the Pulp process CPU usage and time taken to complete export.
> 2. Change the abovementioned Python code in Pulp.
> 3. Run a "hammer export" again and note performance improvement.


**Expected behavior**
A clear and concise description of what you expected to happen.

**Additional context**
https://bugzilla.redhat.com/show_bug.cgi?id=2188504


Discussion:
I don't know that disabling compression entirely is a good idea.  However, the default compression level used is level 9, the most computationally expensive (slow) one.  Given level 9 is (roughly) 6x slower than level 1, but only compresses about 20% better, this is probably a poor tradeoff.

We should look at using levels 1-3 instead.

Resolving patch:
diff --git a/pulpcore/app/monkeypatch.py b/pulpcore/app/monkeypatch.py
new file mode 100644
--- /dev/null
+++ b/pulpcore/app/monkeypatch.py
@@ -0,0 +1,66 @@
+import sys
+
+# This is a monkeypatch for https://github.com/pulp/pulpcore/issues/3869
+if sys.version_info.major == 3 and sys.version_info.minor < 12:
+    # Code copied from the Python 3.12 standard library
+    # We modify the default gzip compression level for writing streams from
+    # 9 to 1, attempting to vendor the minimum amount of code.
+    # -------------------------------------------------------------------
+    # tarfile.py
+    # -------------------------------------------------------------------
+    # Copyright (C) 2002 Lars Gustaebel <lars@gustaebel.de>
+    # All rights reserved.
+    #
+    # Permission  is  hereby granted,  free  of charge,  to  any person
+    # obtaining a  copy of  this software  and associated documentation
+    # files  (the  "Software"),  to   deal  in  the  Software   without
+    # restriction,  including  without limitation  the  rights to  use,
+    # copy, modify, merge, publish, distribute, sublicense, and/or sell
+    # copies  of  the  Software,  and to  permit  persons  to  whom the
+    # Software  is  furnished  to  do  so,  subject  to  the  following
+    # conditions:
+    #
+    # The above copyright  notice and this  permission notice shall  be
+    # included in all copies or substantial portions of the Software.
+    #
+    # THE SOFTWARE IS PROVIDED "AS  IS", WITHOUT WARRANTY OF ANY  KIND,
+    # EXPRESS OR IMPLIED, INCLUDING  BUT NOT LIMITED TO  THE WARRANTIES
+    # OF  MERCHANTABILITY,  FITNESS   FOR  A  PARTICULAR   PURPOSE  AND
+    # NONINFRINGEMENT.  IN  NO  EVENT SHALL  THE  AUTHORS  OR COPYRIGHT
+    # HOLDERS  BE LIABLE  FOR ANY  CLAIM, DAMAGES  OR OTHER  LIABILITY,
+    # WHETHER  IN AN  ACTION OF  CONTRACT, TORT  OR OTHERWISE,  ARISING
+    # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+    # OTHER DEALINGS IN THE SOFTWARE.
+
+    import tarfile
+    from tarfile import NUL
+    import struct
+    import os
+    import time
+
+    class _Stream(tarfile._Stream):
+        """Class that serves as an adapter between TarFile and
+        a stream-like object.  The stream-like object only
+        needs to have a read() or write() method and is accessed
+        blockwise.  Use of gzip or bzip2 compression is possible.
+        A stream-like object could be for example: sys.stdin,
+        sys.stdout, a socket, a tape device etc.
+
+        _Stream is intended to be used only internally.
+        """
+
+        def _init_write_gz(self):
+            """Initialize for writing with gzip compression."""
+            self.cmp = self.zlib.compressobj(
+                1, self.zlib.DEFLATED, -self.zlib.MAX_WBITS, self.zlib.DEF_MEM_LEVEL, 0
+            )
+            timestamp = struct.pack("<L", int(time.time()))
+            self.__write(b"\037\213\010\010" + timestamp + b"\002\377")
+            if self.name.endswith(".gz"):
+                self.name = self.name[:-3]
+            # Honor "directory components removed" from RFC1952
+            self.name = os.path.basename(self.name)
+            # RFC1952 says we must use ISO-8859-1 for the FNAME field.
+            self.__write(self.name.encode("iso-8859-1", "replace") + NUL)
+
+    tarfile._Stream = _Stream
diff --git a/pulpcore/app/tasks/export.py b/pulpcore/app/tasks/export.py
--- a/pulpcore/app/tasks/export.py
+++ b/pulpcore/app/tasks/export.py
@@ -5,7 +5,6 @@
 import os.path
 import subprocess
 import tarfile
-
 from distutils.util import strtobool
 from gettext import gettext as _
 from glob import glob


Test patch: