Repository: dtrx-py/dtrx (language: python)

Issue:
Official list of supported file types
Hi,

we are using dtrx as part of larger scripts to unroll / extract huge bulks of files. As part of the script dtrx receives input from 'find'. However, this leads to the situation that we call dtrx on files that are obviously no archives. Hence, I would like to create a filter to only call dtrx on file that have actual supported extensions.

Is there an official list of all the file types supported by dtrx? The readme list a bunch in the README but this list is not exhaustive.

Thanks


Discussion / hints:
Hi @an-schall , thanks for filing this! I think it's something we can add- would a command-line flag to dump out the list make sense?

Something like:

```bash
$ dtrx --dump-extension-map
Z
bz2
bzip2
bzip2
compress
gz
gzip
gzip
lrz
lrz
lrzip
lz
lzma
lzma
lzma
tar.Z
tar.bz2
tar.gz
tar.lrz
tar.lz
tar.lzma
tar.xz
tar.zst
taz
tb2
tbz
tbz2
tgz
tlz
txz
xz
xz
xz
zstd
```

Note that `dtrx` also maps by mime-type and using the output of the `file` command, but I think extension matching is what would be the most useful to you?
Oh also note that `dtrx` relies on the matching decompression application for the actual decompression (eg you'd need to have `tar`, etc), so the list would indicate which extensions dtrx supports but might not match what the current system can decompress.
Hi,

indeed extension matching is what we are looking for. Also your last point is rather valid: We experienced that dtrx does allow for decompressing jar files, while this is not explicitly listed. Would there be a straight-forward war to get a comprehensive list of all the file types covered even including those supported by the 3rd party applications?

Cheers
I think we could provide a list of:
- built in extensions
- extensions from MIME type mapping

Let me take a crack at this, I'll post here when I have something.

Fix patch:
diff --git a/Dockerfile b/Dockerfile
index bc80242..0b24bb1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -5,6 +5,7 @@ ARG DEBIAN_FRONTEND=noninteractive
 RUN apt-get update && apt-get install -y --no-install-recommends \
     arj \
     binutils \
+    brotli \
     cpio \
     file \
     gzip \
@@ -26,13 +27,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     software-properties-common && \
     add-apt-repository ppa:deadsnakes/ppa && apt-get update && \
     bash -c "\
-        apt-get install -y \
-            python2.7{,-dev} \
-            python3.6{,-dev} \
-            python3.7{,-dev,-distutils} \
-            python3.8{,-dev} \
-            python3.9{,-dev,-distutils}\
-            python3-distutils" \
+    apt-get install -y \
+    python2.7{,-dev} \
+    python3.6{,-dev} \
+    python3.7{,-dev,-distutils} \
+    python3.8{,-dev} \
+    python3.9{,-dev,-distutils}\
+    python3-distutils" \
     && rm -rf /var/lib/apt/lists/*
 
 # create a user inside the container. if you specify your UID when building the
diff --git a/dtrx/dtrx.py b/dtrx/dtrx.py
index 6d01fdc..16dfa1a 100755
--- a/dtrx/dtrx.py
+++ b/dtrx/dtrx.py
@@ -214,6 +214,7 @@ class BaseExtractor(object):
         "lrzip": ["lrzcat", "-q"],
         "lrz": ["lrzcat", "-q"],
         "zstd": ["zstd", "-d"],
+        "br": ["br", "--decompress"],
     }
     name_checker = DirectoryChecker
 
@@ -613,7 +614,16 @@ def __init__(self, filename, encoding):
         self.user_stdin = True
 
     def extract_archive(self):
-        self.extract_pipe = self.extract_command + [self.filename]
+        # the commands provided by the child class have optional format codes
+        # that will be replaced here
+        extract_fmt_args = {
+            "OUTPUT_FILE": os.path.splitext(os.path.basename(self.filename))[0],
+        }
+        formatted_extract_commands = [
+            x.format(**extract_fmt_args) for x in self.extract_command
+        ]
+
+        self.extract_pipe = formatted_extract_commands + [self.filename]
         BaseExtractor.extract_archive(self)
 
     def get_filenames(self):
@@ -722,6 +732,17 @@ def get_filenames(self):
         self.archive.close()
 
 
+class BrotliExtractor(NoPipeExtractor):
+    file_type = "brotli file"
+    extract_command = ["brotli", "--decompress", "--output={OUTPUT_FILE}"]
+    # brotli command line doesn't support this mode
+    list_command = ["false"]
+
+    def get_filenames(self):
+        # just raise an error, this is not supported
+        raise ExtractorError
+
+
 class CABExtractor(NoPipeExtractor):
     file_type = "CAB archive"
     extract_command = ["cabextract", "-q"]
@@ -1233,6 +1254,10 @@ class ExtractorBuilder(object):
             ),
             "magic": ("Zstandard compressed data",),
         },
+        "brotli": {
+            "extractors": (BrotliExtractor,),
+            "extensions": ("br",),
+        },
         "compress": {"extractors": (CompressionExtractor,)},
     }
 


Test patch:
diff --git a/tests/test-text.br b/tests/test-text.br
new file mode 100644
index 0000000..86e96a7
Binary files /dev/null and b/tests/test-text.br differ
diff --git a/tests/tests.yml b/tests/tests.yml
index b163ce1..91911e9 100644
--- a/tests/tests.yml
+++ b/tests/tests.yml
@@ -855,3 +855,8 @@
   error: true
   grep: "cannot extract encrypted archive"
   cleanup: stty -F /dev/stdout echo
+
+- name: brotli
+  filenames: test-text.br
+  baseline: |
+    brotli --decompress $1 --output=test-text