Repository: pulp/pulpcore

Issue:
Wrong ContentType header returned for some files via content app
**Version**
All versions

**Describe the bug**
.xml.gz Files being served by the content app have a `ContentType` header of `text/xml` instead of `application/gzip`.

**To Reproduce**
Sync and publish any RPM repo, use HTTPie to request one of the files, observe the headers

**Expected behavior**
`ContentType` should be `application/gzip`, the type of the outer file rather than that of the decompressed contents.

**Additional context**
https://bugzilla.redhat.com/show_bug.cgi?id=2064092


Discussion:
This is caused by an invalid value returned from `mimetypes.guess_type` (https://docs.python.org/3/library/mimetypes.html#mimetypes.guess_type).

Seems like the `mimetypes` module fetches types from the system (https://stackoverflow.com/a/40540381/3907906).
Should the mimetype not be determined by the content being served? Or maybe the published artifact. Guessing the mimetype from the artifact is probably not a good idea. Also when on cloud storage, it means that we need to load the artifact into pulp for guessing. That is even worse.
There is also a good discussion about the similar problem of serving `.xml.gz` files in pulp2: https://pulp.plan.io/issues/1781#note-24. Adjacent comments reference the PR that resolves the issue.

Their workflow however does not work in our use case. In addition to that, I cannot simply add a new mime type to the existing database of types like this: `mimetypes.add_type("application/gzip", ".xml.gz")`. The type is still not correctly detected and it returns `('text/xml', 'gzip')` all the time.

Loading artifacts from cloud storage is also a no-go for me.
Actually, it looks like we are guessing the mime-type just by means of the filename requested. So it is not a data from cloud issue. But still we are guessing where the content object could tell us the right answer.
> Should the mimetype not be determined by the content being served?

Currently, the type is determined by a file extension: https://github.com/pulp/pulpcore/blob/01557ca70f0863ec006977fb9bca6f8af8285dd6/pulpcore/content/handler.py#L337


Resolving patch:
diff --git a/pulpcore/app/mime_types.py b/pulpcore/app/mime_types.py
new file mode 100644
--- /dev/null
+++ b/pulpcore/app/mime_types.py
@@ -0,0 +1,197 @@
+import os
+
+# The mapping was retrieved from the following sources:
+# 1. https://docs.python.org/3/library/mimetypes.html#mimetypes.types_map
+# 2. https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types
+TYPES_MAP = {
+    ".3g2": "audio/3gpp2",
+    ".3gp": "audio/3gpp",
+    ".3gpp": "audio/3gpp",
+    ".3gpp2": "audio/3gpp2",
+    ".7z": "application/x-7z-compressed",
+    ".a": "application/octet-stream",
+    ".aac": "audio/aac",
+    ".abw": "application/x-abiword",
+    ".adts": "audio/aac",
+    ".ai": "application/postscript",
+    ".aif": "audio/x-aiff",
+    ".aifc": "audio/x-aiff",
+    ".aiff": "audio/x-aiff",
+    ".arc": "application/x-freearc",
+    ".ass": "audio/aac",
+    ".au": "audio/basic",
+    ".avif": "image/avif",
+    ".azw": "application/vnd.amazon.ebook",
+    ".bat": "text/plain",
+    ".bcpio": "application/x-bcpio",
+    ".bin": "application/octet-stream",
+    ".bmp": "image/x-ms-bmp",
+    ".bz": "application/x-bzip",
+    ".bz2": "application/x-bzip2",
+    ".c": "text/plain",
+    ".cda": "application/x-cdf",
+    ".cdf": "application/x-netcdf",
+    ".cpio": "application/x-cpio",
+    ".csh": "application/x-csh",
+    ".css": "text/css",
+    ".csv": "text/csv",
+    ".dll": "application/octet-stream",
+    ".doc": "application/msword",
+    ".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+    ".dot": "application/msword",
+    ".dvi": "application/x-dvi",
+    ".eml": "message/rfc822",
+    ".eot": "application/vnd.ms-fontobject",
+    ".eps": "application/postscript",
+    ".epub": "application/epub+zip",
+    ".etx": "text/x-setext",
+    ".exe": "application/octet-stream",
+    ".gif": "image/gif",
+    ".gtar": "application/x-gtar",
+    ".gz": "application/gzip",
+    ".gzip": "application/gzip",
+    ".h": "text/plain",
+    ".h5": "application/x-hdf5",
+    ".hdf": "application/x-hdf",
+    ".heic": "image/heic",
+    ".heif": "image/heif",
+    ".htm": "text/html",
+    ".html": "text/html",
+    ".ico": "image/vnd.microsoft.icon",
+    ".ics": "text/calendar",
+    ".ief": "image/ief",
+    ".jar": "application/java-archive",
+    ".jpe": "image/jpeg",
+    ".jpeg": "image/jpeg",
+    ".jpg": "image/jpeg",
+    ".js": "application/javascript",
+    ".json": "application/json",
+    ".jsonld": "application/ld+json",
+    ".ksh": "text/plain",
+    ".latex": "application/x-latex",
+    ".loas": "audio/aac",
+    ".m1v": "video/mpeg",
+    ".m3u": "application/vnd.apple.mpegurl",
+    ".m3u8": "application/vnd.apple.mpegurl",
+    ".man": "application/x-troff-man",
+    ".me": "application/x-troff-me",
+    ".mht": "message/rfc822",
+    ".mhtml": "message/rfc822",
+    ".mid": "audio/x-midi",
+    ".midi": "audio/x-midi",
+    ".mif": "application/x-mif",
+    ".mjs": "application/javascript",
+    ".mov": "video/quicktime",
+    ".movie": "video/x-sgi-movie",
+    ".mp2": "audio/mpeg",
+    ".mp3": "audio/mpeg",
+    ".mp4": "video/mp4",
+    ".mpa": "video/mpeg",
+    ".mpe": "video/mpeg",
+    ".mpeg": "video/mpeg",
+    ".mpg": "video/mpeg",
+    ".mpkg": "application/vnd.apple.installer+xml",
+    ".ms": "application/x-troff-ms",
+    ".nc": "application/x-netcdf",
+    ".nws": "message/rfc822",
+    ".o": "application/octet-stream",
+    ".obj": "application/octet-stream",
+    ".oda": "application/oda",
+    ".odp": "application/vnd.oasis.opendocument.presentation",
+    ".ods": "application/vnd.oasis.opendocument.spreadsheet",
+    ".odt": "application/vnd.oasis.opendocument.text",
+    ".oga": "audio/ogg",
+    ".ogv": "video/ogg",
+    ".ogx": "application/ogg",
+    ".opus": "audio/opus",
+    ".otf": "font/otf",
+    ".p12": "application/x-pkcs12",
+    ".p7c": "application/pkcs7-mime",
+    ".pbm": "image/x-portable-bitmap",
+    ".pdf": "application/pdf",
+    ".pfx": "application/x-pkcs12",
+    ".pgm": "image/x-portable-graymap",
+    ".php": "application/x-httpd-php",
+    ".pl": "text/plain",
+    ".png": "image/png",
+    ".pnm": "image/x-portable-anymap",
+    ".pot": "application/vnd.ms-powerpoint",
+    ".ppa": "application/vnd.ms-powerpoint",
+    ".ppm": "image/x-portable-pixmap",
+    ".pps": "application/vnd.ms-powerpoint",
+    ".ppt": "application/vnd.ms-powerpoint",
+    ".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
+    ".ps": "application/postscript",
+    ".pwz": "application/vnd.ms-powerpoint",
+    ".py": "text/x-python",
+    ".pyc": "application/x-python-code",
+    ".pyo": "application/x-python-code",
+    ".qt": "video/quicktime",
+    ".ra": "audio/x-pn-realaudio",
+    ".ram": "application/x-pn-realaudio",
+    ".rar": "application/vnd.rar",
+    ".ras": "image/x-cmu-raster",
+    ".rdf": "application/xml",
+    ".rgb": "image/x-rgb",
+    ".roff": "application/x-troff",
+    ".rtf": "application/rtf",
+    ".rtx": "text/richtext",
+    ".sgm": "text/x-sgml",
+    ".sgml": "text/x-sgml",
+    ".sh": "application/x-sh",
+    ".shar": "application/x-shar",
+    ".snd": "audio/basic",
+    ".so": "application/octet-stream",
+    ".src": "application/x-wais-source",
+    ".sv4cpio": "application/x-sv4cpio",
+    ".sv4crc": "application/x-sv4crc",
+    ".svg": "image/svg+xml",
+    ".swf": "application/x-shockwave-flash",
+    ".t": "application/x-troff",
+    ".tar": "application/x-tar",
+    ".tcl": "application/x-tcl",
+    ".tex": "application/x-tex",
+    ".texi": "application/x-texinfo",
+    ".texinfo": "application/x-texinfo",
+    ".tif": "image/tiff",
+    ".tiff": "image/tiff",
+    ".tr": "application/x-troff",
+    ".ts": "video/mp2t",
+    ".tsv": "text/tab-separated-values",
+    ".ttf": "font/ttf",
+    ".txt": "text/plain",
+    ".ustar": "application/x-ustar",
+    ".vcf": "text/x-vcard",
+    ".vsd": "application/vnd.visio",
+    ".wasm": "application/wasm",
+    ".wav": "audio/x-wav",
+    ".weba": "audio/webm",
+    ".webm": "video/webm",
+    ".webmanifest": "application/manifest+json",
+    ".webp": "image/webp",
+    ".wiz": "application/msword",
+    ".woff": "font/woff",
+    ".woff2": "font/woff2",
+    ".wsdl": "application/xml",
+    ".xbm": "image/x-xbitmap",
+    ".xhtml": "application/xhtml+xml",
+    ".xlb": "application/vnd.ms-excel",
+    ".xls": "application/vnd.ms-excel",
+    ".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+    ".xml": "text/xml",
+    ".xpdl": "application/xml",
+    ".xpm": "image/x-xpixmap",
+    ".xsl": "application/xml",
+    ".xul": "application/vnd.mozilla.xul+xml",
+    ".xwd": "image/x-xwindowdump",
+    ".xz": "application/x-xz",
+    ".zip": "application/zip",
+    ".zst": "application/zstd",
+    ".zstd": "application/zstd",
+}
+
+
+def get_type(url):
+    """Return the mime-type of a file specified by its URL."""
+    _, ext = os.path.splitext(url)
+    return TYPES_MAP.get(ext.lower())
diff --git a/pulpcore/content/handler.py b/pulpcore/content/handler.py
--- a/pulpcore/content/handler.py
+++ b/pulpcore/content/handler.py
@@ -1,6 +1,5 @@
 import asyncio
 import logging
-import mimetypes
 import os
 import re
 from gettext import gettext as _
@@ -36,6 +35,8 @@
     Remote,
     RemoteArtifact,
 )
+from pulpcore.app import mime_types  # noqa: E402: module level not at top of file
+
 from pulpcore.exceptions import UnsupportedDigestValidationError  # noqa: E402
 
 from jinja2 import Template  # noqa: E402: module level not at top of file
@@ -307,7 +308,7 @@ def response_headers(path):
         Returns:
             headers (dict): A dictionary of response headers.
         """
-        content_type, encoding = mimetypes.guess_type(path)
+        content_type = mime_types.get_type(path)
         headers = {}
         if content_type:
             headers["Content-Type"] = content_type


Test patch: