Repository: beetbox/beets

Issue:
fetchart: Use the right filename extension according to the image's MIME type
This is something I noticed today when using fetchart with the fanarttv plugin - one random albumart file it downloaded was actually a PNG file, not a JPG. This wasn't fetchart's fault - I verified that the source image was indeed a PNG from fanarttv despite having the .jpg extension.

But, I think a nice feature would be to include some form of image validty check and perhaps offer to convert to the image to the desired default format.

fetchart: Use the right filename extension according to the image's MIME type
This is something I noticed today when using fetchart with the fanarttv plugin - one random albumart file it downloaded was actually a PNG file, not a JPG. This wasn't fetchart's fault - I verified that the source image was indeed a PNG from fanarttv despite having the .jpg extension.

But, I think a nice feature would be to include some form of image validty check and perhaps offer to convert to the image to the desired default format.



Discussion:
Wow! That's too bad. Is there a way we can reproduce this (i.e., can we fake fetching art for the album you have to get the same image)?

Ha, it's hardly a troubling issue, but thought it was worth mentioning! The album in question was 9 by Damien Rice. If you grab that from fanarttv you'll see what I mean. 

Gnome image viewer wouldn't open it. Gimp opened it (I assume it ignores file types and goes by the file header instead) and when I checked it with the file command, it said it was a PNG. 

``` bash
$ wget -q https://fanart.tv/detailpreview/fanart/music/2119beb8-6ac5-4f21-82a4-b831c90c0024/albumcover/9-4f31935ae2a25.jpg
$ file 9-4f31935ae2a25.jpg 
9-4f31935ae2a25.jpg: PNG image data, 280 x 280, 8-bit/color RGB, non-interlaced
```

Yep, certainly seems like an issue! Thanks @broddo 😄.

That's the one. Sorry I should have provided you with the link. I'm half asleep here. 

Haha, no problem! 👍 @sampsyo Do we actually make an attempt at detecting the file type currently and converting if it's not a JPEG?

Aha!

We don't check the data itself, but we do check the MIME type that the server reports. Fortunately, it looks like fanart.tv doesn't lie there:

```
$ curl -I "https://fanart.tv/detailpreview/fanart/music/2119beb8-6ac5-4f21-82a4-b831c90c0024/albumcover/9-4f31935ae2a25.jpg"
HTTP/1.1 200 OK
Date: Thu, 16 Jun 2016 23:03:59 GMT
Content-Type: image/png
```

The trick is that our `fetch_image` method only validates that the file is either reported as a PNG or a JPEG; it doesn't assign the extension based on that. In fact, unless I'm reading this wrong, I think the file extension is actually _hard-coded_ as JPEG? In any case, we should instead use the MIME type to decide the extension for the downloaded file.

Assigning this to myself as a reminder, but I'll likely not tackle it until in a few weeks. So if anyone wants to step up, feel free to do so, no need to notify me.

Wow! That's too bad. Is there a way we can reproduce this (i.e., can we fake fetching art for the album you have to get the same image)?

Ha, it's hardly a troubling issue, but thought it was worth mentioning! The album in question was 9 by Damien Rice. If you grab that from fanarttv you'll see what I mean. 

Gnome image viewer wouldn't open it. Gimp opened it (I assume it ignores file types and goes by the file header instead) and when I checked it with the file command, it said it was a PNG. 

``` bash
$ wget -q https://fanart.tv/detailpreview/fanart/music/2119beb8-6ac5-4f21-82a4-b831c90c0024/albumcover/9-4f31935ae2a25.jpg
$ file 9-4f31935ae2a25.jpg 
9-4f31935ae2a25.jpg: PNG image data, 280 x 280, 8-bit/color RGB, non-interlaced
```

Yep, certainly seems like an issue! Thanks @broddo 😄.

That's the one. Sorry I should have provided you with the link. I'm half asleep here. 

Haha, no problem! 👍 @sampsyo Do we actually make an attempt at detecting the file type currently and converting if it's not a JPEG?

Aha!

We don't check the data itself, but we do check the MIME type that the server reports. Fortunately, it looks like fanart.tv doesn't lie there:

```
$ curl -I "https://fanart.tv/detailpreview/fanart/music/2119beb8-6ac5-4f21-82a4-b831c90c0024/albumcover/9-4f31935ae2a25.jpg"
HTTP/1.1 200 OK
Date: Thu, 16 Jun 2016 23:03:59 GMT
Content-Type: image/png
```

The trick is that our `fetch_image` method only validates that the file is either reported as a PNG or a JPEG; it doesn't assign the extension based on that. In fact, unless I'm reading this wrong, I think the file extension is actually _hard-coded_ as JPEG? In any case, we should instead use the MIME type to decide the extension for the downloaded file.

Assigning this to myself as a reminder, but I'll likely not tackle it until in a few weeks. So if anyone wants to step up, feel free to do so, no need to notify me.


Resolving patch:
diff --git a/beetsplug/fetchart.py b/beetsplug/fetchart.py
--- a/beetsplug/fetchart.py
+++ b/beetsplug/fetchart.py
@@ -39,9 +39,11 @@
 except ImportError:
     HAVE_ITUNES = False
 
-IMAGE_EXTENSIONS = ['png', 'jpg', 'jpeg']
-CONTENT_TYPES = ('image/jpeg', 'image/png')
-DOWNLOAD_EXTENSION = '.jpg'
+CONTENT_TYPES = {
+    'image/jpeg': [b'jpg', b'jpeg'],
+    'image/png': [b'png']
+}
+IMAGE_EXTENSIONS = [ext for exts in CONTENT_TYPES.values() for ext in exts]
 
 
 class Candidate(object):
@@ -227,8 +229,8 @@ def fetch_image(self, candidate, extra):
         try:
             with closing(self.request(candidate.url, stream=True,
                                       message=u'downloading image')) as resp:
-                if 'Content-Type' not in resp.headers \
-                        or resp.headers['Content-Type'] not in CONTENT_TYPES:
+                ct = resp.headers.get('Content-Type', None)
+                if ct not in CONTENT_TYPES:
                     self._log.debug(
                         u'not a supported image: {}',
                         resp.headers.get('Content-Type') or u'no content type',
@@ -237,7 +239,7 @@ def fetch_image(self, candidate, extra):
                     return
 
                 # Generate a temporary file with the correct extension.
-                with NamedTemporaryFile(suffix=DOWNLOAD_EXTENSION,
+                with NamedTemporaryFile(suffix=b'.' + CONTENT_TYPES[ct][0],
                                         delete=False) as fh:
                     for chunk in resp.iter_content(chunk_size=1024):
                         fh.write(chunk)


Test patch:
diff --git a/test/test_art.py b/test/test_art.py
--- a/test/test_art.py
+++ b/test/test_art.py
@@ -46,7 +46,7 @@ def setUp(self):
 
 
 class FetchImageTest(UseThePlugin):
-    URL = 'http://example.com'
+    URL = 'http://example.com/test.jpg'
 
     @responses.activate
     def run(self, *args, **kwargs):
@@ -61,18 +61,23 @@ def setUp(self):
         self.dpath = os.path.join(self.temp_dir, b'arttest')
         self.source = fetchart.RemoteArtSource(logger, self.plugin.config)
         self.extra = {'maxwidth': 0}
+        self.candidate = fetchart.Candidate(logger, url=self.URL)
 
     def test_invalid_type_returns_none(self):
         self.mock_response('image/watercolour')
-        candidate = fetchart.Candidate(logger, url=self.URL)
-        self.source.fetch_image(candidate, self.extra)
-        self.assertEqual(candidate.path, None)
+        self.source.fetch_image(self.candidate, self.extra)
+        self.assertEqual(self.candidate.path, None)
 
     def test_jpeg_type_returns_path(self):
         self.mock_response('image/jpeg')
-        candidate = fetchart.Candidate(logger, url=self.URL)
-        self.source.fetch_image(candidate, self.extra)
-        self.assertNotEqual(candidate.path, None)
+        self.source.fetch_image(self.candidate, self.extra)
+        self.assertNotEqual(self.candidate.path, None)
+
+    def test_extension_set_by_content_type(self):
+        self.mock_response('image/png')
+        self.source.fetch_image(self.candidate, self.extra)
+        self.assertEqual(os.path.splitext(self.candidate.path)[1], b'.png')
+        self.assertExists(self.candidate.path)
 
 
 class FSArtTest(UseThePlugin):