Repository: python-pillow/Pillow

Issue:
Exception while reading EXIF data from JPEG
There is one weird image that causes `Image.open()` to generate exception on 32-bit Python on windows. It looks like pillow thinks that EXIF data is corrupted though other applications can read exif data normally. 

### What did you do?

Call `PIL.Image.open("file.jpg")`

### What did you expect to happen?

Call to return normally without exception or complaints

### What actually happened?

Pillow generates multiple UserWarning messages about possibly corrupt EXIF, and it also raises exception on 32-bit Python build.

### What versions of Pillow and Python are you using?

Pillow 5.2.0, Python 3.6

Here is what happens on 32-bit Windows build:

```
(venv-win-py36) PS M:\> python
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import Image
>>> Image.open('046.JPG')
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 1441814 bytes but only got 0. Skipping tag 33
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 428 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 196608 bytes but only got 0. Skipping tag 60
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 196608 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 524288 bytes but only got 0. Skipping tag 240
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 1048576 bytes but only got 0. Skipping tag 13222
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 2097152 bytes but only got 0. Skipping tag 1860
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 458752 bytes but only got 0. Skipping tag 10869
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 458752 bytes but only got 0. Skipping tag 1821
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 393216 bytes but only got 0. Skipping tag 1024
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 786432 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 1572864 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 33752069 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "M:\venv-win-py36\lib\site-packages\PIL\Image.py", line 2609, in open
    im = _open_core(fp, filename, prefix)
  File "M:\venv-win-py36\lib\site-packages\PIL\Image.py", line 2599, in _open_core
    im = factory(fp, filename)
  File "M:\venv-win-py36\lib\site-packages\PIL\JpegImagePlugin.py", line 780, in jpeg_factory
    im = JpegImageFile(fp, filename)
  File "M:\venv-win-py36\lib\site-packages\PIL\ImageFile.py", line 102, in __init__
    self._open()
  File "M:\venv-win-py36\lib\site-packages\PIL\JpegImagePlugin.py", line 339, in _open
    handler(self, i)
  File "M:\venv-win-py36\lib\site-packages\PIL\JpegImagePlugin.py", line 124, in APP
    exif = self._getexif()
  File "M:\venv-win-py36\lib\site-packages\PIL\JpegImagePlugin.py", line 432, in _getexif
    return _getexif(self)
  File "M:\venv-win-py36\lib\site-packages\PIL\JpegImagePlugin.py", line 467, in _getexif
    info.load(file)
  File "M:\venv-win-py36\lib\site-packages\PIL\TiffImagePlugin.py", line 738, in load
    fp.seek(offset)
OverflowError: Python int too large to convert to C ssize_t
>>> import PIL
>>> PIL.__version__
'5.2.0'
```
On 64-bit Linux build I see the same warnings, but exception is not raised:
```
(venv-3.6) @ubu-srv-vm:~$ python
Python 3.6.3 (default, Oct  3 2017, 21:45:48)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import Image
>>> Image.open("046.JPG")
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 1441814 bytes but only got 0. Skipping tag 33
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 428 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 196608 bytes but only got 0. Skipping tag 60
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 196608 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 524288 bytes but only got 0. Skipping tag 240
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 1048576 bytes but only got 0. Skipping tag 13222
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 2097152 bytes but only got 0. Skipping tag 1860
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 458752 bytes but only got 0. Skipping tag 10869
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 458752 bytes but only got 0. Skipping tag 1821
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 393216 bytes but only got 0. Skipping tag 1024
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 786432 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 1572864 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 33752069 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 24113876360 bytes but only got 0. Skipping tag 30719
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 30889119128 bytes but only got 0. Skipping tag 62463
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 238688098 bytes but only got 0. Skipping tag 63316
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 282 bytes but only got 0. Skipping tag 1
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 513 bytes but only got 0. Skipping tag 1
  " Skipping tag %s" % (size, len(data), tag))
/home/user/venv-3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py:764: UserWarning: Corrupt EXIF data.  Expecting to read 12 bytes but only got 4.
  warnings.warn(str(msg))
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=2560x1920 at 0x7F5CEF8DF080>
>>> import PIL
>>> PIL.__version__
'5.2.0'
```

I am attaching image file but that file has actual image section stripped (the whole 0xFFDA section removed, but rest of the content is intact) as this is not my photo and I don't want to upload someone else's pictures here. With this stripped image I can still see warnings and exception on 32-bit build: ![jpegstripped](https://user-images.githubusercontent.com/9602964/45198993-6bac0a00-b21e-11e8-95ca-20aa529a9c31.jpeg)




Discussion:
I think the reason for that is slightly different header which is not processed by Pillow correctly. Here is the TIFF header ("that follows `"Exif\0\0"` in EXIF section) from a file that can be read by pillow:

    $ od -Ax -v -j 12 -to1 ../ok.jpg | head -1
    00000c 115 115 000 052 000 000 000 010 000 013 001 017 000 002 000 000

and here the header of the file which causes troubles:

    $ od -Ax -v -j 12 -to1 ../not-ok.jpg | head -1
    00000c 115 115 000 052 000 000 000 012 132 132 000 013 001 017 000 002

The difference is two extra bytes (132 132) that follow header in latter case. The offset in latter case (012) takes into account those two bytes (normal offset is 010), but apparently Pillow does not take into account that offset when parsing remaining TIFF structure. 

I am not sure what is the best way to fix this, I looked at `ImageFileDirectory_v2.load()` method and it looks like it makes assumptions about current file offset and I'm not sure that fixing this won't break the assumptions. I tried trivial fix by adding `fp.seek(self.next)` call before parsing and that seems to fix the problem for my file:

    712         self.reset()
    713         self._offset = fp.tell()
    714         fp.seek(self.next)

Let me know if you want me to open PR with this fix.

Resolving patch:
diff --git a/src/PIL/JpegImagePlugin.py b/src/PIL/JpegImagePlugin.py
--- a/src/PIL/JpegImagePlugin.py
+++ b/src/PIL/JpegImagePlugin.py
@@ -459,35 +459,36 @@ def _getexif(self):
         data = self.info["exif"]
     except KeyError:
         return None
-    file = io.BytesIO(data[6:])
-    head = file.read(8)
+    fp = io.BytesIO(data[6:])
+    head = fp.read(8)
     # process dictionary
     info = TiffImagePlugin.ImageFileDirectory_v1(head)
-    info.load(file)
+    fp.seek(info.next)
+    info.load(fp)
     exif = dict(_fixup_dict(info))
     # get exif extension
     try:
         # exif field 0x8769 is an offset pointer to the location
         # of the nested embedded exif ifd.
         # It should be a long, but may be corrupted.
-        file.seek(exif[0x8769])
+        fp.seek(exif[0x8769])
     except (KeyError, TypeError):
         pass
     else:
         info = TiffImagePlugin.ImageFileDirectory_v1(head)
-        info.load(file)
+        info.load(fp)
         exif.update(_fixup_dict(info))
     # get gpsinfo extension
     try:
         # exif field 0x8825 is an offset pointer to the location
         # of the nested embedded gps exif ifd.
         # It should be a long, but may be corrupted.
-        file.seek(exif[0x8825])
+        fp.seek(exif[0x8825])
     except (KeyError, TypeError):
         pass
     else:
         info = TiffImagePlugin.ImageFileDirectory_v1(head)
-        info.load(file)
+        info.load(fp)
         exif[0x8825] = _fixup_dict(info)
 
     return exif
@@ -510,6 +511,7 @@ def _getmp(self):
     # process dictionary
     try:
         info = TiffImagePlugin.ImageFileDirectory_v2(head)
+        file_contents.seek(info.next)
         info.load(file_contents)
         mp = dict(info)
     except Exception:


Test patch:
diff --git a/Tests/images/exif-ifd-offset.jpg b/Tests/images/exif-ifd-offset.jpg
new file mode 100644
Binary files /dev/null and b/Tests/images/exif-ifd-offset.jpg differ
diff --git a/Tests/images/sugarshack_ifd_offset.mpo b/Tests/images/sugarshack_ifd_offset.mpo
new file mode 100644
Binary files /dev/null and b/Tests/images/sugarshack_ifd_offset.mpo differ
diff --git a/Tests/test_file_jpeg.py b/Tests/test_file_jpeg.py
--- a/Tests/test_file_jpeg.py
+++ b/Tests/test_file_jpeg.py
@@ -581,6 +581,15 @@ def test_invalid_exif(self):
         # OSError for unidentified image.
         self.assertEqual(im.info.get("dpi"), (72, 72))
 
+    def test_ifd_offset_exif(self):
+        # Arrange
+        # This image has been manually hexedited to have an IFD offset of 10,
+        # in contrast to normal 8
+        im = Image.open("Tests/images/exif-ifd-offset.jpg")
+
+        # Act / Assert
+        self.assertEqual(im._getexif()[306], '2017:03:13 23:03:09')
+
 
 @unittest.skipUnless(sys.platform.startswith('win32'), "Windows only")
 class TestFileCloseW32(PillowTestCase):
diff --git a/Tests/test_file_mpo.py b/Tests/test_file_mpo.py
--- a/Tests/test_file_mpo.py
+++ b/Tests/test_file_mpo.py
@@ -62,6 +62,14 @@ def test_mp(self):
             self.assertEqual(mpinfo[45056], b'0100')
             self.assertEqual(mpinfo[45057], 2)
 
+    def test_mp_offset(self):
+        # This image has been manually hexedited to have an IFD offset of 10
+        # in APP2 data, in contrast to normal 8
+        im = Image.open("Tests/images/sugarshack_ifd_offset.mpo")
+        mpinfo = im._getmp()
+        self.assertEqual(mpinfo[45056], b'0100')
+        self.assertEqual(mpinfo[45057], 2)
+
     def test_mp_attribute(self):
         for test_file in test_files:
             im = Image.open(test_file)