Repository: python-pillow/Pillow

Issue:
AttributeError: 'tuple' object has no attribute 'ljust' when trying to save image
### What did you do?

Ran a snippet of code, see below.

### What did you expect to happen?

The script to complete without error.

### What actually happened?

Stack trace, see below.

### What versions of Pillow and Python are you using?

Pillow==3.4.2
(also fails with Pillow==3.5.0.dev0)
Python 3.5.2


Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, django, or buildout, try to replicate the issue just using Pillow. 

```python
from io import BytesIO
from contextlib import closing
from PIL import Image

image = Image.open('phototest.tif')
print(image)

with closing(BytesIO()) as f:
    image.save(f, image.format or 'JPEG')
    value = f.getvalue()
    

```

Where the image can be found [here](https://github.com/tesseract-ocr/tesseract/blob/master/testing/phototest.tif?raw=true), or download it with this command:

```
curl  "https://github.com/tesseract-ocr/tesseract/blob/master/testing/phototest.tif?raw=true" > phototest.tif
```

The output is:

```
<PIL.TiffImagePlugin.TiffImageFile image mode=1 size=640x480 at 0x7F37D8546E10>
Traceback (most recent call last):
  File "try.py", line 9, in <module>
    image.save(f, image.format or 'JPEG')
  File "/usr/local/lib/python3.5/dist-packages/PIL/Image.py", line 1698, in save
    save_handler(self, fp, filename)
  File "/usr/local/lib/python3.5/dist-packages/PIL/TiffImagePlugin.py", line 1487, in _save
    offset = ifd.save(fp)
  File "/usr/local/lib/python3.5/dist-packages/PIL/TiffImagePlugin.py", line 758, in save
    entries.append((tag, typ, count, data.ljust(4, b"\0"), b""))
AttributeError: 'tuple' object has no attribute 'ljust'
```

ValueError when getting tiff tags
### What did you do?

I tried to extract metadata from a .tif file
### What did you expect to happen?

get the metadata
### What actually happened?

ValueError: too many values to unpack (expected 1)
### What versions of Pillow and Python are you using?

pillow 3.3.0
python 3.5

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. The best reproductions are self-contained scripts with minimal dependencies.

```
from PIL import Image
img = Image.open("myfil.tif")
dict(img.tag)
```

Correct ValueError
Fixes #2006.

When `info.length` is 1 and `len(values) > 1`, the code used to throw: 

```
ValueError: too many values to unpack (expected 1)
```

For example: tag `IMAGEJ_META_DATA_BYTE_COUNTS` (50838) can sometimes have two values (I have .tif files with metadata split between UTF8 and UTF16 => one byte count for each)



Discussion:

What's the image that causes this?

[img_000001856_Default_000.tif.zip](https://github.com/python-pillow/Pillow/files/343854/img_000001856_Default_000.tif.zip)

I figured out why it was doing it: the tag 50838 (MetaData count) has a count of 2 and TiffTags.py expect 1. (I think this is because the first 12 bytes are utf8 and the rest are utf16)

Yeah. That appears to be it.  And as far as I can tell, there's no spec for MetaDataByteCounts, but looking at the ImageJ source, (https://github.com/imagej/imagej1/blob/ce491f23ca56a342576ee13455de60bf018b1015/ij/io/TiffEncoder.java#L309) there's definitely an arbitrary number of items in that field, not a single. I think that the tiffTags entry should be changed to `50838: ("ImageJMetaDataByteCounts", LONG, 0),`.

Yes but on line 553:
https://github.com/python-pillow/Pillow/blob/master/PIL/TiffImagePlugin.py#L553
if len(values) >1 and info.length ==1, it will cause a problem, wouldn't it make sense to also check that len(values) ==1? To avoid similar issues?

FWIW, [tifffile.py](http://www.lfd.uci.edu/~gohlke/code/tifffile.py.html) can parse ImageJ metadata:

``` Python
>>> import tifffile
>>> tf = tifffile.TiffFile("img_000001856_Default_000.tif")
>>> print(tf.pages[0].imagej_tags.info)
{
  "Core-Focus": "",
  "Core-Shutter": "",
  "QCamera-TriggerType": "Freerun",
  "Channel": "Default",
  "Core-Initialize": "1",
  "UUID": "6e16cb7e-b556-4dd4-9c90-6a503854801a",
  "Core-Camera": "QCamera",
  "Core-TimeoutMs": "5000",
  "Core-SLM": "",
  "FrameIndex": 1856,
  "QCamera-OffsetMax": "1023",
  "ElapsedTime-ms": 944743.549987793,
  "QCamera-CameraName": "Rolera EMC2",
  "Frame": 1856,
  "Core-ChannelGroup": "",
  "PositionIndex": 0,
  "Width": 1004,
  "QCamera-ExposureMin": "0.2000",
  "WaitInterval": 500,
  "ROI": "0-0-1004-1002",
  "PositionName": null,
  "QCamera-Camera": "37718 Rolera EMC2 S/N Q37718",
  "Camera": "QCamera",
  "PixelSizeUm": 0,
  "Height": 1002,
  "QCamera-OffsetMin": "0",
  "Core-ImageProcessor": "",
  "QCamera-Cooler": "On",
  "Summary": {
    "Slices": 1,
    "InitialPositionList": null,
    "Interval_ms": 500,
    "UUID": "b3c81756-52e2-4db9-9fdd-1abc1ad29c02",
    "UserName": "LegoScope",
    "Depth": 2,
    "PixelType": "GRAY16",
    "Time": "2016-06-24 11:13:55 +0100",
    "Date": "2016-06-24",
    "z-step_um": 0,
    "MetadataVersion": 10,
    "PositionIndex": 0,
    "SlicesFirst": false,
    "ChContrastMin": [0],
    "Width": 1004,
    "PixelAspect": 1,
    "MicroManagerVersion": "1.4.18",
    "ROI": [
      0,
      0,
      1004,
      1002
    ],
    "ChNames": ["Default"],
    "IJType": 1,
    "GridRow": 0,
    "Comment": "",
    "Height": 1002,
    "GridColumn": 0,
    "Prefix": "e1105qt5b_170S2tf_6mgac2t80_0to5V_800ulhr_2",
    "PixelSize_um": 0,
    "Frames": 7200,
    "BitDepth": 14,
    "KeepShutterOpenChannels": false,
    "Source": "Micro-Manager",
    "Channels": 1,
    "ComputerName": "B62-LegoScope",
    "CustomIntervals_ms": [],
    "KeepShutterOpenSlices": false,
    "ChColors": [-1],
    "TimeFirst": false,
    "ChContrastMax": [65536],
    "Positions": 1,
    "Directory": "C:\\Users\\LegoScope\\Desktop\\Therese\\160623e1105q"
  },
  "QCamera-Exposure": "500.00",
  "Source": "QCamera",
  "SlicePosition": null,
  "QCamera-CameraID": "Serial number Q37718",
  "Exposure-ms": 500,
  "QCamera-TransposeCorrection": "0",
  "QCamera-Offset": "0",
  "Core-AutoShutter": "1",
  "QCamera-CCDTemperatureMin": "-50",
  "QCamera-TransposeXY": "0",
  "Binning": "1",
  "NextFrame": null,
  "Core-XYStage": "",
  "AxisPositions": null,
  "PixelType": "GRAY16",
  "Time": "2016-06-24 11:29:26 +0100",
  "QCamera-EMGainMin": "0",
  "QCamera-Binning": "1",
  "ImageNumber": "1856",
  "QCamera-Easy EM Enabled": "Yes",
  "QCamera-EMGainMax": "4095",
  "QCamera-CCDTemperatureMax": "0",
  "QCamera-TransposeMirrorY": "0",
  "Core-AutoFocus": "",
  "YPositionUm": null,
  "QCamera-CCDTemperature": "-50",
  "XPositionUm": null,
  "QCamera-Description": "QCam 2.0.13",
  "QCamera-ReadoutTime": "10.0 MHz",
  "QCamera-TransposeMirrorX": "0",
  "BitDepth": 14,
  "Slice": 0,
  "QCamera-Easy EM Gain": "0",
  "QCamera-PixelType": "14bit",
  "FileName": "img_000001856_Default_000.tif",
  "Core-Galvo": "",
  "Position": "Default",
  "QCamera-ExposureMax": "1073741.8750",
  "QCamera-Name": "QCamera",
  "CameraChannelIndex": 0,
  "ChannelIndex": 0,
  "SliceIndex": 0,
  "ZPositionUm": null
}
```

There's more issues in the metadata handling generally related to the difference between fields in the file being internally correct and fields being correct with respect to the spec, and what that means for the rest of the code. Specifically, there are some metadata fields where the spec is important, and some where it's advisory. 

Yes but the code will produce an error anyway if len(values) >1 and info.length ==1. So why not try to avoid that? If the file can be open even with a few mistakes in the header why not do it?

Maybe it would be better to leave info.length == 1 instead of info.length > 0?

Good idea, but I think it's the wrong approach. I think the implied test case is useful/necessary, but we need to treat the case of getting a different type from the file than the spec differently than this patch. 


Resolving patch:
diff --git a/PIL/TiffImagePlugin.py b/PIL/TiffImagePlugin.py
--- a/PIL/TiffImagePlugin.py
+++ b/PIL/TiffImagePlugin.py
@@ -550,11 +550,28 @@ def _setitem(self, tag, value, legacy_api):
 
         dest = self._tags_v1 if legacy_api else self._tags_v2
 
-        if info.length == 1:
-            if legacy_api and self.tagtype[tag] in [5, 10]:
+        # Three branches:
+        # Spec'd length == 1, Actual length 1, store as element
+        # Spec'd length == 1, Actual > 1, Warn and truncate. Formerly barfed.
+        # No Spec, Actual length 1, Formerly (<4.2) returned a 1 element tuple.
+        # Don't mess with the legacy api, since it's frozen.
+        if ((info.length == 1) or 
+            (info.length is None and len(values) == 1 and not legacy_api)): 
+            # Don't mess with the legacy api, since it's frozen.
+            if legacy_api and self.tagtype[tag] in [5, 10]: # rationals
                 values = values,
-            dest[tag], = values
+            try:
+                dest[tag], = values
+            except ValueError:
+                # We've got a builtin tag with 1 expected entry
+                warnings.warn(
+                    "Metadata Warning, tag %s had too many entries: %s, expected 1" % (
+                        tag, len(values)))
+                dest[tag] = values[0]
+
         else:
+            # Spec'd length > 1 or undefined
+            # Unspec'd, and length > 1
             dest[tag] = values
 
     def __delitem__(self, tag):
@@ -1011,8 +1028,10 @@ def _decoder(self, rawmode, layer, tile=None):
             args = rawmode, ""
             if JPEGTABLES in self.tag_v2:
                 # Hack to handle abbreviated JPEG headers
-                # FIXME This will fail with more than one value
-                self.tile_prefix, = self.tag_v2[JPEGTABLES]
+                # Definition of JPEGTABLES is that the count
+                # is the number of bytes in the tables datastream
+                # so, it should always be 1 in our tag info
+                self.tile_prefix = self.tag_v2[JPEGTABLES]
         elif compression == "packbits":
             args = rawmode
         elif compression == "tiff_lzw":
diff --git a/PIL/TiffTags.py b/PIL/TiffTags.py
--- a/PIL/TiffTags.py
+++ b/PIL/TiffTags.py
@@ -23,7 +23,7 @@
 class TagInfo(namedtuple("_TagInfo", "value name type length enum")):
     __slots__ = []
 
-    def __new__(cls, value=None, name="unknown", type=None, length=0, enum=None):
+    def __new__(cls, value=None, name="unknown", type=None, length=None, enum=None):
         return super(TagInfo, cls).__new__(
             cls, value, name, type, length, enum or {})
 
@@ -142,6 +142,8 @@ def lookup(tag):
     341: ("SMaxSampleValue", DOUBLE, 0),
     342: ("TransferRange", SHORT, 6),
 
+    347: ("JPEGTables", UNDEFINED, 1),
+    
     # obsolete JPEG tags
     512: ("JPEGProc", SHORT, 1),
     513: ("JPEGInterchangeFormat", LONG, 1),
@@ -158,7 +160,10 @@ def lookup(tag):
     531: ("YCbCrPositioning", SHORT, 1),
     532: ("ReferenceBlackWhite", LONG, 0),
 
+    700: ('XMP', BYTE, 1),
+    
     33432: ("Copyright", ASCII, 1),
+    34377: ('PhotoshopInfo', BYTE, 1),
 
     # FIXME add more tags here
     34665: ("ExifIFD", SHORT, 1),
@@ -188,8 +193,8 @@ def lookup(tag):
 
     50741: ("MakerNoteSafety", SHORT, 1, {"Unsafe": 0, "Safe": 1}),
     50780: ("BestQualityScale", RATIONAL, 1),
-    50838: ("ImageJMetaDataByteCounts", LONG, 1),
-    50839: ("ImageJMetaData", UNDEFINED, 1)
+    50838: ("ImageJMetaDataByteCounts", LONG, 0), # Can be more than one 
+    50839: ("ImageJMetaData", UNDEFINED, 1)       # see Issue #2006
 }
 
 # Legacy Tags structure


Test patch:
diff --git a/Tests/images/issue_2278.tif b/Tests/images/issue_2278.tif
new file mode 100644
Binary files /dev/null and b/Tests/images/issue_2278.tif differ
diff --git a/Tests/test_file_tiff_metadata.py b/Tests/test_file_tiff_metadata.py
--- a/Tests/test_file_tiff_metadata.py
+++ b/Tests/test_file_tiff_metadata.py
@@ -56,7 +56,7 @@ def test_rt_metadata(self):
         loaded = Image.open(f)
 
         self.assertEqual(loaded.tag[ImageJMetaDataByteCounts], (len(bindata),))
-        self.assertEqual(loaded.tag_v2[ImageJMetaDataByteCounts], len(bindata))
+        self.assertEqual(loaded.tag_v2[ImageJMetaDataByteCounts], (len(bindata),))
 
         self.assertEqual(loaded.tag[ImageJMetaData], bindata)
         self.assertEqual(loaded.tag_v2[ImageJMetaData], bindata)
@@ -69,6 +69,16 @@ def test_rt_metadata(self):
         loaded_double = loaded.tag[tag_ids['YawAngle']][0]
         self.assertAlmostEqual(loaded_double, doubledata)
 
+        # check with 2 element ImageJMetaDataByteCounts, issue #2006
+        
+        info[ImageJMetaDataByteCounts] = (8, len(bindata) - 8)
+        img.save(f, tiffinfo=info)
+        loaded = Image.open(f)
+
+        self.assertEqual(loaded.tag[ImageJMetaDataByteCounts], (8, len(bindata) - 8))
+        self.assertEqual(loaded.tag_v2[ImageJMetaDataByteCounts], (8, len(bindata) - 8))
+
+
     def test_read_metadata(self):
         img = Image.open('Tests/images/hopper_g4.tif')
 
@@ -202,8 +212,8 @@ def test_exif_div_zero(self):
         im.save(out, tiffinfo=info, compression='raw')
 
         reloaded = Image.open(out)
-        self.assertEqual(0, reloaded.tag_v2[41988][0].numerator)
-        self.assertEqual(0, reloaded.tag_v2[41988][0].denominator)
+        self.assertEqual(0, reloaded.tag_v2[41988].numerator)
+        self.assertEqual(0, reloaded.tag_v2[41988].denominator)
 
     def test_expty_values(self):
         data = io.BytesIO(
@@ -220,6 +230,27 @@ def test_expty_values(self):
             self.fail("Should not be struct value error there.")
         self.assertIn(33432, info)
 
+    def test_PhotoshopInfo(self):
+        im = Image.open('Tests/images/issue_2278.tif')
+
+        self.assertIsInstance(im.tag_v2[34377], bytes)
+        out = self.tempfile('temp.tiff')
+        im.save(out)
+        reloaded = Image.open(out)
+        self.assertIsInstance(reloaded.tag_v2[34377], bytes)
+
+    def test_too_many_entries(self):
+        ifd = TiffImagePlugin.ImageFileDirectory_v2()
+
+        #    277: ("SamplesPerPixel", SHORT, 1),
+        ifd._tagdata[277] = struct.pack('hh', 4,4)
+        ifd.tagtype[277] = TiffTags.SHORT
+
+        try:
+            self.assert_warning(UserWarning, lambda: ifd[277])
+        except ValueError:
+            self.fail("Invalid Metadata count should not cause a Value Error.")
+
 
 if __name__ == '__main__':
     unittest.main()