Repository: activeloopai/deeplake

Issue:
[BUG] UnicodeDecodeError when accessing Tensor.meta
## 🐛🐛 Bug Report


### ⚗️ Current Behavior

Im getting an error when reading an image file that seems to be fine.

`UnicodeDecodeError: 'utf-8' codec can't decode byte 0xea in position 1: invalid continuation byte`

It happens when accessing Tensor.meta apparently.

I don't think I can attach files. I'll happily share it. The image is 0077 from the NWPU crowd dataset.

Full trace:

```
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
/var/folders/rm/y4mfyltx67d84r0n1h3ccgp80000gn/T/ipykernel_25645/1849073815.py in <module>
      5         pts = [[]]
      6 
----> 7     ds.images.append(hub.read(img_paths[i]))
      8     ds.points.append(np.array(pts, dtype=np.uint32))

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/util/invalid_view_op.py in inner(x, *args, **kwargs)
     14                 type(x).__name__,
     15             )
---> 16         return callable(x, *args, **kwargs)
     17 
     18     return inner

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/tensor.py in append(self, sample)
    321             sample (InputSample): The data to append to the tensor. `Sample` is generated by `hub.read`. See the above examples.
    322         """
--> 323         self.extend([sample])
    324 
    325     def clear(self):

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/util/invalid_view_op.py in inner(x, *args, **kwargs)
     14                 type(x).__name__,
     15             )
---> 16         return callable(x, *args, **kwargs)
     17 
     18     return inner

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/tensor.py in extend(self, samples)
    265         """
    266         self._write_initialization()
--> 267         self.chunk_engine.extend(
    268             samples, link_callback=self._append_to_links if self.meta.links else None
    269         )

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/chunk_engine.py in extend(self, samples, link_callback)
    640             if link_callback:
    641                 for sample in samples:
--> 642                     link_callback(sample, flat=None)
    643 
    644         self.cache.autoflush = initial_autoflush

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/tensor.py in _append_to_links(self, sample, flat)
    682         for k, v in self.meta.links.items():
    683             if flat is None or v["flatten_sequence"] == flat:
--> 684                 self.dataset[k].append(get_link_transform(v["append"])(sample))
    685 
    686     def _update_links(

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/tensor_link.py in __call__(self, *args, **kwargs)
     21             out_kwargs = {k: v for k, v in kwargs.items() if k in self.kwargs}
     22             return self.f(*args, **out_kwargs)
---> 23         return self.f(args[0])
     24 
     25     def __str__(self):

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/tensor_link.py in append_info(sample)
     48 def append_info(sample):
     49     if isinstance(sample, hub.core.sample.Sample):
---> 50         meta = sample.meta
     51         meta["modified"] = False
     52         return meta

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/sample.py in meta(self)
    396         compression_type = get_compression_type(self.compression)
    397         if compression_type == IMAGE_COMPRESSION:
--> 398             meta["exif"] = self._getexif()
    399         if compression_type == VIDEO_COMPRESSION:
    400             meta.update(self._get_video_meta())

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/sample.py in _getexif(self)
    386         else:
    387             img = Image.open(BytesIO(self.buffer))
--> 388         return {
    389             TAGS.get(k, k): f"{v.decode() if isinstance(v, bytes) else v}"
    390             for k, v in img.getexif().items()

~/.pyenv/versions/3.9.9/lib/python3.9/site-packages/hub/core/sample.py in <dictcomp>(.0)
    387             img = Image.open(BytesIO(self.buffer))
    388         return {
--> 389             TAGS.get(k, k): f"{v.decode() if isinstance(v, bytes) else v}"
    390             for k, v in img.getexif().items()
    391         }

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xea in position 1: invalid continuation byte
```


**Input Code**

```python
hub.read("0077.jpg").meta
```

**Expected behavior/code**
The image loads and is appended to the dataset as expected

### ⚙️ Environment

- `Python` version(s):
    - *good*: [e.g. 3.9.9]
- `OS`: [M1 macOS]
- `IDE`: [VS-Code]


Discussion:
Hey @pietz! Thanks a lot for reporting the issue - sorry you've run into this. We will take a look into this straight away! 
Purging the EXIF data solved the problem. I used EXIFPurge from the AppStore.

EDIT: This is still a workaround. I think there might be an issue here. Not being able to add an image because of ill formatted EXIF should have at least the option to be skipped. E.g. `ds.append(file, ignore_exif=True)`
Oh, nice, you beat us to it, hehe. We are taking a look to see if there's something we can adjust on our side to make sure this doesn't happen. :) let us know if you run into any other issues, @pietz ! 
Experiencing the same error with multiple images now.
Might be relevant:

https://github.com/python-pillow/Pillow/issues/1605#issuecomment-167402651
@mikayelh Could you provide me a quick workaround to using hub.read() to append images to the dataset? You docs state:

> Appending the object hub.read(path)is equivalent to appending PIL.Image.fromarray(path)

However, it looks like that's a typo. fromarray() needs an array parameter and not a path. However using `PIL.Image.open(path)` throws a `TypeError: Cannot serialize sample of type <class 'PIL.Image.Image'>` error.

I'm stuck here.
I'm sorry about that, @pietz (thanks for flagging this, tagging @istranic to fix the docs). 

The team is already on it trying to see if there's a workaround. By the way, would you mind sharing a couple of the pictures that are raising errors? 
![0178](https://user-images.githubusercontent.com/13137132/163974386-79fe1736-3697-4c7f-8530-4902e722c43c.jpg)

I hope GitHub keeps the meta data. 
Yepp, that image triggers the error even after downloading it here.

`hub.read(path).meta` should trigger the error.
Nice. Thanks. Fun fact I might actually be somewhere on this exact picture. :D one of my teammates will update you shortly on this, @pietz !
I'll let you know when I see your avatar among the 8359 people, haha.
@pietz thanks for reporting and the test image, will fix soon. In the mean time you can do `ds.tensor.append(np.array(PIL.Image.open(path)))` as a workaround.
@pietz above workaround decompresses the image and recompresses it, which can slow things down. A better work around is to pass `create_sample_info_tensor=False` to your `create_tensor` call to ignore exif and other meta data. Example:

`ds.create_tensor("image", sample_compression="jpeg", create_sample_info_tensor=False)`
Beautiful, cheers @farizrahman4u 

Resolving patch:
diff --git a/hub/core/sample.py b/hub/core/sample.py
--- a/hub/core/sample.py
+++ b/hub/core/sample.py
@@ -19,6 +19,7 @@
     AUDIO_COMPRESSION,
     IMAGE_COMPRESSION,
 )
+from hub.util.exif import getexif
 from hub.core.storage.provider import StorageProvider
 from hub.util.path import get_path_type, is_remote_path
 import numpy as np
@@ -37,6 +38,8 @@
 except ImportError:
     GCSProvider = None  # type: ignore
 
+import warnings
+
 
 class Sample:
     path: Optional[str]
@@ -383,10 +386,13 @@ def _getexif(self) -> dict:
             img = Image.open(self.path)
         else:
             img = Image.open(BytesIO(self.buffer))
-        return {
-            TAGS.get(k, k): f"{v.decode() if isinstance(v, bytes) else v}"
-            for k, v in img.getexif().items()
-        }
+        try:
+            return getexif(img)
+        except Exception as e:
+            warnings.warn(
+                f"Error while reading exif data, possibly due to corrupt exif: {e}"
+            )
+            return {}
 
     @property
     def meta(self) -> dict:
diff --git a/hub/util/exif.py b/hub/util/exif.py
new file mode 100644
--- /dev/null
+++ b/hub/util/exif.py
@@ -0,0 +1,78 @@
+from fractions import Fraction
+
+from PIL import Image  # type: ignore
+import PIL.ExifTags  # type: ignore
+
+
+_LOOKUPS = {
+    "MeteringMode": (
+        "Undefined",
+        "Average",
+        "Center-weighted average",
+        "Spot",
+        "Multi-spot",
+        "Multi-segment",
+        "Partial",
+    ),
+    "ExposureProgram": (
+        "Undefined",
+        "Manual",
+        "Program AE",
+        "Aperture-priority AE",
+        "Shutter speed priority AE",
+        "Creative (Slow speed)",
+        "Action (High speed)",
+        "Portrait ",
+        "Landscape",
+        "Bulb",
+    ),
+    "ResolutionUnit": ("", "Undefined", "Inches", "Centimetres"),
+    "Orientation": (
+        "",
+        "Horizontal",
+        "Mirror horizontal",
+        "Rotate 180",
+        "Mirror vertical",
+        "Mirror horizontal and rotate 270 CW",
+        "Rotate 90 CW",
+        "Mirror horizontal and rotate 90 CW",
+        "Rotate 270 CW",
+    ),
+}
+
+
+def getexif(image: Image):
+    raw_exif = image._getexif()
+    if not raw_exif:
+        return {}
+    exif = {}
+    for k, v in raw_exif.items():
+        tag = k
+        k = PIL.ExifTags.TAGS.get(k, k)
+        v = _process_exif_value(k, v)
+        if v is None:
+            continue
+        exif[k] = v
+    return exif
+
+
+def _process_exif_value(k: str, v):
+    if hasattr(v, "numerator"):  # Rational
+        v = v.numerator / v.denominator if v.denominator else 0
+        if k == "ExposureTime":
+            v = float(Fraction(v).limit_denominator(8000))
+        elif k in ("XResolution", "YResolution"):
+            try:
+                v = int(v)
+            except ValueError:  # nan
+                pass
+        return v
+    elif k in _LOOKUPS:
+        return _LOOKUPS[k][v]
+    elif isinstance(v, bytes):
+        return str(v) if len(v) < 16 else "<bytes>"
+    elif isinstance(v, (list, tuple)):
+        return type(v)(_process_exif_value("_", x) for x in v)
+    elif isinstance(v, dict):
+        return {k: _process_exif_value("_", x) for k, x in v.items()}
+    return str(v)


Test patch:
diff --git a/hub/api/tests/test_api_with_compression.py b/hub/api/tests/test_api_with_compression.py
--- a/hub/api/tests/test_api_with_compression.py
+++ b/hub/api/tests/test_api_with_compression.py
@@ -257,3 +257,16 @@ def test_audio(local_ds, compression, audio_paths):
     for i in range(10):
         decompressed = local_ds.audio[i].numpy()
         np.testing.assert_array_equal(decompressed[: len(arr), :], arr)  # type: ignore
+
+
+def test_exif(memory_ds, compressed_image_paths):
+    ds = memory_ds
+    with ds:
+        ds.create_tensor("images", htype="image", sample_compression="jpeg")
+        for path in compressed_image_paths["jpeg"]:
+            ds.images.append(hub.read(path))
+    for image in ds.images:
+        assert isinstance(image.sample_info["exif"], dict), (
+            type(image.sample_info["exif"]),
+            path,
+        )
diff --git a/hub/api/tests/test_sample_info.py b/hub/api/tests/test_sample_info.py
--- a/hub/api/tests/test_sample_info.py
+++ b/hub/api/tests/test_sample_info.py
@@ -1,19 +1,18 @@
 from PIL import Image  # type: ignore
 from PIL.ExifTags import TAGS  # type: ignore
 from miniaudio import mp3_get_file_info  # type: ignore
+from hub.util.exif import getexif
 import numpy as np
 import pytest
 import os
 import sys
 import hub
+import json
 
 
 def get_exif_helper(path):
     img = Image.open(path)
-    return {
-        TAGS.get(k, k): f"{v.decode() if isinstance(v, bytes) else v}"
-        for k, v in img.getexif().items()
-    }
+    return json.loads(json.dumps(getexif(img)))
 
 
 def test_image_samples(local_ds_generator, compressed_image_paths):