Repository: activeloopai/deeplake

Issue:
Fixes in to_tensorflow method
Observed a couple of problems while converting stored datasets to TensorFlow format that need some small fixes.

to_tensorflow fails when the meta information for a tensor includes dtype="object" ("object" dtype has been used for images, area, id, bbox in Coco dataset - https://github.com/activeloopai/Hub/blob/master/examples/coco/upload_coco2017.py#L24) 
A fix for this is to keep the dtype="uint8" or something similar while uploading. The Coco example needs to be updated to reflect this.

to_tensorflow also fails when it gets shape=(1,) in meta and the actual object has multiple dimensions, for example, an image.
This can be fixed by commenting out this line https://github.com/activeloopai/Hub/blob/master/hub/collections/dataset/core.py#L633, which will set the output_shapes as None by default. 

to_pytorch works fine in both the above cases.



Discussion:
> to_tensorflow fails when the meta information for a tensor includes dtype="object" ("object" dtype has been used for images, area, id, bbox in Coco dataset - https://github.com/activeloopai/Hub/blob/master/examples/coco/upload_coco2017.py#L24)
A fix for this is to keep the dtype="uint8" or something similar while uploading. The Coco example needs to be updated to reflect this.

Changing the values in the dict returned by meta would work?

> The Coco example needs to be updated to reflect this.

And I did not get this part...

> to_tensorflow also fails when it gets shape=(1,) in meta and the actual object has multiple dimensions, for example, an image.
This can be fixed by commenting out this line https://github.com/activeloopai/Hub/blob/master/hub/collections/dataset/core.py#L633, which will set the output_shapes as None by default.

After I comment out the line, how do I test it?


> Changing the values in the dict returned by meta would work?


Yeah should work by changing the dtype in meta, but it's better to change it in _call_ as well, to make it obvious for any user

>And I did not get this part...

When the user doesn't specify the shape and just mentions shape = (1,), to_tensorflow strictly expects shape as (1,). If we comment out https://github.com/activeloopai/Hub/blob/master/hub/collections/dataset/core.py#L633, it will accept any shape given to it.

>After I comment out the line, how do I test it?

Once you make both of these changes, try storing the dataset using https://github.com/activeloopai/Hub/blob/master/examples/coco/upload_coco2017.py.
 (Tip:- if you use ds.store("./path/to/directory"), the dataset will get stored locally, instead of online, might save you some time)
Once you have stored it, try loading it with a file similar to:- https://github.com/activeloopai/Hub/blob/master/examples/load_tf.py
If everything is fine, you shouldn't face any issues in loading.

Also try loading with Pytorch, to ensure that the changes didn't break that:-
https://github.com/activeloopai/Hub/blob/master/examples/load_pytorch.py

Let me know if you face any issues.

Resolving patch:
diff --git a/examples/coco/upload_coco2017.py b/examples/coco/upload_coco2017.py
--- a/examples/coco/upload_coco2017.py
+++ b/examples/coco/upload_coco2017.py
@@ -20,14 +20,14 @@ def __init__(self, args, tag):
 
     def meta(self):
         return {
-            "segmentation": {"shape": (1,), "dtype": "object", "chunksize": 1000},
-            "area": {"shape": (1,), "dtype": "object", "chunksize": 1000},
-            "iscrowd": {"shape": (1,), "dtype": "object", "chunksize": 1000},
+            "segmentation": {"shape": (1,), "dtype": "uint32", "chunksize": 1000},
+            "area": {"shape": (1,), "dtype": "uint32", "chunksize": 1000},
+            "iscrowd": {"shape": (1,), "dtype": "uint8", "chunksize": 1000},
             "image_id": {"shape": (1,), "dtype": "int64"},
-            "bbox": {"shape": (1,), "dtype": "object", "chunksize": 1000},
-            "category_id": {"shape": (1,), "dtype": "object", "chunksize": 1000},
-            "id": {"shape": (1,), "dtype": "object", "chunksize": 1000},
-            "image": {"shape": (1,), "dtype": "object", "chunksize": 100},
+            "bbox": {"shape": (1,), "dtype": "uint16", "chunksize": 1000},
+            "category_id": {"shape": (1,), "dtype": "int64", "chunksize": 1000},
+            "id": {"shape": (1,), "dtype": "uint32", "chunksize": 1000},
+            "image": {"shape": (1,), "dtype": "uint32", "chunksize": 100},
         }
 
     def __call__(self, input):
@@ -36,21 +36,21 @@ def __call__(self, input):
             # print(f"Image id: {input['image_id']}")
             ds["image_id"] = input["image_id"]
             info = input["info"]
-            ds["image"] = np.empty(1, object)
+            ds["image"] = np.empty(1, np.uint32)
             ds["image"][0] = np.array(
                 Image.open(
                     os.path.join(
                         self._args.dataset_path,
                         get_image_name(self._args, self._tag, input["image_id"]),
                     )
-                )
+                ), dtype=np.uint32
             )
-            ds["segmentation"] = np.empty(1, object)
-            ds["area"] = np.empty(1, object)
-            ds["iscrowd"] = np.empty(1, object)
-            ds["bbox"] = np.empty(1, object)
-            ds["category_id"] = np.empty(1, object)
-            ds["id"] = np.empty(1, object)
+            ds["segmentation"] = np.empty(1, np.uint32)
+            ds["area"] = np.empty(1, np.uint32)
+            ds["iscrowd"] = np.empty(1, np.uint8)
+            ds["bbox"] = np.empty(1, np.np.uint16)
+            ds["category_id"] = np.empty(1, np.uint8)
+            ds["id"] = np.empty(1, np.uint32)
 
             ds["segmentation"][0] = [anno["segmentation"] for anno in info]
             ds["area"][0] = [anno["area"] for anno in info]
diff --git a/hub/collections/dataset/core.py b/hub/collections/dataset/core.py
--- a/hub/collections/dataset/core.py
+++ b/hub/collections/dataset/core.py
@@ -630,7 +630,7 @@ def tf_dtype(np_dtype):
         return tf.data.Dataset.from_generator(
             tf_gen,
             output_types=output_types,
-            output_shapes=output_shapes,
+            # output_shapes=output_shapes, # commenting out this so TF takes the shape whatever is passed
         )
 
 


Test patch: