Repository: openvinotoolkit/datumaro

Issue:
Dump Anotations: LabelMe3.0 supports only Flat Image Layout
<!---
Copyright (C) 2020 Intel Corporation

SPDX-License-Identifier: MIT
-->

### My actions before raising this issue
- [  ] Searched [past issues] - https://github.com/openvinotoolkit/cvat/pull/1715/files

<!--- Provide a general summary of the issue in the Title above -->

### Expected Behaviour
Dumping Annotations in LabelMe3.0 Format. Annotations should be dumped with File (Image) name and should remove the base path (head of the file name). 
Eg. root/Folder_containing_images/Image1.jpg -- > Annotations in LabelMe3.0 should be dumped as Image1.xml
<!--- If you're describing a bug, tell us what should happen. If you're
suggesting a change/improvement, tell us how it should work -->

### Current Behaviour
Currently, I am uploading the image data from the shared drive on the CVAT tool. Hence the Images are uploaded with base path + filename (root/Folder_containing_images/Image1.jpg). While Dumping Annotations in LabelMe3.0, it raises an error as it is only expecting a flat layout for the Image name
<!--- If describing a bug, tell us what happens instead of the expected
behavior. If suggesting a change/improvement, explain the difference from
current behavior -->

### Possible Solution
Support Relative Path/ Absolute Path for LabelMe3.0.
Reference - https://github.com/openvinotoolkit/cvat/pull/1715/files
<!--- Not obligatory, but suggest a fix/reason for the bug, or ideas how
to implement the addition or change -->

### Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to
reproduce this bug. Include code to reproduce, if relevant -->
1. Upload the Image Data from Shared Drive.
1. Dump annotations in LabelMe3.0 format


### Context
I am trying to Upload the Annotations file to verify and correct the Labels. LabelMe3.0 suits for the application as it generates an individual XML file for each image and supports polygon labels and attributes as well. Converters are designed to/from LabelMe3.0 to a custom format. It will be great if it is possible to support the relative/absolute path (without flat layout) in the LabelMe3.0 format as well.

Also, Open for suggestions if any other Annotations formats can be suitable (generates a separate file and supports polygon and attributes) and used.

Thanks a lot!
<!--- How has this issue affected you? What are you trying to accomplish?
Providing context helps us come up with a solution that is most useful in
the real world -->

### Your Environment
<!--- Include as many relevant details about the environment you experienced
the bug in -->
CVAT tool is deployed on AWS instance




Discussion:
Hello. Yes, there is a problem in the combination of file share and LabelMe format, which, probably, cannot be fixed easily. If you are not constrained on generating a separate file for each image, you can switch to `COCO` (json), `CVAT for images` (xml) or `Datumaro` (json) formats - all support polygons, labels and attributes, otherwise consider uploading images as a flat zip archive, or just as images.
Hello,

Thanks for the suggestions! But It would be good to have a separate file for each image as the current application is based on it.

So if there is a problem in the combination of the file share and LabelMe format, **it does not raise any error while uploading the annotations for the same directory structure**. Should not it raise the same kind of error (flat layout required)?

I upload the Annotations for the file share and it uploads without any error.
Just wondering how can we solve this!

Thanks a lot!
If you need separate files, try to use another image upload method, like described above.

If I get it right, you are uploading annotations in LabelMe format for this directory structure? In this case, I think, there are 2 options: 
- your layout is like `a/b/c/image_xxx...` and annotations are like `/image_xxx...`, so files can be matched without in issue
- annotations are loaded for only a part of the files

Actually, the main question in how to represent a folded layout with LabelMe format, which is flat.
Hello,

Actually, without sharing the folder (file share mounting) there's a limit to uploading data (500 MB). SO it is not feasible to upload large data. Hence I am uploading via the shared drive.



> If I get it right, you are uploading annotations in LabelMe format for this directory structure? In this case, I think, there are 2 options:
> 
> * your layout is like `a/b/c/image_xxx...` and annotations are like `/image_xxx...`, so files can be matched without in issue
> * annotations are loaded for only a part of the files

Yes. 
Currently, the Image layout is `a/b/c/image_xxx`. But Annotations that are uploaded have the structure  `image_xxx.xml` directly under the zip directory.



> Actually, the main question in how to represent a folded layout with LabelMe format, which is flat.

Correct. 
If while uploading, it can match the image_name (by either of the 2 options), It should do the same for Downloading the Images as well right?

Please correct me and help me understand if I am missing something here.

Thanks a lot!
We could support such tasks in LabelMe by dropping everything except the file name (and it was done so), but it creates wrong output if the task contains files with same names in different folders. Maybe, we should check for this situation and raise an error only in the case of conflicts.
> We could support such tasks in LabelMe by dropping everything except the file name (and it was done so), but it creates wrong output if the task contains files with same names in different folders.

Yes. This is an issue.
Not sure completely, but can it be possible, while dumping the Annotations, dump them with the layout they have `(a/b/c/image_xxx)` --> `(b/c/image_xxx.xml)` if the original layout is not flat. Otherwise, dump them as usual. This might eliminate the issue of similar names if present in multiple folders of the same Task.

Thanks a lot!


@zhiltsov-max , I believe we need to support the case. There are multiple options. A couple of them below:
- Use `a/b/c/image_xxx.xml` is a good idea. I see only one problem here. If we have an image in the same directory but with different extensions, it can be a problem. In this case Datumaro should raise an issue.
- Don't use "flat" layout. I mean use flat layout only if all images are flat.
- When loading annotations try to use `a/b/c/image_xxx.xml` first and if it doesn't exist use `image_xxx.xml`
- ...
@zhiltsov-max , I will transfer the issue to Datumaro repo.
The problem is how to save it (so we could adequately parse it), if the format looks like this: https://github.com/openvinotoolkit/datumaro/tree/develop/tests/assets/labelme_dataset

I see the only solution in truncating paths to names (without extensions) and check for collisions.

Resolving patch:
diff --git a/datumaro/plugins/labelme_format.py b/datumaro/plugins/labelme_format.py
--- a/datumaro/plugins/labelme_format.py
+++ b/datumaro/plugins/labelme_format.py
@@ -49,10 +49,14 @@ def _parse(self, path):
         }
 
         items = []
-        for p in sorted(p for p in os.listdir(path) if p.endswith('.xml')):
+        for p in os.listdir(path):
+            if not p.endswith('.xml'):
+                continue
             root = ElementTree.parse(osp.join(path, p))
 
-            image_path = osp.join(path, root.find('filename').text)
+            item_id = osp.join(root.find('folder').text or '',
+                root.find('filename').text)
+            image_path = osp.join(path, item_id)
             image_size = None
             imagesize_elem = root.find('imagesize')
             if imagesize_elem is not None:
@@ -63,7 +67,7 @@ def _parse(self, path):
 
             annotations = self._parse_annotations(root, path, categories)
 
-            items.append(DatasetItem(id=osp.splitext(p)[0],
+            items.append(DatasetItem(id=osp.splitext(item_id)[0],
                 subset=self._subset, image=image, annotations=annotations))
         return items, categories
 
@@ -298,8 +302,8 @@ def apply(self):
             os.makedirs(osp.join(subset_dir, LabelMePath.MASKS_DIR),
                 exist_ok=True)
 
-            for item in subset:
-                self._save_item(item, subset_dir)
+            for index, item in enumerate(subset):
+                self._save_item(item, subset_dir, index)
 
     def _get_label(self, label_id):
         if label_id is None:
@@ -307,15 +311,11 @@ def _get_label(self, label_id):
         return self._extractor.categories()[AnnotationType.label] \
             .items[label_id].name
 
-    def _save_item(self, item, subset_dir):
+    def _save_item(self, item, subset_dir, index):
         from lxml import etree as ET
 
         log.debug("Converting item '%s'", item.id)
 
-        if '/' in item.id:
-            raise Exception("Can't export item '%s': "
-                "LabelMe format only supports flat image layout" % item.id)
-
         image_filename = self._make_image_filename(item)
         if self._save_images:
             if item.has_image and item.image.has_data:
@@ -324,8 +324,8 @@ def _save_item(self, item, subset_dir):
                 log.debug("Item '%s' has no image", item.id)
 
         root_elem = ET.Element('annotation')
-        ET.SubElement(root_elem, 'filename').text = image_filename
-        ET.SubElement(root_elem, 'folder').text = ''
+        ET.SubElement(root_elem, 'filename').text = osp.basename(image_filename)
+        ET.SubElement(root_elem, 'folder').text = osp.dirname(image_filename)
 
         source_elem = ET.SubElement(root_elem, 'source')
         ET.SubElement(source_elem, 'sourceImage').text = ''
@@ -384,7 +384,8 @@ def _save_item(self, item, subset_dir):
                 ET.SubElement(poly_elem, 'username').text = \
                     str(ann.attributes.pop('username', ''))
             elif ann.type == AnnotationType.mask:
-                mask_filename = '%s_mask_%s.png' % (item.id, obj_id)
+                mask_filename = '%s_mask_%s.png' % \
+                    (item.id.replace('/', '_'), obj_id)
                 save_image(osp.join(subset_dir, LabelMePath.MASKS_DIR,
                         mask_filename),
                     self._paint_mask(ann.image))
@@ -424,7 +425,7 @@ def _save_item(self, item, subset_dir):
                 ET.SubElement(parts_elem, 'hasparts').text = ''
                 ET.SubElement(parts_elem, 'ispartof').text = str(leader_id)
 
-        xml_path = osp.join(subset_dir, '%s.xml' % item.id)
+        xml_path = osp.join(subset_dir, 'item_%09d.xml' % index)
         with open(xml_path, 'w', encoding='utf-8') as f:
             xml_data = ET.tostring(root_elem, encoding='unicode',
                 pretty_print=True)


Test patch:
diff --git a/tests/test_labelme_format.py b/tests/test_labelme_format.py
--- a/tests/test_labelme_format.py
+++ b/tests/test_labelme_format.py
@@ -30,7 +30,7 @@ def _test_save_and_load(self, source_dataset, converter, test_dir,
 
     def test_can_save_and_load(self):
         source_dataset = Dataset.from_iterable([
-            DatasetItem(id=1, subset='train',
+            DatasetItem(id='dir1/1', subset='train',
                 image=np.ones((16, 16, 3)),
                 annotations=[
                     Bbox(0, 4, 4, 8, label=2, group=2),
@@ -54,7 +54,7 @@ def test_can_save_and_load(self):
         })
 
         target_dataset = Dataset.from_iterable([
-            DatasetItem(id=1, subset='train',
+            DatasetItem(id='dir1/1', subset='train',
                 image=np.ones((16, 16, 3)),
                 annotations=[
                     Bbox(0, 4, 4, 8, label=0, group=2, id=0,
@@ -96,18 +96,6 @@ def test_can_save_and_load(self):
                 partial(LabelMeConverter.convert, save_images=True),
                 test_dir, target_dataset=target_dataset)
 
-    def test_cant_save_dataset_with_relative_paths(self):
-        expected_dataset = Dataset.from_iterable([
-            DatasetItem(id='dir/1', image=np.ones((2, 6, 3))),
-        ], categories={
-            AnnotationType.label: LabelCategories(),
-        })
-
-        with self.assertRaisesRegex(Exception, r'only supports flat'):
-            with TestDir() as test_dir:
-                self._test_save_and_load(expected_dataset,
-                    LabelMeConverter.convert, test_dir)
-
 
 DUMMY_DATASET_DIR = osp.join(osp.dirname(__file__), 'assets', 'labelme_dataset')
 
@@ -139,7 +127,7 @@ def test_can_import(self):
         ]
 
         target_dataset = Dataset.from_iterable([
-            DatasetItem(id='img1', image=img1,
+            DatasetItem(id='example_folder/img1', image=img1,
                 annotations=[
                     Polygon([43, 34, 45, 34, 45, 37, 43, 37],
                         label=0, id=0,