Repository: openvinotoolkit/datumaro

Issue:
Images missing after merging two datasets
Hello all,

I am trying to merge two datasets using the merge command, but unfortunately this is not successful. 

The following is present:
- Two datasets -> coco_instances
- Both have a different number of images
- In the categories part (anotation file) the names are identical
- Folder structure is the same for both (project can import the datasets without any problems).

Now comes my problem:
With the merge command I try to merge both datasets.

date merge path/to/dataset1:coco_instances path/to/dataset2:coco_instances

I get a folder with the annotation file, a report and a folder for the images. However, this does not contain the images. But in the annotation file there is a path for every single image. I would like to have the images in the folder and remove the paths in the annotation file. How can I do this?

I cannot import the merged dataset (created folder after merge command) into any empty datum project. 

Datumaro version is 0.2



Discussion:
Hi, thank you for the problem report. Currently, the recommended way to merge datasets in simple cases is using the [`patch` command](https://openvinotoolkit.github.io/datumaro/docs/user-manual/command-reference/patch/). This command will also allow to pass options for how to save the results, including an option to `--save-images`. 
``` python
datum patch \ # use either --overwrite or -o output_dir
  path/to/dataset1:coco_instances path/to/dataset2:coco_instances -- --save-images
```

Probably, we need to allow export options in the merge command too, because currently it only produces annotations in Datumaro format. If you need complex merging functionality, consider using API like in the example [here](https://github.com/openvinotoolkit/datumaro/blob/develop/tests/test_ops.py#L231-L263).
@zhiltsov-max thanks for the quick response. The solution works in part for me, as the second dataset contains a few labels that the first does not. 

For example:
1st dataset -> Labels -> [a, b, c]
2nd dataset -> Labels -> [a, b, c, d, e]

The merged dataset should contain all images and all labels. 

Unfortunately I don't see any option now with the patch command to unify this information. 

Is there another option?
I think, we could extend the `patch` command to support label addition in future. Currently, you can try to use API:

``` python
from datumaro.components.dataset import Dataset
from datumaro.components.annotation import AnnotationType

dataset1 = Dataset.import_from('path1/', 'format')
dataset2 = Dataset.import_from('path2/', 'format')
dataset1.transform('project_labels', 
    set(label.name for label in dataset1.categories()[AnnotationType.label]) |
    set(label.name for label in dataset2.categories()[AnnotationType.label])
) # merge labels
dataset1.update(dataset2)
dataset1.export('path/', 'format', save_images=True)
```

Resolving patch:
diff --git a/datumaro/cli/commands/merge.py b/datumaro/cli/commands/merge.py
--- a/datumaro/cli/commands/merge.py
+++ b/datumaro/cli/commands/merge.py
@@ -10,13 +10,15 @@
 import os.path as osp
 
 from datumaro.components.dataset import DEFAULT_FORMAT
+from datumaro.components.environment import Environment
 from datumaro.components.errors import (
     DatasetMergeError, DatasetQualityError, ProjectNotFoundError,
 )
 from datumaro.components.operations import IntersectMerge
+from datumaro.components.project import ProjectBuildTargets
 from datumaro.util.scope import scope_add, scoped
 
-from ..util import MultilineFormatter
+from ..util import MultilineFormatter, join_cli_args
 from ..util.errors import CliException
 from ..util.project import (
     generate_next_file_name, load_project, parse_full_revpath,
@@ -26,13 +28,17 @@
 def build_parser(parser_ctor=argparse.ArgumentParser):
     parser = parser_ctor(help="Merge few projects",
         description="""
-        Merges multiple datasets into one and produces a new
-        dataset in the default format. This can be useful if you
-        have few annotations and wish to merge them,
-        taking into consideration potential overlaps and conflicts.
-        This command can try to find a common ground by voting or
+        Merges multiple datasets into one and produces a new dataset.
+        The command can be useful if you have few annotations and wish
+        to merge them, taking into consideration potential overlaps and
+        conflicts. This command can try to find common ground by voting or
         return a list of conflicts.|n
         |n
+        In simple cases, when dataset images do not intersect and new
+        labels are not added, the recommended way of merging is using the
+        "patch" command. It will offer better performance and provide the same
+        results.|n
+        |n
         This command has multiple forms:|n
         1) %(prog)s <revpath>|n
         2) %(prog)s <revpath> <revpath> ...|n
@@ -56,6 +62,13 @@ def build_parser(parser_ctor=argparse.ArgumentParser):
         It can be useful for dataset paths in targets. When not specified,
         the current project's working tree is used.|n
         |n
+        The output format can be specified with the '-f/--format' option.
+        Each dataset format has its own export
+        options, which are passed after the '--' separator (see examples),
+        pass '-- -h' for more info. If not stated otherwise, by default
+        only annotations are exported; to include images pass
+        '--save-images' parameter.|n
+        |n
         Examples:|n
         - Merge annotations from 3 (or more) annotators:|n
         |s|s%(prog)s project1/ project2/ project3/|n
@@ -72,12 +85,17 @@ def build_parser(parser_ctor=argparse.ArgumentParser):
         |n
         - Merge a source from a previous revision and a dataset:|n
         |s|s%(prog)s HEAD~2:source-2 path/to/dataset2:yolo
+        |n
+        - Merge datasets and save in different format:|n
+        |s|s%(prog)s -f voc dataset1/:yolo path2/:coco -- --save-images
         """,
         formatter_class=MultilineFormatter)
 
     def _group(s):
         return s.split(',')
 
+    parser.add_argument('_positionals', nargs=argparse.REMAINDER,
+        help=argparse.SUPPRESS) # workaround for -- eaten by positionals
     parser.add_argument('targets', nargs='+',
         help="Target dataset revpaths (repeatable)")
     parser.add_argument('-iou', '--iou-thresh', default=0.25, type=float,
@@ -97,8 +115,14 @@ def _group(s):
         help="Output directory (default: generate a new one)")
     parser.add_argument('--overwrite', action='store_true',
         help="Overwrite existing files in the save directory")
+    parser.add_argument('-f', '--format', default=DEFAULT_FORMAT,
+        help="Output format (default: %(default)s)")
     parser.add_argument('-p', '--project', dest='project_dir',
         help="Directory of the 'current' project (default: current dir)")
+    parser.add_argument('extra_args', nargs=argparse.REMAINDER,
+        help="Additional arguments for converter (pass '-- -h' for help). "
+            "Must be specified after the main command arguments and after "
+            "the '--' separator")
     parser.set_defaults(command=merge_command)
 
     return parser
@@ -110,6 +134,22 @@ def get_sensitive_args():
 
 @scoped
 def merge_command(args):
+    # Workaround. Required positionals consume positionals from the end
+    args._positionals += join_cli_args(args, 'targets', 'extra_args')
+
+    has_sep = '--' in args._positionals
+    if has_sep:
+        pos = args._positionals.index('--')
+        if pos == 0:
+            raise argparse.ArgumentError(None,
+                message="Expected at least 1 target argument")
+    else:
+        pos = len(args._positionals)
+    args.targets = args._positionals[:pos] or [ProjectBuildTargets.MAIN_TARGET]
+    args.extra_args = args._positionals[pos + has_sep:]
+
+    show_plugin_help = '-h' in args.extra_args or '--help' in args.extra_args
+
     dst_dir = args.dst_dir
     if dst_dir:
         if not args.overwrite and osp.isdir(dst_dir) and os.listdir(dst_dir):
@@ -123,9 +163,22 @@ def merge_command(args):
     try:
         project = scope_add(load_project(args.project_dir))
     except ProjectNotFoundError:
-        if args.project_dir:
+        if not show_plugin_help and len(args.targets) == 1 and args.project_dir:
             raise
 
+    if project is not None:
+        env = project.env
+    else:
+        env = Environment()
+
+    try:
+        converter = env.converters[args.format]
+    except KeyError:
+        raise CliException("Converter for format '%s' is not found" % \
+            args.format)
+
+    export_args = converter.parse_cmdline(args.extra_args)
+
     source_datasets = []
     try:
         if len(args.targets) == 1:
@@ -144,7 +197,8 @@ def merge_command(args):
         output_conf_thresh=args.output_conf_thresh, quorum=args.quorum
     ))
     merged_dataset = merger(source_datasets)
-    merged_dataset.export(save_dir=dst_dir, format=DEFAULT_FORMAT)
+
+    merged_dataset.export(save_dir=dst_dir, format=converter, **export_args)
 
     report_path = osp.join(dst_dir, 'merge_report.json')
     save_merge_report(merger, report_path)


Test patch:
diff --git a/tests/cli/test_merge.py b/tests/cli/test_merge.py
--- a/tests/cli/test_merge.py
+++ b/tests/cli/test_merge.py
@@ -62,9 +62,9 @@ def test_can_run_self_merge(self):
             with Project.init(proj_dir) as project:
                 project.import_source('source', dataset2_url, 'voc')
 
-            result_dir = osp.join(test_dir, 'cmp_result')
-            run(self, 'merge', dataset1_url + ':coco', '-o', result_dir,
-                '-p', proj_dir)
+            result_dir = osp.join(test_dir, 'result')
+            run(self, 'merge', '-o', result_dir, '-p', proj_dir,
+                dataset1_url + ':coco')
 
             compare_datasets(self, expected, Dataset.load(result_dir),
                 require_images=True)
@@ -111,9 +111,51 @@ def test_can_run_multimerge(self):
             dataset1.export(dataset1_url, 'coco', save_images=True)
             dataset2.export(dataset2_url, 'voc', save_images=True)
 
-            result_dir = osp.join(test_dir, 'cmp_result')
-            run(self, 'merge', dataset2_url + ':voc', dataset1_url + ':coco',
-                '-o', result_dir)
+            result_dir = osp.join(test_dir, 'result')
+            run(self, 'merge', '-o', result_dir,
+                dataset2_url + ':voc', dataset1_url + ':coco')
 
             compare_datasets(self, expected, Dataset.load(result_dir),
                 require_images=True)
+
+    @mark_requirement(Requirements.DATUM_542)
+    def test_can_save_in_another_format(self):
+        dataset1 = Dataset.from_iterable([
+            DatasetItem(id=100, subset='train', image=np.ones((10, 6, 3)),
+                annotations=[
+                    Bbox(1, 2, 3, 3, label=0),
+                ]),
+        ], categories=['a', 'b'])
+
+        dataset2 = Dataset.from_iterable([
+            DatasetItem(id=100, subset='train', image=np.ones((10, 6, 3)),
+                annotations=[
+                    Bbox(1, 2, 3, 4, label=1),
+                    Bbox(5, 6, 2, 3, label=2),
+                ]),
+        ], categories=['a', 'b', 'c'])
+
+        expected =  Dataset.from_iterable([
+            DatasetItem(id=100, subset='train', image=np.ones((10, 6, 3)),
+                annotations=[
+                    Bbox(1, 2, 3, 4, label=2),
+                    Bbox(5, 6, 2, 3, label=3),
+                    Bbox(1, 2, 3, 3, label=1),
+                ]),
+        ], categories=['background', 'a', 'b', 'c'])
+
+        with TestDir() as test_dir:
+            dataset1_url = osp.join(test_dir, 'dataset1')
+            dataset2_url = osp.join(test_dir, 'dataset2')
+
+            dataset1.export(dataset1_url, 'coco', save_images=True)
+            dataset2.export(dataset2_url, 'voc', save_images=True)
+
+            result_dir = osp.join(test_dir, 'result')
+            run(self, 'merge', '-o', result_dir, '-f', 'yolo',
+                dataset2_url + ':voc', dataset1_url + ':coco',
+                '--', '--save-images')
+
+            compare_datasets(self, expected,
+                Dataset.import_from(result_dir, 'yolo'),
+                require_images=True)
diff --git a/tests/requirements.py b/tests/requirements.py
--- a/tests/requirements.py
+++ b/tests/requirements.py
@@ -28,6 +28,7 @@ class Requirements:
     DATUM_399 = "Implement import for ADE20K dataset"
     DATUM_475 = "Support import for CelebA dataset"
     DATUM_497 = "Support import for SYNTHIA dataset"
+    DATUM_542 = "Images missing after merging two datasets"
 
     # GitHub issues (bugs)
     # https://github.com/openvinotoolkit/datumaro/issues