Repository: openvinotoolkit/datumaro

Issue:
[Performance] Converting mask to RLE comparison with COCO tools
I've got the following results for datumaro utils and COCO tools:

![image](https://user-images.githubusercontent.com/40690378/196153157-34b51eef-0a25-410e-8e56-773a3a7282db.png)

It shows that DM implementation ~15 times slower (at least in my case).
Need to investigate in advance.

My system specs are:

Intel i7 12700K
Python 3.8.10
Ubuntu 20.04 (Run in VMWare Player 16)
pip freeze:

<details>
> absl-py==1.3.0
> asgiref==3.5.2
> astroid==2.11.7
> astunparse==1.6.3
> attrs==21.4.0
> av==8.0.2
> azure-core==1.26.0
> azure-storage-blob==12.13.0
> boto3==1.17.61
> botocore==1.20.112
> cachetools==5.2.0
> certifi==2022.9.24
> cffi==1.15.1
> charset-normalizer==2.0.12
> click==7.1.2
> contourpy==1.0.5
> coreapi==2.3.3
> coreschema==0.0.4
> croniter==1.3.7
> cryptography==38.0.1
> cycler==0.11.0
> datumaro==0.3
> defusedxml==0.7.1
> Deprecated==1.2.13
> dill==0.3.5.1
> diskcache==5.0.2
> dj-pagination==2.5.0
> dj-rest-auth==2.2.4
> Django==3.2.15
> django-allauth==0.50.0
> django-appconf==1.0.4
> django-auth-ldap==2.2.0
> django-compressor==2.4
> django-cors-headers==3.5.0
> django-extensions==3.0.8
> django-filter==2.4.0
> django-rq==2.3.2
> django-sendfile2==0.7.0
> djangorestframework==3.12.4
> dnspython==2.2.0
> drf-spectacular==0.22.1
> EasyProcess==0.3
> entrypoint2==1.1
> flatbuffers==22.9.24
> fonttools==4.37.4
> furl==2.1.0
> gast==0.5.3
> gitdb==4.0.9
> GitPython==3.1.27
> google-api-core==2.10.2
> google-auth==2.12.0
> google-auth-oauthlib==0.4.6
> google-cloud-core==2.3.2
> google-cloud-storage==1.42.0
> google-crc32c==1.5.0
> google-pasta==0.2.0
> google-resumable-media==2.4.0
> googleapis-common-protos==1.56.4
> grpcio==1.49.1
> h5py==3.6.0
> idna==3.4
> importlib-resources==5.10.0
> inflection==0.5.1
> isodate==0.6.1
> isort==5.10.1
> itypes==1.2.0
> Jinja2==3.1.2
> jmespath==0.10.0
> jsonschema==4.16.0
> keras==2.8.0
> Keras-Preprocessing==1.1.2
> kiwisolver==1.4.4
> lazy-object-proxy==1.7.1
> libclang==14.0.6
> limits==2.7.0
> lxml==4.9.1
> Markdown==3.2.2
> MarkupSafe==2.1.1
> matplotlib==3.6.1
> mccabe==0.7.0
> mistune==2.0.4
> msrest==0.7.1
> natsort==8.0.0
> networkx==2.8.7
> nibabel==4.0.2
> numpy==1.22.0
> oauthlib==3.2.1
> opencv-python==4.6.0.66
> opencv-python-headless==4.5.5.62
> opt-einsum==3.3.0
> orderedmultidict==1.0.1
> orjson==3.8.0
> packaging==21.3
> pandas==1.5.0
> patool==1.12
> pdf2image==1.14.0
> Pillow==9.0.1
> pkgutil-resolve-name==1.3.10
> platformdirs==2.5.2
> protobuf==4.21.7
> pyasn1==0.4.8
> pyasn1-modules==0.2.8
> pycocotools==2.0.5
> pycparser==2.21
> Pygments==2.7.4
> PyJWT==2.5.0
> pylint==2.14.5
> pylint-django==2.5.3
> pylint-plugin-utils==0.7
> pylogbeat==2.0.0
> pyparsing==3.0.9
> pyrsistent==0.18.1
> python-dateutil==2.8.2
> python-ldap==3.4.0
> python-logstash-async==2.2.0
> python3-openid==3.2.0
> pytz==2020.1
> pyunpack==0.2.1
> PyYAML==6.0
> rcssmin==1.0.6
> redis==3.5.3
> requests==2.26.0
> requests-oauthlib==1.3.1
> rjsmin==1.1.0
> rope==0.17.0
> rq==1.5.1
> rq-scheduler==0.10.0
> rsa==4.9
> ruamel.yaml==0.17.21
> ruamel.yaml.clib==0.2.6
> rules==2.2
> s3transfer==0.4.2
> scipy==1.9.2
> Shapely==1.7.1
> six==1.16.0
> smmap==5.0.0
> snakeviz==2.1.0
> sqlparse==0.4.2
> tensorboard==2.8.0
> tensorboard-data-server==0.6.1
> tensorboard-plugin-wit==1.8.1
> tensorboardX==2.5.1
> tensorflow==2.8.1
> tensorflow-estimator==2.8.0
> tensorflow-io-gcs-filesystem==0.27.0
> termcolor==2.0.1
> tomli==2.0.1
> tomlkit==0.11.5
> tornado==6.2
> types-cryptography==3.3.23
> typing-extensions==4.4.0
> uritemplate==4.1.1
> urllib3==1.26.12
> Werkzeug==1.0.1
> wrapt==1.14.1
> zipp==3.9.0

</details>


Discussion:
Hi @bsekachev , I took a look for this issue. But, I found that two outputs from `mask_to_rle()` and `pycocotools.mask.encode()` are different. This seems because `pycocotools` compresses `counts` array again with `LEB128` compression algorithm ([link](https://github.com/cocodataset/cocoapi/blob/8c9bcc3cf640524c4c20a9c40e89cb6a2f2fa0e9/PythonAPI/pycocotools/mask.py#L12)). To improve performances, I think we can just replace Datumaro's custom logic to compute uncompressed RLE in `mask_to_rle()` with a wrapper function to compute compressed RLE by `pycocotools.mask.encode()`. Is there a potential harm if we introduce the compressed RLE in `mask_to_rle()` rather than using the uncompressed RLE?

```bash
import numpy as np
from datumaro.util.mask_tools import mask_to_rle
import pycocotools.mask as mask_utils

mask = np.zeros([5, 5])

for i in range(5):
    mask[i, :i] = 1

mask = mask.astype(dtype=np.uint8)

print(mask)
print(mask_to_rle(mask))
print(mask_utils.encode(np.asfortranarray(mask)))
```

```
[[0 0 0 0 0]
 [1 0 0 0 0]
 [1 1 0 0 0]
 [1 1 1 0 0]
 [1 1 1 1 0]]
{'counts': array([1, 4, 2, 3, 3, 2, 4, 1, 5]), 'size': [5, 5]}
{'size': [5, 5], 'counts': b'142O1O1O1'}
```

Resolving patch:
diff --git a/datumaro/plugins/synthetic_data/image_generator.py b/datumaro/plugins/synthetic_data/image_generator.py
--- a/datumaro/plugins/synthetic_data/image_generator.py
+++ b/datumaro/plugins/synthetic_data/image_generator.py
@@ -66,7 +66,13 @@ def generate_dataset(self) -> None:
 
         mp_ctx = get_context("spawn")  # On Mac 10.15 and Python 3.7 fork leads to hangs
         with mp_ctx.Pool(processes=self._cpu_count) as pool:
-            params = pool.map(self._generate_category, [Random(i) for i in range(self._categories)])
+            try:
+                params = pool.map(
+                    self._generate_category, [Random(i) for i in range(self._categories)]
+                )
+            finally:
+                pool.close()
+                pool.join()
 
         instances_weights = np.repeat(self._weights, self._instances, axis=0)
         weight_per_img = np.tile(instances_weights, (self._categories, 1))
@@ -88,7 +94,11 @@ def generate_dataset(self) -> None:
             generation_params.append((param, w, indices))
 
         with mp_ctx.Pool(processes=self._cpu_count) as pool:
-            pool.starmap(self._generate_image_batch, generation_params)
+            try:
+                pool.starmap(self._generate_image_batch, generation_params)
+            finally:
+                pool.close()
+                pool.join()
 
     @scoped
     def _generate_image_batch(
diff --git a/datumaro/util/mask_tools.py b/datumaro/util/mask_tools.py
--- a/datumaro/util/mask_tools.py
+++ b/datumaro/util/mask_tools.py
@@ -8,6 +8,7 @@
 
 import numpy as np
 
+from datumaro._capi import encode
 from datumaro.util.image import lazy_image, load_image
 
 
@@ -153,6 +154,10 @@ def lazy_mask(path, inverse_colormap=None):
 
 
 def mask_to_rle(binary_mask):
+    return encode(binary_mask)
+
+
+def mask_to_rle_py(binary_mask):
     # walk in row-major order as COCO format specifies
     bounded = binary_mask.ravel(order="F")
 
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -8,6 +8,7 @@
 from distutils.util import strtobool
 
 import setuptools
+from pybind11.setup_helpers import Pybind11Extension, build_ext
 
 # Snyk scan integration
 here = None
@@ -54,6 +55,15 @@ def parse_requirements(filename=CORE_REQUIREMENTS_FILE):
 with open("README.md", "r") as fh:
     long_description = fh.read()
 
+ext_modules = [
+    Pybind11Extension(
+        "datumaro._capi",
+        ["datumaro/capi/pybind.cpp"],
+        define_macros=[("VERSION_INFO", find_version(here))],
+        extra_compile_args=["-O3"],
+    ),
+]
+
 setuptools.setup(
     name="datumaro",
     version=find_version(here),
@@ -79,11 +89,13 @@ def parse_requirements(filename=CORE_REQUIREMENTS_FILE):
         "tf-gpu": ["tensorflow-gpu"],
         "default": DEFAULT_REQUIREMENTS,
     },
+    ext_modules=ext_modules,
     entry_points={
         "console_scripts": [
             "datum=datumaro.cli.__main__:main",
         ],
     },
+    cmdclass={"build_ext": build_ext},
     package_data={"datumaro.plugins.synthetic_data": ["background_colors.txt"]},
     include_package_data=True,
 )


Test patch: