Repository: Lightning-AI/torchmetrics

Issue:
Missing documentation for mask format for segmentation IOU type in mAP detection metric
The [documentation](https://torchmetrics.readthedocs.io/en/v0.9.0/detection/mean_average_precision.html) seems to be missing the required format for masks when using the "segm" option for `iou_type`.

I'm assuming there has to be a dict entry with key "masks", but what should the shape and dtype of the masks be? I tried a few different options, but always got cryptic errors from pycocotools. The [example script](https://github.com/PyTorchLightning/metrics/blob/567fa3405d5a60ba0a6052e57afe5a5d34c376a7/tm_examples/detection_map.py) doesn't mention masks at all. I tried to figure it out from the [unit tests](https://github.com/PyTorchLightning/metrics/blob/567fa3405d5a60ba0a6052e57afe5a5d34c376a7/tests/detection/test_map.py), but I couldn't really follow what was going on.

I'd be happy to create a PR and improve the documentation myself if someone can help me figure out the required format.

Related question: I was very much looking forward to the torchmetrics implementation of mAP with masks because I thought it meant not having to deal with the poorly documented pycocotools. I've been following #822 and saw pycocotools is still required. Besides improving documentation, are there plans for removing this dependency entirely?


Discussion:
Hi! thanks for your contribution!, great first issue!
cc: @gianscarpe can you comment what the format should be then @JeroenDelcour can do the actual PR? :]
Hi! let me clarify this.
Predictions are a list of dictionaries. Each dictionary has 3 keys, "masks", "labels", and "scores". Labels and scores are the same as in bounding-boxes IOU. "masks" is a tensor of shape N_predxHxW, as the example below
```
{'masks': tensor([[[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]],

        [[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]]]), 'scores': tensor([0.3180, 0.7260]), 'labels': tensor([3, 2], dtype=torch.int32)}
```
In unit-test was easier to use the RLE version and decode it when giving it as input to `update`. Thanks for pointing out!
Regarding the dependency, unfortunately, mask-IOU is time and computation-consuming, and pycocotools have a CPP implementation. What I suggest I implementing a RLE toolbox that also works with CUDA and integrate it in torchmetrics, removing pycocotools depency
edit: Fixed in #1080 

Just wanted to follow up on this. Using the `segm` flag and providing masks (attempted in both BoolTensor, ByteTensor), scores and labels, I am getting the following issue:

```
File "/usr/local/lib/python3.8/site-packages/torchmetrics/metric.py", line 508, in wrapped_func
  value = compute(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/torchmetrics/detection/mean_ap.py", line 902, in compute
  precisions, recalls = self._calculate(classes)
File "/usr/local/lib/python3.8/site-packages/torchmetrics/detection/mean_ap.py", line 728, in _calculate
  eval_imgs = [
File "/usr/local/lib/python3.8/site-packages/torchmetrics/detection/mean_ap.py", line 729, in <listcomp>
  self._evaluate_image(img_id, class_id, area, max_detections, ious)
File "/usr/local/lib/python3.8/site-packages/torchmetrics/detection/mean_ap.py", line 568, in _evaluate_image
  return self.__evaluate_image_gt_no_preds(gt, gt_label_mask, area_range, nb_iou_thrs)
File "/usr/local/lib/python3.8/site-packages/torchmetrics/detection/mean_ap.py", line 487, in __evaluate_image_gt_no_preds
  areas = box_area(gt)
File "/usr/local/lib/python3.8/site-packages/torchvision/ops/boxes.py", line 241, in box_area
  boxes = _upcast(boxes)
File "/usr/local/lib/python3.8/site-packages/torchvision/ops/boxes.py", line 220, in _upcast
  if t.is_floating_point():
AttributeError: 'tuple' object has no attribute 'is_floating_point'
```

The reuqirements.txt includes (latest stable at pypi):
```
torch==1.11.0
torchvision==0.12.0
torchmetrics==0.9.0
```

We already have the logic for appending an empty mask in case there are none (this was an unwritten requirement in the bbox case for metrics.mAP). It doesn't change the failure method above.

Is there anything we are missing on our side?

<details>
<summary>
Expand to see how preds/targets look like
</summary>

```

[{'masks': tensor([[[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]],

        [[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]],

        [[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]],

        ...,

        [[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]],

        [[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]],

        [[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]]]), 'labels': tensor([ 1,  1,  1, 10,  1, 31, 31, 28, 27, 62,  1], dtype=torch.int32), 'scores': tensor([0.9992, 0.9986, 0.9971, 0.9676, 0.7950, 0.5312, 0.4368, 0.4220, 0.3327,
        0.3197, 0.2246])}]
 ```
</details>

Sidenote:
- digging into `gt` just before the area call, I get 
> print: ((428, 640), b'a]`43X=5K2O1O1N20O00XCD`<=]CEc<a03N[_f3')
> type: <class 'tuple'>
> len: 2

@23pointsNorth thanks for trying this out! It seems from  your reported issue that you're using IOU type bboxes. Can you check what IOU type are you passing as argument at initialization ? should be "segm" for using segmentation IOU
```
File "/usr/local/lib/python3.8/site-packages/torchvision/ops/boxes.py", line 241, in box_area
  boxes = _upcast(boxes)
```

Hi @gianscarpe , it was the case that if there are 0 predictions made by the model, another call was made that had some leftover area calculation that wasn't converted to use the `self.iou_type`. See here for the fix #1080. 

Thanks for doing #822 in the first place!
Hi @gianscarpe,
one thing is not clear to me about the input format.

The masks must necessarily contain only one element for each mask?

e.g.
```python
import torch

masks = torch.tensor(
    [
        [
             [0, 0, 0, 0, 0],
             [0, 0, 1, 1, 0],
             [0, 0, 1, 1, 0],
             [0, 0, 0, 0, 0],
             [0, 0, 0, 0, 0],
        ],
    ],
)
```
or it could also be something like
```python
import torch

masks = torch.tensor(
    [
        [
             [1, 1, 0, 0, 0],
             [1, 1, 0, 0, 0],
             [0, 0, 0, 0, 0],
             [1, 1, 0, 1, 1],
             [1, 1, 0, 0, 0],
        ],
    ],
)
```
Hi @faber6911, thanks for reaching out. I believe you're referring to multi-predictions, but correct me if I'm wrong. In that case, you should have a mask of N channels (where N number of predictions), similarly to what it's already done with bounding-boxes. So that:
```py
masks = torch.tensor(
    [
        [
             [0, 0, 0, 0, 0],
             [0, 0, 0, 0, 0],
             [0, 0, 0, 0, 0],
             [0, 0, 0, 1, 1],
             [0, 0, 0, 0, 0],
        ], # mask for pred. 1
        [
             [1, 1, 0, 0, 0],
             [1, 1, 0, 0, 0],
             [0, 0, 0, 0, 0],
             [0, 0, 0, 0, 0],
             [0, 0, 0, 0, 0],
        ], # mask for pred. 2
    ],
)

labels = torch.tensor([1, 2]) # labels for preds 
```
and therefore:
```
>>> masks.shape
torch.Size([2, 5, 5])
```
@gianscarpe Yep, thank you.
With bboxes it was easier for me to figure out how to feed the metric but with segmentation maps, it was difficult for me to have the same intuition.
Hi, sorry for not responding sooner. Thank you @gianscarpe for explaining. I was hoping to have the time to help write the documentation, but unfortunately priorities at my work have changed. I'd still like to help, but there's not telling when I might have the time. Apologies!

Resolving patch:
diff --git a/examples/detection_map.py b/examples/detection_map.py
--- a/examples/detection_map.py
+++ b/examples/detection_map.py
@@ -16,12 +16,19 @@
 python detection_map.py.
 """
 
-from torch import IntTensor, Tensor
+from torch import BoolTensor, IntTensor, Tensor
 
 from torchmetrics.detection.mean_ap import MeanAveragePrecision
 
 # Preds should be a list of elements, where each element is a dict
 # containing 3 keys: boxes, scores, labels
+mask_pred = [
+    [0, 0, 0, 0, 0],
+    [0, 0, 1, 1, 0],
+    [0, 0, 1, 1, 0],
+    [0, 0, 0, 0, 0],
+    [0, 0, 0, 0, 0],
+]
 preds = [
     dict(
         # The boxes keyword should contain an [N,4] tensor,
@@ -34,23 +41,35 @@
         # The labels keyword should contain an [N,] tensor
         # with integers of the predicted classes
         labels=IntTensor([0]),
+        # The masks keyword should contain an [N,H,W] tensor,
+        # where H and W are the image height and width, respectively,
+        # with boolean masks. This is only required when iou_type is `segm`.
+        masks=BoolTensor([mask_pred]),
     )
 ]
 
 # Target should be a list of elements, where each element is a dict
-# containing 2 keys: boxes and labels. Each keyword should be formatted
-# similar to the preds argument. The number of elements in preds and
-# target need to match
+# containing 2 keys: boxes and labels (and masks, if iou_type is `segm`).
+# Each keyword should be formatted similar to the preds argument.
+# The number of elements in preds and target need to match
+mask_tgt = [
+    [0, 0, 0, 0, 0],
+    [0, 0, 1, 0, 0],
+    [0, 0, 1, 1, 0],
+    [0, 0, 1, 0, 0],
+    [0, 0, 0, 0, 0],
+]
 target = [
     dict(
         boxes=Tensor([[214.0, 41.0, 562.0, 285.0]]),
         labels=IntTensor([0]),
+        masks=BoolTensor([mask_tgt]),
     )
 ]
 
 if __name__ == "__main__":
     # Initialize metric
-    metric = MeanAveragePrecision()
+    metric = MeanAveragePrecision(iou_type="bbox")
 
     # Update metric with predictions and respective ground truth
     metric.update(preds, target)
diff --git a/src/torchmetrics/detection/mean_ap.py b/src/torchmetrics/detection/mean_ap.py
--- a/src/torchmetrics/detection/mean_ap.py
+++ b/src/torchmetrics/detection/mean_ap.py
@@ -226,7 +226,8 @@ class MeanAveragePrecision(Metric):
             Input format of given boxes. Supported formats are ``[`xyxy`, `xywh`, `cxcywh`]``.
         iou_type:
             Type of input (either masks or bounding-boxes) used for computing IOU.
-            Supported IOU types are ``[`bboxes`, `segm`]``.
+            Supported IOU types are ``["bboxes", "segm"]``.
+            If using ``"segm"``, masks should be provided (see :meth:`update`).
         iou_thresholds:
             IoU thresholds for evaluation. If set to ``None`` it corresponds to the stepped range ``[0.5,...,0.95]``
             with step ``0.05``. Else provide a list of floats.
@@ -355,6 +356,8 @@ def update(self, preds: List[Dict[str, Tensor]], target: List[Dict[str, Tensor]]
                 - ``scores``: ``torch.FloatTensor`` of shape ``[num_boxes]`` containing detection scores for the boxes.
                 - ``labels``: ``torch.IntTensor`` of shape ``[num_boxes]`` containing 0-indexed detection classes
                   for the boxes.
+                - ``masks``: ``torch.bool`` of shape ``[num_boxes, image_height, image_width]`` containing boolean
+                  masks. Only required when `iou_type="segm"`.
 
             target: A list consisting of dictionaries each containing the key-values
                 (each dictionary corresponds to a single image):
@@ -364,6 +367,8 @@ def update(self, preds: List[Dict[str, Tensor]], target: List[Dict[str, Tensor]]
                   ``[xmin, ymin, xmax, ymax]`` in absolute image coordinates.
                 - ``labels``: ``torch.IntTensor`` of shape ``[num_boxes]`` containing 0-indexed ground truth
                    classes for the boxes.
+                - ``masks``: ``torch.bool`` of shape ``[num_boxes, image_height, image_width]`` containing boolean
+                  masks. Only required when `iou_type="segm"`.
 
         Raises:
             ValueError:


Test patch: