metadata

app_file: app.py
colorFrom: gray
colorTo: green
description: 'TODO: add a description here'
emoji: 📚
pinned: false
runme:
  id: 01HPS3ASFJXVQR88985QNSXVN1
  version: v3
sdk: gradio
sdk_version: 4.36.0
tags:
  - evaluate
  - metric
title: mot-metrics

How to Use

>>> import evaluate
>>> from seametrics.fo_utils.utils import fo_to_payload
>>> b = fo_to_payload(
>>>         dataset="SENTRY_VIDEOS_DATASET_QA",
>>>         gt_field="ground_truth_det",
>>>         models=['volcanic-sweep-3_02_2023_N_LN1_ep288_TRACKER'],
>>>         sequence_list=["Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39"],
>>>         tracking_mode=True
>>>    )
>>> module = evaluate.load("SEA-AI/mot-metrics")
>>> res = module._calculate(b, max_iou=0.99)
>>> print(res)
{'Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39': {'volcanic-sweep-3_02_2023_N_LN1_ep288_TRACKER': {'idf1': 0.9543031226199543,
   'idp': 0.9804381846635368,
   'idr': 0.9295252225519288,
   'recall': 0.9436201780415431,
   'precision': 0.9953051643192489,
   'num_unique_objects': 2,
   'mostly_tracked': 1,
   'partially_tracked': 0,
   'mostly_lost': 1,
   'num_false_positives': 6,
   'num_misses': 76,
   'num_switches': 1,
   'num_fragmentations': 4,
   'mota': 0.9384272997032641,
   'motp': 0.5235835810268012,
   'num_transfer': 0,
   'num_ascend': 1,
   'num_migrate': 0}}}

Metric Settings

The max_iou parameter is used to filter out the bounding boxes with IOU less than the threshold. The default value is 0.5. This means that if a ground truth and a predicted bounding boxes IoU value is less than 0.5, then the predicted bounding box is not considered for association. So, the higher the max_iou value, the more the predicted bounding boxes are considered for association.

Output

The output is a dictionary containing the following metrics:

Name	Description
idf1	ID measures: global min-cost F1 score.
idp	ID measures: global min-cost precision.
idr	ID measures: global min-cost recall.
recall	Number of detections over number of objects.
precision	Number of detected objects over sum of detected and false positives.
num_unique_objects	Total number of unique object ids encountered.
mostly_tracked	Number of objects tracked for at least 80 percent of lifespan.
partially_tracked	Number of objects tracked between 20 and 80 percent of lifespan.
mostly_lost	Number of objects tracked less than 20 percent of lifespan.
num_false_positives	Total number of false positives (false-alarms).
num_misses	Total number of misses.
num_switches	Total number of track switches.
num_fragmentations	Total number of switches from tracked to not tracked.
mota	Multiple object tracker accuracy.
motp	Multiple object tracker precision.

Citations

@InProceedings{huggingface:module,
title = {A great new module},
authors={huggingface, Inc.},
year={2020}}

@article{milan2016mot16,
title={MOT16: A benchmark for multi-object tracking},
author={Milan, Anton and Leal-Taix{\'e}, Laura and Reid, Ian and Roth, Stefan and Schindler, Konrad},
journal={arXiv preprint arXiv:1603.00831},
year={2016}}

Further References

Github Repository - py-motmetrics