Spaces:
Sleeping
Sleeping
| app_file: app.py | |
| colorFrom: gray | |
| colorTo: green | |
| description: 'TODO: add a description here' | |
| emoji: "\U0001F4DA" | |
| pinned: false | |
| runme: | |
| id: 01HPS3ASFJXVQR88985QNSXVN1 | |
| version: v3 | |
| sdk: gradio | |
| sdk_version: 4.36.0 | |
| tags: | |
| - evaluate | |
| - metric | |
| title: mot-metrics | |
| # How to Use | |
| ```python {"id":"01HPS3ASFHPCECERTYN7Z4Z7MN"} | |
| >>> import evaluate | |
| >>> from seametrics.fo_utils.utils import fo_to_payload | |
| >>> b = fo_to_payload( | |
| >>> dataset="SENTRY_VIDEOS_DATASET_QA", | |
| >>> gt_field="ground_truth_det", | |
| >>> models=['volcanic-sweep-3_02_2023_N_LN1_ep288_TRACKER'], | |
| >>> sequence_list=["Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39"], | |
| >>> tracking_mode=True | |
| >>> ) | |
| >>> module = evaluate.load("SEA-AI/mot-metrics") | |
| >>> res = module._calculate(b, max_iou=0.99) | |
| >>> print(res) | |
| {'Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39': {'volcanic-sweep-3_02_2023_N_LN1_ep288_TRACKER': {'idf1': 0.9543031226199543, | |
| 'idp': 0.9804381846635368, | |
| 'idr': 0.9295252225519288, | |
| 'recall': 0.9436201780415431, | |
| 'precision': 0.9953051643192489, | |
| 'num_unique_objects': 2, | |
| 'mostly_tracked': 1, | |
| 'partially_tracked': 0, | |
| 'mostly_lost': 1, | |
| 'num_false_positives': 6, | |
| 'num_misses': 76, | |
| 'num_switches': 1, | |
| 'num_fragmentations': 4, | |
| 'mota': 0.9384272997032641, | |
| 'motp': 0.5235835810268012, | |
| 'num_transfer': 0, | |
| 'num_ascend': 1, | |
| 'num_migrate': 0}}} | |
| ``` | |
| ## Metric Settings | |
| The `max_iou` parameter is used to filter out the bounding boxes with IOU less than the threshold. The default value is 0.5. This means that if a ground truth and a predicted bounding boxes IoU value is less than 0.5, then the predicted bounding box is not considered for association. So, the higher the `max_iou` value, the more the predicted bounding boxes are considered for association. | |
| ## Output | |
| The output is a dictionary containing the following metrics: | |
| | Name | Description | | |
| | :------------------- | :--------------------------------------------------------------------------------- | | |
| | idf1 | ID measures: global min-cost F1 score. | | |
| | idp | ID measures: global min-cost precision. | | |
| | idr | ID measures: global min-cost recall. | | |
| | recall | Number of detections over number of objects. | | |
| | precision | Number of detected objects over sum of detected and false positives. | | |
| | num_unique_objects | Total number of unique object ids encountered. | | |
| | mostly_tracked | Number of objects tracked for at least 80 percent of lifespan. | | |
| | partially_tracked | Number of objects tracked between 20 and 80 percent of lifespan. | | |
| | mostly_lost | Number of objects tracked less than 20 percent of lifespan. | | |
| | num_false_positives | Total number of false positives (false-alarms). | | |
| | num_misses | Total number of misses. | | |
| | num_switches | Total number of track switches. | | |
| | num_fragmentations | Total number of switches from tracked to not tracked. | | |
| | mota | Multiple object tracker accuracy. | | |
| | motp | Multiple object tracker precision. | | |
| ## Citations | |
| ```bibtex {"id":"01HPS3ASFJXVQR88985GKHAQRE"} | |
| @InProceedings{huggingface:module, | |
| title = {A great new module}, | |
| authors={huggingface, Inc.}, | |
| year={2020}} | |
| ``` | |
| ```bibtex {"id":"01HPS3ASFJXVQR88985KRT478N"} | |
| @article{milan2016mot16, | |
| title={MOT16: A benchmark for multi-object tracking}, | |
| author={Milan, Anton and Leal-Taix{\'e}, Laura and Reid, Ian and Roth, Stefan and Schindler, Konrad}, | |
| journal={arXiv preprint arXiv:1603.00831}, | |
| year={2016}} | |
| ``` | |
| ## Further References | |
| - [Github Repository - py-motmetrics](https://github.com/cheind/py-motmetrics/tree/develop) | |