metadata

title: box-metrics
tags:
  - evaluate
  - metric
description: built upon yolov5 iou functions. Outputs metrics regarding box fit
sdk: gradio
sdk_version: 4.36.0
app_file: app.py
pinned: false
emoji: 🕵️

SEA-AI/det-metrics

This hugging face metric uses seametrics.detection.PrecisionRecallF1Support under the hood to compute coco-like metrics for object detection tasks. It is a modified cocoeval.py wrapped inside torchmetrics' mAP metric but with numpy arrays instead of torch tensors.

Getting Started

To get started with det-metrics, make sure you have the necessary dependencies installed. This metric relies on the evaluate and seametrics libraries for metric calculation and integration with FiftyOne datasets.

Installation

First, ensure you have Python 3.8 or later installed. Then, install det-metrics using pip:

pip install evaluate git+https://github.com/SEA-AI/seametrics@develop

Basic Usage

Here's how to quickly evaluate your object detection models using SEA-AI/box-metrics:

import evaluate

# Define your predictions and references (dict values can also by numpy arrays)
predictions = {
              "model1": [torch.tensor[n,6], torch.tensor[n,6]],
              "model2": [torch.tensor[n,6], torch.tensor[n,6]]
              }

#predictions box format: x1, y1, x2, y2, conf, label  (torch metrics format)

references = [torch.tensor[n,5], torch.tensor[n,5]]

#refernces box format: label, x1, y1, x2, y2 (torch metrics format)

# Load SEA-AI/det-metrics and evaluate
module = evaluate.load("SEA-AI/box-metrics")
module.add_batch(prediction=predictions, reference=references, sequence_name="sequence")
results = module.compute()

print(results)

This will output the evaluation metrics for your detection model.

{'sequence': {'model1':
  {'iou': '0.6',
  'bep': 0.5,
  ...
  }}}

FiftyOne Integration

Integrate SEA-AI/det-metrics with FiftyOne datasets for enhanced analysis and visualization:

import evaluate
import logging
from seametrics.payload.processor import PayloadProcessor

logging.basicConfig(level=logging.WARNING)

# Configure your dataset and model details
processor = PayloadProcessor(
    dataset_name="SENTRY_VIDEOS_DATASET_QA",
    gt_field="ground_truth_det",
    models=["ahoy-IR-b2-whales__XAVIER-AGX-JP46_CNN"],
    sequence_list=["Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39"],
    data_type="thermal",
)

# Evaluate using SEA-AI/det-metrics
module = evaluate.load("SEA-AI/box-metrics")
module.add_payload(processor.payload)
results = module.compute()

print(results)

{'Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39': {'ahoy-IR-b2-whales__XAVIER-AGX-JP46_CNN':
  {'iou': '0.6',
  'bep': 0.5,
  ...
  }}}

Further References

seametrics Library: Explore the seametrics GitHub repository for more details on the underlying library.
Understanding Metrics: For a deeper understanding of precision, recall, and other metrics, read this comprehensive guide.

Contribution

Your contributions are welcome! If you'd like to improve SEA-AI/det-metrics or add new features, please feel free to fork the repository, make your changes, and submit a pull request.