Suggestions for Datasets and Domains for Benchmarking
Hello everyone 👋,
We're actively working on enhancing our 🏅 Object Detection Leaderboard and would like to hear your thoughts!
One of the key aspects we want to improve is the range of datasets and domains that we're currently benchmarking. Our goal is to make this leaderboard as broad and useful as possible for everyone.
We would love to hear your suggestions on:
Domains: The object detection task can vary greatly across different domains (e.g., medical imaging, autonomous driving, aerial imaging, etc.). Are there any specific domains you're interested in?
Applications: Are there any specific applications or use-cases that you think should be addressed by our benchmarks?
Datasets: Are there specific datasets that you believe should be included in our benchmarks? Please let us know which ones and why you think they're important.
Feel free to share your thoughts and ideas in this discussion. All suggestions are welcome! 🤗
Best,
Rafael
Great work on this leaderboard!
It could be interesting to add some datasets containing historical images; in particular, there is ongoing interest in using object detection approaches for parsing historical documents. One example of such a dataset is https://huggingface.co/datasets/biglam/yalta_ai_segmonto_manuscript_dataset, but I could suggest others. The challenge for many of these datasets is that there won't already be a strong baseline on the hub, so that the first model would be the best!
From my (anecdotal) experience, I've found that models which perform well on the standard benchmarks don't always do well when fine-tuned on downstream tasks slike this one, so it could be interesting to include these sorts of datasets/tasks to give people a better sense of which models might end up being better for fine-tuning on out-of-domain data compared to the original model.
Great work!
Is it possible to add some model for object detection in mobile or low-end hardware? which may focus on balancing the fps and accuracy.