Package: detection_datasets
Detection datasetsEasily load and transform datasets for object detection.Documentation:https://blinjrm.github.io/detection-datasets/Source Code:https://github.com/blinjrm/detection-datasetsDatasets on Hugging Face Hub:https://huggingface.co/detection-datasetsdetection_datasetsaims to make it easier to work with detection datasets.This library works alongside theDetection datasetorganisation on the 🤗 Hub, where some detection datasets have been uploaded in the format expected by the library, and are ready to use.The main features are:Readthe dataset :From disk if it has already been downloaded.Directly from the Hugging Face Hub if italready exist.Transformthe dataset:Select a subset of data.Remap categories.Create new train-val-test splits.Visualizethe annotations and images.Writethe dataset:To disk, selecting the target detection format:COCO,YOLOand more to come.To the Hugging Face Hub for easy reuse in a different environment and share with the community.Read the quick start bellow, or directly jump to the tutorials:GoalTutorialColabLoad from disk and upload to the HubOpen in the docsLoad from the Hub and transformOpen in the docsGetting started0. SetupRequirementsPython 3.8.1+detection_datasetsis upon the great work of:Pandasfor manipulating data.Hugging Face Datasetsto store and load datasets from the Hub.Installation$pipinstalldetection_datasetsImportfromdetection_datasetsimportDetectionDataset1. ReadFrom local filesystemconfig={'dataset_format':'coco',# the format of the dataset on disk'path':'path/do/data/on/disk',# where the dataset is located'splits':{# how to read the files'train':('train.json','train'),# name of the split (annotation file, images directory)'test':('test.json','test'),},}dd=DetectionDataset()dd.from_disk(**config)# note that you can use method cascading as well:# dd = DetectionDataset().from_disk(**config)From the Hugging Face HubThedetection_datasetlibrary works alongside theDetection datasetorganisation on the Hugging Face Hub, where some detection datasets have been uploaded in the format expected by the library, and are ready to use.dd=DetectionDataset().from_hub(name='fashionpedia')Currently supported format for reading datasets are:COCOmore to comeThe list of datasets available from the Hub is given by:# Search in the "detection-datasets" repository on the Hub.DetectionDataset().available_in_hub()# Search in another repository on the Hub.DetectionDataset().available_in_hub(repo_name=MY_REPO_OR_ORGANISATION)2. TransformThe supported transformations are:# Select a subset of images, perserving the splits and their proportionsdd.select(n_images=1000)# Shuffle the dataset, perserving the splits and their proportionsdd.shuffle(seed=42)# Create new train-val-test splits, overwritting the splits from the original datasetdd.split(splits=[0.8,0.1,0.1])# Map existing categories to new categories.# The annotations with a category absent from the mapping are dropped.dd.map_categories(mapping={'existing_category':'new_category'})These transformations can be chained; for example here we select a subset of 10.000 images and create new train-val-test splits:dd=DetectionDataset()\.from_hub(name='fashionpedia')\.select(n_images=10000)\.split(splits=[0.8,0.1,0.1])3. VisualizeTheDetectionDatasetobjects contains several properties to analyze your data:dd.data# This is equivlent to calling `dd.get_data('image')`,# and returns a DataFrame with 1 row per imagedd.get_data('bbox')# Returns a DataFrame with 1 row per annotationdd.n_images# Number of imagesdd.n_bbox# Number of annotationsdd.splits# List of split namesdd.split_proportions# DataFrame with the % of iamges in each splitdd.categories# DataFrame with the categories and thei idsdd.category_names# List of categoriesdd.n_categories# Number of categoriesYou can also visualize a image with its annotations in a notebook:dd.show()# Shows a random image from the datasetdd.show(image_id=42)# Shows the select image based on image_id4. WriteTo local filesystemOnce the dataset is ready, you can write it to the local filesystem in a given format:dd.to_disk(dataset_format='yolo',name='MY_DATASET_NAME',path='DIRECTORY_TO_WRITE_TO',)Currently supported format for writing datasets are:YOLOCOCOMMDETmore to comeTo the Hugging Face HubThe dataset can also be easily uploaded to the Hugging Face Hub, for reuse later on or in a different environment:dd.to_hub(dataset_name='MY_DATASET_NAME',repo_name='MY_REPO_OR_ORGANISATION')The dataset viewer on the Hub will work out of the box, and we encourage you to update the README in your new repo to make it easier for the comminuty to use the dataset.