DOI: 10.5281/zenodo.15823741
title: Balanced Dataset

description: the dataset was obtained as a pre-annotated collection introduced in [1]. This dataset can be accessedat https://doi.org/10.5281/zenodo.7528566. The researcherscreated this dataset using high-resolution aerial imagery(10 cm GSD) obtained from the SWISSIMAGE dataset,which is freely provided by the Swiss Federal Office ofTopography (swisstopo) https://www.swisstopo.admin.ch/de/orthobilder-swissimage-10-cm.  In the data preprocessing stage, to reduce class imbalance andimprove object detection model performance, the initial datasetwas reviewed and balanced. Initially, using the available labelfiles, the sample distribution across each class was calculated.Then, to equalize the number of samples in each class to atarget value (2,500 samples), synthetic data were generated forunderrepresented classes using data augmentation techniques.This augmentation process was performed using the imgauglibrary, and included the following transformations:• Rotation: Between −30◦to +30◦to simulate differentimaging angles.• Scaling: Zooming in or out between 80% and 120% ofthe original image size.• Translation: Shifting the image horizontally and vertically up to 10% of the image dimensions.• Horizontal Flip: Applied with a 50% probability.• Vertical Flip: Applied with a 50% probability.• Gaussian Blur: Used to reduce noise and simulatevarying imaging conditions.• Additive Gaussian Noise: Added to enhance model A sample of these augmented images is shown in Figure 9.The coordinates of the bounding boxes were also updated toreflect the applied image transformations. After the balancingstage was completed, the final images and their correspondinglabels were split into three sets: training, validation, andtesting. These were then saved in standard CSV format.This approach, aimed at achieving a more uniform classdistribution, played a significant role in reducing model biasand improving accuracy during both training and evaluationphases. The number of samples in this newly constructeddataset is illustrated in Figure
description: import osimport shutilimport randomimport pandas as pdimport cv2from imgaug import augmenters as iaafrom imgaug.augmentables.bbs import BoundingBox, BoundingBoxesOnImagefrom tqdm import tqdmfrom sklearn.model_selection import train_test_split

# Initial configurationinput_image_dir = r"E:\FASTRCNN\FASTRCNN\dataset2\images"  # Path to the original imagesinput_annotations = r"E:\FASTRCNN\FASTRCNN\dataset2\train_labels.csv"  # Path to the annotations CSVoutput_dir = r"E:\FASTRCNN\FASTRCNN\dataset2\balanced_dataset"  # Output directorytarget_count = 2500  # Target number of samples per class

# Create output foldersos.makedirs(os.path.join(output_dir, "images"), exist_ok=True)os.makedirs(os.path.join(output_dir, "labels"), exist_ok=True)

# Define augmentationsaugmenter = iaa.Sequential([    iaa.Fliplr(0.5),  # 50% chance of horizontal flip    iaa.Flipud(0.5),  # 50% chance of vertical flip    iaa.Affine(        rotate=(-30, 30),         # Rotation between -30 and 30 degrees        scale=(0.8, 1.2),         # Scaling between 80% and 120%        translate_percent=(-0.1, 0.1)  # Translation up to 10%    ),    iaa.GaussianBlur(sigma=(0, 1.0)),  # Gaussian blur    iaa.AdditiveGaussianNoise(scale=(0, 0.05 * 255))  # Gaussian noise])

def load_annotations(annotations_path):    """Load annotations and calculate class distribution"""    df = pd.read_csv(annotations_path)    class_dist = df['class'].value_counts().to_dict()    return df, class_dist

def balance_dataset(df, class_dist, target_count):    """Balance the dataset using augmentation"""    balanced_data = []

    for class_id, count in class_dist.items():        class_samples = df[df['class'] == class_id]        current_count = len(class_samples)

        if current_count < target_count:            # Number of samples to be generated            needed = target_count - current_count            augmented_samples = []

            # Randomly select samples for augmentation            for _ in range(needed):                sample = class_samples.iloc[random.randint(0, current_count - 1)]                augmented_samples.append(sample)

            balanced_data.extend(class_samples.to_dict('records'))            balanced_data.extend(augmented_samples)        else:            # If class is already sufficient, sample randomly            balanced_data.extend(class_samples.sample(target_count).to_dict('records'))

    return pd.DataFrame(balanced_data)

def process_images(df, input_dir, output_dir):    """Process images and apply augmentations with bounding box adjustments"""    new_annotations = []

    for idx, row in tqdm(df.iterrows(), total=len(df)):        img_path = os.path.join(input_dir, row['filename'])        img = cv2.imread(img_path)

        if img is None:            continue

        height, width = img.shape[:2]

        # For augmented samples        if idx >= len(df.drop_duplicates()):            bbs = BoundingBoxesOnImage([                BoundingBox(                    x1=row['xmin'], y1=row['ymin'],                    x2=row['xmax'], y2=row['ymax']                )            ], shape=img.shape)

            seq_det = augmenter.to_deterministic()            new_img = seq_det.augment_image(img)            new_bbs = seq_det.augment_bounding_boxes([bbs])[0]

            if len(new_bbs.bounding_boxes) == 0:                continue

            new_bbox = new_bbs.bounding_boxes[0]

            # Clip bounding box coordinates to image bounds            x1 = max(0, min(new_bbox.x1, width))            y1 = max(0, min(new_bbox.y1, height))            x2 = max(0, min(new_bbox.x2, width))            y2 = max(0, min(new_bbox.y2, height))

            # Validate bounding box            if (x1 >= x2) or (y1 >= y2):                continue

            row['xmin'], row['ymin'] = x1, y1            row['xmax'], row['ymax'] = x2, y2

            # Create new filename            base_name = os.path.splitext(row['filename'])[0]            ext = os.path.splitext(row['filename'])[1]            new_filename = f"{base_name}_aug_{idx}{ext}"        else:            new_filename = row['filename']

        # Save image        cv2.imwrite(os.path.join(output_dir, "images", new_filename),                    new_img if 'new_img' in locals() else img)

        new_row = row.copy()        new_row['filename'] = new_filename        new_annotations.append(new_row)

    return pd.DataFrame(new_annotations)

def split_dataset(df):    """Split the dataset into train/val/test"""    train, test = train_test_split(df, test_size=0.2, stratify=df['class'])    train, val = train_test_split(train, test_size=0.1, stratify=train['class'])    return train, val, test

def save_annotations(df, split_name, output_dir):    """Save annotations for each split"""    output_path = os.path.join(output_dir, "labels", f"{split_name}_annotations.csv")    df.to_csv(output_path, index=False)

def main():    # Load original annotations    df, class_dist = load_annotations(input_annotations)    print("Original class distribution:")    print(class_dist)

    # Balance the dataset    balanced_df = balance_dataset(df, class_dist, target_count)    print("\nClass distribution after balancing:")    print(balanced_df['class'].value_counts())

    # Process and augment images    processed_df = process_images(balanced_df, input_image_dir, output_dir)

    # Split dataset    train_df, val_df, test_df = split_dataset(processed_df)

    # Save annotations    save_annotations(train_df, "train", output_dir)    save_annotations(val_df, "val", output_dir)    save_annotations(test_df, "test", output_dir)

    print("\nBalancing process completed successfully!")    print(f"Number of train samples: {len(train_df)}")    print(f"Number of validation samples: {len(val_df)}")    print(f"Number of test samples: {len(test_df)}")

if __name__ == "__main__":    main()
description: 📄 Report: Dataset Balancing and Augmentation Pipeline for Object Detection

🔹 Overview

This Python script is designed to preprocess and balance an object detection dataset by applying data augmentation techniques and equalizing the number of samples per class. The balanced dataset is then split into training, validation, and test subsets for model training and evaluation.



🔹 Main Objectives





Equalize the number of samples for all object classes using both real and augmented data.




Apply controlled image and bounding box augmentations to increase dataset variability.




Save the resulting dataset in a structured format with corresponding annotations.




Split the balanced dataset into stratified training, validation, and testing sets.





🔹 Key Functional Components

1. Configuration Parameters





input_image_dir: Path to the directory containing original images.




input_annotations: CSV file containing original bounding box annotations.




output_dir: Destination directory for the balanced dataset.




target_count: Desired number of samples per class (e.g., 2500).



2. Data Augmentation

Uses the imgaug library to apply a combination of transformations:





Horizontal and vertical flipping.




Rotation between -30 and 30 degrees.




Scaling (0.8x to 1.2x).




Translation (±10%).




Gaussian blur and noise.



Bounding boxes are adjusted accordingly using BoundingBoxesOnImage.

3. Balancing Logic





If a class has fewer than target_count samples, it is oversampled using augmentation.




If a class has more than target_count, it is downsampled randomly.



4. Image Processing





Augmented images are saved in a new directory.




Updated bounding box annotations are recorded.




Filenames of augmented samples are modified to avoid conflicts.



5. Dataset Splitting





Uses stratified sampling to divide the dataset into:





80% train




10% validation




10% test





6. Annotation Saving





Outputs CSV files: train_annotations.csv, val_annotations.csv, test_annotations.csv in the labels subdirectory.





🔹 Output Structure



 




🔹 Applications

This script is well-suited for object detection tasks that suffer from class imbalance and data scarcity. It helps improve generalization performance for models like Faster R-CNN, YOLO, and SSD by enriching training data and ensuring equal representation across classes.



✅ Conclusion

This pipeline automates the task of dataset balancing and augmentation for object detection problems. By integrating random sampling, augmentation, and validation splitting, it produces a high-quality dataset ready for training deep learning models with improved robustness and accuracy.