Using python package to build your custom dataset — MLDatasetBuilder | by Karthick Nagarajan | Towards Data Science

As an ML Newbie, I need to figure out the best way to prepare the dataset for our machine learning training model. As per my last article, I came up with a Python package for this process!
Whenever you are training a custom model the important thing is the dataset. Yes, of course, the dataset plays the main role in deep learning. The accuracy of your model will be based on the dataset. So, before you train a custom model, you need to plan how to build a dataset? Here, I’m going to share my ideas on the easy way to build your dataset.
pypi.org
Whenever we begin a machine learning project, the first thing that we need is a dataset. Datasets will be the pillar of the training model. You can build the dataset either automatically or manually. MLDatasetBuilder is a python package which is helping to prepare the image for your ML dataset.
github.com
We can install MLDatasetBuilder package using the below command
pip install MLDatasetBuilder
When you run python3 in the terminal, it will produce output like this:
Python 3.6.9 (default, Apr 18 2020, 01:56:04) [GCC 8.4.0] on linuxType "help", "copyright", "credits" or "license" for more information.>>> 
Run the following code to you can get the Initialize process output for the MLDatasetBuilder package.
>>> from MLDatasetBuilder import *>>> MLDatasetBuilder()
PrepareImage — Remove unwanted format images and Rename your images
#PrepareImage(folder_name, image_name)PrepareImage('images', 'dog')
ExtractImages — Extract images from video file
#ExtractImages(video_path, file_name, frame_size)ExtractImages('video.mp4', 'frame', 10)#OR#ExtractImages(video_path, filename)ExtractImages('video.mp4', 'frame')#Default FPS will be 5
Yes, we can get images from Google. Using the Download All Images browser extension we can easily get images in a few minutes. You can check out here for more details about this extension!
Once you have downloaded the images using this extension, you can create a python file called test.py the same directory as below.
download_image_folder/|   _14e839ba-9691-11ea-a968-2ed746e9a968.jpg|   5e5f7af12600004018b602c0.jpeg|   A471529_Alice_b-1.jpg|   image1.png|   image2.png|   ...test.py
Inside the images folder, you can see lots of PNG images and random filenames.
MLDatasetBuilder provides a method called PrepareImage. Using this method to we can remove the unwanted images and rename your image files which are already you have downloaded from the browser’s extensions.
PrepareImage(folder_path, class_name)#PrepareImage('download_image_folder', 'dog')
As per the above code, we need to mention the image folder path and class name.
After completing the process your image folder structure will look like below
download_image_folder/|   dog_0.jpg|   dog_1.jpg|   dog_2.jpg|   dog_3.png|   dog_4.png|   ...test.py
This process very helps to annotate your images while labeling. And of course, it will be like one of the standardized things.
MLDatasetBuilder also provides a method called ExtractImages. Using this method we can extract the images from the video files.
download_image_folder/video.mp4test.py
As per the below code, we need to mention the video path, folder name, and framesize. Folder name will the class name and framesize’s default value 5 and it’s not mandatory.
ExtractImages(video_path, folder_name, framesize)#ExtractImages('video.mp4', 'frame', 10)ExtractImages(video_path, folder_name)#ExtractImages('video.mp4', 'frame')
After completing the process your image folder structure will look like below
download_image_folder/dog/|   dog_0.jpg|   dog_1.jpg|   dog_2.jpg|   dog_3.png|   dog_4.png|   ...dog.mp4test.py
I have planned to release version 2.0.0 on next month, This will include some additional features.
I mean this package will provide more than 100 objects images with annotations file :)
All issues and pull requests are welcome! To run the code locally, first, fork the repository and then run the following commands on your computer:
git clone https://github.com/<your-username>/ML-Dataset-Builder.gitcd ML-Dataset-Builder# Recommended creating a virtual environment before the next steppip3 install -r requirements.txt
When adding code, be sure to write unit tests where necessary.
MLDatasetBuilder was created by Karthick Nagarajan. Feel free to reach out on Twitter Linkedin or through Email!
Originally published at www.spritle.com