LiDAR point cloud based 3D object detection implementation with colab{Part 2 of 2} | by Gopalakrishna Adusumilli | Towards Data Science

VoxelNet a point cloud based 3D object detection algorithm is implemented using google colab.
In my previous article, I have explained crucial concepts required to implement the VoxelNet an end-to-end learning model for the 3d object detection you can find here
In continuation with the previous article, we will work out to implement the VoxelNet algorithm for the 3d object detection using KITTI point cloud data
Step 1: Processing the KITTI dataset to train the model[Detailed steps]
The following steps are crucial to prepare the data to train the model. KITTI dataset needs to be downloaded, cropped, processed, and save in the drive.
login to google colab, Open a notebook, GPU is not required at the moment.
Note: in case if GPU is selected, note that each user is provided with ~30GB of virtual space and the entire dataset has a size of ~40+ GB and then to unzip dataset memory will run out. In the next steps we will download the KITTI datset, process, crop and create .zip and move to the drive to use the data for future
Required dataset
Velodyne point clouds (29 GB): input data to VoxelNet
Velodyne point clouds (29 GB): input data to VoxelNet
2. Training labels of the object data set (5 MB): input label to VoxelNet
3. Camera calibration matrices of the object data set (16 MB): for visualization of predictions
4. Left color images of the object data set (12 GB): for visualization of predictions
#clone the voxelnet git repo!git clone https://github.com/gkadusumilli/Voxelnet.git#change the cwd%cd /content/drive/My Drive/Voxelnet/crop_data
To download the datasets (.zip) directly to the colab virtual machine [~ 15–20 mins]
#Data label file!wget (enter the link)#Calib file!wget (enter the link )#Velodyne file!wget (enter the link)# Image file!wget (enter the link)
Unzip the datasets(.zip) to the folder (~20- 30 mins)
#Unzip the velodyne training folder!unzip /content/Voxelnet/crop_data/data_object_velodyne.zip 'training/*' -d /content/Voxelnet/crop_data#Unzip the image training folder!unzip /content/Voxelnet/crop_data/data_object_image_2.zip 'training/*' -d /content/Voxelnet/crop_data#unzip the object label!unzip /content/Voxelnet/crop_data/data_object_label_2.zip#unzip the data object calib!unzip /content/Voxelnet/crop_data/data_object_calib.zip 'training/*' -d /content/Voxelnet/crop_data
cropping point cloud data for training and validation. Point clouds outside the image coordinates are removed. cropped point cloud will overwrite the existing raw point cloud [40–45 mins]
#to run the 'crop.py' lower version of scipy is needed!pip install scipy==1.1.0#run crop.py!python crop.py
Create the validation data to evaluate the model[~10–15 mins]
#create a folder 'validation' and copy the content in training #folder!mkdir /content/Voxelnet/crop_data/validation%cp -av /content/Voxelnet/crop_data/training /content/Voxelnet/crop_data/validation/
we will split the training as per the protocols here [2–3 mins]
Final step...
create .zip the folder of the processed, cropped data [30–40mins]
!zip -r /content/VoxelNet/data/data_lidar.zip /content/VoxelNet/crop_data
Move the .zip folder to the drive [~5 mins]
#rename the folder as you need in the drive, I've stored in dtive with the folder named 'AI'!mv "/content/VoxelNet/data/data_lidar.zip" "/content/gdrive/My Drive/AI"
Here is the link to the colab jupyter notebook with all the above-mentioned steps.
log in to google colab , create a new notebook
To access the GPU: Click on runtime > Change runtime type > GPU
Note: The colab GPU runtime is approximate ~12 hours post that, the runtime will be disconnected, the data stored will be lost. Since in our case each epoch would take ~2 hours and need to train for more than 20 epochs to observe the preliminary results. So will use google drive as a path to store the code, checkpoints, prediction results, and so on
Change the current working directory(CWD) to drive
#voxelnet is the folder name, you can rename as you need%cd /content/drive/My Drive/Voxelnet
VoxelNet implementation needs several dependencies. So we will clone the entire repository.
!git clone https://github.com/gkadusumilli/Voxelnet.git
Building the files
%cd /content/drive/My Drive/Voxelnet!python setup.py build_ext --inplace
Unzip the processed dataset in the crop_data folder [35–40 mins]
%cd /content/drive/My Drive/Voxelnet/crop_data#Locate the zip folder in the drive and unzip using !unzip command!unzip "/content/drive/My Drive/AI/data_lidar.zip"
Training the model
%cd /content/drive/My Drive/Voxelnet/
Important argument parse
!python train.py \--strategy="all" \--n_epochs=16 \--batch_size=2 \--learning_rate=0.001 \--small_addon_for_BCE=1e-6 \--max_gradient_norm=5 \--alpha_bce=1.5 \--beta_bce=1 \--huber_delta=3 \#if dump_vis == yes, boolean to save visualization results--dump_vis="no" \--data_root_dir="/content/drive/My Drive/Voxelnet/crop_data" \--model_dir="model" \--model_name="model6" \--dump_test_interval=3 \--summary_interval=2 \--summary_val_interval=40 \--summary_flush_interval=20 \--ckpt_max_keep=10 \
%load_ext tensorboard#summary_logdir is the logdir name%tensorboard --logdir summary_logdir
The below snap is the logdir results @ epoch6
!python predict.py \--strategy="all" \--batch_size=2 \--dump_vis="yes" \--data_root_dir="../DATA_DIR/T_DATA/" \--dataset_to_test="validation" \--model_dir="model" \--model_name="model6" \--ckpt_name="" \
The code to download, crop, process the KITTI dataset can be found here
The code to implement from step 2 can be found here
Here are some of the results obtained after training the model for 30 epochs.
Special thanks to Tsinghua Robot Learning Lab, Qiangui Huang, David Stephane for their valuable contribution in the implementation of VoxelNet
Special thanks:
Dr.Uma K Mudenagudi, KLE Technological University for the project mentorship.
References:
KITTI raw dataset: @ARTICLE{Geiger2013IJRR,author = {Andreas Geiger and Philip Lenz and Christoph Stiller and Raquel Urtasun},title = {Vision meets Robotics: The KITTI Dataset},journal = {International Journal of Robotics Research (IJRR)},year = {2013}}