Announcing YOLTv4: Improved Satellite Imagery Object Detection
By: Adam Van Etten
Tags: Machine Learning, Geospatial, Open Source, Computer Vision, Data Science

Announcing YOLTv4: Improved Satellite Imagery Object Detection

Preface: Though CosmiQ Works (and its associated blog: The DownLinQ) has unfortunately been shut down, there remains much to be done in the geospatial analytics domain. Accordingly, this blog details work performed independently of IQT and in my spare time.

In a number of previous blogs [e.g. 1, 2, 3] and academic papers [e.g. 4, 5, 6] we’ve demonstrated the striking efficacy of adapting YOLO to detect objects in satellite imagery. Recall that YOLO is a leading deep learning object detection framework, designed to detect objects in imagery. YOLO maxes out at image sizes of a few thousand pixels in size, far too small to handle large scale satellite imagery which can exceed 100 million pixels. Therefore we built YOLT (and extended YOLT with SIMRDWN) to optimize this object detection framework for satellite images of arbitrarily large size.

We are now pleased to announce that an improved version, YOLTv4, has been released that is both faster and more performant. The code is open-sourced at github.com/avanetten/yoltv4. In the sections below we detail this new repository and provide example results.

1. Introduction

YOLTv4 is designed to rapidly detect objects in aerial or satellite imagery in arbitrarily large images that far exceed the ~600×600 pixel size typically ingested by deep learning object detection frameworks. This repository is built upon the impressive work of AlexeyAB’s YOLOv4 implementation, which improves both speed and detection performance compared to YOLOv3 (which is implemented in SIMRDWN). We use YOLOv4 insead of “YOLOv5”, since YOLOv4 is endorsed by the original creators of YOLO, whereas “YOLOv5” is not; furthermore YOLOv4 appears to have superior performance.

YOLTv4 therefore combines the impressive YOLOv4 backbone with the pre- and post-processing scripts of SIMRDWN to create an improved, lighter-weight alternative SIMRDWN. Recall that SIMRDWN allows users to select from a multitude of backbones (e.g. YOLO, SSD, Faster-RCNN, etc.). While this flexibility is quite useful, it does balloon the code base; furthermore multiple studies [5, 6] have shown that YOLO backbones outperform other methods for satellite imagery object detection. Therefore, in YOLTv4 we opt to forge a simpler codebase and focus only on the YOLOv4 backbone.

Below, we provide examples of how to use this repository with the open-source Rareplanes dataset.

2. Running YOLTv4

In this section we demonstrate how to train and test YOLTv4 using the Rareplanes dataset to localize aircraft.

2.1. Installation

YOLTv4 is built to execute within a docker container on a GPU-enabled machine. The docker command creates an Ubuntu 16.04 image with CUDA 9.2, python 3.6, and conda.

Clone this repository (e.g. to /yoltv4/). Download model weights to yoltv4/darknet/weights (e.g. see: https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.conv.137). Install nvidia-docker. Build docker file. nvidia-docker build -t yoltv4_image /yoltv4/docker Spin up the docker container (see the docker docs for options). nvidia-docker run -it -v /yoltv4:/yoltv4 -ti — ipc=host — name yoltv4_gpu0 yoltv4_image Compile the Darknet C program. First Set GPU=1 CUDNN=1, CUDNN_HALF=1, OPENCV=1 in /yoltv4/darknet/Makefile, then:

cd /yoltv4/darknet

make

2.2. Train

A. Prepare Data

Make YOLO images and labels (see yoltv4/notebooks/prep_data.ipynb for further details). Create a txt file listing the training images. Create file obj.names file with each desired object name on its own line. Create file obj.data in the directory yoltv4/darknet/data containing necessary files. For example /yoltv4/darknet/data/rareplanes_train.data:

classes = 30

train = /local_data/cosmiq/wdata/rareplanes/train/txt/train.txt

valid = /local_data/cosmiq/wdata/rareplanes/train/txt/valid.txt

names = /yoltv4/darknet/data/rareplanes.name

backup = backup Prepare config files.

See instructions here, or tweak /yoltv4/darknet/cfg/yoltv4_rareplanes.cfg.

B. Execute Training

Execute.

cd /yoltv4/darknet

time ./darknet detector train data/rareplanes_train.data cfg/yoltv4_rareplanes.cfg weights/yolov4.conv.137 -dont_show -mjpeg_port 8090 -map Review progress (see Figure 1 below):

Figure 1.Progress for YOLTv4 training (saved as /yoltv4/darknet/chart_yoltv4_rareplanes.png)

2.3. Test

A. Prepare Data

Make sliced images (see yoltv4/notebooks/prep_data.ipynb for further details). Create a txt file listing the training images. Create file obj.data in the directory yoltv4/darknet/data containing necessary files. For example /yoltv4/darknet/data/rareplanes_test.data:

classes = 30

train =

valid = /local_data/cosmiq/wdata/rareplanes/test/txt/test.txt

names = /yoltv4/darknet/data/rareplanes.name

backup = backup/

B. Execute Testing

Execute (proceeds at >80 frames per second on a Tesla P100):

cd /yoltv4/darknet time ./darknet detector valid data/rareplanes_test.data cfg/yoltv4_rareplanes.cfg backup/ yoltv4_rareplanes_best.weights Post-process detections, stitch images back together, and make plots:

time python /yoltv4/yoltv4/post_process.py \

— pred_dir=/yoltv4/darknet/results/rareplanes_preds_v0/orig_txt/ \

— raw_im_dir=/local_data/cosmiq/wdata/rareplanes/test/images/ \

— sliced_im_dir= /local_data/cosmiq/wdata/rareplanes/test/yoltv4/images_slice/ \

— out_dir= /yoltv4/darknet/results/rareplanes_preds_v0 \

— detection_thresh=0.25 \

— slice_size=416 \

— n_plots=8

3. Outputs

Outputs will look something like the figures below.