# Small Object Detection in Optical Remote Sensing Images via Modified Faster R-CNN
## 4 Data Pre-Processing  
In this section, we present a simple yet effective approach to augment our available optical remote sensing data while applying a sampling strategy to solve the problem of non-uniform class distribution during training. As a matter of fact, the available optical remote sensing data are very scarce while they can be gathered difficultly. We only focus on two remote sensing object instances, ship and plane. Currently only the NWPU VHR-10 dataset [28], which contains totally 800 very-high-resolution (VHR) optical remote sensing images that were cropped from Google Earth and Vaihingen dataset and then manually annotated by experts, is available to the public. This dataset includes 10 categories--which are airplane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge, and vehicle--but there are only 90 images involved in the plane class and 57 images for the ship class, which is not enough for training. Getting more data is essential for our current situation. We augment our training data by two methods: manual annotation and data augmentation. A set of 2608 images containing the'ship' category in our dataset are collected from multiple sensors and platforms such as Google Earth with multiple resolutions. In addition, the flip operation is usually used for data augmentation in object detection. Built on Random Erasing (RE) [29], we introduce Random Rotation (RR) which is a simple yet effective data augmentation technique for training our modified Faster R-CNN. In particular, RR happens in a certain probability. An image within a mini-batch is randomly chosen to undergo either RR with probability \(p\) or kept unchanged with probability \(1-p\). RR randomly rotates an image by an angle \(\theta\). Notably, the four points of the ground truth, annotated by a rectangle region, are rotated by the angle \(\theta\). The original coordinates of these points are denoted anticlockwise as \(\{(x_{i},y_{i}),i=1,2,3,4\}\), respectively. Hence, the rotated points \(\{(x_{i}\prime,y_{i}\prime),i=1,2,3,4\}\) can be calculated under the following Equation (1). \[\left[ \begin{array}{c}x_{i}\prime\\ y_{i}\prime\end{array} \right]=\left[ \begin{array}{cc}\cos\theta&-\sin\theta \\ \sin\theta&\cos\theta\end{array} \right]\left[ \begin{array}{c}x_{i}\\ y_{i}\end{array} \right]\ \ \ i=1,2,3,4 \tag{1}\]  
What is noteworthy is that the original bounding box of each ground truth becomes parallelogram after RR operation so that we employ their minimum enclosing rectangle (MER), whose upper left corner \((x_{min},y_{min})\) and lower right corner \((x_{max},y_{max})\) can be calculated by the following Equation (2), as the rotated bounding box. This has the advantage of leveraging the context information to some extent because the MER crops more regions than the original bounding box. \[\left\{ \begin{array}{l}x_{min}=\min(x_{1}\prime,x_{2}\prime,x_{3}\prime,x_{ 4}\prime)\\ y_{min}=\min(y_{1}\prime,y_{2}\prime,y_{3}\prime,y_{4}\prime)\\ x_{max}=\max(x_{1}\prime,x_{2}\prime,x_{3}\prime,x_{4}\prime)\\ y_{max}=\max(y_{1}\prime,y_{2}\prime,y_{3}\prime,y_{4}\prime)\end{array} \right. \tag{2}\]  
Fortunately, in total we collected 5922 optical remote sensing images, 5216 for'ship' and 706 for 'plane'. Obviously, the numbers of images in different classes are imbalanced, which pose great challenges for training. To address this, we apply a sampling strategy named 'balanced sampling' (BS) [30] during training. This strategy is aim at iterating as uniform as possible within an epoch with respect to classes. In reality, we use one type of list, namely a training list. A typical example for  
Figure 3: The contextual detection model. three classes is shown in the Figure 4. Firstly, we sort our training list class by class and count the largest category number denoted as K\({}_{3}\) in the Figure 4. Then we generate a random list of K\({}_{3}\) integers with the interval \([0,K_{3}-1]\) for each class.