# An Intelligent Real-Time Object Detection System on Drones
## 3 Materials and Methods
### Intelligent Object Detection  
We note that when performing object detection, the objects may be misclassified in some cases. Therefore, we propose intelligent object detection techniques, as displayed in Figure 5. It generates the image and performs object detection on it. Only when the object detection is higher than or equal to the preset threshold, will the system report the image remotely. This way, there is no need to send every image to the remote machine, which significantly reduces the transferred image data.  
Figure 4: The proposed intelligent system.  
Figure 3: The architecture of the proposed intelligent system.  
Specifically, in the workflow of Figure 5, the drone first starts flying in an area of interest and then looks around to search for different types of targets. Depending on the battery, a drone can fly typically between 10 to 30 minutes and power consumption is a concern for drones.  
While flying, the drone captures images using equipped sensors, including pressure sensors, humidity sensors, accelerometer sensors, image cameras, etc. With the help of deep learning, it is feasible to detect objects using images. Therefore, we use the camera to capture images during the flight.  
Upon the image data generation, the host transfers the image data to the FPGA for object detection. The FPGA contains the NN engine that can run various NN models, including ResNet-50, Yolov3, and Yolov3-tiny models. These models can be used to perform object classification and detection tasks with low power consumption.  
Based on the object detection results, the system evaluates if there is any target of interest (TOI) in the search area. The TOI is the target that the user specifies in advance, and it may be a person, a car, or other objects. If no TOI is found, the drone moves to the next area of interest.  
When TOI is found, our system looks into the confidence score of the object detection. We use a confidence-score threshold as the tuning hyperparameter for the user to adjust. When the confidence score is greater than or equal to the threshold, the system is confident that the image provides enough information and then sends the image to the remote machine for the report. The drone can then start searching the next area.  
If the confidence score is below the threshold value, the drone flies around and attempts to take images of the object from different angles. By approaching the target, the system can generate images that may contain more pixels for the TOI. In this case, the confidence score will be effectively increased.  
Note that the confidence score denotes the uncertainty of the object detection result from Yolo. By using a preset threshold, the uncertainty in decision-making can be reduced. When the threshold is set to a low value (e.g., 0.60), the uncertainty is high and the object may not be that clear in the image; when it is set to a high value (e.g., 0.95), the uncertainty decreases but the system may not be able to reach the threshold, which results in a deadlock.  
Figure 6 illustrates the principle of intelligent object detection. The target of interest is the person, and in the first image, the object detection confidence score is 0.60 and it is below the threshold (e.g., 0.8). By providing more pixels of the person, the system can produce a more effective object detection with a higher confidence score--i.e., 0.90--and report the image accordingly.  
Figure 5: The workflow of the proposed intelligent object detection.