# TuSeSy: An Intelligent Turntable Servo System for Tracking Aircraft and Parachutes Automatically
## 4 The Design of the Tracking Algorithm
### Tracking Switch Algorithms  
**Tracking the Aircraft. ** When TuSeSy is started, the frame difference starts to detect moving objects and the frame difference detects the aircraft. First, a box covering the target could be obtained using the method of positioning the captured foreground target, as shown in Figure 10a. The coordinates and size of the box are transferred to the corresponding original image, as demonstrated in Figure 10b. Then, the area covered by the box in the original image is taken as the initialization of optical flow for tracking target characteristics. Finally, the optical flow is utilized to track the target continuously, as depicted in Figure 11. The specific aircraft tracking process is shown in Algorithm 3. **Tracking from the Aircraft to the Pilot Chute. ** When tracking the aircraft using the optical flow, it takes the tracking box as the center and divides the image into five blocks, as shown in Figure 12. Before the pilot chute is thrown, the aircraft is tracked by the optical flow. At the same time, the moving target that is out of the aircraft tracking box is detected by the frame difference. That is to say, the movements of areas 1,2,3,4 shown in Figure 12 are detected so that the aircraft is not detected again, and it can avoid the interference of the aircraft when detecting the pilot chute. To improve the speed of the algorithm, the area of the pilot chute can be predicted according to the direction of the flight. Figure 11: The optical flow to track the aircraft target. Figure 10: Capture and track the aircraft targets. (**a**) The aircraft target capture; (**b**) transfer the coordinates and size of the box to the corresponding original image. Figure 13a,b demonstrate that when the plane enters the camera field from left to right, the thrown pilot chute will appear at the bottom left of the image, which is shown in area 1.4 of Figure 12. In contrast, when the plane enters the camera field from right to left, the parachute will appear at the bottom right of the image, which is shown in area 3.4 of Figure 12. Therefore, according to the flight direction of the aircraft, the target is searched for in the predicted region to reduce the data that need to be processed and improve the speed of the algorithm. Since the frame difference detects moving targets in the predicted area all of the time, once the pilot chute is thrown, it will be detected immediately, as shown in Figure 14. From the pilot chute being thrown to opening, the detected area greatly changes. When the detected area of the pilot chute is greater than the set threshold (the threshold is 48.4 times larger than the first detected area of the parachute), the pilot chute is positioned with the proposed method of positioning the captured foreground target, as shown in Figure 15a. The coordinates and sizes of the current bounding box are transferred to the current original image. In addition, the area covered by the box frame in the original image is used as the initialization of the optical flow to track the target features. Therefore, the switching of the tracking target from the aircraft to the pilot parachute is realized, and the optical flow is used to track the new target (pilot chute), as shown in Algorithm 4. Figure 14: Detecting the pilot chute in the predicted area. Figure 12: The image is divided into five blocks. Figure 13: Target tracking switch. (**a**) The tracking switch when the aircraft enters the camera field, from right to left, (**b**) The tracking switch when the aircraft enters the camera field, from left to right. **Tracking from the Pilot Chute to the Main Chute**. The pilot chute throws the main chute; the main chute appears below the pilot chute. The tracking switch from the pilot chute to the main chute is similar to that from the aircraft to the point chute. Therefore, the area below the pilot chute; that is, area 4, needs to be detected. The detailed process is shown in Algorithm 5.