# General Moving Object Localization from a Single Flying Camera
## 4 Mathematical Formulation  
We will now present our formulation for addressing the general moving object localization problem--i.e., estimating the depth of an object moving in an arbitrary direction--from a single flying camera. The simple cases in our previous work [16] are just special cases in the general moving object localization problem, where the motion of the object is limited to certain axes. We will first present the assumptions that will reduce the general moving object localization problem to a linear motion problem, and then we will show that, by using four images, we are able to recover the distance of an object moving linearly in an arbitrary direction. Figure 7 shows the geometry of our localization problem. The movement of the drone is from the right to the left. Note that the first image (the first black line from the right) is taken when the drone has just completed its rotation, but before any horizontal translation has taken place. Notice also that in subsequent images, the distances between the drones are further and further apart. This is because the drone is accelerating towards its maximum speed. We make three important assumptions as follows:  
1. The drone is moving much faster than the object;
2. The drone's camera's frame rate is high;
3. The drone is always in a state of acceleration. The first two assumptions will reduce the general motion of the object to that of a linear motion. Usually, we can expect a moving object to follow a straight line of motion and at constant speed, but there are situations where it may not be doing so. If the first two assumptions are true, the movement of the object can be approximated to a linear motion during the time interval that the drone is flying and taking a burst of images of the object. If the object continues to move in a curved or non-linear fashion, the error can be corrected when the drone flies the repeated pattern again, as described in Figure 2.  
Figure 7: Overview of the arbitrary movement of an object. The drone is flying from the right to the left, accelerating towards maximum speed. The distances \(C_{1}\), \(C_{2}\), etc., are increasing as the drone moves with increasing speed. The object may move in an arbitrary fashion, subtending angles \(\theta_{1}\), \(\theta_{2}\), etc., with the camera axes at each position. The third assumption is needed so as to remove the depth ambiguity illustrated in Figure 6. In our experiments, we use a Parrot AR Drone 2.0, which has a maximum speed of 11.11 m/s. We also instructed the drone to capture a sequence of 40 images at 200 ms intervals, which is a high enough frame rate. By the end of the 40 images, the drone would have reached its maximum speed and will not accelerate anymore. Thus, we believe that all these three assumptions are valid and reasonable. Using all three assumptions, we can represent the object's movement as a straight line and set up the geometry as shown in Figure 8.  
We can simplify Figure 8 and label our variables as shown in Figure 9. Let the origin \((0,0)\) be at the bottom right hand corner. Line \(L_{0}\) is the line in the direction of movement of the object, where \((x_{1},\ y_{1})\) to \((x_{4},\ y_{4})\) are the positions of the object at time periods 1 to 4, respectively. Lines \(L_{1}\) to \(L_{4}\) are the lines joining the center of the object to the image center of the camera for the respective camera positions. \(D_{A}\), \(D_{B}\), and \(D_{C}\) are the distances between the objects in successive time periods in the direction of the object's motion. To achieve the largest baseline in order to achieve the best accuracy, we should use the first two images and the last two images in the sequence. However, it is possible that the pedestrian may disappear from the drone's view before the end of the sequence. In such situations, we will use the last two images where the pedestrian is still detected (along with the first two images of the sequence) to compute the distance estimation. Figure 8: Geometry for linear object movement in an arbitrary direction.