# Integration of the 3D Environment for UAV Onboard Visual Object Tracking
## 3 Single Visual Object Tracking Pipeline for UAV
### Particle Filter for Modeling the Object in 3D  
In order to overcome the drawbacks of the common maximum-likelihood approach used in SOTs, we switch to detection-by-tracking by applying an additional state estimator. Additionally, to robustly handle multiple high similarity scores in the similarity score map, a state estimator with a multimodal representation of the probability density function is favored. To that end, a particle filter [56] is used, which will estimate the state of the object--i.e., the position and velocity--in the 3D scene for every time step \(t\). The particle filter estimates the posterior density function of the object \(p(s_{t})\) based on a transition model \(f\), the current observation \(z_{t}\), and the prior density function \(p(s_{t-1})\). The general idea is shown in Equation (1):  
\[p(s_{t})=f(p(s_{t-1}),z_{t}). \tag{1}\]  
In order to approximate the probability density function \(p(s_{t})\), the particle filter uses particles. Each particle denotes a hypothesis on the state. Particles from the prior probability density function \(p(s_{t-1})\) are propagated through the transition model \(f\). Weights \(w_{t}^{i}\) at time step \(t\) are assigned to \(n\) particles with \(i\in I=\{1,...,n\}\), mirroring how strongly particles match with the current observation \(z_{t}\). Let \(p(s_{t}|z_{1:t})\) represent the probability density function of the posterior state given all observations \(z_{1:t}\) up to time step \(t\) in Equation (2). Each particle has a corresponding weight \(w_{t}^{i}\) at time step \(t\). Let \(s_{t}^{i}\) denote the hypothesis on the state of the \(i\)-th particle and \(s_{t}\) the state estimation at time \(t\). \(\delta\) is the Dirac delta function. The weights are normalized such that \(\sum_{i}^{n}w_{t}^{i}=1\). Particles are weighted according to their matching similarity with the current observation \(z_{t}\).  
\[p(s_{t}|z_{1:t})\approx\sum_{i\in I}w_{t}^{i}\delta(s_{t}-s_{t}^{i}). \tag{2}\]  
A resampling is performed after the weighting of the particles whenever \(\hat{n}_{eff}\), presented in Equation (3), is below a certain threshold. The resampling allows the particle filter to discard low-weighted particles and create new particles based on the stronger-weighted ones, allowing a refined approximation of \(p(s_{t}|z_{1:t})\).  
\[\hat{n}_{eff}=\frac{1}{\sum_{i\in I}{(w_{t}^{i})}^{2}}. \tag{3}\]  
In our case, the observation \(z_{t}\) is the similarity score map produced by ATOM or DiMP. We apply a constant velocity model for the transition model \(f\) and fix the velocity \(v_{\mathbf{z}_{g}}\) on the \(\mathbf{z}_{g}\) axis, as \(v_{\mathbf{z}_{g}}~{}=~{}0\)--where \(\mathbf{z}_{g}\) is perpendicular to the estimated ground plane \((\mathbf{x}_{g},\mathbf{y}_{g})\) of the reconstructed scene. This results in particles being only able to move on the ground.  
Figure 8: Comparison between a depth map approximation (**left**) and the corresponding frame (**right**). Points in yellow on the frame indicate visible points from the point cloud reconstruction aside from the ground estimation.