## 4 Remaining Research Challenges and Future Research Directions  
Although numerous researchers have made efforts to develop visual target tracking systems for autonomous vehicles navigation, many issues are still open and deserve further research, especially in the  
Figure 10: These figures are taken from [85]. [PERSON] _et al._’s proposed data fusion based target tracking system is described schematically in (a). The cameras are mounted on an autonomous vehicle moving in the environment as shown in (b). Estimation of the \(3D\) velocity of the interested target in the world coordinate can be achieved using the procedures explained in (c). following areas. First, as the visual tracking system designed for autonomous vehicles works mainly in an unknown environment, the target's specific features are generally unavailable or unpredictable. Also as the cameras are moving with the vehicle, the visual features of the environment may also change continuously. For example as the light attenuates exponentially with the distance in the air in an outdoor environment, it makes the quality of the camera images poor. In such a situation the color of the object of interest will be attenuated and the normal tracking algorithms will meet great difficulties. Also there may be many unknown colorful objects in the same environment, which can be misinterpreted for the object of interest. So it is not suitable to develop a tracking system mainly based on the target's special visual features or models such as color or shape. Secondly, in the normal visual target tracking systems the target's relative dynamics to the cameras is estimated for navigation purposes. This relative information is obtained from the vehicle's onboard sensors without considering the vehicle's self movement. The relative information is not available for other applications in the same environment. Also the relative information is less accurate due to the vehicle's self complex rotational and translational motions (such as pan and tilt motions), which are normally ignored in the estimation processes. Thirdly, as the autonomous vehicle is working in both the structured and unstructured environment, it is not always applicable to put additional sensors in the environment to help autonomous vehicles track the objects and navigate the environment. Also when the autonomous vehicle works in an unknown or unexpected environment alone, it doesn't have a prior knowledge of itself location or the environment conditions. The \(3D\) visual target tracking will be difficult when there are very few initial or beforehand information that can be used. Fourthly, in the visual target tracking, the dynamic information aids the tracking process in the presence of occlusions and measurement noises [87]. However, the tracking is complicated by the fact that natural moving targets do not exhibit one type of motion but rather have complex, unknown, highly non-linear or time-varying dynamics. The single model approaches do not make full use of the information available and often rely on somewhat ad hoc methods of incorporating uncertainty about the mode estimates (like adjusting the filter noise covariance in proportion to the variance in the model estimate [85]), so the traditional single target linear dynamic models are not quite applicable. Fifthly, according to the object tracking survey paper [29], there are still some major remaining research challenges: 1. One challenge in tracking is to develop algorithms for tracking objects in unconstrained environments, for example, videos obtained from crowded streets. 2. An important issue that has been neglected in the development of tracking algorithms is integration of contextual information. For example, in a vehicle tracking application, the location of vehicles should be constrained to paths on the ground as opposed to vertical walls or the sky.