dataesr/scientific-paragraphs-categorization huggingface dataset:
Bayesian estimation is a classical, well-mastered technique and, in the case of visual-based estimation, the large number of processed data justifies that the visual-based estimation may be considered as a realization of a Gaussian process (with ellipsoidal confidence domains). On the other hand, set-membership estimation is the correct framework to deal with geographic databases in which the geometric statements are verified and guaranteed. Ellipsoidal algorithms make it possible to update ellipsoidal confidence domains in this context. The structure of the paper is as follows. Variables, notations and the evolution model of the vehicle are introduced in the second section, while the third section presents the experimental context that is of primary importance for this paper. Section 4 then details the visual localization method in three steps: 1) visual database construction, 2) real-time localization, and 3) good results and a few problems to be dealt with.
Elements of the GIS In the present work, a 3D city model of the environment is available in a GIS managed by SIVNav SDK from BeNomad  [20]. This software development kit has been designed for applications with augmented reality, 3D map rendering and vision for robotics. It includes tools for both database management and the accurate geometrical description of buildings, roads (actually, streets) and side walks. Geographical data are provided by the French National Geographical Institute (IGN), in the national French plane projection (Lambert 93) plus the mean sea level (MSL) altitude (also called the GIS reference frame). Their accuracy is: 5 cm horizontally and 25 cm vertically in Paris  [21, 22]. The components that are added into the GIS for the present study are: i) a layer containing the visual landmarks that are collected in an initial learning step detailed in Section 4.
The shape of this domain is compatible with statements of accuracy in both Bayesian  [15, 16]  and set-membership  [17] [18] [19]  estimation. Bayesian estimation is a classical, well-mastered technique and, in the case of visual-based estimation, the large number of processed data justifies that the visual-based estimation may be considered as a realization of a Gaussian process (with ellipsoidal confidence domains). On the other hand, set-membership estimation is the correct framework to deal with geographic databases in which the geometric statements are verified and guaranteed. Ellipsoidal algorithms make it possible to update ellipsoidal confidence domains in this context. The structure of the paper is as follows. Variables, notations and the evolution model of the vehicle are introduced in the second section, while the third section presents the experimental context that is of primary importance for this paper.
Section 4 then details the visual localization method in three steps: 1) visual database construction, 2) real-time localization, and 3) good results and a few problems to be dealt with. Section 5 details the map-aided with tight coupling GPS/odometry (MATCGO) method, and finally Section 6 explicates how the collaboration of the two methods brings integrity to the global localization process. M: M, i 3 , j 3 , k 3 as the vehicle reference frame (Figure  1 ), where j 3 is aligned with the common axle of the rear wheels, M is the middle of the centres of the rear wheels, k 3 is normal to the osculating plane in the up In other words, the configuration may be considered as the isometric transformation 0 R m 0 T m from M to W , where 0 T m is the (OM → ) translation and 0 R m denotes the 3D rotation generated by the successive rotations (ψ,k → 0 ), (θ, j → 1 ) and (ϕ,i → 3 ).
Location and Geographic Database The data comes from the CityVIP  [25]  validation campaign data. Indeed, in order to finalize the project with a final demonstration, in September 2011 in Paris, a recorded experiment was carried out. Two datasets are available, each one composed by at least two laps (about one kilometre per lap). The first lap is used to build the visual landmark database; the second and further laps are used for evaluation purposes. The urban environment in the vicinity of the 12 th district city hall is very dense, with some narrow streets bordered by high Haussmann-style buildings.
Building the Georeferenced Visual Landmark Database With respect to previous works  [26]  where vision-based localization has been introduced, the originality of the present work relies on the fact that the coordinates of the landmarks are defined in the reference frame of the GIS and not in a reference frame specifically devoted to visual localization. The database is built in an initial learning phase during which video frames are recorded with the onboard camera while the trajectory is recorded with a high accuracy IMU associated with a differential GPS (DGPS) sensor. This initial dataset is composed of a list of (t i ,I i ,IM U i ) for 1 ≤ i ≤ N , where t i is a time stamp, I i is a video frame and IM U i is the pose of the camera according to the DGPS and the IMU. Our goal is to compute C i = R i T i the pose of the camera at frame I i as well as a set of 3D landmarks extracted from the video frames.
The landmark database we used in this experiment has 856 cameras and 156, 008 points were tracked in the images (an aerial view is shown in Figure  17 ). If we stop at the first step (left of Figure  3 ), 123, 476 3D points were correctly triangulated and the mean reprojection error was 0. 873 pixels. If we carry out a full bundle adjustment, we get 139, 723 correct 3D points, and the mean re-projection error is reduced to 0. 700 pixels. However, introducing this additional degree of freedom implies that the positions given by the vision algorithm are no longer in the same coordinate system as the GIS database. idea was used as in  [28]  to improve the quality of the 3D reconstructions from the images. The solution we used is a global bundle adjustment initialized by the poses recorded in the reference path.
Real-time localization The real-time localization algorithm is designed to compute the pose of the camera from a single image. An initial estimation of the pose of the vehicle is provided by the propagation of the previous estimate obtained either from previous visual data or from map/satellitebased localization, further detailed in Section 5. In this last case, the accuracy of the estimation is about a few metres but it is useful to select a set of candidate key frames on the reference trajectory which are close to the current position of the vehicle. Matching interest points between these key frames and the current image yields associations between 2D points and 3D points in the landmark database (Figure  6 ). The pose is then computed in a RANSAC scheme to eliminate outliers and is improved with an iterative minimization of the reprojection error  [31].
The current position of the vehicle P geo given in the world reference frame defined in Section 2. 2 is given by P geo = P • Select the closest key frame to know which landmarks can be observed; • Match the interest points detected in the current frame to the interest points of the closest key frame; • Retrieve 3D coordinates of the points matched in the key frame from the visual landmark database; • Compute the pose of the camera and the associated covariance matrix. mes, a closed-form solution exists. If it is visible on an two frames, an iterative refinement of the point n is done after an initial triangulation. st step gives an initial estimate of both the poses of era and the 3D landmarks. These data are further zed by using a bundle adjustment algorithm  [30]. done by using the Levenberg-Marquardt algorithm mize the re-projection error of every point in every age according to the cost function F of Equation  4 : F(C i , P j ) = N ∑ i=1 ∑ j∈L i π i (P j ) -p i,j 2 (4) riables which can be adjusted in the minimization pose of each camera C i (associated with the ion function π i ) and the 3D position of each visual rk P j.
The landmark se we used in this experiment has 856 cameras and points were tracked in the images (an aerial view n in Figure  17 ). If we stop at the first step (left of 3), 123,476 3D points were correctly triangulated e mean re-projection error was 0. 873 pixels. If we ut a full bundle adjustment, we get 139,723 correct nts, and the mean re-projection error is reduced to ixels. However, introducing this additional degree om implies that the positions given by the vision m are no longer in the same coordinate system as database. rcome this problem, we propose to compute a ansformation from the coordinates defined in the   visual reconstruction into the georeferenced ones. This transformation is defined locally in the Frenet-Serret frame associated with the reference trajectory (Figure  5 ).
Results When enough interest points are matched between the current image and the closest key frame, the localization is very similar to the reference trajectory as can be seen in Figure  8. The accuracy of the result is below one metre, as shown in the cumulative distribution function of the absolute error in 3D (Figure  7 ) computed when visual localization is available -that is to say, 78% of the entire trajectory. the current position of the vehicle. Matching interest points between these key frames and the current image yields associations between 2D points and 3D points in the landmark database (Figure  6 ). The pose is then computed in a RANSAC scheme to eliminate outliers and is improved with an iterative minimization of the re-projection error  [31]. Afterwards, the kinematic motion model (Section 2.
3) is used to predict the pose of the vehicle for each new video frame. The process to update the pose of the vehicle is the same: • Select the closest key frame to know which landmarks can be observed; • Match the interest points detected in the current frame to the interest points of the closest key frame; • Retrieve 3D coordinates of the points matched in the key frame from the visual landmark database; • Compute the pose of the camera and the associated covariance matrix.
MATCGO localization method The use of the geographical data is based upon the assumption that the vehicle moves on a road with a known geometric specification stored in the GIS. Usually, a road network is represented with set of connected polylines. Taking an estimation of the road width and the altitude inaccuracy of the map into account, each line segment forming the road can be extended to a box (Figure  12 ). In this study, we consider that the set of all these boxes represents the set of all acceptable positions of the car. This set can be used to constrain the ellipsoidal confidence domain obtained with tight-coupled GPS (Figure  11 ). For this, two steps are needed: i) to select the box in which the length is Figure 12                       .
MATCGO Localization Method The use of the geographical data is based upon the assumption that the vehicle moves on a road with a known geometric specification stored in the GIS. Usually, a road network is represented with set of connected polylines. Taking an estimation of the road width and the altitude inaccuracy of the map into account, each line segment forming the road can be extended to a box (Figure  12 ). In this study, we consider that the set of all these boxes represents the set of all acceptable positions of the car. This set can be used to constrain the ellipsoidal confidence domain obtained with tight-coupled GPS (Figure  11 ). For this, two steps are needed: i) to select the box in which the vehicle is assumed to be, and ii) an ellipsoidal confidence domain update using this knowledge.
Collaboration results The 2D and 3D results (Figure  17  and 18 interest of the collaboration. Indeed, th database and the visual localization probl compensated for. Moreover, except during continuity is maintained. Figure  19 , where the cumulative distributi the absolute error in 3D for the three meth clearly demonstrates the limits of each ind and the benefit of the collaboration. The vis is accurate but cannot protect from d incoherence, whereas the MATCGO loca precise enough when used alone but it reliable confidence tests of the estimates to the database of visual landmarks to qu www. intechopen. com Collaborative Methods for Real-time Localization.
Collaboration of Both MATCGO and Visual Localizations In order to deal with the visual localization problems reported in Figure  9 , three criteria are checked prior to any visual estimation: 1. The number of landmarks: the localization is valid when there are at least 30 inliers. This corresponds to a lack of data in Figure  9   Assuming that the MATCGO localization algorithm always updates a localization estimate (q ^,P), a simple collaboration strategy (illustrated by Figure  16 ) is as follows: • When visual localization is available (i. e. , the database exists and the method is not lost) and validates some measurement criteria, the collaboration returns the estimate of this method; In conclusion, although the use of map-aiding enhanced the accuracy of the 3D localization, it is not sufficient to localize the vehicle at the lane-level.
Collaboration of both MATCGO and visual localizations In order to deal with the visual localization problems reported in Figure  9 , three criteria are checked prior to any visual estimation: 1. The number of landmarks: the localization is valid when there are at least 30 inliers. This corresponds to a lack of data in Figure  9   Assuming that the MATCGO localization algorithm always updates a localization estimate ( q, P), a simple collaboration strategy (illustrated by Figure  16 ) is as follows: • When visual localization is available (i. e. , the database exists and the method is not lost) and validates some measurement criteria, the collaboration returns the estimate of this method; • Otherwise, the collaboration returns an estimate of the 3D MATCGO localization algorithm (Section 5. 3).
Collaboration results The 2D and 3D results (Figure  17  and 18) illustrate the interest of the collaboration. Indeed, the absence of a database and the visual localization problems have been compensated for. Moreover, except during the first switch, continuity is maintained. Figure  19 , where the cumulative distribution functions of the absolute error in 3D for the three methods are plotted, clearly demonstrates the limits of each individual method and the benefit of the collaboration. The visual localization is accurate but cannot protect from discontinuity or incoherence, whereas the MATCGO localization is not precise enough when used alone but it provides both reliable confidence tests of the estimates and a window to the database of visual landmarks to quickly initialize  In conclusion, although the use of map-aiding enhanced the accuracy of the 3D localization, it is not sufficient to localize the vehicle at the lane-level.
Collaboration of both MATCGO and visual localizations In order to deal with the visual localization problems reported in Figure  9 , three criteria are checked prior to any visual estimation: 1. The number of landmarks: the localization is valid when there are at least 30 inliers. This corresponds to a lack of data in Figure  9     Assuming that the MATCGO localization algorithm always updates a localization estimate ( q, P), a simple collaboration strategy (illustrated by Figure  16 ) is as follows: • When visual localization is available (i. e. , the database exists and the method is not lost) and validates some measurement criteria, the collaboration returns the estimate of this method; • Otherwise, the collaboration returns an estimate of the 3D MATCGO localization algorithm (Section 5.
Collaboration results The 2D and 3D results (Figure  17  and 18) illustrate the interest of the collaboration. Indeed, the absence of a database and the visual localization problems have been compensated for. Moreover, except during the first switch, continuity is maintained. Both methods deliver an ellipsoidal confidence domain  (11). The estimated vehicle configuration is the centre of this ellipsoid associated with a positive definite matrix defining the ellipsoid. Switching is simply performed by: 1. When the visual localization does not validate one of the three measurement criteria, the MATCGO localization is used after having been updated by the last "good" visual localization in order to keep continuity; 2. When the visual localization again validates the three measurement criteria, the detection threshold is temporarily increased to ensure the stability of the solution.
Conclusion This paper has demonstrated the interest of the collaboration between two localization methodologies based upon different data sources to supervise the displacement missions of a vehicle in dense urban environments. One method is vision-based and uses a database of visual landmarks acquired in a preliminary learning phase. The other method combines odometry with data from a digital map of the road network together with data from satellites of a GNSS when it is available, On the other hand, the MATCGO method is obviously more reliable since: i) the databases it uses are already available almost everywhere, and ii) due to the tight fusion algorithm, it is robust against long periods of poor satellite visibility. The vision-based method may fail in certain uncommon situations. This can be the case not only where there are not enough visual landmarks but also when the optimization algorithm does not converge in due time, as well as in special cases where the result of the visual localization is discontinuous since its principle does not guarantee the unicity of the solution.
In this case, the MATCGO method running in parallel helps to detect this discontinuity and maintain a localization, which is important since the visual database of the GIS will certainly not be immediately available everywhere in the near future. In addition, the continuous result of the MATCGO method speeds up the initialization of the visual method where it can be used by restricting the numbers of landmarks in the database to be matched. From a technical viewpoint: 1) it has been useful to design a 3D kinematical model because the images are very sensitive to the variations of the axis of the camera, 2) the fact that the visual landmarks are referenced in the coordinates of the GIS has received a careful attention, which is important since the visual control of the trajectory must be achieved within the same reference frame as the lane marks, side walks and pedestrian crossings that materialize on the planned path.
The method was implemented in a full-scale experiment of 4 km in the centre of Paris which is a typical dense environment. Figure  19 , where the cumulative distribution functions of the absolute error in 3D for the three methods are plotted, clearly demonstrates the limits of each individual method and the benefit of the collaboration. The visual localization is accurate but cannot protect from discontinuity or incoherence, whereas the MATCGO localization is not precise enough when used alone but it provides both reliable confidence tests of the estimates and a window to the database of visual landmarks to quickly initialize the visual method. This collaboration adds integrity to the accuracy of the visual-based localization and gives a continuous evolution of the accuracy. the visual method.
Conclusion This paper has demonstrated the interest of the collaboration between two localization methodologies based upon different data sources to supervise the displacement missions of a vehicle in dense urban environments. One method is vision-based and uses a database of visual landmarks acquired in a preliminary learning phase. The other method combines odometry with data from a digital map of the road network together with data from satellites of a GNSS when it is available, and execution control purposes. Both methods define the results of a localization as a feasible ellipsoidal domain in the configuration space. Vision-based localization has been shown to be sufficiently accurate to locate the vehicle at the lane-level whereas the accuracy of the MATCGO method is not so good. On the other hand, the MATCGO method is obviously more reliable since: i) the databases it uses are already available almost everywhere, and ii) due to the tight fusion algorithm, it is robust against long periods of poor satellite visibility.
The vision-based method may fail in certain uncommon situations. This can be the case not only where there are not enough visual landmarks but also when the optimization algorithm does not converge in due time, as well as in special cases where the result of the visual localization is discontinuous since its principle does not guarantee the unicity of the solution. In this case, the MATCGO method running in parallel helps to detect this discontinuity and maintain a localization, which is important since the visual database of the GIS will certainly not be immediately available everywhere in the near future. In addition, the continuous result of the MATCGO method speeds up the initialization of the visual method where it can be used by restricting the numbers of landmarks in the database to be matched.
Conclusion This paper has demonstrated the interest of the collaboration between two localization methodologies based upon different data sources to supervise the displacement missions of a vehicle in dense urban environments. One method is vision-based and uses a database of visual landmarks acquired in a preliminary learning phase. The other method combines odometry with data from a digital map of the road network together with data from satellites of a GNSS when it is available, that is to say rarely in environments with poor satellites visibility. The geometry of the road network and the visual landmarks are defined in the same coordinates and constitute two layers of the GIS, used both for planning and execution control purposes. Both methods define the results of a localization as a feasible ellipsoidal domain in the configuration space.
Vision-based localization has been shown to be sufficiently accurate to locate the vehicle at the lane-level whereas the accuracy of the MATCGO method is not so good. On the other hand, the MATCGO method is obviously more reliable since: i) the databases it uses are already available almost everywhere, and ii) due to the tight fusion algorithm, it is robust against long periods of poor satellite visibility. The vision-based method may fail in certain uncommon situations. This can be the case not only where there are not enough visual landmarks but also when the optimization algorithm does not converge in due time, as well as in special cases where the result of the visual localization is discontinuous since its principle does not guarantee the unicity of the solution. In this case, the MATCGO method running in parallel helps to detect this discontinuity and maintain a localization, which is important since the visual database of the GIS will certainly not be immediately available everywhere in the near future.
In addition, the continuous result of the MATCGO method speeds up the initialization of the visual method where it can be used by restricting the numbers of landmarks in the database to be matched. From a technical viewpoint: 1) it has been useful to design a 3D kinematical model because the images are very sensitive to the variations of the axis of the camera, 2) the fact that the visual landmarks are referenced in the coordinates of the GIS has received a careful attention, which is important since the visual control of the trajectory must be achieved within the same reference frame as the lane marks, side walks and pedestrian crossings that materialize on the planned path. The method was implemented in a full-scale experiment of 4 km in the centre of Paris which is a typical dense environment.