TopoLogic: An Interpretable  Pipeline for Lane Topology Reasoning on Driving Scenes

Authors: yanping  fu
wenbin  liao
xinyuan  liu
hang  xu
yike  ma
yucheng  zhang
feng  dai
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 16:04:18

Abstract: As an emerging task that integrates perception and reasoning, topology reasoning in autonomous driving scenes has recently garnered widespread attention.  However, existing work often emphasizes "perception over reasoning": they typically boost reasoning performance by enhancing the perception of lanes and directly adopt vanilla MLPs to learn lane topology from lane query. This paradigm overlooks the geometric features intrinsic to the lanes themselves and are prone to being influenced by inherent endpoint shifts in lane detection.
 To tackle this issue, we propose an interpretable method for lane topology reasoning based on lane geometric distance and lane query similarity, named TopoLogic. This method mitigates the impact of endpoint shifts in geometric space, and introduces explicit similarity calculation in semantic space as a complement. By integrating results from both spaces, our methods provides more comprehensive information for lane topology.  Ultimately, our approach significantly outperforms the existing state-of-the-art methods on the mainstream benchmark OpenLane-V2 (23.9 v.s. 10.9 in TOP$_{ll}$ and 44.1 v.s. 39.8 in OLS on subsetA). Additionally, our proposed geometric distance topology reasoning method can be incorporated into well-trained models without re-training, significantly enhancing the performance of lane topology reasoning. The code is released at https://github.com/Franpin/TopoLogic.

Main paper: 
 introduction: 
 In recent years, the field of autonomous driving has witnessed numerous milestone achievements and has progressively shifted from pure research to practical applications. In complex driving scenarios, vehicles need to perceive lanes & traffic elements and reason their topological relationships (i.e., lane connectivity and correspondence with traffic elements), which provides comprehensive information for downstream path planning and motion control. Under the trend of end-to-end autonomous driving, abovementioned perception and reasoning are integrated into a single task, referred to as topology reasoning [1] in autonomous driving scenes. This challenge has attracted widespread attention within the ego planning [2, 3, 4] and high-definition map learning [5, 6, 7, 8] communities.The topology reasoning task has garnered significant attention recently, since it is closer to the real needs. Some works have explored lane centerline representation [1, 9, 10, 11] and lane segment representation [12] , while others have introduced SDMap (Standard-Definition Map) [13] to provide additional clues for learning. However, existing works primarily focus on enhancements in the perception part, with scarce modifications made to the reasoning part. Irrespective of the approach details, existing studies typical employ vanilla MLP to learn lane topology directly from lane query. This paradigm has its shortcoming: since each lane is encoded independently through distinct query, it is challenging to ensure strictly overlap at the endpoints of two connected lanes, as shown in Figure 1 (b). In contrast, it is evident that the endpoints of two connected lanes in the ground truth actually overlap perfectly, as shown in Figure 1 (a). Lanes with slightly shifted endpoints may be erroneously classified by MLP as disconnected. This leads to MLP easily predicting fewer lane topology, as shown in Figure 1(c) .To tackle the aforementioned issues, we introduce TopoLogic, an interpretable method for lane topology reasoning that is based on lane geometric distances and the similarity of lane query in semantic space. The geometric distance-based approach aims to mitigates the impact of endpoint shift, thereby more robustly learning lane topology. This approach first calculates the geometric distance between lanes and then uses a learnable mapping function to map the distance to connectivity probability. Notably, for two given lanes, their geometric distance is defined as the distance between the ending point of one lane and the starting point of subsequent lane. This distance itself can serve as a strong criterion: when this distance is within a certain range, the predicted endpoints should be considered overlapping, and the lanes connected; otherwise, they are not. In this way, the lane topology reasoning becomes more tolerant of endpoint shifts, thus becoming more accurate. It's worth noting that even when the geometric distance method is merely applied as post-processing without re-training, the performance of SOTA model in lane topology reasoning is significantly improved, as shown in Figure 1 (d). Moreover, reasoning lane topology completely based on geometric distances can lead to inaccuracies when lane detection is imprecise as is shown in Figure 4 , since the calculation of lane geometric distance heavily relies on the accuracy of lane detection. To make up for the deficiency of geometric distances approach, we design a extra topology approach based on the similarity of lane queries as a complement. This approach projects lane queries into a high-dimensional semantic space, and involves explicitly computing the dot product between lane query to determine similarity, and then mapping this similarity onto lane topology using sigmoid [14] . The approach for calculating lane query similarity complements the approach used for computing lane geometric distance topology and similarly boasts high interpretability. The final lane topology is obtained by fusing the topology matrix derived from both approachs. By the way, the lane topology is also used in GNN to enhance lane learning through the aggregation of features from adjacent lanes. In summary, our contributions are as follows:1. We identify the current state of research in topology reasoning as "perception over reasoning", and reveal that the lane topology is easily disturbed by the endpoint shifts in lane detection when MLP employed solely for lane topology reasoning.We propose an interpretable method, referred to as TopoLogic, which conducts lane topology reasoning by calculating lane geometric distances and semantic similarity of lane query in a highdimensional semantic space.Extensive experiments on the mainstream benchmark OpenLane-V2 for topology reasoning task indicate that our method significantly outperforms existing state-of-the-art methods, especially in lane topology metric. Even if employed solely as a post-processing step without re-training, proposed geometric distance approach can significantly enhance well-trained lane topology reasoning models.2 Related WorkQuestion: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: We have clearly stated this in the introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
lane detection: 
 Lane detection plays an important role in autonomous driving, which has been a fundamental aspect of lane topology reasoning. In the realm of lane detection, some works [15, 16, 17] attempt to perform lane detection on a segmentation map. Moreover, some researchers use vector-based methods to perform 3D lane detection [18, 19, 20, 21] , however, these methods rely on a predetermined series of Y-axis coordinates within the query for forecasting 3D lanes, thereby lacking the capability to exclusively predict 3D lane positions along the Y-axis. In recent study, TopoNet [1] leverages Graph Neural Network (GNN) [22] to enhance the perception of lane centerline, while TopoMLP [9] utilizes PETR [23] for centerline detection. LaneSegNet [12] designs a Lane Attention mechanism to reinforce the perception of lane segment, and SMERF [13] introduces Standard-definition (SD) Map as an additional input to bolster the perception of lane centerline. In our work, we enhance lane learning by aggregating features of adjacent lanes through GNN, which involves computing lane geometric distance and lane query similarity.In lane topology reasoning, accurate comprehension of lane topology is imperative for effective navigation and decision-making in autonomous driving. Some methods [24, 25, 26, 27, 28] have been proposed to address this. The STSU [29] model drew inspiration from DETR [30] and employed a neural network architecture, complemented by a multi-layer perceptron (MLP) to establish line connectivity. Building upon this foundation, Can et al. [31] introduced minimal cycle queries to refine centerline, ensuring accurate ordering of overlapping lines and thereby enhancing precision. Further advancements include the perception of centerline [1, 13, 11, 9] and the perception of lane segment [12] . Among them, CenterLineDet [11] and TopoNet [1] , Both treat lane line as vertices and leverage an graph-based model to update lane representation and lane topology. While these methods have predominantly relied on MLP for generating adjacency matrices to represent lane topology. In our work, we calculate the lane topology matrices based on the geometric distances between lanes and the similarity of lane query within high-dimensional semantic space, respectively, and then fuse them to form the final lane topology. The fusion of geometric and semantic space enriches the model's understanding of lane topology, thereby culminating in improved performance in driving scene analysis and decision-making. 3 MethodQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: We have discussed this in the conclusion of the paper.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
problem definition: 
 Given images captured by the surround-view cameras of a vehicle, lane topology reasoning needs to perceive lane instances in BEV(Bird's Eye View) and then infer the topology between these lane instances. The enhancement of lane instance perception assists in the reasoning of lane topology. Lane instances are described as a set of directed lane lines which is denoted as L = [l 0 , . . . , l n-1 ].Each lane line is composed of a series of ordered points, and it is denoted as l = [p 0 , . . . , p n-1 ], p = (x, y, z) ∈ R 3 . The topology between lane instances signifies the connectivity of the directed lanes and it is depicted as a topology graph (V, E) , where the edge set E ⊆ V × V . An entry (i, j) in E is positive if and only if the ending point of lane l end i connects to the starting point of lane l start j .As is depicted in the Figure 2 , our proposed TopoLogic takes multi-view images from onboard cameras as input. These images are processed by a backbone to generate multi-scale image features. Multi-scale image features are transformed into BEV features through a view transformation module, and then passed to a lane deformable decoder to generate lane query Q l for lane detection. The proposed lane geometric distance approach and the lane similarity approach compute the lane topology respectively. Ultimately, the two topologies are fused and fed into GNN to augment the learning of lane line in the next decoder layer.Lane Geometric Distance Matrix. Lane query Q l can generate multiple directed lane lines through lane head. We can assess the connectivity between these lanes by computing the geometric distance between the ending point of one directed lane line and the starting point of the following lane line. where D is the lane geometric distance, l i and l j represent preceding and subsequent lane lines, d ij denotes the geometric distance between l i and l j , l end i signifies the last point of lane line l i , and l start j indicates the first point of lane line l j .EQUATIONEQUATIONEQUATIONDistance to Topology Mapping Function. After obtaining the geometric distance matrix D for the lanes, it is necessary to map the lane geometric distance into the lane topology. The lane topology can be represented by a matrix ranged in 0∼1. Zero indicates that there is no connection between two lanes, while one indicates that there is a connection. This mapping function needs to capture the following notion: when the input x → 0, it meaning the two lanes are very close to each other, the output y → 1, suggesting that these two lanes are very likely to be connected. Conversely, as x → ∞, y → 0. Inspired by the Gaussian function, we design a learnable mapping function as follows:EQUATIONwhere x = d ij . σ is the standard deviation of the geometric distance matrix D. α, λ are learnable parameters. With the help of such mapping, we can get a lane topology as follows:EQUATIONThere also exists some common alternative functions that meet the criteria, for example Gaussian function, sigmoid-based function and tanh-based function as Equation 6(a,b,c). We make a comparison between them with f ours in Figure 3 . Obviously, f ours sets a larger geometric distance threshold for determining topological connectivity compared to f gau , f sig and f tan , which makes the lane topology more robust to the endpoint shifts. Ablation study in Table 3 also verifies this opinion.EQUATIONLane topology reasoning based on the geometric distance of lane lines can achieve commendable results when the detection of lane lines is accurate. However, since this topology reasoning method heavily relies on the detected lane lines, inaccuracies in lane line detection can interfere with the geometric approach and lead to erroneous reasoning outcomes, as demonstrated in the Figure 4 . In light of this situation, we reason lane topology by calculating the similarity between lane query Q l within high-dimensional semantic space. A higher similarity between Q l indicates a greater likelihood of connectivity between the lanes, while lower similarity suggests an absence of connectivity. We initially encode Q l using two distinct MLP, and then represent the similarity by computing the inner product between the two encoding results. Finally, we require a function to map the similarity between Q l to lane topology. Given the correlation between lane similarity and lane topology, we employ sigmoid to map the lane similarity onto lane topology. This process is as follows:EQUATIONwhereQ i l ∈ R N l ×C , S represents the similarity of Q l . G sim ∈ R N l ×N l ,while N l represents the number of lane query.Both the lane topology reasoned from the geometric distance between lane lines and the lane topology reasoned from the similarity of lane query Q l in the high-dimensional semantic space can indicate the connectivity of lanes. These two methods are complementary in the task of lane topology reasoning. In this context, we merge the two lane topology reasoning results together as the final and more accurate lane topology using learnable coefficients as follow:EQUATION)where λ 1 , λ 2 are learnable parameters, and G is the final lane topology prediction.Similar to Transfomer-based networks [30, 32] , the supervision is applied on each decoder layer to optimize the query feature iteratively. The overall loss of TopoLogic is:EQUATIONThe lane detection loss L det consists of a focal loss [33] for lane classification and an L1 loss for lane regression. The lane topology reasoning loss L top includes only the loss computed for G sim .As for the calculation of G dis , since it is enhanced by GNN to facilitate lane learning, we update its learnable parameters through L det .4 ExperimentQuestion: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
dataset and metric: 
 Dataset. We have evaluated TopoLogic on the OpenLane-V2 [34] , which is currently the only large-scale perception and topology reasoning dataset devised for autonomous driving scenarios. This dataset was developed by Argogorse2 [35] and nuScenes [36] respectively. It provides annotations for both lane centerline tasks and lane segment detection tasks. OpenLane-V2 consists of two subsets: subset_A and subset_B, each comprising 1000 scenes with 2Hz multi-view images and annotations. Each subset includes annotations for the lane centerline, traffic element, lane topology, as well as the topology between traffic element and lane. In subset_A, there are seven views as input, together with an additional Standard-definition Map input, and the annotations for lane segment have been expanded; subset_B contain only six views as input.Metric. OpenLane-V2 evaluates perception tasks for both lane centerline and lane segment.(1) In the task of lane centerline perception, the metrics include DET l , DET t , TOP ll , and TOP lt . DET l quantifies similarity by averaging the Frechet distance at matching thresholds of 1.0, 2.0, 3.0. DET t uses Intersection over Union (IoU) as a measure of similarity and calculates the average across different traffic categories. TOP ll and TOP lt respectively compute the topology matrix similarity between lanes and between lanes and traffic elements, with the overall evaluation metric for lane centerlines being denoted as OLS. The OLS is carried out as OLS = 1 4 [DET l + DET t + f (TOP ll ) + f (TOP lt )], where f is the square root function.(2) In the task of lane segment perception, we adopt the metric proposed by LaneSegNet for evaluating lane segment perception. These include the lane segment distance D ls , the corresponding Average Precision AP ls and AP ped , with the mAP calculated as the average of AP ls and AP ped . The lane segment topology metric is denoted as TOP lsls .(3) For centerline, OpenLane-V2 has two versions available for evaluation on TOP ll , TOP lt , and OLS: v1.0.0 and v2.1.0. For lane segment, OpenLane-V2 has versions v2.0.0 and v2.1.0 on TOP lsls . 2 Since the ultimate goal of perception is reasoning, we believe that topology metrics are what should be paid more attention. Moreover, our modifications mainly involve lane topology reasoning, so we primarily focus on the lane topology metric TOP ll and TOP lsls .Feature Extractor. All images are resized into the same resolution of 1550 × 2048. For reproducibility, we utilize the official implementations of TopoNet, SMERF, and LaneSegNet models. Both models use ResNet-50 [37] backbone pretrained on ImageNet [38] paired with a Feature Pyramid Network (FPN) [39] to extract multi-scale features. The number of output channels is set to 256. We employ the view transformer from BEVformer [40] encoder to transform multi-scale features into BEV features. The size of BEV grids is set to 200×100. TopoLogic is configured identically.Lane Detector. We employ Deformable DETR [32] for the detection of lane line. The number of query is set to 200. After passing through each decoder layer of Deformable DETR, the query undergo GNN using the lane topology matrix. We predict the offset of the lane lines by setting reference points, with each lane line consisting of 11 three-dimensional points. For LaneHead, the classification head adopts a three-layer MLP with LayerNorm and ReLU to predict the confidence score of the lane line. The regression head is a three-layer MLP combined with ReLU, used to predict the 11×3 offset of the lane line. For lane detection loss L det l , the weight of the classification part is 1.5, and the weight of the regression part is 0.025.Lane Topology Head. The Lane Topology Head consists of a lane geometry distance predictor and a lane similarity predictor. For lane geometry distance predictor, we first calculate the geometric distance between the terminating point of the previous lane line and the starting point of the subsequent lane line to obtain a 200×200 distance matrix. Then the distance matrix is mapped to a lane topology matrix through a learnable mapping functionf (x) = e -x α λ•σ , where the size of α, λ, σ is 1×1, σ is the standard deviation of x, α, λ are learnable parameters, α is initialized to 2, λ is initialized to 0.2. For the calculation of lane similarity, given a 200×256 lane query, it is encoded through two different three-layer MLP. The we compute the similarity between the encoded results, resulting in a 200×200 lane similarity matrix. The similarity matrix is transformed into a lane topology matrix through sigmoid. The two lane topology matrices are fused into the final lane topology using learnable coefficients, which are initialized to 1 and have a size of 1×1.Training. We train TopoLogic utilizing the AdamW optimizer [43] with a weight decay of 0.01 with an initial learning rate of 2 × 10 -4 and employ a cosine annealing schedule for the learning rate. All experiment is trained for 24 epochs on 8 NVIDIA RTX 3090 GPUs with a batch size of 16.Centerline. We compared TopoLogic with existing state-of-the-art methods such as STSU, Vec-torMapNet, MapTR, TopoNet, SMERF on centerline. Table 1 shows the results on the subset_A and subset_B datasets. Without any additions, our method achieves state-of-the-art performance. Compared with TopoNet, our method achieves decent detection accuracy (29.9 v.s. 28.6 on subset_A, 25.9 v.s. 24.4 on subset_B), especially with a significant improvement in TOP ll (18.6 v.s. 4.1 on subset_A for v1.0.0, 23.9 v.s. 10.8 on subset_A for v2.1.0), which is the lane topology reasoning score. There is also an improvement in OpenLane-V2 overall score OLS (41.6 v.s. 35.6 on subset_A for v1.0.0, 44.1 v.s. 39.8 on subset_A for v2.1.0). Even when using SDMap, our proposed TopoLogic still manages to achieve state-of-the-art performance and realizes a significant improvement in TOP ll (23.4 v.s. 7.5 for v1.0.0, 28.9 v.s. 15.4 for v2.1.0). Lane Segment. Concurrently, we compared TopoLogic with existing state-of-the-art methods such as TopoNet, MapTR, MapTRv2, LaneSegNet on lane segment. In Table 2 , it indicates that our method exhibits improvements in the Mean Average Precision (mAP) for lane segment detection compared We have studied several important components of TopoLogic and conducted ablation experiments on the OpenLane-V2 subset_A. In the following text, we employ evaluation metrics from the latest v2.1.0 release for our assessment.The design of mapping function. We have studied the effect of different mapping functions in the transformation from lane geometric distances to lane topology. The approach of lane topology reasoning. We have investigated the impact of various lane topology computation approaches on lane topology reasoning, specifically using MLP, lane query similarity, and geometric distance. The results in the Table 4 indicate that the integration of a lane topology method, combining both lane geometric distance and lane query similarity calculations, yields optimal results on TOP ll (23.9 v.s. 20.1 v.s. 12.9 v.s. 10.8) and also performs best in terms of lane line detection scores indicated by DET l (29.9 v.s. 28.6 v.s. 28.1 v.s. 27.8) . This demonstrates that the topology obtained from the fusion of these two methods can also improve the learning of lane centerline through GNN feature enhancement.The approach of similarity topology. We investigated the impact of using MLPs to encode lane queries for computing lane similarity on lane topology reasoning. As Lane similarity topology, its subtlety lies in that it uses two independent MLPs to map the lane query rather than a single MLP, which can decouple a lane into two queries of start and end point, achieving an analogous effect to the geometric distance approach in semantical space. The results presented in Table 5 indicate that encoding lane query with two independent MLPs to obtain features for the starting and ending points of lanes allows for better computation of lane similarity topology. This approach exhibits the best performance in terms of both lane centerline detection score and lane topology reasoning score (29.9 v.s. 27.5 v.s. 25.6 on DET l , and 23.9 v.s. 21.2 v.s. 18.7 on TOP ll ).The post-processing mode of geometric distance approach. We have investigated the effectiveness of geometric distance approach as a post-processing module on well-trained model. In Table 6 , we conducted experiments by adding a post-processing to the already trained TopoNet, SMERF, and LaneSegNet, respectively. The results indicate that our proposed approach, which calculates lane topology based on lane geometric distances, can be integrated into well-trained models without any additional modifications and can significantly enhance the performance of lane topology reasoning(22.3 v.s. 10.9 on TopoNet, 26.2 v.s. 15.4 v.s. on SMERF).As shown in Figure 5 , we presents a qualitative comparison between TopoLogic and TopoNet. Specifically, two traffic scenes are selected for analysis, and the results of lane line detection and topology reasoning are visualized. The first row displays multi-view inputs of realistic scenes, while the second row shows the lane detection results of TopoLogic and TopoNet alongside the ground truth. Notably, TopoLogic demonstrates superior accuracy in lane line detection compared to TopoNet.Lane Graph. The inherent complexity of topology reasoning makes intuitive representation of results challenging. To address this issue, as shown in third row in Figure 5 , we construct a lane graph where nodes represent lane lines, and their relative positions in the graph correspond one-to-one with the relative positions of lane lines. This layout enhances the connection between lane line detection results and facilitates subsequent analysis. Additionally, we use directed edges to represent the lane topology, with red indicating error predictions and blue indicating missing predictions. Topo-Logic consistently exhibits proficient lane topology reasoning across various intersection scenarios, showcasing significantly enhanced performance in the topology graph compared to TopoNet.  
conclusion: 
 In this paper, we reveal the limitation of using vanilla MLP in lane topology reasoning task and propose TopoLogic, which is the first to employ an interpretable approach for lane topology reasoning. TopoLogic fuses the geometric distance of lane line endpoints mapped through a designed function and the similarity of lane query in a high-dimensional semantic space to reason lane topology. Experiments on the large-scale autonomous driving dataset OpenLane-V2 benchmark demonstrate that TopoLogic significantly outperforms existing methods in topology reasoning in complex scenarios.Limitations. Due to the GNN's role in merely aggregating features from adjacent lanes to enhance the learning of the current lane, our proposed method significantly improves the performance of lane topology but does not substantially elevate lane detection.Impact. Based on the previous sections, it is evident that our proposed method is intended for research purposes. It should not be directly used in or deployed within any actual autonomous driving application. Notably, we cannot provide any guarantee in safety-critical situations.NeurIPS Paper Checklist 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We have provided the computer resources necessary to reproduce the experiments in implementation detail of the experiment section.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We have mentioned the impact in the conclusion.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: They are properly credited and properly respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: Justification: We adhere to the NeurIPS Code of Ethics in every respect.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).See the official repository for version differences of metrics: https://github.com/OpenDriveLab/ OpenLane-V2/issues/76d 


Decision: The paper proposes TopoLogic, an interpretable framework for topology reasoning in autonomous driving. It introduces several modifications to mitigate several issues in existing approaches. e.g. explicit similarity calculation. It brings significant improvement over previous SOTA on OpenLane-v2 benchmark.

Authors have carefully addressed reviewers comments and provide more details. There are two positive reviews and two negative ones. The negative comments are mainly focused on the lack of implementaion details / explaination on the experimental settings / typos / etc. Since it falls into the category of borderline paper, AC steps in, read the paper carefully and rebuttal feedbacks. 

There are concerns on the technical contribution as pointed out by reviewers. In AC's view, the field of lane detection is **slowly** evolving over the years. There are not too many research on topology reasoning, which is a critical issue for the next-level of lane detection. Given the importance of the topic, great improvement over previous SOTA and detailed feedback on the rebuttal, AC would champion this work for this time. Please incorporate all the concerns, release the code and contribute to the community.

Reviews: review: paper_summary: paper_summary: This paper proposes an interpretable method for lane topology reasoning based on lane geometric distance and lane query similarity. The authors reveal that the lane topology is easily disturbed by the endpoint shifts. Based on this, the proposed post-processing module improves the robustness and performance of lane topology reasoning, which can be plugged into other methods without re-training. And it also alleviates the influence of inaccuracies in lane detection on topology reasoning. Extensive experiments on OpenLane-V2 demonstrate its state-of-the-art performance.
strength_weakness: strength_weakness: - The proposed method is relatively simple and involves only post-processing modules, which at the same time leads to limited contributions and incremental improvements.

- Considering that TopoNet is just a simple baseline model, the boost in topological reasoning is not surprising. Moreover, as can be seen in Table 1, the improvement of $TOP_{lt}$ is very small and $DET_{t}$ even decreases, which needs further analysis and discussion.
questions: questions: - As can be seen in Figure 3, compared to other mapping functions, the final result of the learnable Gaussian mapping function has a much higher threshold. Therefore, compared to the present comparison experiments ($f_{gau}, f_{sig}, f_{tan}$), it may be better to manually set the hyperparameters for a series of gradients to observe its effect on the final performance. And I'm wondering why $f_{gau}, f_{sig}, f_{tan}$ are very close but differ dramatically on $TOP_{ll}$ in Table 3.

- According to L146-149, the motivation of Lane Similarity Topology is to alleviate the influence of inaccuracies in lane detection on topology reasoning. However, the proposed method as a post-processing step does not significantly change the lane line detection results. In addition, the comparison results of the Lane Similarity Topology module should be added to Figure 4 to increase its persuasiveness.

There are many typos in the paper, which should be further checked and polished.  
For example:  
L6: "are prone to" -> "is prone to"  
L12: "our methods provides" -> "our methods provide"  
L17: "boost" -> "boosting"  
Figure 1 caption: "reasoing"
limitations: limitations: This paper is limited in the post-processing for topology reasoning.
paper_summary: paper_summary: The authors claim previous topology reasoning methods typically boost reasoning performance by enhancing the perception of lanes and directly adopt MLP to learn lane topology from lane query. In this work,  the authors propose to make full use of lane geometric distance and lane query similarity for topology reasoning. The proposed method, named TopoLogic, mitigates the impact of endpoint shifts in geometric space, and introduces explicit similarity calculation in semantic space as a complement. The method achieves SOTA results in OpenLane-V2 dataset.
strength_weakness: strength_weakness: 1. Like an incremental work based on TopoNet. The contribution is limited. The contributed part is only a similarity module for mapping lane line endpoints. But the design is simple and straightforward, not surprising enough.
2. The proposed GeoDist module is a post-processing algorithm. It's like an engineering improvement rather than academic contribution.
questions: questions: When compared with previous methods (like TopoNet, SMERF), what about the computation overhead and the difference in network architecture?
limitations: limitations: Refer to the Weaknesses part.
paper_summary: paper_summary: Reasoning about lane topology is becoming more and more important for the autonomous driving community. Current methods mostly focus on improving the perception performance and ignore the reasoning part of the task. The authors raise the importance of the relationship between lanes, namely the lane distance and lane similarity. The lane distance is determined by the geometry distance between the starting and ending points of the lanes. Then, the connectivity is decided based on the result of a learnable threshold function. The lane similarity is calculated based on the lane queries, and a large similarity indicates a large possibility of connectivity. Both lane distance and lane similarity modules serve as plug-in modules to existing networks. Experimentally, both modules have significantly improved the existing methods.
strength_weakness: strength_weakness: - In the experiment session, the authors provide scores on lane segment prediction. However, the lane segment is different from the lane centerline. Only the lane centerline is discussed in the methodology session.
- It is not stated which split of the dataset is used for experiments.
- The mixture of using different versions of the metric is confusing (vx.0 and vx.1). As stated in the repository of the dataset, this difference leads to different TOP scores. For instance, in Table, metrics in v2.0 and v2.1 are both listed, which is unreasonable.
- The two proposed modules focus on topology reasoning. However, experimentally, the detection scores of the proposed pipeline also improved compared to other methods. This improvement is not discussed.
questions: questions: - The two proposed modules are plug-in modules on current networks, and the proposed pipeline is trained in an end-to-end manner. However, as these two modules focus on topology reasoning, and the final topology prediction is decided by these two modules, it is curious to know how these two modules benefit existing networks. Namely, only parameters in the proposed modules are trained, and the remaining weights of the existing networks are frozen.
- For the second proposed module, the lane similarity module. Logically, the lane queries usually encode positional information of lanes. It is curious to know why improving the similarity between two connected lanes would improve topology reasoning performance. That is, the only similar thing between two connected lanes is the ending point of one lane and the starting point of another. Two parallel lanes should have a larger similarity than two connected lanes.
limitations: limitations: N/A
paper_summary: paper_summary: The paper investigates topological reasoning for lane graph prediction for autonomous driving. The paper focuses on improving the prediction of the topological structure of the lane graph. Therefore, they propose two new mechanisms, the first is a geometric approach that estimates connectivity based on the distance between the end point and starting point of center lines. Secondly, they propose a semantic approach which uses the similarities of lane queries in a high dimensional space. These two approaches stand in contrast to existing works which mainly use MLPs to predict the connectivity of the lanes. They show how these approaches can be interleaved into a lane graph network and how it improves results, especially the topological reasoning results. Even adding only the geometric approach to existing methods improves the topological reasoning results.
strength_weakness: strength_weakness: - It is not clear to me why the paper does not compare with TopoMLP, the numbers for Table 2 v1.0.0 can just be taken out of the paper.
- When comparing with TOP_lt TopoMLP outperforms TopoLogic and overall performance is very similar, in this case both methods use the same backbone.
- The paper highlights that geometric cues in predicting the topology of the lane graph are underutilized which is interesting, but I am not fully sure if this is a Neurips paper, since technical contributions are marginal.
- See questions
questions: questions: - Given that f_ours is just a gaussian function with an adapted decision boundary. I think the fair comparison would be with a hand tuned decision boundary, for example by looking at end point errors of a trained model. 
- f_tanh in equation 6 seems to map to [0, 2] and not to [0, 1]
- Equation 10 is not a [0, 1] probability anymore, to remain this property you would need to take a convex combination of the two values. Do you perform some sort of normalization?
- If you say a variable is 1x1 do you mean it is a real number? If so just write \in R
- The batch size of 2 on 8 GPUs seems strange, is it a batch size of 2 per GPU, so an actual batch size of 16?
- How do you handle traffic elements?

- Typos: 
- L90: . some methods - not capitalized after full stop
- L204: Both models - three models are listed before
- L236: ),especially 
- L259: Table 4 Indicate show
limitations: limitations: Has been discussed

Comments: title: Authors' Response to All Reviews
Further questions
comment: Thanks for the reviewers' careful reading of our paper, and giving valuable suggestions for us to improve the paper. We hope our response and clarifiction can ease some of your concern. If you have any further questions, we are more than happy to discuss with you if time permits. 

If you are satisfied after reading our response, we sincerely hope that you could increase the rating of our paper. Thank you.
**Thank you for your insightful question.**

**Q4**: Why does TopoNet-based TopoLogic and SMERF-based TopoLogic run faster and
 have less params and flops compared with original TopoNet and SMERF?

**A4**: TopoNet and SMERF both compute lane topology directly using MLP. In contrast, TopoLogic calculates lane topology through a geometric distance module and a similarity module. This distinction highlights the structural differences between TopoLogic and both TopoNet and SMERF.

In the TopoNet and SMERF, two MLPs are used to generate $Q_{emb}  \in \mathbb{R}^{N \times C}$, where $N$ is the number of query. Subsequently, $Q_{emb}$ are repeated to $Q_{emb}^{'} \in  \mathbb{R}^{N \times N \times C}$. **They  are then concatenated and processed through $\operatorname{MLP_3}$ to compute the lane topology**, as described by the following Equations:
$$
Q_{emb_1}, Q_{emb_2}=\operatorname{MLP_1}(Q_l^i), \operatorname{MLP_2}(Q_l^i) \in \mathbb{R}^{N \times C}
$$
$$
Q_{emb_1}^{'},Q_{emb_2}^{'}=\operatorname{Repeat}(Q_{emb_1}),\operatorname{Repeat}(Q_{emb_2}) \in \mathbb{R}^{N \times N \times C}
$$
$$
\operatorname{G_{sim}}=\operatorname{MLP_3}(\operatorname{Concat}(Q_{emb_1}^{'}, Q_{emb_2}^{'})) \in \mathbb{R}^{N \times N}
$$

For TopoLogic, we calculates lane topology through a geometric distance module and a similarity module. In similarity module, We directly compute lane topology via matrix multiplation between $Q_{emb_1}$ and the transposition of $Q_{emb_2}$, followed by applying a sigmoid activation function, as detailed in our paper:


$$
Q_{emb_1}, Q_{emb_2}=\operatorname{MLP_1}(Q_l^i), \operatorname{MLP_2}(Q_l^i) \in \mathbb{R}^{N \times C}
$$
$$
\operatorname{S}=\operatorname{matmul}(Q_{emb_1},\operatorname{transpose}(Q_{emb_2})) \in \mathbb{R}^{N \times N}
$$
$$
\operatorname{G_{sim}}=\operatorname{sigmoid}(\operatorname{S}) \in \mathbb{R}^{N \times N}\\
$$

In geometric distance module, as illustrated in Equation (4) of our paper, we utilize only two parameters, $\alpha$ and $\lambda$. Additionally, the merge process involves two parameters, $\lambda_1$ and $\lambda_2$, as shown in Equation (10) of our paper. All these parameters are scalar values,  i.e., $\alpha,\lambda, \lambda_1, \lambda_2\in\mathbb{R}$. 

To summarize, while TopoNet and SMERF models involve **three MLPs** with associated parameters, TopoLogic utilize only **two MLPs** in the similarity module, two parameters $\alpha$, $\lambda$ in geometric distance m odule, and $\lambda_1$ and $\lambda_2$ in merge process. **In TopoLogic, the scalar parameters in the geometric distance module are significantly fewer in number compared to the MLP$_3$ parameters in TopoNet and SMERF.**  Additionally, the MLP$_3$ results in a higher computational complexity. Therefore, TopoLogic has a smaller number of parameters (#PARAMS) and lower computational complexity (FLOPS), leading to faster speeds (FPS) compared to TopoNet and SMERF. 

**We hope this explanation addresses your concern, thank you again for your valuable feedback.**
**We are very grateful that you have carefully reviewed our rebuttal response.**

Although there are deficiencies in the details, we still believe that this work has sufficient contributions to the field: **we have significantly improved the core metrics (TOP$_{ll}$ & OLS) of the emerging task of topology reasoning on autonomous driving with a easily understandable method.**

Regarding your further questions, we will answer them one by one below. We hope you can reconsider the rating of our article.

----

**Q6.1:** It seems that the results of TOP$_{ll}$ are very sensitive to the hyperparameters.

**A6.1:** As you said, TOP$_{ll}$ is sensitive to hyperparameters, but **its actual negative effects are tiny**. 

- On the one hand, the learnable hyperparameters we used can effectively **self-adapt** to the data distribution and **avoid** the tedious manual parameter tuning. 
- On the other hand, the results under **all** hyperparameter configurations in our experiments are actually always **significantly higher** than the baseline (**16.0~23.9** v.s. 10.9 on Q3.1, **15.1~23.9** v.s. 10.9 on Table 3 in the paper), which further implies the effectiveness of our methods.

----

**Q6.2:** As shown in Figure 3 of the global rebuttal, the introduction of the similarity module removes many predictions of topology. But both results of (b) and (c) are far away from GT. I think that the existing metrics may not reflect the results well, and the filtering could lead to large variations.

**A6.2:** We are very sorry that our **improper presentation** in Figure 3 gave you the misunderstanding. 

- To avoid the **visual confusion** caused by excessive topological connections, during visualization in Figure 4 of the main paper and its extended version Figure 3(b,c) of the Rebuttal PDF , only **the right lanes** (blue lines), **the incorrect lanes** (yellow lines) and **the incorrect topological connections** (red arrows) are drawn (as is description in the caption of Figure 4 of the main paper), while **the correct topological connections** are **omitted**, which is why the results of (b,c) look away from GT.

- You can find that **all** the arrows in the figure are red (wrong), but it is **obviously impossible** that all connections are wrong (red), so please believe that we **deliberately visualized it this way** (although it looks stupid now), rather than **the result itself being bad**. In this perspective,  what the figure shows is that the similarity module can further reduce **false predictions**, rather than **all predictions**.
- Thank you very much for pointing out this issue. We will definitely add the correct predictions in the figure in the final version (using other color, e.g. green), but unfortunately, we **cannot upload** any pictures again during the discussion stage limited by Open Review system. If you still have doubts about the visualization results, you can also pay attention to **Figure 5** in the main paper again. We believe both the **Lane Topology** and the **Lane Graph** in it can definitely make people clearly understand the effect of our method.

----

**Q6.3:** The authors are not able to clearly explain the results of the DET.

**A6.3:** It is indeed hard to provide a more clear explanation in a short time, however, compared with the little drop on DET$\_{t}$(**-1.4**),  the **improvement in lane & topology** (**+13.0** TOP$_{ll}$, **+4.3** OLS, **+1.3** DET$_l$ [Topologic v.s. TopoNet]) is **more significant**. Besides, the latter is our **core research motivation and improvement goal**. Objectively speaking, compared with the **reproduced TopoNet**, our method is not inferior in terms of DET$\_t$ (**47.2** v.s. 46.9). It means that the issue lies in the reproduction of TopoNet (**46.9** v.s. 48.6) rather than our TopoLogic, which has also puzzled other researchers (e.g., the reproduction of TopoNet has lower DET$\_{t}$ (**44.5** v.s. 48.6) in Table 1 of SMERF paper). If there is still a chance, we will definitely explore this issue in depth and fix it.

----

**If you have any further questions or suggestions about our article, we'd love to discuss our content with you.**
Thanks for the authors' positive feedback. 

It seems that the results of $TOP_{ll}$ are very sensitive to the hyperparameters. And as shown in Figure 3 of the global rebuttal, the introduction of the similarity module removes many predictions of topology. But both results of (b) and (c) are far away from GT. I think that the existing metrics may not reflect the results well, and the filtering could lead to large variations. In addition, the authors are not able to clearly explain the results of the $DET$. The above makes the quantitative experiment less convincing.

Overall, I still have concerns about incremental improvements and contributions, which are also mentioned by other reviewers.
Thanks for the authors' feekback. My further questions are as follows:
Why does TopoNet-based TopoLogic and SMERF-based TopoLogic run faster and have less params and flops compared with original TopoNet and SMERF? Is the post-processing step (GeoDist module) included？
**Thank you very much for your quick response.**

**Q7:** For Q2, are the scores reported on the validation set or test set?

**A7:** We're sorry we didn't mention that detail, and we will add more clarification in the final version. In fact, all the results are reported on the **validation set**, which is **completely consistent** with the protocol of the **all reference paper**, e.g., TopoNet, SMERF, LanSegNet, etc. As for test set, it is only used for the closed-evaluation of the OpenLane-V2's challenge(https://opendrivelab.com/challenge2024/#mapless_driving), which was not available at the time of our research.

----

**Q8:** For Q5, the DET$_t$ of the frozen TopoNet drops compared to the baseline version, are there modules which could affect the traffic element detection branch?

**A8:** We're sorry to confuse you.  Either similarity module or geometric distance module just involves the **lane**, not **traffic elements**, so DET$_t$ really shouldn't drop. However, in the freezing experiment we loaded the weights of the TopoNet that we reproduced, whose DET$_t$ in itself is lower (**46.9** v.s. **48.6**) than reported in the TopoNet paper. It seems to be **a bug in baseline's official code**. The reproduction about DET$_t$ was always **unstable and lower** than the value reported in the original paper, and this phenomenon can also be observed in the results of other paper (e.g., **44.5** DET$_t$ of TopoNet in Table 1 in paper SMERF$^{[1]}$)

[1] Luo K Z, Weng X, Wang Y, et al. Augmenting lane perception and topology understanding with standard definition navigation maps[J]. arXiv preprint arXiv:2311.04079, 2023.

**If you have any further questions or suggestions about our article, we'd love to discuss our content with you.**
Thank you for your feedback. I still got some question.

For Q2, are the scores reported on the validation set or test set?

For Q5, the DET_t of the frozen TopoNet drops compared to the baseline version, are there modules which could affect the traffic element detection branch?