Reasoning Multi-Agent Behavioral Topology for Interactive Autonomous Driving

Authors: haochen  liu
chen  li
yu  qiao
chen  lv
hongyang  li
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-24 17:00:32

Abstract: Autonomous driving system aims for safe and social-consistent driving through the behavioral integration among interactive agents. However, challenges remain due to multi-agent scene uncertainty and heterogeneous interaction. Current dense and sparse behavioral representations struggle with inefficiency and inconsistency in multi-agent modeling, leading to instability of collective behavioral patterns when integrating prediction and planning (IPP). To address this, we initiate a topological formation that serves as a compliant behavioral foreground to guide downstream trajectory generations. Specifically, we introduce Behavioral Topology (BeTop), a pivotal topological formulation that explicitly represents the consensual behavioral pattern among multi-agent future. BeTop is derived from braid theory to distill compliant interactive topology from multi-agent future trajectories. A synergistic learning framework (BeTopNet) supervised by BeTop facilitates the consistency of behavior prediction and planning within the predicted topology priors. Through imitative contingency learning, BeTop also effectively manages behavioral uncertainty for prediction and planning. Extensive verification on large-scale real-world datasets, including nuPlan and WOMD, demonstrates that BeTop achieves state-of-the-art performance in both prediction and planning tasks. Further validations on the proposed interactive scenario benchmark showcase planning compliance in interactive cases. Code and model is available at https://github.com/OpenDriveLab/BeTop.

Main paper: 
 introduction: 
 Autonomous driving system aspires to safe, humanoid, and socially compatible maneuvers [1] . This drives for formulation, prediction, and negotiation of collective future behaviors among interactive agents and autonomous vehicles (AVs) [2] . Remarkable accuracy is achieved by learning-based paradigms [3] , including end-to-end modular design [4] [5] [6] [7] , social modeling [8, 9] , and trajectorylevel integration [10] [11] [12] [13] . However, substantial challenges arise in real-world cases due to scene uncertainty and volatile interactive patterns for multi-agent future behaviors.To embrace compliant patterns for multi-agent future behaviors, current formulations fall into two mainstreams, dense representation and sparse representation (Fig. 1 ). Dense representation quantizes agent behaviors under ego-centric rasterization, forecasting bird's eye view (BEV) occupancy probabilities [14, 7, 15] or temporal flow [16] [17] [18] . It is easy to deduce interactions, perform scalable behaviors for agents [19] , and align with BEV perceptions [20] . Still, dense representation is hindered by frozen receptions. It causes safety-vulnerable intractability and occlusions potentially interacting with ego maneuvers [16, 21] . Contrary to pixel-wise behavioral probability, sparse representation forecasts agent-anchored set of trajectories [22] [23] [24] [25] or intention distributions [10, 26, 27] . Its multimodal formulation for each agent marks the elasticity in diverse behavioral uncertainty and tractability under flexile spatial semantics. However, behavioral misalignment [28] and modality collapse [24] Figure 1 : Multi-agent Behavioral Formulation. (a) A typical driving scenario in Arizona, US [35] ; (b) Dense representation conducts scalable occupancy prediction jointly, but restrained reception leads to unbounded collisions with planning; (c) Sparse supervision derives multi-agent trajectories with multi-modalities, while it struggles with conflicts among integrated prediction and planning; (d) BeTop reasons future topological behaviors for all scene-agents through braids theory, funneling interactive eventual agents (in highlighted colors) and guiding compliant joint prediction and planning. impede compliant multi-agent modeling, requiring exponential computations with growth agent numbers [29] . Issues particularly result in unstable and slow behavioral learning when exposed to predictions and planning (IPP) [30] . Typical solutions by conditional prediction [31, 28, 32] or game-theoretic reasoning [8, 33] often lead to nonstrategic maneuvers [34] due to non-compliant rollouts in adjusting interactive behaviors. This calls for a re-formulation for multi-agent behaviors, which should stabilize collective behavioral patterns in a compliant manner for IPP objectives.The decision-making process for human drivers provides valuable insights. Humans primarily determine the future behavior of interacted agents for decision-making without relying on their specific states [2, 36] . Thus, an effective strategy involves assessing agent-wise behavioral impact on planning maneuvers, and reasoning about compliant interactions. Our fundamental insight is that compliant multi-agent behaviors exhibit topological formations, which can be identified by distilling consensual interactions from future behaviors. Prior works have approached this challenge through structural design [37, 38] or implicit relational learning [39] [40] [41] using GNNs [42] or Transformers [43] . Other studies quantify uncertainty by topological properties [44, 45] . Nevertheless, current literature is scarce in formulating explicit future supervision of compliant multi-agent behavioral patterns.To this end, we launch the multi-agent behavior formulation termed as Behavioral Topology (BeTop). At its core, BeTop explicitly forms the topological supervision of consensual multi-agent future interactions, and reasons to guide prediction and planning. BeTop stems from braid theory [46] , which infers compliant interactions of multiple paths from the intertwining of their braids. This empowers BeTop to intuitively distill forward intertwines (occupancy) as joint topology from braided multi-agent future trajectories (Fig. 1 ), marrying dense and sparse representation. With the aid of BeTop, we introduce a synergistic Transformer-based learning stack, BeTopNet, for learning IPP objectives. To implement, an iterative decoding strategy simultaneously reasons about Behavioral Topology and generates trajectory sets. Then the topology-guided local attention, embedded in each decoder layer, selectively queries behavioral semantics from social-compliant agents within the predicted BeTop priors. To further alleviate multi-agent uncertainty through topological guidance, a contingency planning paradigm is fitfully deployed. We lay out the imitative contingency learning process, which regulates the safety-ensured short-term plan. It maintains the long-range uncertainty by reasoned joint predictions from BeTop. Experimental results exhibit enhanced consistency and accuracy for prediction and planning in real-world scenarios. Testing in proposed interactive cases further highlights the planning ability of BeTopNet. To sum up, our contributions are three-fold:• We bring in the concept of Behavioral Topology, a multi-agent behavioral formulation for topological reasoning that explicitly supervises consensual future interactions jointly for the IPP system.• A synergistic learning framework BeTopNet, offering joint planning and prediction guided by topology reasoning, is devised. Topology-guided local attention and imitative contingency planning could resolve scene compliance and multi-agent uncertainty.• Benchmarking on nuPlan [47] and WOMD [35] , our approach demonstrates strong performance in both planning strategy and prediction accuracy. BeTopNet witnesses evident improvement over previous counterparts, e.g., +7.9% in general planning score, +3.8% under interactive cases, +4.1% mAP for joint prediction, and +2.3% mAP for marginal prediction. 
related work: 
 Multi-agent behavioral modeling. Carving the collective future behavior of diverse agents is imperative for socially-consistent driving maneuver. Earlier approaches centered around occupancy prediction [14, 5, 4, 48] . Forecasting the spatial presence under dense BEV representation [49, 20] offers flexibility of arbitrary agents [17, 50, 19] and alignment with perception [7, 51] . However, rigidity in resolution induces scenario occlusion [16] , rendering intractable occupancy [21] . In parallel, sparse representation consolidates multi-agent behavior into cohesive modalities across future trajectories [52, 22, 53, 54] or intentions [27, 10, 26, 55] . Joint future behaviors are derived through goal-based sampling or recombination from marginal predictions [56] [57] [58] . However, the collection of joint modalities is susceptible to mode collapse and entails exponential complexity [29, 59] . Meanwhile, topological representation has garnered traction as motion primitives [60, 61] or tools [44, 45] for scenario quantification, yet topological properties delineating collective future behaviors remain largely unexplored. Our BeTop targets the issue, marrying dense behavior probabilities by topology with the sparse motion from joint predictions to present structured future behaviors.On the other hand, inconsistent communal agent behaviors have motivated leveraging future interactions. Implicit approaches obtain tacit interactions with attention [62, 32, 63] or GNN [24, 64, 37, 65] from final motion regressions. Nonetheless, the implicit supervisions are found inefficient in dynamic scenarios [22] . Contrarily, explicitly reasoning mutual behaviors by conditional factorization [31, 66] , relation reasoning [41, 40] , or entropy-based methods [67, 68] offer consistent behavioral priors. However, hefty variance across agent dynamics and scenario geometries yield unstable inference. Distinguished from them, BeTop crafts a compact topological supervision that stabilize future interactions among multi-agent behaviors. Derived from topological braids, BeTop offers a topological equivalent behavioral representation to guide compliant forecasting and planning.Integrated prediction and planning. IPP system aims to harmonize trajectory-based learning of future interactive behaviors between the ego vehicle and social agents. Rule-based approaches [69] [70] [71] integrate handcrafted future interactions to evaluate candidate planning profiles, offering remarkable outcomes in rule-powered reactive simulation [47] . Still, the absence of real-world behaviors exhibits significant gaps in interactive scenarios. Learning-based methods yield imitative planning by integrating predictions within holistic modeling [72] [73] [74] . However, history-based coalitions pose challenges in supervising future homology among agents. Recently, hybrid pipelines [8, 75] have utilized post-processing and optimization upon learning-based models to realize behavior interactions among predictions and planning. This can entail significant computational overhead, and imitative planning tends to overestimate hereditary uncertainty in behavior predictions. Tree-based [10, 27] and contingency-enabled [76, 11] works seek to balance planning preemption and aggression in the face of behavior uncertainty. Nonetheless, pipelines without holistic interactions fall into passive planning maneuvers and incur high exponential cost for predictions. In our work, BeTop provides an explicit prior for future interactive behaviors which enhances compliant trajectory generations. Moreover, the synergistic prediction and contingency planning networks with BeTop effectively manage behavioral uncertainty. 
behavioral topology: 
 Presenting BeTop, we commence the Behavioral Topology formulation and task statement of IPP for autonomous driving in Sec. 3.1. Then, we demonstrate the BeTopNet network architecture (Fig. 3 ) for topological reasoning and IPP generation in Sec. 3.2. Finally, in Sec. 3.3, we propose the imitative contingency learning process by topological guidance for the proposed network.Problem formulation. We consider the driving scenario with N a agents as A 1:Na at presence t = 0, along with the scenario map M. The states over historical horizon T h are denoted as X 1 for AV and as X 2:Na for scenario agents, respectively, whereX n = {x -T h :0 } n , n ∈ [1, N a ].The objective for integrated prediction and planning is to jointly predict scene agents' trajectories Y 2:Na as well as AV planning Y 1 over a future horizon T f as Topological formulation. We leverage the braid theory [46] , which probes explicit formulations for compliant multi-agent interactions from future data Y 1:Na . Intuitively, it denotes a transform process for Y 1:Na with respective agent coordinates, and then gathers each future forward intertwine (occupancy) as joint interactions. Formally, consider the braid group B Na = {σ n } by N a primitive braids σ n , each of whichY n = {y 1:T f } n , n ∈ [1, N a ].σ n = (f n 1 , • • • , f n Na) denotes a tuple of monotonically increased functions f : R 3 × Y → R 2 × I mapping from Cartesian ⃗ x, ⃗ y, ⃗ t to lateral coordinate ⃗ y, ⃗ t for agent future Y. Specifically, the function f n i in σ n is defined asf n i → (Y i -b n )R n ; 1 ≤ i, n ≤ N a, where b n and R n denote the left-hand transform matrix to local coordinate of agent A n . The joint interactive behaviors are identified as a set of braids having intertwines {σ ± n } ⊂ B Na over others [45] , as shown in Fig. 2 . Opposite to implicit methods [22, 67, 41] banking on future distance heuristic, each intertwine in the braid can signify an explicit behavioral response, distinguishing between assertive (σ + n , elicit yielding from others) and passive (σ - n , yield to others) maneuvers. To avert difficulties in dynamic braid set inference, we redraft multi-agent braids from a topology reasoning perspective.Named by BeTop, the goal is to reason a topological graph G = (V, E) for multi-agent future behaviors (Fig. 2 ). Expressly, node topology V = {Y n } is denoted by multi-agent future trajectories. We can then reformulate the braid set {σ ± n } as an edge topology e ij → E ∈ R Na×Na ; 1 ≤ i, j ≤ N a for future interactive behaviors. Each topology element e ij can be defined by two braid functions f i i , f i j ∈ σ i assessing the future interwines along with Y i , Y j as: e ij = max t I f i i (y t i ), f i j (y t j ) . Here I is an intertwine indicator by segment intersection [77] under lateral coordinates. With favorable properties proved in Appendix B, we can formulate the reasoning task as:EQUATIONAgent future Ŷ in node term V is defined by Gaussian mixtures (GMM) and optimized in Sec. 3.3.The edge topology reasoning Ê can be specified as a probabilistic inference problem by:EQUATIONwhere 1 ≤ i, j ≤ N a . Synergistic reasoning structures are then established optimizing G * .Comparative analysis. To highlight BeTop's position among various formulations, we first conduct a preliminary analysis to assess behavioral similarity by retrieving future interactive agent pairs using human annotations [35] . Human likeness is quantified by classification metrics, including accuracy and the area under the curve (AUC), with annotated interactive IDs. As depicted in Table 1 , labeled BeTop achieves the closest behavioral similarity compared with other well-accepted formulations in the community. Compared with retrieving k nearest strategy (k = 6) by ground-truth future states, we observe advanced differentiation in non-interactive behaviors (+16.1% Acc., +4.13% AUC) by BeTop. We then look into the generic learning-based structure by attention [22] or dynamic graph [67] for interactive behaviors. Despite high accuracy, their inferior AUC scores imply difficulties in retrieving precise interactivity compared with BeTop (+19.9 AUC). We refer analytical content in Appendix B. This draft for a reasoning framework BeTop prompting joint behaviors.As presented in Fig. Scene encoder. We leverage a scene-centric coordinate system following planning-oriented principle [7] . Scene attributes comprise historical agent states X ∈ R Na× T h ×Da and map polyline inputs M ∈ R Nm× Lm×Dm , where we portion N m map segments with length L m from full scene map. Both attributes are encoded separately as S A ∈ R Na×D and S M ∈ R Nm×D and concatenated as scene features S = [S A ; S M ] ∈ R (Na+Nm)×D . A stack of Transformer encoders with local attention are directly employed in capturing regional interactions from encoded scene semantics S A , S M . Synergistic decoder. Retaining encoded scene features S A , S M , we zoom in the decoding strategy that asks for: 1) interactively reason simultaneous BeTop formulations; 2) selectively decoding of compliant interactive semantics leveraging reasoned topology priors. To this end, we introduce the iterative process of N Transformer decoder layers contributed to all agents, pursuing the basis from [78] . To iron out the scene uncertainties, a multi-modal set of M decoding queries Q 0 A ∈ R M ×D are initialized for multi-agent future trajectories. Meanwhile, relative attributes S R ∈ R Na×Na×D R are deployed through MLPs as topology features Q 0 R ∈ R Na×Na×D for edge topology reasoning. Next, we devise dual infostreams to the iterative decoding process for V of future trajectories and Ê of future topology. Given agent A n , the decoding process in layer l follows:EQUATIONEQUATIONwhere both future trajectories Ŷn ∈ V and interactive topology ên ∈ Ê in BeTop are decoded in synergistic manners. Reasoned edge topology êl n ∈ R M ×Na are garnered by topological decoder with query broadcasting Q l-1,n A ; Reasoning nodes for Ŷn , a Transformer decoder with topology-guided local attention are drafted serving êl n as priors. We provide further details in Appendix C.1. Topology-guided local attention. Querying whole-scene agent semantics results in misaligned interactive agents and sparse attention. This motivates our design for local attention guided by the reasoned topology êl n ∈ R M ×Na as priors. Specifically, we retrieve the top-K index ϵ l n ∈ R M ×K priored from êl n for eventual interactive agents behaviors with A n . Interactive indices are directly leveraged in gathering S A selectively for local cross-attention. This process is formed as:EQUATIONwhere ϵ l n = argmax K (ê l n ). Topology-guided agent features C l,n A are then aggregated in each layer.Reason heads. Given respective decoding features Q l,n R and Q l,n A for each layer, we affix reason heads accustomed to corresponding formulations for ên and Ŷn . Referred in Eq. ( 3), the topology head, planning head, and prediction head (IPP heads) are jointly devised by stacked MLPs in reasoning BeTop results. For agent A n in each layer, reason heads decode GMM components of future states ŷn ∈ R M ×T f ×5 (referring to (µ x , µ y , log σ x , log σ y , ρ) per step) with mixture score pn ∈ R M ,{ŷ n , pn } ∈ Ŷn , as well as interactive edge topology êl n ∈ R M ×Na for BeTop.Pursuing the target in Eq. ( 1), BeTopNet learns end-to-end objectives imitating human-like multi-agent behaviors, integrating compliant behaviors by contingency planning under scenario uncertainties.Imitation learning. Imitation objectives are firstly established in regulating multi-agent behavioral states { Ŷn } ⊂ V while maximizing their interactive distributions Ê. The imitative objective for Ŷ is defined by the negative log-likelihood (NLL) from best-reasoned components m * closest to ground-truths, as denoted: 2), the behavioral distributions for edge topology are computed by binary cross-entropy (BCE) given gathered êm * n ∈ R Na , formulated asL V = T f t L NLL (ŷ m * ,t n , pm * n , Y n ). Followed Eq. (L E = Na j H(ê m * n,j , e n,j) over N a agents jointly.Integrated contingency planning. To integrate compliant behavior learning for G amidst multiagent scenario uncertainties, contingency planning [79, 76] is turned out an apt solution. Bridging immediate safe maneuvers τ M to branched planning sets {τ J } with joint prediction, it adjourns uncertain decisions and ensures actual safety. While direct joint prediction may lose diversity [11] , reasoned topology Ê serves as a suitable medium distilling future interactive agents for efficient joint combination. Given imitative AV planning outputs τ ⊂ Ŷ1 with branching time t b ∈ (1, T f ), integrating contingency learning asks for a safe short-term planτ M ∈ T M , T M ∈ R M ×t b ×2 to full marginal predictions ŶM = Ŷ2:Na , as well as M branched planning sets T m J = {τ 1:M b J } m guided by joint predictions Ŷ mJ . This is defined by:EQUATIONwhere max Ŷ C M denotes worst-case cost fir τ M ; Joint predictions ŶJ with scene probabilities P ( ŶJ ) are recombined by K M interactive agent subsets, indexing ϵ AV ∈ R K M from sorted AV topology: ϵ AV = argmax K M (max M ê1 ). It is described by joint costs C J in guiding branched planning maneuvers. Specifically, both cost functions are defined by the repulsive potential field [8] discouraging planning proximity with respective prediction formulations.Training loss. BeTopNet is trained end-to-end through imitative objectives and contingency planning costs by weighted integration for each layer, whenever applicable (for the datasets). Please refer to Appendix C.2 for additional details. 
experiment: 
 With preliminary analysis in Sec. 3.1, this section further discovers the following questions: 1) Can BeTop perform compliant planning via BeTopNet, especially in interactive scenarios? 2) Can BeTop achieve accurate marginal and joint predictions of heterogeneous agents under diverse real-world cases? 3) Can the formulated BeTop facilitate existing state-of-the-art prediction and planning methods? and 4) How do the functionalities in BeTopNet affect the performance?Benchmark and metrics. BeTop is verified on diverse benchmarks. We leverage two large-scale real-world datasets, i.e., nuPlan [47] and Waymo Open Motion Dataset (WOMD) [35] , which are presently the most diverse motion datasets in manifesting planning and prediction performance. For planning tasks in nuPlan, there are in total 1M training cases with 8s horizons. 8,300 separated testing set are chosen by Test14-Hard and Test14-Random benchmarks [73] for hard-core and general driving scenes. With further demands verifying maneuvers under interactive cases, we build the Test14-Inter benchmark filtering 1,340 scenes by testing set. Scenarios ranging 15 seconds are tested under three tasks: 1) open-loop (OL), 2) close-loop non-reactive (CL-NR) simulations, and 3) reactive (CL-R) ones by nuPlan simulator. We report the official Planning Scores [80] computed by each task. The motion prediction tasks in WOMD share 487k training scenarios, with 44k validation and 44k testing set separately partitioned under two challenges: 1) The Marginal prediction challenge [81] forecasting multiple scene agents independently; 2) The Joint prediction challenge [82] predicting joint trajectory collections by two interactive agents. Primary metrics of mAP and Soft mAP are ranked for official leaderboards [81, 82] . We leave experimental details in Appendix D.Performance for interactive planning. [53, 86] for compliant predictions. Exhibited strong prediction displacement metric (-4.3% minFDE) over methods using extra pretraining [85] , it should be noted that displacement metric is less illustrative as it discounts uncertainty scoring. BeTopNet further outperforms +6.0% and +26.1% Soft mAP over multi-agent predictors solely leveraging scenario attention [25] or graph [24] . N. [32] , boosting +25.4% mAP with game-theoretic methods [8] by a large margin. Fig. 5 (e-h ) demonstrates the qualitative prediction performance by BeTopNet. At the time of submission, BeTopNet ranked 1 st on both WOMD prediction leaderboards [82, 81] .Instructed by the last two motivating questions, we investigate the effect of BeTop formulations and components inside BeTopNet. For efficient study, we randomly partition 20% of WOMD train set for prediction, and directly report the planning results by Test14-Random benchmark, which are both representative for the original datasets as verified by [22, 73] .Synergy with existing state-of-the-art methods. We first study the effect adjoining BeTop as synergistic objectives over existing SOTA methods in planning and prediction. Described in  
conclusion: 
 In this paper, we present BeTop, a topological new-look for multi-agent behavioral formulation. Derived by braid theory, the reasoning tasks for BeTop are drafted supervising joint interactive patterns with integrated prediction and planning. A synergistic network, BeTopNet, is established with an imitative contingency learning process to boost compliant BeTop reasoning. Experiments on nuPlan and WOMD verify BeTopNet's state-of-the-art performance in prediction and planning.Limitation and Future work. Current BeTop considers one-step future topology alone, and focuses on prediction and planning. Future work would be centered on developing a recursive version of BeTop in multi-step, multi-agent reasoning and coordination. Another promising direction would be the connectivity of BeTop upon perceptions as tracking for the end-to-end paradigm, as well as an extension on reasoning behaviors under 3D scenarios for multiple autonomous agents. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the papers for the used datasets and models in the paper, and list corresponding licenses in Appendix H.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Towards a better understanding of this work, we supplement intuitive questions that may raise. Note that the following list does not indicate the manuscript was submitted to a previous venue or not.Q1: How does BeTop bridge and discern with dense, sparse, and topological representations?BeTop is derived from braid theory [46] , reasoning the topology that explicitly labels consensual interactions as dense, occupancy-like intertwines from sparse, braided future trajectories of multiagent behaviors. Unlike fixed occupancy [14, 21, 18] , BeTop dynamically forecasts behavioral interactions by agents collectively, serving as a guiding medium for node reasoning in joint planning and prediction compared to standard sparse predictions [25, 24, 66] . Differentiating from typical topological approaches, BeTop explicitly formulates coordinated multi-agent behaviors and reasons topology in occupancy manners, rather than relying on implicit relation graph learning [41, 67, 40] or complex braid inference [44, 45, 61] .Q2: Why is BeTop allied with contingency instead of conditional or game-theoretic reasoning?Contingency plays the most suitable role in BeTopNet, addressing multi-agent uncertainty by deferring uncertain planning with long-term joint predictions from interactive agents under behavioral compliance. This aligns with BeTop's reasoning targets, which aim to achieve a one-shot consensus among all behaviors by formulating future behavior as coordinated joint interactions. This synergy alleviates the challenges led by game-theoretic reasoning [8, 33] or conditional integration [31, 28, 32] , which are struggled by multi-step interactive rollouts and unstable joint behavioral patterns.Q3: What would be the broader impact and future direction?BeTop steps the first trial towards an explicit topological formulation and reasoning paradigm for multi-agent interactive behaviors. This serves as a basis in exploring immense interactive behaviors in the real-world, and is reasoned by the autonomous agents jointly in a coordinated manner. For instance, BeTop with enlarged scalability and dimension may summarize the topology for all behaviors in 3D scenarios or even larger spaciality, and may reasoned through end-to-end BeTopNet as a foundation behavioral model. Moreover, we can consider BeTop's capability as collective maneuvers, which can be further leveraged in coordinating naturalistic and efficient decision-making for multiple autonomous agents.In this section, we supplement additional properties that characterize the formulation of Behavioral Topology (BeTop). Analytically, BeTop is highlighted with: 1) geometric invariant (Theorem B.1); 2) approximated topological invariant (Theorem B.2); and 3) asymmetric topology (Theorem B.3). Theorem B.1 (Geometric Invariant). The topology results of E ⊂ G in BeTop remain unchanged given arbitrary geometrical transformations for the collective scene trajectories Y n .Proof. Given arbitrary rotation H ∈ R 2×2 and shifting b ∈ R 2 . Consider the mapping g for elementary topology e ij ∈ E from future trajectories(Y i , Y j ), i ̸ = j ∈ [1, N a ], the local transformation f : g → h • f to i's coordinate is invariant, such that h(f (Y i , Y j )) = h(f (HY i + b, HY j + b)).Hence, given the function sets f i j ∈ σ i ⊂ f, I ∈ h defined in Sec. 3.1; e ij ∈ E is also invariant. Remark 1. The Theorem B.1 proves the behavioral stability of BeTop given arbitrary multi-agent trajectories patterns for planning and predictions. Any rotations and movement of the original scene will not interfere with the formulated results of BeTop. Definition B.1 (Topological Invariant). Given future trajectory pairs(Y i , Y j ), i ̸ = j ∈ [1, N A ]with certain current heading (θ 0 i , θ 0 j ), the sum of future relative angles (winding number)w ij = 1 2πT f 0 ∆θ t ij form its first-order [87] topological invariant.Proof. Consider the polar representation for the closed form ψ i (t) = ||ψ i (t)||e iθi(t) . Whereψ i : [0, T m ] → C\{0}, i ∈ [0, n],we can define the winding function λ i (t) = 1 2πi ψi dz/z, z = ψ i (t), t ∈ [0, T m ]. This Cauchy formula [2] can be further integrated as:EQUATIONWe are interested in the real (first-order) part of λ i (t) which is an invariant topologically. Hence, the trajectory pairs (Y i , Y j ), 0 < t ≤ T f < T m can be described as:Y i = ψ i : (0, T f ]. The joint invariant across future w ij = T f t (λ i (t) -λ j (t)) then becomes:EQUATIONAs the current heading pair (θ 0 i , θ 0 j ) is certain, the invariant becomesw ij = 1 2π T f 0 ∆θ t ij which proofs the definition. Corollary B.1.1. Given any ∆θ t ij ∈ [-π 2 , π 2 ], where 0 < i.j < N a , t ∈ (0, T f ], and the constantη i , η j ∈ R, the transformed w T ij = T f 0 (η j sin ∆θ t ij -η i sin θ t i ) is also topological invariant.Proof. The defined function sin(•) is a monotone mapping under [-π 2 , π 2 ]. Hence, this firstly enables T f 0 η i sin θ t i uniquely defines Y i . More than that, w T ij is the unique mapping value of w ij with Y i under defined transformation, and thereby keep the invariant property.Theorem B.2. The edge topology E ⊂ G is an approximate of topological invariant, so that e ij ∈ E, 0 < i, j ≤ N a is characterized by w ij .Proof. Given future trajectories Y i , Y j , we consider the braid functions σ i maps monotonically increased transformations f i i , f i j to i's local coordinate,as defined in Sec. 3.1. We assume a continuous future horizon (0, T f ] where the headings for f i j (Y j ) is defined by relative angles ∆θ ij (t). Thereby, the transformed lateral trajectory for agent j can be formed as: t 0 η j sin ∆θ ij (t)dt. Similarly, f i i (Y i ) can be formed as t 0 η j sin θ i (t)dt, where η i , η j denotes small constant step lengths.These form the original intersection function I in Sec. 3.1 as:EQUATIONwhere I(•) = 0 denotes braid intertwine for interactive behaviors. As the term in Corollary B.1.1 of w T ij (the sum of the right term) is an approximate (discretization) of the right term, this proves the edge topology e ij ∈ E → max T f I (•) as the approximation of topological invariant.Remark 2. The Theorem B.2 proves the generality of BeTop in terms of future interactive behavioral patterns. The approximated topological invariant property prompts a representative of various future states sharing similar behavioral or identical interactive patterns by BeTop.Edge topology E ⊂ G is not symmetric such that ∃i, j, e ij ̸ ≡ e ji , 0 < i, j ≤ N a .Proof. Given Eq. ( 9) defined in Theorem B.2,we can always construct a case ∃t i , t j ∈ (0, T f ], the intersection I f i i (y ti i ), f i j (y ti j ) = 0, but I(f j i (y tj j ), f j i (y tj i )) ̸ = 0, which prove the claim.Remark 3. The Theorem B.3 proves a more naturalistic interactive behavior of BeTop. It is likely in a real-world scenario that the future behavior of agent A i is interacted by agent A j , while A j does not.Computational complexity. The complexity in computing full E is O(N 2 a ). In practice, we downscale the sourced as the agents of interests N I < N a , such that O(N I N a ). It is much less than braid sequence inference [45] with maximum O((N a -1) Na ) computational costs. Further analytical proof of computational efficiency leveraging braids can be found in [44] . In this section, we instantiate further details for BeTop on the configurations for BeTopNet structure, and provide contingency learning paradigms for both prediction and planning challenges.Subjecting to different testing requirements defined in nuPlan and WOMD, we set up two model variants for BeTopNet in formulating planning and prediction challenges. Apart from topology reasoning for interactive behaviors, for planning challenges, BeTopNet integrates both tasks of prediction and planning. In the prediction challenges under marginal and joint settings, BeTopNet is allocated only with the prediction parts. The structural details are illustrated below.Scene inputs. Carving the driving scenarios involves historical agent states X and map polylines M as scene inputs. For planning settings, we collect scene agent states with past T h = 2 seconds at 10Hz, leaving basic kinematic states as (x, y, v x , v y , θ), joining with agent shapes and types. We only keep the current state for ego vehicle (AV) in preventing closed-loop gap [73, 69, 88] with open-loop training as recently discussed in the community. N m = 256 segments of the map with length L m = 20 are gathered by scene-centric manners considering positions, traffic lights, and speed limits. The prediction task is built on WOMD considering T h = 1 seconds at 10Hz with scenarios of larger scalability. It is followed by full scene agents with N m = 768 map segments of identical states for both X and M.Scene encoder. Both scene attributes X, M are firstly encoded leveraging layered point encoder [89] to hidden dimension D shared throughout the BeTopNet structures, with D = 128 for planning and D = 256 for prediction tasks. A stack of Transformer encoders is then devised with 4 layers in planning and 6 in prediction for S A , S M . Due to scalable settings for prediction tasks, local attention with the nearest 16 keys is built in each layer. Following [22, 73] , the dense prediction head is adopted for all agents after the encoder, enhancing future semantics.Synergistic decoder. Depicted in Fig. 6 , the decoder structure is founded by an iterative stack of L Transformer decoders querying M modes of future trajectories Ŷ with dual stream in reasoning topology Ê. Consisting of L = 6 and L = 4 decoders for prediction and planning respectively, it is initialized jointly by relative features Q 0,n R and decoding queries Q 0,n A . Relative attributes S R are computed efficiently following [64] for relative distance and headings. M = 6 learnable embedding are devised as Q 0,n A for planning, and we utilize the anchored ones [22] with M = 64 in prediction tasks. As displayed in Fig. 6 (a), dual queries Q j-1,n A , Q j-1,n R are served from the last level. The decoding process (following Eq. ( 3)) iteratively reasons êl n from TopoDecoder, serving as a prior guiding agent semantics by TopoAttn inside TransDecoder, which concurrently aggregates scene semantics from agents S A and maps S M . Expressly, the structure of TopoDecoder (Fig. 6(b) ) comprises simple MLPs and update Q l,n R by concatenation sourcing query features Q j-1,n A with agent semantics S A , and connect residuals Q l-1,n R from last layer. Decoding queries Q l,n A is updated by a concatenation from aggregated agent feature C l,n A , map features C l,n M , and agent semantics S n A directly from encoder. Agent feature C l,n A is aggregated by TopoAttn, where the local attention is devised using deployments from [90] indexing K = 32 agents from reasoned topology êl n . We omit the aggregation process for map features, which performs the vanilla Transformer decoder structure for planning and dynamic collection form by [22] under prediction tasks for hefty map features.Reason heads. Following the contents in Sec. 3.2, reason heads in decoding prediction and topology follow simple MLPs given respective decoding queries. For the planning head, it leverages a cascaded design for branched contingency planning with multi-modalities (Fig. 6(c) ). Specifically, the short-planning τ M is decoded by the AV future states {ŷ 1:t b 1 } m from first stage head with m ∈ M modes, where t b = 3 denotes the branching time. They are then detached and leveraged as prior for the branched planning. Successive MLPs project and reshape the short-term contingency prior by R M ×M J ×D for M J = 6 branches planning T m J under each of τ m M . Further concatenated by broadcasted decoding queries, the planning head generates M • M J trajectories Ŷ1 for AV.Efficient joint prediction recombination. Retrieving the top-performed joint predictions from full marginal predictions ŶM sequentially is time-consuming with exponentially complexity. Hence, we firstly downscale the potentially interested agents N I by sorting the AV-reasoned topology êL 1 with the largest K M = 4 value as index given the planning task. For the joint prediction task, N I = 2 is annotated already from the original data. Then, we leverage the tensor broadcasting mechanism in efficiently retrieving M = 6 largest joint distributions P ( Ŷ M J ) and joint trajectories Ŷ M J from N I interacted agents. Given a tensor P J ∈ R M N I initialized by ones, the joint score is computed by N I times of iterative broadcasting pn on the n-th dimension for P J as P J = max M N I n P (n) J ⊗ pn . This process only costs 1.6ms in computing N I = 4 joint predictions for contingency planning.Imitative contingency objectives. Followed by learning objectives derived in Sec. 3.3, the imitation objectives for each layer can be represented as L IL = L V + λ 1 L E . λ 1 = 50 weighting BCE loss for edge topology reasoning, the NLL loss for L V is formulated as:L NLL = log σ x + log σ y + log(1-ρ 2 ) 2 + 1 2(1-ρ 2 ) dx σx 2 + dy σy 2 -2ρ dxdy σxσy-log p(m * ), (10) where d x , d y denotes the difference with ground-truths. In determining the component m * , we leverage a winner-take-all (WTA) strategy [91] in planning by measuring the average displacements (ADE) with groung-truths. For prediction tasks, m * is selected from the closest anchor as in [53] . For the learnable cost functions max C M (•), C J (•) in contingency planning, we leverage the repulsive potential field [8] delineating planning with prediction by ϕ = min d 1/(1+d(τ, ŷ)). For max C M (•), ϕ is gathered across T f considering the worst case under full marginal prediction ŶM comprising N a = 32 scene agents. For the branched cost C J (•), ϕ for each branch is computed considering joint prediction from N I = 4 agents. Following the objective defined in Eq. ( 6), the learnable contingency cost is defined as:L CL = C M + M m P ( Ŷ m J )C m J .Hence, the general objectives for planning become:EQUATIONwhere λ 2 = 5 is the contingency costs weight. Prediction tasks are updated only by L IL .Inference. Different from the training process, for the planning task we directly select the full planning trajectory of T f = 8 seconds by highest scoring τ * = argmax C Ŷ1 , subjecting to the original task settings in nuPlan. The scoring results are a combination from original confidence p1 and the short-term cost C M [69] : C = p1 + λ m C M , where λ m = 0.5 facilitates short-term planning compliance [92] . For the prediction task, a post-processing module following [53] is leveraged in selecting M = 6 marginal or joint trajectories of T f = 8 seconds among 3 agent types in WOMD.In this section, we provide extra details demonstrated in Sec. 4 for the experiment setups, including detailed settings for the proposed benchmark, testing metrics, state-of-the-art baselines, and training.Testing metrics. For open-loop planning tests, the open-loop score (OLS) serves as the general statistics weighting displacement metrics and miss rates. For closed-loop simulations, both metrics (CLS, CLS-NR) are weighted by a series of statistics measuring 1) driving safety, 2) planning progress, 3) driving comforts, and 4) rule obeying. The PDMScore [93, 94] compared in Table 3 is basically a replica of the closed-loop score for efficient computations. It is denoted as:EQUATIONwhere the sub-metrics are abbreviations referred in Table 3 . All general metrics range from 0 to 1.Test14-Inter. We launch the Test14-Inter benchmark in verifying the planning systems under typical corner cases containing rich social interactions, or dynamic profiles by complex map forms. This is highly motivated by the issues raised in [69, 88] , that massive scenarios may also be completed by a simple motion model. Specifically, we adopt a mining heuristic defining corner cases by which human experts excel but the motion model (constant acceleration vehicle, CAV) fails. For efficient mining, we directly assess planning results by PDMScore and define the criteria as:EQUATIONwhere γ = 0.1 denotes a scoring threshold for cases that cannot be easily solved by regular motion profiles of the planning maneuvers. As future work, we aim to explore more interactive scenarios aggravating by BeTop as an enhancement.Val14. In pursuing comprehensive comparisons with current methods, we also manifest BeTopNet in the Val14 set proposed in [69] . It is a subset of 1040 scenes from the validation set. However, since a portion of validation scenes are shared with the training set in nuPlan [80] , we argue this is less representative of testing fairness for learning-based methods. Hence, we only place it as supplementary.Baselines. For all baselines presented in the planning task, we directly report their previous benchmark results. Additional results in the proposed benchmark (Table 3 ) and ablation studies (Table 6 ) are re-implemented by the official releases [69, 72, 47, 69] . Expressly, we study the state-of-the-art planning systems categorized by: 1) Rule-based: performing maneuvers by designate rules with the reactive agents [70] or mimicking the planning score [69] ; 2) Hybrid: incorporating rules [69] or post-optimizations [8] with a learning-based model; and 3) Learning-based: end-to-end planning with GNN [83, 74] or Transformer [72, 73] enabled models, as well as concurrent methods [92] augmented by representation learning. For ablation studies in Table 6 , BeTop is trained directly by the proposed topology decoder with the original PlanT [73] pipeline. For the PDM [69] as a rule-based planning system, we integrate BeTop by replacing the original rule-based motion model with predictions generated from BeTopNet.Testing metrics. For the prediction task, the mean AP (mAP) and Soft-mAP scores are assigned as the primary metrics in computing multi-modal predictions modeled by marginal or joint distributions [35, 81, 82] . Displacement metrics of minADE and minFDE provide the multi-modal prediction errors closest to ground truths without considering the prediction scores.Baselines. We also directly provide the prediction results displayed on the official leaderboards in Tables 4 and 5 . Ablation studies in Table 7 are reproduced by the official codes [22, 53] . The prediction performance of BeTopNet is compared against SOTA baselines by: 1) GNN-enabled interactive graph [37, 24] ; 2) conditional or game-theoretic behavioral interactions [31, 8] ; 3) DETRbased Transformer attentions [22, 25, 53, 86] ; and 4) auto-regressive modeling [32] . Temporal granularity in BeTop. Table 16 explores the effect of varied temporal granularity in BeTop, with minimal adjustments to BeTopNet. In our study, future interactions are split into multistep BeTop labels. Topology reasoning task is then deployed through expanded MLP Topo Head for output steps. Compared to the baseline 1-step reasoning, multi-step BeTop reasoning slightly improves performance (e.g., 2-steps, +0.2 mAP), with a corresponding increase in computational costs for additional steps. This highlights the potential of multi-step reasoning to enhance BeTopNet in interactive scenarios, while refining temporal granularity for more accurate and efficient interactions remains an open question. We believe how to effectively leverage multi-step BeTopNet represents an interesting area for future exploration.Synergy with additional model foundation. To understand the generalization under different model foundations, we conduct additional ablations integrating BeTop with reproduced Wayformer [62] in [95] . As reported in Additional planning results. We provide the qualitative closed-loop simulations for all of the benchmarks in Test14, as shown in Figs. 7 and 8 .Additional prediction results. We provide the qualitative prediction results for BeTop with reasoned edge topology under both marginal (Fig. 10 ) and joint (Fig. 9 ) challenge settings.Data for nuPlan [80] and WOMD [35] are complied with CC-BY-NC 4.0 licence and Apache License 2.0; The code for re-implementations are under Apache License 2.0 for PDM [69] and MTR [22] , and MIT License for EDA [53] and PlanTF [73] , respectively. The source code and our trained models will be publicly available under the Apache License 2.0. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: This paper introduces Behavioral Topology (BeTop), a novel approach to modeling multi-agent behaviors in autonomous driving using braid theory. The authors present BeTopNet, a framework that incorporates topological reasoning into a synergistic learning model for both behavioral prediction and planning.

The reviewers generally agree on the innovative nature of the approach. Reviewer PWLC notes that "the use of braid theory to represent multi-agent interactions is innovative and provides a solid theoretical foundation," while Reviewer aQzs states that it "lays a solid mathematical foundation for BeTop." The comprehensive evaluation is another strong point, with Reviewer PWLC highlighting the "extensive experiments on large-scale real-world datasets" and Reviewer PsGi confirming that these experiments "demonstrate that BeTop achieves state-of-the-art performance in prediction and planning tasks."

However, the paper has some limitations. Reviewer PWLC points out that it "lacks an in-depth discussion on how this method scales and handles various multi-agent settings." A major concern raised by multiple reviewers (M7XC, PsGi, and aQzs) is the use of only one-step topology. Reviewer M7XC questions, "Could you provide some basis to the idea why one step topology will be enough?" Reviewer M7XC also suggests benchmarking against prior work using braid topology for planning to provide a clearer picture of the method's contribution.

The presentation of the paper received mixed reviews. While most reviewers found it well-organized, Reviewer M7XC notes that "The paper is generally not well-written, with extensive use of ChatGPT leading to paragraphs that are hard to follow."

Several important questions were raised by the reviewers. Reviewer PsGi asks how BeTop is used during inference when future trajectories for surrounding agents are unavailable, and how the robustness of BeTopNet is ensured in highly dynamic and unpredictable driving environments. Reviewer PWLC suggests providing a clearer explanation of how the method integrates with different numbers and types of agents.

Recommendation:
Despite the concerns raised, the paper's innovative approach and comprehensive experiments are seen as significant contributions to the field. Three out of four reviewers recommend acceptance (two "Accept" and one "Weak Accept"), while one reviewer gives a "Borderline Accept."

Based on these reviews, the recommendation is to accept the paper, contingent on the authors addressing the following key points in the camera-ready version:

Provide a more detailed discussion on multi-step topologies and their potential benefits.
1. Clarify how the method scales to various multi-agent settings.
2. Include a comparison with existing methods, particularly those using braid topology for planning.
3. Address the questions raised about inference and robustness in dynamic environments.
4. Improve the clarity and readability of the paper, addressing the writing concerns raised by Reviewer M7XC.

Reviews: review: paper_summary: paper_summary: This paper introduces a new approach, called Behavioral Topology (BeTop), to address the challenges in modeling multi-agent behaviors in autonomous driving. By utilizing braid theory, BeTop explicitly represents the consensual behavioral patterns among multiple agents, facilitating better prediction and planning. The framework, BeTopNet, incorporates this topological reasoning into a synergistic learning model that guides both behavioral prediction and planning. Good experiments on large-scale datasets, such as nuPlan and WOMD, demonstrate the superior performance of BeTopNet in both prediction and planning tasks, showcasing significant improvements over existing methods.
strength_weakness: strength_weakness: Lack of Discussion on Multi-Agent Settings: While the paper introduces a topological approach for multi-agent behavior modeling, it lacks an in-depth discussion on how this method scales and handles various multi-agent settings. More insights into the limitations and potential scalability issues would strengthen the paper.

Formulation for Multi-Agent Settings: The paper could benefit from a more detailed formulation of the multi-agent setting. While the braid theory is used to model interactions, a clearer and more comprehensive explanation of how this integrates with different numbers and types of agents would be helpful.
questions: questions: Lack of Discussion on Recursive Settings: While the paper introduces a topological approach for multi-agent behavior modeling, it lacks an in-depth discussion on how this method scales and handles various steps of multi-agent settings. More insights into the limitations and potential scalability issues would strengthen the paper. While the braid theory is used to model interactions, a clearer and more comprehensive explanation of how this integrates with different numbers and types of agents would be helpful.
limitations: limitations: Yes.
paper_summary: paper_summary: The paper addresses the challenges of autonomous driving by integrating behavior among interactive agents, specifically focusing on issues caused by multi-agent scene uncertainty and heterogeneous interactions.  To tackle this, the paper introduces a topological formation called Behavioral Topology (BeTop), derived from braid theory, to represent consensual behavioral patterns among multiple agents. This formulation guides downstream trajectory generations and enhances the consistency of behavior prediction and planning.
strength_weakness: strength_weakness: The paper is generally not well-written, with extensive use of ChatGPT leading to paragraphs that are hard to follow.

- As far as I understand, the paper uses braid topology for just one step in planning and prediction. In long-horizon planning and prediction, this may not be sufficient as the motion of vehicles is  no longer independent. Could you provide some basis to the idea why one step topology will be enough? 

-  There has been prior work using braid topology for planning (https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9812118). Benchmarking your method against this prior work would provide a clearer picture of this method. 

- The method appears very similar to existing methods like Wayformer, with the main difference being the use of braid topology. I would like to see a detailed comparison showing how much the encoding of braid topology improves performance compared to Wayformer, especially given that only one-step braid topology is used instead of long-horizon topology.

Overall, the paper has potential, and I would be happy to discuss more with you. I would be willing to increase my score if my questions and concerns are addressed satisfactorily.
questions: questions: See weaknesses
limitations: limitations: N/A
paper_summary: paper_summary: The paper introduces a novel approach to enhance the safety and social consistency of autonomous driving systems through improved multi-agent behavioral integration. To address inefficiencies and inconsistencies in current behavioral representations, the authors propose Behavioral Topology (BeTop), a topological framework derived from braid theory that captures consensual behavioral patterns among multiple agents. This framework guides downstream trajectory generations and ensures stable collective behavior when integrating prediction and planning. The paper also presents BeTopNet, a synergistic learning framework supervised by BeTop that manages behavioral uncertainty and enhances prediction and planning consistency. Extensive experiments on large-scale real-world datasets, including nuPlan and WOMD, demonstrate that BeTop achieves state-of-the-art performance in prediction and planning tasks, showcasing its effectiveness in interactive scenarios
strength_weakness: strength_weakness: In general, the paper is well written and there are no major weakness, however, there are some aspects that can be further discussed in the paper: 1.While the paper demonstrates effectiveness on specific datasets, it remains uncertain how well the method generalizes to diverse driving environments and conditions not covered in the training data. 2. The computational overhead associated with the topological framework and synergistic learning might be higher compared to simpler models, possibly affecting real-time performance.
questions: questions: 1. It is unclear how is BeTop used during inference when future trajectories for surrounding agents are unavailable.

2. It is unclear how are $Q_R$ and $Q_A$ initialized and defined.

3. How do you ensure the robustness of BeTopNet in highly dynamic and unpredictable driving environments, such as those with sudden changes or unexpected behaviors from other agents?
limitations: limitations: Please refer to the weakness section.
paper_summary: paper_summary: The paper presents a approach to address the challenges of autonomous driving in multi-agent scenarios and heterogeneous interaction. It introduces the concept of Behavioral Topology (BeTop) and its corresponding network BeTopNet. BeTop is based on braid theory and aims to provide a topological representation of multi-agent interactions to enhance the prediction and planning of autonomous vehicles (AVs). The proposed method focuses on creating a compliant behavioral pattern among agents, which guides the trajectory generation for AVs. Extensive experiments on large-scale datasets such as nuPlan and WOMD demonstrate the good performance of BeTop in both prediction and planning tasks.
strength_weakness: strength_weakness: * The proposed method infer one type of agent behavior topology from one mode of future trajectories (e.g., 8s), while the topology of multi-agent interaction for real-world autonomous driving is usually multi-modal and dynamic. A discussion of modeling multi-step and dynamic topologies of vehicles for a long horizon would be beneficial.
questions: questions: * The proposed method infer one type of agent behavior topology from one mode of future trajectories (e.g., 8s), while the topology of multi-agent interaction for real-world autonomous driving is usually multi-modal and dynamic. A discussion of modeling multi-step and dynamic topologies of vehicles for a long horizon would be beneficial.
limitations: limitations: * The current implementation of BeTop considers only one-step future topology. Extending this to multi-step reasoning could provide more robust predictions and planning.

Comments: title: Response to the Reviewer
Response
Response to the Reviewer
Response to the Reviewer
Response to the Reviewer
Thank for your rebuttal.
comment: Thank you for the kind feedback. Your time and effort in reviewing our work are truly appreciated! We will revise the manuscript according to your valuable comments. During the remaining author-reviewer discussion period, we would be glad to provide further clarifications for any concerns you may have.
Thanks for the detailed response. I do not have other questions, though some of my concerns remain. But in general, I see fit to accept this work.
Thanks for the feedback and raising the score! We do appreciate your helpful review and will update the paper accordingly.
Thank you for the response and recognition! We appreciate your valuable review for improving our work.
Thanks for your feedback and raising the score! We will integrate your insightful comments in our revision accordingly.
I thank the authors for the extra results and clarification and I would raise my score.
Hi Authors, 

Thank you for the clarification and doing experiments within this short time. I am increasing my score. All the best!
Thank you for the clarification and additional information. I will remain my score.