Drones Help Drones: A Collaborative Framework for Multi-Drone Object Trajectory Prediction and Beyond

Authors: zhechao  wang
peirui  cheng
mingxin  chen
pengju  tian
zhirui  wang
xinming  li
xue  yang
xian  sun
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 13:51:11

Abstract: Collaborative trajectory prediction can comprehensively forecast the future motion of objects through multi-view complementary information. However, it encounters two main challenges in multi-drone collaboration settings. The expansive aerial observations make it difficult to generate precise Bird's Eye View (BEV) representations. Besides, excessive interactions can not meet real-time prediction requirements within the constrained drone-based communication bandwidth. To address these problems, we propose a novel framework named "Drones Help Drones" (DHD). Firstly, we incorporate the ground priors provided by the drone's inclined observation to estimate the distance between objects and drones, leading to more precise BEV generation. Secondly, we design a selective mechanism based on the local feature discrepancy to prioritize the critical information contributing to prediction tasks during inter-drone interactions. Additionally, we create the first dataset for multi-drone collaborative prediction, named "Air-Co-Pred", and conduct quantitative and qualitative experiments to validate the effectiveness of our DHD framework. The results demonstrate that compared to state-of-the-art approaches, DHD reduces position deviation in BEV representations by over 20\% and requires only a quarter of the transmission ratio for interactions while achieving comparable prediction performance. Moreover, DHD also shows promising generalization to the collaborative 3D object detection in CoPerception-UAVs.

Main paper: 
 introduction: 
 Multi-drone object trajectory prediction [1] [2] [3] aims to collaboratively identify objects and forecast their future movements using multiple drones with overlapping observations, which can overcome the single-drone limitations of occlusions, blur, and long-range observations. Given the safety and reliability of task execution, the role of multi-drone object trajectory prediction is indispensable. It contributes to early warning of potential accidents and path planning in drone operations, better serving for intelligent cities [4] , transportation [5] , aerial surveillance [6] and response systems [7] .Current methods in collaborative object trajectory prediction are primarily categorized into two frameworks: multi-stage prediction [8, 9] and end-to-end prediction [10, 11] . The multi-stage paradigm achieves collaborative prediction based on individual perception results. Specifically, it begins with object detection in each view, such as oriented object detection [12] . Then, the multi-view objects are correlated, and the generated trajectories are fed into a regression model for predictions. In contrast, end-to-end approaches transform each view's 2D features into BEV features at a unified 3D coordinate system and conduct feature-level correlation. Subsequently, future instance segmentation and motion are jointly predicted. The end-to-end paradigm significantly enhances accuracy and computational efficiency compared to multi-stage detect-track-predict pipelines, leading to its widespread popularity and application.Nevertheless, end-to-end approaches are primarily designed for autonomous driving scenarios and may not directly apply to aerial perspectives. Specifically, they adopt the prevalent "Lift-Splat-Shoot" (LSS) [13] to predict pixel-wise categorical depth distribution and reconstruct objects' distribution in 3D space using the camera's intrinsics and extrinsics. In aerial contexts, observation distances are considerably more extensive, as depicted in Fig. 1 . This expanded range increases depth categories and poses substantial challenges to view transformation. Crucially, mistaken feature projection distorts BEV representations, severely undermining the reliability of subsequent multi-drone collaboration. Furthermore, efficient interaction is essential for real-time collaborative prediction. The prevalent sparse collaborative strategy, where2comm [14] , utilizes the downstream detection head for information selection. However, its mode is inflexible and overlooks valuable contextual cues beyond the object information, which are equally essential for motion forecasting. To address the challenge of view transformation, we observe that the drone's inclined observation leads to intersections between the sight and the ground plane. Consequently, this geometric attribute assigns a theoretical maximum depth to each pixel, providing a constraint for depth estimation. Moreover, given the noticeable gap between objects and the ground plane, it is an alternative to derive intricate depth estimation from the simpler height estimation. For the design of the collaborative strategy, we advocate that a flexible collaborative strategy should dynamically assess the information volume contributed by each region to downstream tasks based on the model's feedback. This adaptive methodology ensures a thorough understanding of the environment, encompassing both foreground objects and their broader surroundings, thus facilitating more accurate decision-making in collaborative trajectory prediction.Building upon the mentioned solutions, this research presents a collaborative prediction framework for drones called DHD, consisting of a Ground-prior-based BEV Generation (GBG) module and a Sparse Interaction via Sliding Windows (SISW) module. The GBG module calculates the viewing angle of each pixel with the camera's intrinsic and extrinsic parameters. Subsequently, based on the flight altitude, this module determines the theoretical maximum depth for each pixel. With the guidance of the viewing angle and maximum depth information, a relatively simple height estimation can derive more precise depth, thus achieving more accurate BEV representations. The SISW module employs sliding windows to analyze discrepancies between the central and surrounding features, thereby quantifying the information volume at each position. Regions exhibiting significant feature variations are assigned higher information scores, indicating that they contain objects or crucial environmental information. Additionally, due to the lack of relevant datasets for multi-drone collaborative prediction, this study utilizes the CARLA [15] simulation platform to generate a novel dataset, "Air-Co-Pred", comprising cooperative observations from four drones across 200 varied scenes. This simulated dataset serves to validate the effectiveness of our proposed DHD framework in aerial collaborations.In summary, our contributions are listed as follows:A Ground-prior-based Bird's Eye View (BEV) Generation module is presented to achieve more accurate BEV representations guided by the ground prior and height estimation.A Sparse Interaction via Sliding Windows module is proposed to improve the accuracy of multi-drone object trajectory prediction through efficient information interaction. A simulated dataset, "Air-Co-Pred", designed for multi-drone collaborative prediction, is introduced to validate the effectiveness of the proposed DHD framework.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes]Justification: The abstract and introduction succinctly outline the challenges in the multidrone object trajectory prediction and introduce the Drones Help Drones (DHD) framework to address these issues.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 BEV Generation. Benefiting from the scale consistency and unified coordinate system, BEV is extensively utilized in collaborative perception. The generation of BEV features is currently categorized into two types: explicit 2D-3D mapping and implicit 3D-2D mapping [16] . In the explicit mapping paradigm, PON [17] introduces a dense transformer module to learn the mappings between image and BEV representations. LSS [13] employs a learnable categorical depth distribution to lift the 2D features to 3D space, and BEVDepth [18] further utilizes explicit supervision on the predicted depth. In the implicit mapping paradigm, BEVFormer [19] leverages the deformable attention mechanism for BEV feature generation, while PolarFormer [17] adopts polar coordinates for precise feature localization. This study adopts a 2D to 3D generation manner similar to LSS. Depth estimation plays a crucial role in the BEV generation, and we are committed to its enhancement.Collaborative Strategy. Current collaborative methods include raw-measurement-based early collaboration, result-based late collaboration, and feature-based intermediate collaboration [20] . Due to the performance-bandwidth trade-off, intermediate collaboration is extensively studied. For example, Who2com [21] introduces a handshake mechanism to select collaborative partners, while When2com [22] further determines when to initiate collaboration. Where2comm [14] employs the detection head to direct regions for sparse interactions. V2VNet [23] achieves multi-round message exchange via graph neural networks. V2X-ViT [24] explore correlations among heterogeneous collaborators through transformer blocks. DiscoNet [25] employs knowledge distillation, whereas CORE [26] utilizes reconstruction concepts to facilitate feature interactions. UMC [27] and SCOPE [28] leverage temporal information to guide feature fusion. However, these strategies are mostly optimized for detection tasks or merely improving feature-level representations. Concerning collaborative forecasting, V2X-Graph [29] utilizes intricate graph structures to forecast motion based on vector maps instead of the vision-based inputs as in this study.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: The paper discusses the limitations of this work in the Section 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
problem formulation: 
 This section presents DHD, a well-designed collaborative framework for multi-drone object trajectory prediction, as depicted in Fig. 2 . The proposed DHD enables multiple drones to share visual information, promoting more holistic perception and prediction. Conceptually, we consider a scenario with N drones, each capable of sending and receiving collaboration messages from others, and storing T 1 historical frames while predicting T 2 future trajectories. For the k-th drone, X ti k denotes the raw observation at input frame t i , and Y to k represents the ground truth at output frame t o . The objective of DHD is to optimize the performance of multi-drone object trajectory prediction under a total transmission budget B:EQUATIONwhere g(•, •) represents the evaluation metric, and Ŷ to k is the prediction outcome of drone k at output frame t o . C θ (•) is the collaborative framework parameterized by θ, and X ti j→k denotes the message transmitted from the j-th drone to the k-th drone at input frame t i . The subsequent subsections of Section 3 detail the major components. Initially, each drone captures observations and individually processes its range view (RV) images through a shared encoder Φ Enc (•) to extract semantic information. We adopt the backbone of EfficientNet-B4 [30] as the encoder for each drone because of its low inference latency and optimized memory usage, consistent with the literature [31] . For the k-th drone, given its observation X ti k at the input frame t i , the extracted 2D feature map is denoted asF ti k,2D = Φ Enc (X ti k ) ∈ R H×W×C, where H, W, C denote its height, width and channel, respectively.Depth estimation plays a critical role in generating BEV representations. However, the vanilla LSS method struggles to approximate the precise depth of each pixel due to the vast observation range of drones. Fortunately, the oblique perspective of drones intersects with the ground plane, providing a theoretical upper bound for depth and a nearly zero-altitude reference for each pixel, as shown in Fig. 3 . Guided by this principle, we propose the GBG module, which refines depth at each pixel to generate more accurate BEV representations, with its methodology illustrated in Fig. 4 . The process below omits the subscript k and t i for concision.Derivation of Depth Upper-bound. Given a drone at altitude H, the pinhole camera model is utilized to determine the depth upper-bound D (u,v) upperbound at each pixel coordinate (u, v), with the help of its camera's intrinsic matrix K, rotation matrix R, and translation vector T:EQUATIONFurthermore, the viewing angle θ (u,v) of each pixel can be represented as arcsin H/D (u,v)upperbound . Integrating Height into Depth Estimation. In aerial observations, objects are usually closer to the ground than to the drones. Therefore, it's more feasible to estimate the distance from the objects to the intersection point between their respective viewing ray and the ground, as indicated by line D 2 in Fig. 3 . Nevertheless, as the viewing angles diminish, the corresponding depth significantly increases. It becomes progressively challenging to estimate the distance to the intersection point due to less visual information available at greater distances. Given the static height of objects and apparent feature discrepancy against the zero-altitude ground plane, it's an alternative to estimate the object's height h pred and deduce the distance to the intersection points with the help of viewing angles. Accordingly, depth estimation Φ Depth (•) can be succinctly expressed as:EQUATIONBEV Generation Process. We utilize a parametric network Φ Height (•) for height estimation, which generates a pixel-wise categorical distribution of height, represented as h pred = Φ Height (F 2D ) ∈ R H×W×D . Here, D signifies the number of discretized height bins. Subsequently, the estimated height are input into Φ Depth to compute the estimated depth d pred . Then, an outer product between image features F 2D and all potential depth candidates d pred is conducted to generate frustum features, which are then projected into the 3D space based on the inverted pinhole camera model. This process results in 3D voxel representation V ∈ R X×Y×Z×C , where Z refers to the height of voxel. Finally, through sum pooling Φ Sp , these voxel features V are condensed into a single height plane to generate the BEV features: F BEV = Φ Sp (V ) ∈ R X×Y×C , serving for the subsequent collaboration process.In aerial observations, objects are typically small and sparsely distributed, leading to critical information occupying only a minor portion of the panoramic view. Given this context, the communication overhead can be considerably reduced by transmitting solely the complementary objects during collaborations [14] . Nonetheless, environmental information is also crucial for prediction tasks, with studies [32, 33] demonstrating that dynamic entities and static textures within surroundings can provide insights for forecasting future trends. Hence, it's necessary to include essential environmental elements within limited data transmissions for better forecasting. To this end, we introduce a novel sparse interaction module named SISW for collaboration among drones. This module assesses the information volume across different areas through sliding windows, determining regions for inter-drone interactions. The SISW module's workflow is illustrated in Fig. 5 and delineated below:Information Volume Assessment. The original BEV features F k,BEV ∈ R X×Y×C are compressed to F ′ k,BEV ∈ R X×Y×1 with a trainable 1 × 1 convolution, reducing the transmission overhead for inter-drone comparisons of complementary information. Then, a K × K sliding window traverses over the compressed features, calculating the discrepancy between each position's features and the central features. The calculated discrepancy is then normalized to the interval (0, 1) via a sigmoid function σ, with the average discrepancy across the window serving as the indicator of the information volume I:EQUATIONIn this setting, the information volume is almost zero in areas of homogeneity, like empty backgrounds.In contrast, the information volume is much higher in regions with significant feature variations, such as when the area contain the objects or crucial environmental information.Selective Sparse Interaction. Efficient collaboration among drones aims to share complementary information that supplements receivers' current knowledge, thus enhancing the overall system's accuracy. When k-th drone receives the information volume from j-th drone, complementary scores S j→k among drones are calculated as I j ⊙ (1 -I k ). To achieve sparse interactions, the interaction mask M j→k is determined by the top K from the ranked scores S j→k and sparse collaborative features C j→k are denoted as M j→k ⊙ F j,BEV .Collaborative Features Aggregation. Upon receiving sparse collaborative features, drone k implements a geometric transformation T to align C j→k with its local coordinate system, ensuring spatial congruence. Subsequently, a learnable Gaussian-based interpolation ϕ inter is applied to infill undefined values, smoothing the spatial distribution of the features. This step is essential to mitigate the impacts of numerical zero vectors that could adversely affect subsequent feature fusion pro-cesses [27] . The refined collaborative features are then given by: C ′ j→k = ϕ inter (T(Z j→k )). Guided by local features F k,BEV , the contribution weight W j of C ′ j→k towards constructing aggregated features A k is quantified by: Wj = ϕconv([F k,BEV ;C ′ j→k ]) N j=1 ϕconv([F k,BEV ;C ′ j→k ]) ∈ R X×Y×1, where ϕ conv represents the multi-layer convolution operations. Moreover, a pixel-level weighted fusion is executed to generate the aggregated features A k for subsequent downstream tasks:A k = N j=1 W j C ′ j→k ∈ R X×Y×C .Aligned with PowerBEV [34] , the state-of-the-art approach for joint perception and prediction, a temporal U-Net architecture is used to interpret a series of T 1 aggregated features {A ti } T1 ti=1 and forecast further T 2 dynamics, such as instance segmentation {y seg to } T2 to=1 and future flow {y flow to } T2 to=1 . For the end-to-end training, we adopt a cross-entropy loss for the segmentation task and a smooth l 1 distance for the flow loss. The overall loss function L is formulated as:EQUATIONwhere γ = 0.95 is a temporal discount parameter and λ 1 , λ 2 are the balancing coefficients adjusted dynamically through uncertainty weighting. In the inference phase, Hungarian algorithm correlates multi-frame instances to synthesize present instance segmentation and future motion prediction.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] .Justification: The source codes of the core modules and a demo dataset are provided in the supplemental materials zip file. The complete source code will be released upon acceptance.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
datasets: 
 The available datasets for multi-drone collaboration [14] primarily focus on detection and segmentation, lacking the support for the prediction tasks. To bridge this gap, we create a simulated dataset "Air-Co-Pred" for collaborative trajectory prediction based on CARLA [15] . Specifically, four collaborative drones are positioned at intersections to monitor traffic flow from different directions.These drones fly at an altitude of 50 meters, covering an area of approximately 100m × 100m. They capture images at a frequency of 2Hz to support the temporal prediction task. The collected dataset includes 32k synchronous images with a resolution of 1600 × 900 and is split into 170 training scenes and 30 validation scenes. Each frame is well-annotated with both 2D and 3D labels, comprising three major object categories: vehicles, cycles, and pedestrians. Given the challenges of tiny objects from aerial perspectives, this study mainly concentrates on the vehicle category, which includes many sub-categories to augment the robustness of identifying various vehicles. To illustrate the challenges of aerial observations intuitively, we present several charts to reflect the characteristics of "Air-Co-Pred", such as occlusion, long-distant observations, small objects, etc., as shown in Fig. 6 .To evaluate the localization accuracy of generated BEV representations, we utilize the metrics of precision and recall to reflect matches between predicted and ground-truth instance centers. Additionally, the L2 distance is used to quantify the position deviations for successfully matched instances. Regarding downstream tasks, we adopt two metrics widely used in previous works [31, 34] .For frame-level evaluation, Intersection-over-Union (IoU) evaluates the segmentation quality of objects at the present and future frames. The specific calculation is as follows:EQUATIONwhere N denotes the number of output frames, ŷseg t and y seg t denote the predicted and ground truth semantic segmentation at timestamp t , respectively. As for the video-level evaluation, Video Panoptic Quality (VPQ) reflects the quality of the segmentation and ID consistency of the instances through the video, expressed as: where T P t , F P t , and F N t correspond to true positives, false positives, and false negatives at timestamp t, respectively. True positives are predicted instances with the IoU greater than 0.5 and consistent instance IDs with the ground truth, whereas false positives and false negatives are incorrectly predicted and missed instances, respectively.EQUATIONWe follow the setup from the existing study [10] for collaborative trajectory prediction. Initially, the raw images, with a resolution of 900 × 1600 pixels, are scaled and cropped to a size of 224 × 480.As to view transformation, the height estimation range is configured from 0 to 10 meters, discretized into 100 intervals. Subsequently, for BEV representations, the spatial ranges for the x, y, and z axes are set to [-50, 50], [-50, 50], and [-60, -40] meters, respectively. We evaluate the model performance across various perceptual scopes: a 100 m×100 m area with 0.5 m resolution (long) and a 50 m×50 m area with 0.25 m resolution (short). Temporally, we utilize three frames from the past 1s (including the present frame) to predict both semantic segmentation and instance motion for the future four frames (2s). Besides, we select a 7×7 window size and set the transmission ratio as 25% for the SISW module for comprehensively optimal performance. The relative details on hyper-parameter ablation studies are provided in the supplementary material. Our DHD framework is trained with Adam optimizer at an initial learning rate of 3 × 10 -4 . It runs on four RTX 4090 GPUs, handling a batch size of 4 for 20 epochs. Benchmark Comparison in BEV Generation. We select the vanilla LSS [13] and the drone-specific DVDET [35] as baselines. For fairness, their depth estimation range is set from 1 to 100 meters, divided into 100 intervals. As shown in Table 1 , DHD outperforms the vanilla LSS regarding downstream performance, showing a 9.06% improvement in IoU and a 12.11% increase in VPQ within short-range observations. Furthermore, it demonstrates a 5.06% improvement in IoU and a 6.42% increase in VPQ for the long-range setting. While DVDET incorporates a deformable attention mechanism to refine BEV representations, it exhibits gains in the short-range setting but a notable decline in the long-range setting in contrast to the vanilla LSS. Inaccurate depth estimation causes the projected objects to shift from their correct positions, resulting in mismatches with the ground truth. Specifically, DHD achieves fewer mistaken and missing matches and better localization, with over a 20% reduction in position deviation. Remarkably, the DHD without height estimation still achieves performance gains and reduces long-range localization errors solely by relying on ground priors.Benchmark Comparison in Collaboration. Table 2 shows that our DHD achieves performance comparable to early collaboration, particularly within the long perceptual range setting. It also significantly outperforms No-Collaboration, with an increase of 42.61% in IoU and 45.90% in VPQ for the short-range setting, and 74.29% in IoU and 79.36% in VPQ for the long-range setting, respectively, revealing the effectiveness of collaboration. In contrast to fully connected baselines, DHD can approach and even exceed similar performance with only a quarter of the transmission ratio. When compared to the partially connected baselines, DHD surpasses When2com, the upgraded version of Who2com, by almost 20% in both IoU and VPQ. Furthermore, DHD also beats the previous state-of-the-art method, Where2com, by a considerable margin, for example, 5.82% IoU and 3.85% VPQ in the long-range setting. Because our DHD not only considers the foreground objects but also incorporates relevant environmental information for predictions. Notably, the recent state-of-the-art method, UMC, with GRU-based feature fusion, exhibits 3∼ 4% lower performance than ours. We find that its temporal fusion deteriorates original downstream tasks. Although this temporal prior has enhanced object detection, it may lead to perception aliasing for prediction tasks.Visualization of Prediction Results. Fig. 7 demonstrates that collaboration among drones, when compared to no collaboration, facilitates the acquisition of the location and state of occluded and out-of-range objects through feature-level interactions. Furthermore, DHD correctly forecasts the trajectory of multiple objects in challenging intersections and achieves more accurate segmentation and prediction results compared to the well-known baseline, Where2com. This is attributed to Where2comm's transmitted features to focus on foreground objects while neglecting the surroundings that contributed to downstream tasks. These findings are consistent with our quantitative evaluations.Figure 7 : A comparative analysis of visualizations among various collaboration baselines. Each instance is allocated a distinct color, and its predicted trajectory is represented with the same color and slight transparency. Red circles highlight the areas where other baselines make mistaken predictions.Figure 8 : The visualization of feature-level interactions. Green circles represent the primary complementary area, including missing objects and crucial environmental cues.Visualization of Sparse Interactions. To examine the interaction during the collaboration, we analyze sparse features transmitted between Drone 1 and Drone 2 via the SISW module. Fig. 8 shows that transmissions include not only object features (in dark gray) but also surrounding features (in light white), which is more prominent in column 5. Columns 2 and 3 highlight an obvious discrepancy in the mean of features, which could affect subsequent fusion processes. We implement a learnable interpolation to fill in blank features to alleviate the gap. Above all, these visualizations are in agreement with our designs and expectations. Effectiveness of Proposed Modules. Our DHD framework introduces two innovative components: the GBG and SISW modules. We evaluate these modules based on their ability to enhance prediction accuracy and optimize the performance-transmission trade-off, as depicted in Table 3 . The GBG-only variant significantly enhances prediction accuracy with improvements of approximately 10% in short-range and 5% in long-range predictions, primarily attributable to the more accurate BEV representations guided by the ground prior. The SISW-only variant reduces transmission costs by 75% with only a marginal performance decrease of about 1% relative to the baseline. Overall, our DHD, integrated with both modules, can achieve a balance between prediction enhancement and transmission cost.Performance Changes with the Number of Collaborative Drones. To explore how the number of collaborating drones affects performance, we conducted relevant experiments, as shown in Table 4 . For the short-range area of 50m × 50m, three drones are sufficient to predict trajectories, with performance comparable to that of four drones. However, for the long-range area of 100m × 100m, predictive performance improves as the number of drones increases. This difference can be attributed to the positional layout of the drones in the dataset, Air-Co-Pred. The drones are situated near intersections to monitor traffic flow, facilitating comprehensive coverage of the short-range areas close to these intersections. In contrast, much of the information in the long-range areas extends along specific road branches, which may only be captured by a single drone. Therefore, increasing the number of drones results in more comprehensive coverage of the long-range areas.We also conduct generalization validation for collaborative 3D object detection in CoPerception-UAVs [14] , a publicly available multi-drone collaborative dataset. We select several models for BEV generation baselines: BEVDet [36] (a modified LSS model for detection), its temporal version BEVDet4D [37] , and DVDET [35] , the official detector for CoPerceptionUAVs. All of these models adopt the SISW module for inter-drone feature-level interactions. The evaluation metrics [38]  
conclusion and limitation: 
 This paper presents DHD, a collaborative framework for multi-drone object trajectory prediction. Its GBG module leverages the ground prior and simpler height estimation for more accurate BEV representations. Meanwhile, the SISW module adaptively selects regions for collaborative interactions, guided by the sliding window's information volume calculation. Additionally, we construct the first simulated dataset of multi-drone collaborative prediction, named "Air-Co-Pred", to evaluate the effectiveness of DHD through quantitative and qualitative experiments.Limitation and Future Work. The current work only exploits the simulated setting, an idealized scenario, for multi-drone object trajectory prediction. For practical applicability, future efforts will extend to real-world environments, taking into account realistic challenges such as flight turbulence, rough terrain, camera extrinsic noise, latency, communication frequency etc. Specifically, flight turbulence and rough terrain can impact the derivation of depth upper-bound in the GBG module, while inaccurate camera parameters may compromise multi-view projection transformations. Additionally, effectively managing latency and optimizing communication frequency are crucial for collaboration within the SISW module. Addressing these issues is essential for deploying the proposed model in real-world scenarios.6 Appendix 
air-co-pred dataset details: 
 Map Creation. The simulation scenes, including the road layouts, static objects, and traffic flow, are generated using the CARLA simulation platform. We utilize the "Town10" map provided by CARLA as the foundational road layout. This scenario is similar to the intelligent urban setting, with high-rise buildings and roadside trees leading to observation occlusions. These challenges can be effectively addressed through multi-drone collaboration.Traffic Flow Creation. Moving vehicles in the scene are controlled via CARLA, with hundreds of vehicles spawned in each scene using the official script provided by CARLA. The map's road layout determines each vehicle's initial location and motion trajectory.Sensor Setup. Each drone is equipped with a forward-facing RGB camera with a 90°field of view and a resolution of 1600 × 900. The cameras are fixed, with their internal positions and rotation degrees remaining constant. During data collection, each camera's translation (x, y, z) and rotation (w, x, y, z in quaternion) are recorded in global and ego coordinates. With this sensor configuration, four collaborative drones flying at a height of 50 meters can effectively cover an area of 100m × 100m.Data Collection. Our proposed dataset is collected by the CARLA simulation platform under the MIT license. We utilize CARLA to create complex simulation scenes and traffic flow. Aerial observation samples are collected at a frequency of 2 Hz. We synchronously collect images from four drones, resulting in four images per sample. Additionally, camera intrinsics and extrinsics in global coordinates are provided to support coordinate transformation. A total of 32,000 images have been collected to support our experiments. Our ground truth labels for collaborative prediction are derived from 3D bounding boxes of observed targets. Thus, we take advantage of lidar sensors to collect these 3D bounding boxes, which include location (x, y, z), rotation (represented with quaternion), and dimensions (length, width and height), amounting to nearly 430,000 3D bounding boxes.Data Usage. We randomly split the samples into training and validation sets, yielding 25.6k images for training and 4.4k for validation. The dataset is structured similarly to the widely used autonomous driving dataset, nuScenes, so that it can be directly used with the well-established nuScenes-devkit.Figure 9 : The visualization of our Air-Co-Pred dataset along with the detailed annotations for collaborative detection, segmentation and prediction tasks.To demonstrate the contributions of our dataset, Air-Co-pred, we conduct a comparative analysis of existing datasets in the multi-drone collaboration domain. By the submission deadline, several datasets are available for multi-drone collaboration, including two real-world datasets (VisDrone-MDOT [2] and VisDrone-MDMT [1] ) and two simulation datasets (Airsim-Map [22] and CoPerception-UAVs [14] ).Regarding the existing real-world datasets, the VisDrone series has collected a substantial amount of real-world video data. However, these datasets are constructed solely from a visual perspective and do not provide any information about the drones' poses or camera parameters. As a result, they are limited to supporting 2D visual algorithms [39] such as ReID and object tracking, and cannot be used to evaluate our proposed collaborative prediction framework, which integrates both visual and spatial information.The simulation dataset, Airsim-Map, is designed to demonstrate the effectiveness of who2com and when2com in mitigating image degradation. However, it only provides multi-view 2D semantic segmentation masks. The dataset most similar to ours is CoPerception-UAVs, proposed by where2comm.While this dataset focuses exclusively on multi-drone collaborative 3D object detection and has been used to validate frameworks like Where2comm [14] and CoCa3D [40] , it falls short in addressing joint temporal tasks. Additionally, its large sampling intervals are inadequate for validating our DHD.To bridge this gap, we propose a more comprehensive dataset that includes detailed annotations and an appropriate sampling frequency. This dataset supports a wide range of tasks, including 2D/3D detection [41] [42] [43] , BEV segmentation [31] , multi-object tracking [44] , and trajectory prediction [34] , and facilitates the preliminary validation of multi-drone collaboration across various scenarios within a simulation environment.The Advantage of Depth Estimations from a Near-Ground Perspective. In aerial observations, objects are generally closer to the ground rather than to the drones, which introduces particular considerations for depth estimation. The estimated range determined by the distance to the drone is represented by[H -h max , D (u,v) upperbound ], where H denotes the altitude of the drone, h max represents the maximum height of the objects, and D upperbound (u, v) corresponds to the depth upper-bound of pixel (u, v). In contrast, considering the distance to the ground, the required range is h max , hmax sin(θlb) , where θ lb denotes the lower-bound of the viewing angle. The discrepancy between these two ranges is significant:EQUATION)Here we substitute D (u,v) with H and set H = k • h max , where k ≫ 1. Depth estimation as an n-class problem becomes increasingly complex as the interval range expands. Therefore, it is more feasible to estimate the distance from objects to the point where their sight intersects the ground.Rationality for Height-Based Depth Estimation. As a car of height h moves away from the camera, its viewing angle θ (u,v) diminishes, as depicted in Fig. 3 . A small angular change δ incurs a depth variation ∆d, given by:EQUATIONAdditionally, ∆d is a monotonically decreasing function of the angle θ. As θ decreases, the corresponding depth significantly increases, which exacerbates the challenge of depth estimation due to less visual information available at greater distances. In contrast, an object's height remains constant regardless of its position, making it relatively easier to estimate. Furthermore, the task of height estimation can even be simplified to an object classification task. For instance, the ground plane maintains an altitude of zero, while different types of vehicles are associated with specific height values.Impact of Flight Turbulence and Rough Terrain on DHD Performance. We acknowledge that flight vibrations and uneven terrain can interfere with the drone's relative height to the ground, affecting the BEV generation from the GBG module. Therefore, we introduce perturbations to the drone's altitude to simulate these conditions.Specifically, we introduce Gaussian noise to the drone's altitude, with noise levels ranging from 0.002 to 0.01. At the highest level, this results in a maximum altitude variation of 0.5 meters, which is significant for drone flight. As illustrated in Fig. 10 , the results demonstrate that when the noise level exceeds 0.01, the depth estimation advantage conferred by the geometric prior in the GBG module diminishes. However, this limitation of terrain variations is not unique to our GBG module. Mainstream BEV generation methods, such as LSS, BEVFormer, and PETR, also assume a flat ground. Complex terrain requires further study and can be considered a distinct research direction.The primary goal of our GBG is to explore BEV generation specifically designed for drones. While this is an initial attempt, we recognize the need to account for more complex real-world conditions in future deployments. To address this, we propose a simple yet potentially effective solution: developing a ground flatness estimation module to assess variations in the ground plane, allowing the estimated object height to be adaptively adjusted and thereby mitigating the impact of uneven terrain on subsequent BEV generation.Impact of Sensor Noise on DHD Performance. Sensor noise primarily affects the accurate acquisition of camera extrinsic parameters. To examine this effect, we introduce Gaussian noise to simulate varying levels of disturbance in the extrinsic calibration.As illustrated in Fig. 11 , increasing noise results in a gradual decline in both IoU and VPQ. In short-range settings, noise ratios below 0.003 cause negligible performance drops. However, IoU decreases by about 25% when the noise ratio is between 0.003 and 0.013. Between 0.013 and 0.020, the decline slows, with an additional reduction of approximately 10%. VPQ exhibits a similar trend. In long-range settings, noise ratios below 0.003 also result in acceptable performance declines. However, when the ratio reaches 0.005, noticeable performance degradation occurs, with IoU dropping by 21.3% and VPQ by 32.9%. Overall, noise has a more pronounced impact on VPQ than on IoU, indicating that camera extrinsic bias more severely affects the consistency of future trajectory predictions. Furthermore, the greater impact of extrinsic noise in long-range observations can be attributed to objects at long distances often being observed from a single perspective, lacking the multi-view validation available in short-range scenarios.These results demonstrate that our DHD can tolerate a small amount of sensor-based extrinsic noise. Besides, larger biases in extrinsic parameters can significantly impact collaborative prediction. Therefore, accurate estimation of these parameters is crucial for maintaining high performance in collaborative perception systems. This finding is equally applicable to real-world scenarios. Visualization of Collaborative 3D Object Detection. As illustrated in Fig. 12 , although our DHD framework is not specially designed for collaborative 3D object detection, it still effectively overcomes the limitations of single-drone perception, such as occlusion and out-of-range observations, through multi-drone collaboration.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The paper comprehensively outlines all relevant details pertaining to the training and testing procedures in Section 4, including the specific hyper-parameter settings, optimizer configurations, and other pertinent information.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification:We have not repeated experiments many times to get error bars since experiments of multi-drone object trajectory prediction on large scale datasets is time-consuming.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: This research has no potential harm, societal impact, or harmful consequences.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA]Justification: The research presented in this paper remains purely at the theoretical and experimental levels and has not yet resulted in any societal impacts.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper is unrelated to the mentioned high-risk data or models.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] .Justification: The license and terms of use for our simulated dataset, "QuadDrones200", are thoroughly detailed in Section 4.1 and the supplemental material. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The size of the window determines the contextual range around the central point. Smaller windows are more sensitive to capturing high-frequency local feature variations, but they may also be susceptible to noise. Conversely, larger windows encompass a broader range of information but potentially neglect valuable local discrepancies. After investigating the effects of various window sizes, as shown in Fig. 13 , we select a 7×7 window size to assess the information volume in the SISW module for optimal performance.Performance Enhancement Brought by Transmission Ratio. Fig. 8 shows that each drone's effective BEV feature coverage is approximately 50%. Accordingly, our transmission ratio decreases from 50% to 0.1% to explore the balance between performance and bandwidth. In the curve shown in Fig. 13 , as the ratio decreases from 50% to 25%, there is negligible performance variation, maintaining a relatively high value. However, a significant decline in performance occurs below 10%, with no discernible benefit below 0.25%. To consider comprehensively, we set the transmission ratio as 25% for DHD's SISW module.NeurIPS Paper Checklist 


Decision: The paper has undergone thorough evaluation by the reviewers, most of whom have raised their scores after the authors addressed their initial concerns. The reviewers are now generally satisfied with the paper, acknowledging its strengths and contributions to the field. After considering the updated reviews and the paper's overall quality, we recommend accepting this submission.

Reviews: review: paper_summary: paper_summary: This paper proposes a framework, Drones Help Drones (DHD), which tackles trajectory prediction of objects in the scene. DHD consists of a Ground Prior Based Bird’s Eye View (BEV) Generation (GBG) module, which provides depth estimation from the drone to an object using ground priors to create an accurate BEV representation of the features. It also utilises Sparse Interaction via Sliding Window (SISW) to minimize the data transmission cost between drones. In addition, the authors develop a new dataset for multi-drone collaboration prediction, "Air-Co-Pred". The paper is interesting in the field of collaborative AI.
strength_weakness: strength_weakness: The paper has several weaknesses:
- It investigates methods to overcome single-drone issues, such as occlusions and blurs; nevertheless, it is also important to discuss a bigger picture of their use cases, including accident prevention and path planning, in greater details with certain limitations. For example, in accident prevention, if multiple drones collaborate and predict an accident is about to happen, what can it do? Does it then communicate/inject commands over the air to the vehicle causing certain actions?
- The DHD framework consists of feature extraction, BEV, Sparse Interaction via Sliding Windows, and Trajectory Prediction. The idea of having sliding windows for sparse interaction is quite interesting; nevertheless, relying on a local coordinate system and pixel-level weight fusion can be a weak spot when it comes to real-world settings. It is also important to examine how each of the modules contributes to the overall performance in ablation studies.
- The development of Air-Co-Pred remains questionable, based on CARLA. There are indeed many CARLA-based datasets and it is important to compare your dataset with others to prevent overlappings/duplications.
- It is vital to discuss the following papers, in relation to the work:
1. Wei, S., Wei, Y., Hu, Y., Lu, Y., Zhong, Y., Chen, S., & Zhang, Y. (2024). Asynchrony-robust collaborative perception via bird's eye view flow. Advances in Neural Information Processing Systems, 36.
2. Lu, Y., Hu, Y., Zhong, Y., Wang, D., Chen, S., & Wang, Y. (2024). An extensible framework for open heterogeneous collaborative perception. arXiv preprint arXiv:2401.13964.
3. Liang, J., Jiang, L., & Hauptmann, A. (2020). Simaug: Learning robust representations from simulation for trajectory prediction. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16 (pp. 275-292). Springer International Publishing.
questions: questions: There are many questions to address as the following:
- With the DHD framework, given objects to track/predict, what is the optimal number of drones that need to be “watching” the object in order to produce the best prediction? Is it always the case that the more drones that have the object in view, the better the trajectory prediction? Why four collaborative drones in Air-Co-Pred? Can we get away with just 2 drones? 
- Can you explain more about why CARLA is used to simulate and produce the dataset? Has anyone attempted trajectory prediction with multiple drones in real-world settings? Is it feasible? What needs to happen before DHD can be deployed in production in the real-world setting? Maybe it can be included in a discussion section.
Is there an optimal drone height/altitude? Is 50 meters the best height value, so it’s used in the dataset?
- How often should the drones be communicating/transmitting data to each other? Is it the same as the aerial observation samples being collected (frequency of 2 Hz)? Can it be reduced to further lower transmission data?
- How does the DHD framework respond to noises, such as flight turbulence?
- What will be the performance if we turn on/off multiple components, such as BEV or Sparse Interaction? This should be included in ablation studies.
- How far are we from real-world experiments?
- What will be the impacts of your study to the field?
limitations: limitations: The authors have raised concerns about the use of simulated settings, which limit the research's practicality. It is important to develop real-world scenarios for future validations of the research.
paper_summary: paper_summary: The paper proposes a ground-prior-based BEV generation for drone trajectory prediction. The methods also propose an efficient information interaction via a Sliding Windows module. It accesses information volume across different areas through sliding windows. The authors provide experiments on the simulator-based Air-Co-Pred dataset and 3D object detection on CoPerception-UAVs.
strength_weakness: strength_weakness: The Ground-prior-based BEV Generation (GBG) module relies on certain geometric assumptions for depth estimation. In real-world scenarios, these assumptions might not hold due to variations in terrain, drone stability, and other environmental factors, potentially affecting the accuracy of BEV representations.


Although the paper mentions using EfficientNet-B4 for feature extraction due to its low latency and optimized memory usage, it does not provide a detailed analysis of the computational efficiency and scalability of the overall framework.


The Air-Co-Pred dataset, while a valuable contribution, is still limited to simulated data. The paper could benefit from additional datasets or validation for the main task.


The proposed Sparse Interaction via Sliding Windows (SISW) module, although innovative, might oversimplify the information selection process. The method assumes that feature discrepancies are the primary indicators of valuable information, which may not always be the case, especially in highly dynamic or cluttered environments.


There is a lack of baseline in the main experiment; the authors have compared their method with only two other methods for BEV generation.

There is a lack of trajectory prediction evaluation; the authors should provide the average displacement error as well.

Typo in Table 3 caption.

SISW has shown little improvement, and the performance declines when the transition rate is 0.25. Additionally, the authors do not provide the individual result of SISW on 1 with GBG still present.
questions: questions: Can the authors provide a component ablation of only the SISW component, like three rows on a 1:1 ratio? Furthermore, the results of missing GBG and SISW on a 0.25 transition rate are not provided.
limitations: limitations: The paper relies on simulations using the CARLA platform. Although the simulated environment provides controlled and varied scenarios, it may not fully capture the complexities and unpredictabilities of real-world environments.

It seems like the sliding window module is not that significant, and the authors have not provided a complete ablation study to show its efficacy. 

The component ablation is not thorough.
paper_summary: paper_summary: The paper presents the "Drones Help Drones" (DHD) framework for improving collaborative trajectory prediction in multi-drone systems. It addresses challenges with aerial observations and communication bandwidth by: (i) Using ground priors from inclined drone observations to enhance Bird's Eye View (BEV) accuracy; (ii) Implementing a selective mechanism to prioritize essential information, reducing communication needs; (iii) Introducing the "Air-Co-Pred" dataset for multi-drone collaborative prediction.
Experiments show that DHD improves BEV accuracy, reduces required transmission bandwidth, and maintains high prediction performance.
strength_weakness: strength_weakness: W1. The strategy follows the classic collaborative framework composed of three modules: BEV estimation, construction of interaction, and prediction. The methodology utilizes well-known strategies from the literature for this task. Specifically, for BEV estimation, it uses part of the framework from [13], although it is well adapted with the inclusion of height in the depth estimation. The Selective Sparse Interaction used in the SISW module appears similar to the approach in [14], as well as the aggregation performed. Finally, the prediction mechanism is taken from [34], as mentioned by the authors in L206.
Although the construction of a unified framework is not trivial, the overall strategy deviates little from existing ones.

W2. The authors have contributed to the creation of a new dataset that could benefit future research, but this will require appropriate adjustments. There is no comparison with other datasets used in the same or related fields, making it impossible to evaluate the quality of the dataset.

W3. The lack of additional datasets also makes the effectiveness of the framework less convincing, as there is no evidence of generalization on other established benchmarks. I understand the uniqueness of the task and wonder if it is possible to test the framework on established benchmarks (e.g., DAIR-V2X, V2XSet, and OPV2V datasets).

W4. In Table 2, the reasoning for separating V2X-ViT and V2VNet from other methodologies is not clear. The distinction between fully/partially connected is not a convincing rationale. Could the authors clarify this point?

W5. [Minor] Some findings in the appendix, such as Figure 9 and Section 6.4, could have been included in the main paper to strengthen certain claims.
questions: questions: See Weaknesses section for questions, especially I would appreciate if the authors highlighted the differences with the other mentioned methodologies.
limitations: limitations: The authors adequately addressed the limitations.
paper_summary: paper_summary: This paper studied the problem of collaborative multi-drone object trajectory prediction. The authors proposed a framework named "Drones Help Drones" (DHD) that can improve accuracy compared to existing methods while reducing the required communication bandwidth. To be more specific, the authors leveraged a Ground-prior-based BEV Generation module that can generate more accurate BEV representations, and used a Sparse Interaction via Sliding Windows module that enables efficient information sharing between drones. The authors also constructed a new dataset named "Air-Co-Pred" for multi-drone collaborative prediction tasks.
strength_weakness: strength_weakness: * The primary evaluation is conducted on simulated data. While the authors acknowledge this limitation, it raises questions about the framework's performance in more complex, real-world scenarios with sensor noise, communication delays, environmental variability, and other common difficulties in real-world experiments. 
* It would be interesting to see how the performance scales with the number of collaborating drones, which could provide valuable insights into the method's scalability. Is there an optimal or maximum number of drones for effective collaboration?
questions: questions: see above
limitations: limitations: Limitations are discussed
paper_summary: paper_summary: This paper presents a collaborative framework for multi-drone object trajectory prediction, named DHD, which consists of two specifically designed components: the GBG module, aimed at generating more accurate BEV representations in aerial scenes, and the SISW module, which adaptively selects regions for collaborative interactions. Additionally, a simulated multi-drone collaborative observation dataset is created to demonstrate the effectiveness of the proposed DHD framework.
strength_weakness: strength_weakness: - The authors need to carefully review the paper for grammar mistakes, typos, and formatting errors. Specific issues include:
    - Line 82: BEV's full-term appears redundantly.
    - Line 119: The subscript in$ Y_{k}^{o} $ is incorrect and should be$ Y_{k}^{t_o} $.
    - Line 152: Figure reference is incorrect.
    - Line 221: "2HZ" is improperly formatted and should be "2Hz."
    - Equation 6: The subscript next to $ \Sigma $ should be $ xy $ instead of $ hw $.  
Addressing these issues will improve the paper's readability.

- The transmission volumes for various collaboration strategies in Table 2 are unclear. It would be beneficial to supplement this information to better reflect the balance between the performance improvements in prediction and the cost of collaborative interactions. Additionally, it appears that the results of DHD in Table 2 are not the best in every column, so why are they bolded?
- Please consider exploring scenarios where the number of drones varies from 1 to 4. This would help investigate the impact of drone quantity on perception enhancement and also reflect the influence of potential drone failures on performance metrics.
- In reality, extrinsic parameters are derived through computation and approximation, leading to certain biases. Please explore how the extent of extrinsic bias affects the performance of collaborative prediction.
questions: questions: - Could you provide more details about Air-Co-Pred, including the drone attitude settings, flight patterns, the speed of objects, etc.? Why were four drones chosen to collect data? 
- In Section 3.3, the equation regarding the derivation of the depth upper-bound is presented directly without any intermediate steps in the main text or supplementary materials. Please provide the detailed derivation process to help readers understand it better.
- The reviewer found that the proposed GBG module is highly relevant to the applications of camera intrinsics and extrinsic in UAVs. The reviewer encourages the authors to acknowledge the existence of related works and clarify the differences. 
For reference, see:  
Shen H, Lin D, Yang X, et al. Vision-Based Multi-Object Tracking through UAV Swarm[J]. IEEE Geoscience and Remote Sensing Letters, 2023.  
Pan T, Dong H, Deng B, et al. Robust Cross-Drone Multi-Target Association Using 3D Spatial Consistency[J]. IEEE Signal Processing Letters, 2023.  
- Considering the complexity of the proposed framework, could you provide additional details or code for better understanding and reproducibility?
limitations: limitations: The authors acknowledge that the current focus of this paper is on simulated environments to investigate multi-drone object trajectory prediction without considering latency and camera noise. These are reasonable limitations, and addressing them in future work would be beneficial

Comments: title: Thank you for the feedback
Thank you for the feedback
Thank you for the feedback
Comment after reading the response
comment: Dear Reviewer nEkT. We hope that our rebuttals have clarified your concerns. if there are any specific analyses or complementary experiment that could clear your doubts, we would be happy to try and provide them. We sincerely thank you again for your time and feedback.
Thank you for patiently pointing out the weaknesses in our initial manuscript and reviewing our response. If the submission is accepted, we will include a detailed analysis of our proposed DHD's real-world limitations and scalability in the revised version, ensuring a more comprehensive presentation of our paper.
Thank you very much for reading our responses! If the submission is accepted, we will incorporate more details about the differences between our DHD and existing methodologies, and highlight the comparison of our proposed Air-Co-Pred with other datasets in the final version.
Thank you very much for taking the time to read our responses and provide your valuable feedback. As mentioned, we are committed to sharing our resources with the community. If the submission is accepted, we will release the dataset, along with the relevant codes and model checkpoints, to facilitate further research and development.
Thank you to the authors for the thorough explanation of the differences with existing methodologies. I also appreciated the comparison of the dataset with others in the literature; it would be helpful if this could be included in the revised version of the paper. 

In general, my concerns have been addressed, and I am inclined to raise the score.
I would like to thank the authors for their detailed explanation. I've raised the score accordingly.
Thanks for the authors’ time and effort in responding to the weaknesses and questions raised.

Overall, this revised version is more comprehensive, incorporating some previously missing details and effectively resolving all my concerns. Moreover, the innovations presented in the paper have been fully recognized by reviewers CzWU, nEkT, and rEnT, and partially acknowledged by reviewer nhVA. Consequently, I am convinced to maintain my initial judgment.