Drones Help Drones: A Collaborative Framework for Multi-Drone Object Trajectory Prediction and Beyond
Authors: Zhechao Wang
Peirui Cheng
Minxing Chen
Pengju Tian
Zhirui Wang
Xinming Li
Xue Yang
Xian Sun
Venue: NeurIPS 2024 poster (2024)
Keywords: Multi-drone Collaboration
Perception and Prediction
Primary area: machine_vision
TL;DR: We present a collaborative framework for multi-drone object trajectory prediction, including a drone-specific BEV generation module and a selective interaction strategy based on the local feature discrepancy.

Abstract: Collaborative trajectory prediction can comprehensively forecast the future motion of objects through multi-view complementary information. However, it encounters two main challenges in multi-drone collaboration settings. The expansive aerial observations make it difficult to generate precise Bird's Eye View (BEV) representations. Besides, excessive interactions can not meet real-time prediction requirements within the constrained drone-based communication bandwidth. To address these problems, we propose a novel framework named "Drones Help Drones" (DHD). Firstly, we incorporate the ground priors provided by the drone's inclined observation to estimate the distance between objects and drones, leading to more precise BEV generation. Secondly, we design a selective mechanism based on the local feature discrepancy to prioritize the critical information contributing to prediction tasks during inter-drone interactions. Additionally, we create the first dataset for multi-drone collaborative prediction, named "Air-Co-Pred", and conduct quantitative and qualitative experiments to validate the effectiveness of our DHD framework. The results demonstrate that compared to state-of-the-art approaches, DHD reduces position deviation in BEV representations by over 20\% and requires only a quarter of the transmission ratio for interactions while achieving comparable prediction performance. Moreover, DHD also shows promising generalization to the collaborative 3D object detection in CoPerception-UAVs.

Drones Help Drones: A Collaborative Framework for Multi-Drone Object Trajectory Prediction and Beyond
Zhechao Wang 1.2, Peirui Cheng', Mingxin Ch2 Pengju Tian Zhirui Wang'
Xinming Li Xue Yang". Xian Sun 1.2
Key Laboratory of Network Information System Technology,
Aerospace Information Research Institute, Chinese Academy of Sciences University of Chinese Academy of Sciences Shanghai AI Laboratory fvangzhechao21, chenmingxin22, tianpengjuu2)@maaile wua.llll..c..... ucas. ac. cn
yangxue@pjlab.. ong.cn (chengpr, wangzr, 1ixm004499, sunxian)daircas ac.
Abstract
Collaborative trajectory prediction can comprehensively forecast the future motion of objects through multi-view complementary information. However, it encoun- ters two main challenges in multi-drone collaboration settings. The expansive aerial observations make it difficult to generate precise Bird's ' Eye View (BEV) representations. Besides, excessive interactions can not meet real-time prediction requirements within the constrained drone-based communication bandwidth. To ad- dress these problems, we propose a novel framework named "Drones Help Drones" (DHD). Firstly, we incorporate the ground priors provided by the drone's 's inclined observation to estimate the distance between objects and drones, leading to more precise BEV generation. Secondly, we design a selective mechanism based on the local feature discrepancy to prioritize the critical information contributing to prediction tasks during inter-drone interactions. Additionally, we create the first dataset for multi-drone collaborative prediction, named Air-Co-Predd, and conduct quantitative and qualitative experiments to validate the effectiveness our DHD framework. The results demonstrate that compared to state-of-the-art approaches, DHD reduces position deviation in BEV representations by over 20% and requires only a quarter of the transmission ratio for interactions while achieving comparable prediction performance. Moreover, DHD also shows promising generalization to the collaborative 3D object detection in CoPerception-UAN Our source code is available at https: //g thub. com/ con/VangzcBruue/DDi)
Introduction
Multi-drone object trajectory prediction [1-3] aims to collaboratively identify objects and forecast their future movements using multiple drones with overlapping observations, which overcome the single-drone limitations of occlusions, blur, and long-range observations. Given the safety and reliability of task execution, the role of multi-drom contributes to early warning of potential accidents and path planning in drone operations, better serving for intelligent cities [4], transportation [5], aerial surveillance [6] and response systems [7].
object trajectory prediction is indispensable.
Current methods in collaborative object trajectory prediction are primarily categorized into two frameworks: multi-stage prediction [8, 9] and end-to-end prediction [10, 11]. The multi-stage paradigm achieves collaborative prediction based on individual perception results. Specifically,
"Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

it begins with object detection in each view, such as oriented object detection [12]. Then, the multi-view objects are correlated, and the generated trajectories are fed into a regression model for predictions. In contrast, end-to-end approaches transform each view's 2D features into BEV features at a unified 3D coordinate system and conduct feature-level correlation. Subsequently, future instance segmentation and motion are jointly predicted. The end-to-end paradigm significantly enhances accuracy and computational efficiency compared to multi-stage detect-track pracc-predicct pipelines, leading to its widespread popularity and application.
Nevertheless, end-to-end approaches are primarily designed for autonomous driving scenarios and may not directly apply to aerial perspectives. Specifically, they adopt the prevalent "Lift-Splat-Shoot Splat-Shoot" (LSS) [13] to predict pixel-wise categorical depth distribution and reconstruct objects' distribution in 3D space using the camera's 's intrinsics and extrinsics. In aerial contexts, observation distances are considerably more extensive, as depicted in Fig.1. This expanded range increases depth categories and poses substantial challenges to view transformation. Crucially, mistaken feature projection distorts BEV representations, severely undermining the reliability of subsequent multi-drone col- laboration. Furthermore, efficient interaction is essential for real-time collaborative prediction. The prevalent sparse collaborative strategy, where2comm [14], utilizes the downstream detection head for information selection. However, its mode is inflexible and overlooks valuable contextual cues beyond the object information, which are equally essential for motion forecasting.
To address the challenge of view transformation, we ob- serve that the drone's inclined observation leads inter- sections between the sight and the ground plane. Con- sequently, this geometric attribute assigns a theoretical maximum depth to each pixel, providing a constraint for depth estimation. Moreover, given the noticeable gap be- tween objects and the ground plane, it is an alternative to derive intricate depth estimation from the simpler height estimation. For the design of the collaborative strategy, we advocate that a flexible collaborative strategy should dynamically assess the information volume contributed by each region to downstream tasks based on the model's s feedback. This adaptive methodology ensures thorough understanding of the environment, encompassing both foreground objects and their broader surroundings, thus fa- cilitating more accurate decision-making in collaborative trajectory prediction.
VS
Figure 1: Comparative visualization of observations in autonomous driving ver- sus aerial surveillance.
Building upon the mentioned solutions, this research presents collaborative prediction framework for drones called DHD, consisting of a Ground-prion prino---ro---bsedd BEV Gen- eration (GBG) module and Sparse Interaction via Sliding Windows (SISW) module. The GBG module calculates the viewing angle of each pixel with the camera's intrinsic and extrinsic parameters. Subsequently, based on the flight altitude, this module determines the theoretical maximum depth for each pixel. With the guidance of the viewing angle and maximum depth information, relatively simple height estimation can derive more precise depth, thus achieving more accurate BEV representations. The SISW module employs sliding windows to analyze discrepancies between the central and surrounding features, thereby quantifying the in- formation volume at each position. Regions exhibiting significant feature variations are assigned higher information scores, indicating that they contain objects or crucial environmental information. Additionally, due to the lack of relevant datasets for multi -rone collaborative prediction. this study utilizes the CARLA [15] simulation platform to generate novel dataset, "Air-Co-Pred"" comprising cooperative observations from four drones across 200 varied scenes. This simulated dataset serves to validate the effectiveness of our proposed DHD framework in aerial collaborations.
In summary, our contributions are listed as follows:
A Ground-prior-based Bird's Eye View (BEV) Generation module is presented to achieve more accurate BEV representations guided by the ground prior and height estimation.
A Sparse Interaction via Sliding Windows module is proposed to improve the accuracy of multi-drone object trajectory prediction through efficient information interaction.

---

View
Festure Appregatioon
U-Witt Predictor
Depth
<+1x
N drones:
Spetial Worping
Decoder
Selection
Segmentation Motion Prediction
Depth
Transform
Feature
Ground-prion sere---rr-b-eeeddd
Sparse Interaction via Sliding Windows Module
Trajectory Prediction
Extraction
BEV Generation Module
Figure 2: The overall architecture of our proposed DHD framework. For clarity, we just present the collaboration between two drones.
A simulated dataset, "Air-Co-Pred", designed for multi-drone collaborative prediction, is introduced to validate the effectiveness of the proposed DHD framework.
2 Related Work
Benefiting from the scale consistency and unified coordinate system, BEV extensively utilized in collaborative perception. The generation of BEV features currently categorized into two types: explicit 2D-3D mapping and implicit 3D-2D mapping [16]. In the explicit mapping paradigm, PON [17] introduces a dense transformer module to learn the mappings between image and BEV representations. LSS [13] employs learnable categorical depth distribution to lift the 2D) features to 3D space, and BEVDepth [18] further utilizes explicit supervision on the predicted depth. In the implicit mapping paradigm, BEVFormer [19] leverages the deformable attention mechanism for BEV feature generation, while PolarFormer [17] adopts polar coordinates for precise feature localization. This study adopts 2D 3D generation manner similar LSS. Depth estimation plays a crucial role in the BEV generation, and we are committed to its enhancement.
BEV Generation.
Collaborative Strategy. Current collaborative methods include www.mmmemeet-eeeaeeeeeee based early col- laboration, result- late collaboration, and feature-based intermediate collaboration [20]. Due the nerformance-bandvee
trade-off, intermediate collaboration is extensively studied. For ample, Who2com [21] introduces handshake mechanism to select collaborative partners, while When2com [22] further determines when to initiate collaboration. Where2comm [14] employs the detection head to direct regions for sparse interactions, V2VNet [23] achieves multi-round message exchange via graph neural networks. ViX-ViT [24] explore correlations among hetero- geneous collaborators through transformer blocks. DiscoNet [25] employs leowledge distillation, whereas CORE [26] utilizes reconstruction concepts facilitate feature interactions. UMC [27] and SCOPE [28] leverage temporal information to guide feature fusion. However, these strategies are mostly optimized for detection tasks or merely improving feature-level representations. Concerning collaborative forecasting, V2X-Grap [29] utilizes intricate graph structures to forecast motion based on vector maps instead of the vision-based inputs as this study.
Methodology
3.1 Problem Formulation
This section presents DHD, well-designed collaborative framework for multi-drone object trajectory prediction, as depicted in Fig. 2. The proposed DHD enables multiple drones to share visual information, promoting more holistic perception and prediction. Conceptually, we consider scenario with N drones, each capable of sending and receiving collaboration messages from others, and storing Tj historical frames while predicting Tद future trajectories. For the k-th drone, X denotes the raw observation at input frame 4, and y represents the ground truth at output frame lg The objective of DHD is to optimize the performance of multi-drone object trajectory prediction under

---

total transmission budget B:
1,=1 co (x T2
max
(v2 (z)), subject to {1r)) f==1
ΣΣ 4=1
x
S
B,
5=1
(1)
where g(,, represents the evaluation metric, and 2 is the prediction outcome of drone l output frame to. Co( is the collaborative framework parameterized by and X denotes the message transmitted from the j-th drone the k-th drone input frame 4. The subsequent subsections of Section 3 detail the major components.
3.2 2D Feature Extraction of Observations
Initially, each drone captures observations and individually processes its range view (RV) images through a shared encoder F (-) to ex- tract semantic information. We adopt the backbone of EfficientNet- B4 [30] as the encoder for each drone because of its low infer- ence latency and optimized memory usage, consistent with the literature [31]. For the k-th drone, given its observation X at the input frame 4, the extracted 2D feature map is denoted Fk:20 = Fne ( E HR where H, W, C denote its height, width and channel, respectively.
Dapartont
Figure 3: Illustration of the theoretical depth upper-bound and the impact of various viewing angles on depth es- timation.The The depth of ob- jects, D1. can not exceed D Dupperbound-
3.3 Ground-prior-bant based BEV Generation Module
Depth estimation plays critical role in generating BEV represen- tations. However, the vanilla LSS method struggles to approximate the precise depth of each pixel due to the vast observation range of drones. Fortunately, the oblique perspective drones intersects with the ground plane, providing theoretical upper bound for depth and a nearly zero-altitude reference for each pixel, as shown Fig. 3. Guided by this principle, we propose the GBG module, which refines depth at each pixel to generate more accurate BEV representations, with its methodology illustrated in Fig. 4. The process below omits the subscript l and t, for concision.
Derivation of Depth Upper-bound. Given a drone altitude H, the pinhole camera model is utilized to determine the depth upper-bound D D upperhound at each pixel coordinate (u, v), with the help of its camera's intrinsic matrix K, rotation matrix R, and translation vector T:
((u.e) (u.e)
-H H
R ' (-T))]
upporbound
RK 12112
R
22 + R R- 1R-K-1122 1/23"
K
(2)
Furthermore, the viewing angle (m,,)) of each pixel can be represented as arcsin
H
(
upperbound)
(u..e)
Integrating Height into Depth Estimation. In aerial observations, objects are usually closer the ground than to the drones. Therefore, it's more feasible to estimate the distance from the objects the intersection point between their respective viewing ray and the ground, as indicated by line D2 in Fig. 3. Nevertheless, as the viewing angles diminish, the corresponding depth significantly increases. It becomes progressively challenging to estimate the distance to intersection point due to less visual information available at greater distances. Given the static height of objects and apparent feature discrepancy against the alto-uu ground plane, it's an alternative to estimate the object's height hpred and deduce the distance to the intersection points with the help of viewing angles. Accordingly, depth estimation & (-) can be succinctly expressed as:
Depih (u, v) = (2,0) upperboumdd sin (Q..)) upperbound (1- /Preed
Apred
(u,,)
(3)
We utilize parametric network PHeight (-) for height estimation, which generates a pixel-wise categorical distribution height, represented as /tpred = HeHighh((/i (Fi)) HxD, Here, D signifies the number of discretized height bins. Subsequently, the estimated height input into Depth to compute the estimated depth dpred- Then, outer product between
BEV Generation Process.

---

Volume
View
Transform
Feature
Height
Compression
Aggregation
BEV Features
Drone;
Division
Drone
Depth
Subtraction
Compression
Top-k
Depth
Depth
Viewing Angle
Dupar
Interaction
Upper-bound
L Complementary
Figure 4: Operation flow GBG module.
Figure Operation flow of SISW Module.
image features F'D and all potential depth candidates dpred is conducted to generate frustum features, which are then projected into the 3D space based on the inverted pinhole camera model. This process results in 3D voxel representation V R xxxxxxxc where Z refers to the height of voxel. Finally, through sum pooling 2sp. these voxel features V are condensed into a single height plane to generate the BEV features:
FREV = (V) E RXXYXC serving for the subsequent collaboration process.
3.4 Sparse Interaction via Sliding Windows Module
In aerial observations, objects are typically small and sparsely distributed, leading to critical informa- tion occupying only a minor portion of the panoramic view. Given this context, the communication overhead can be considerably reduced by transmitting solely the complementary objects during collaborations [14]. Nonetheless, environmental information also crucial for prediction tasks, with studies [32, 33] demonstrating that dynamic entities and static textures within surroundings can provide insights for forecasting future trends. Hence, it's necessary to include essential environmental elements within limited data transmissions for better forecasting. To this end, introduce novel sparse interaction module named SISW for collaboration among drones. This module assesses the information volume across different areas through sliding windows, determining regions for inter-drone interactions. The SISW module's workflow is illustrated in Fig. 5 and delineated below:
Information Volume Assessment. The original BEV features FR,BEV XxYxC compressed to F'EBEV E xx x1 with trainable X convolution, reducing the transmission overhead for inter-drone comparisons of complementary information. Then, a K x K sliding window traverses over the compressed features, calculating the discrepancy between each position's 's features and the central features. The calculated discrepancy is then normalized to the interval (0, 1) via sigmoid function o, with the average discrepancy across the window serving as the indicator of the information volume T:
K-1K Σ
Σ
(F'.Ev aan + b) Fl.BEv(,,))) where (m, n) E XXX
(4)
K2
In this setting, the information volume is almost zero in areas of homogeneity, like empty backgrounds. In contrast, the information volume is much higher in regions with significant feature variations, such as when the area contain the objects or crucial environmental information.
Selective Sparse Interaction. Efficient collaboration among drones aims to share complementary information that supplements receivers' current knowledge. thus enhancing the overall system's accuracy. When k-th drone receives the information volume from j-th drone, complementary scores Sy-* among drones are calculated as I5 O (1 - 1x). .. To achieve sparse interactions, the interaction mask M,-k is determined by the top K from the ranked scores Sy-t and sparse collaborative features Cy-ck are denoted as My-* F,,BEV
Collaborative Features Aggregation. Upon receiving sparse collaborative features, drone k im- plements a geometric transformation to align Cf-. with local coordinate system, ensuring spatial congruence. Subsequently, a learnable Gaussian-basee interpolation Qinee is applied to infill undefined values, smoothing the spatial distribution of the features. This step is essential to mitigate the impacts of numerical zero vectors that could adversely affect subsequent feature fusion pro-
5

---

cesses [27]. The refined collaborative features are then given by: C² (T( Guided by local features BE,, the contribution weight W, of C'j+k towards constructing aggregated
Puar (IF (evvi'
features A is quantified by: W, Σe Poom (IFs mev; multi-layer convolution operations. Moreover, pixel-level weighted fusion is executed to generate
E RXXYX1 where Pcone represents the
the aggregated features Aş for subsequent downstream tasks: A& - W RRXYXC
3.5 Prediction Decoders and Objective Optimization
Aligned with PowerBEV [34), the state-of-the-art approach for joint perception and prediction, temporal U-Net architecture is used to interpret a series of T1 aggregated features {Ai, 1 and forecast further T2 dynamics, such as instance segmentation {yt, 400 Ta = and future flow (yAow) T3 =1,=!-
For the end-to-end training, we adopt a cross-entropy loss for the segmentation task and smooth I distance for the flow loss. The overall loss function is formulated as:
T2 Σ
L T2
Le Me () + X2Lit (9tt
flow flow
(5)
where y 0.95 is temporal discount parameter and A1 X2 are the balancing coefficients adjusted dynamically through uncertainty weighting. the inference phase, Hungarian algorithm correlates multi-frame instances to synthesize present instance segmentation and future motion prediction.
4
Experiments
4.1
Datasets
The available datasets for multi-drone collaboration [14] primarily focus on detection and segmenta- tion, lacking the support for the prediction tasks. To bridge this gap, we create a simulated dataset "AirCCo-Pred" for collaborative trajectory prediction based on CARLA[1S 15]. Specifically, four col- laborative drones are positioned at intersections to monitor traffic flow from different directions. These drones fly at an altitude of 50 meters, covering area of approximately 100m x 100m. They capture images at a frequency of 2Hz to support the temporal prediction task. The collected dataset includes 32k synchronous images with resolution of 1600 x 900 and is split into 170 training scenes and 30 validation scenes. Each frame well-annotated with both 2D and 3D labels, comprising three major object categories: vehicles, cycles, and pedestrians. Given the challenges of tiny objects from aerial perspectives, this study mainly concentrates on the vehicle category, which includes many sub-categories to augment the robustness of identifying various vehicles. To illustrate the challenges of aerial observations intuitively, we present several charts to reflect the characteristics of "Air-Co-Pred such occlusion, long-distant observations, small objects, etc., shown in Fig.6.
4.2 Evaluation Metrics
To evaluate the localization accuracy of generated BEV representations, we utilize the metrics of precision and recall to reflect matches between predicted and ground-truth instance centers. Additionally, the L2 distance is used to quantify the position deviations for successfully matched instances. Regarding downstream tasks, adopt two metries widely used in previous works [31, 34]. For frame-level evaluation, Intersection-over. Union (IoU) evaluates the segmentation quality of objects the present and future frames. The specific calculation is follows:
N-1
En. D.
20g
IoU (1e
(6)
N
En.
3/ 4r
De
where denotes the number of output frames, Dr and y denote the predicted and ground truth semantic segmentation at timestamp t, respectively. As for the video-level evaluation, Video Panoptic Quality (VPQ) reflects the quality of the segmentation and ID consistency of the instances through the video, expressed as:
N
Σ (P.P IoU (pe. qe) FNN
VPQ (t
inst inst)
Σ
(7)
TP
FP
t=u

---

Multi-drone Collaborative Observations
Occlusion Ratio
Statistical Distribution Various Types
Podostaan
12%
Visiiliity
Ambulance
605-885%
Polic
40%
0% 40%
67.9%
30000 12000
Distance between Objects and Drones
Occupancy of Each Object in Aerial Perspectives Occupancy of All Objects in Each 2D Observation
25000
Frequency
15000
10000
20000
30
Distance
20
100 8.
Propartion (%)
3.0
3.5
3.0
20
3/0 2.3
Proportion (%)
10.5
22.
15.8 275
Figure 6: Statistical charts for the Air-Co-Pred dataset, depicting the occlusion within a single view, the number of various object types, the distribution of distances between objects and drones, and the proportion of objects within observations, respectively.
Table I: An analysis of trajectory prediction and localization error across different BEV generation baselines. They follow the SISW-based interaction mechanism for subsequent multi-drone collabora- tion. DHD (w/o H) denotes the DHD variant without integrating height estimation.
IoU (%)
VPQ t
Precision (%) f
Recall
Deviation (m)
Models
Short
Long
Short
Long
Short
Long
Short
Long
Short
Long
LSS [13]
56.29
51.76
44.98
43.13
70.97
75.68
77.88
80.49
1.08
2.18
DVDET [35]
59.37
48.91
49.98
41.84
77.59
56.62
77.85
84.24
1.00
2.28
DHD (wlo H)
58.28
52.73
48.01
44.37
83.37
76.21
77.14
81.28
1.07
1.67
DHD (Ours)
61.39
53.95
50.43
46.15
81.47
85.45
80.25
77.98
0.82
1.46
where TP, FP,, and FN, correspond to true positives, false positives, and false negatives at timestamp f. respectively. True positives are predicted instances with the loU greater than 0.5 and consistent instance IDs with the ground truth, whereas false positives and false negatives are incorrectly predicted and missed instances, respectively.
4.3 Implementation Details
We follow the setup from the existing study [10] for collaborative trajectory prediction. Initially, the raw images, with resolution of 900 1600 pixels, scaled and cropped to size of 224 480. As to view transformation, the height estimation range is configured from 0 to 10 meters, discretized into 100 intervals. Subsequently, for BEV representations, the spatial ranges for the æ, y, and axes are set to [-50, 50). [-50, 50]. and [-60, -40) meters, respectively. We evaluate the model performance across various perceptual scopes: 100 mx 100 m area with 0.5 resolution (long) and 50 mx area with 0.25 m resolution (short). Temporally, we utilize three frames from the past is (including the present frame) predict both semantic segmentation and instance motion for the future four frames (2s). Besides, we select 7 7 window size and set the transmission ratio as 25% for the SISW module for comprehensively optimal performance. The relative details on hyper-parameter ablation studies are provided in the supplementary material. Our DHD framework is trained with Adam optimizer at an initial learning rate of 3 x 10-1, runs on four RTX 4090 GPUs, handling batch size of 4 for 20 epochs.
7

---

Table 2: A comparison of collaboration baselines for enhanced prediction performance. Early collaboration refers to the collaboration of raw observations, where multi-view images jointly generate BEV representations. Intermediate collaboration focuses on the feature-level interactions to achieve comprehensive BEV representations. The fully connected paradigm shares complete features among all members, while the partially connected one restricts interactions to certain members or regions. Late collaboration merges the individual prediction results from multiple drones, All the collaboration approaches adopt the GBG module for BEV generation. Our DHD performs best in the partially connected intermediate collaboration paradigm.
IoU (%) f
VPQ (%)* +
Models
Short
Long
Short
Long
Collaboration
38.1
32.6
31.1
27.8
Early Collaboration
62.8
54.5
51.7
45.9
Late Collaboration
57.1
51.4
47.5
43.6
Intermediate Collaboration
V2X-ViT [24]
61.5
53.3
50.2
45.7
(Fully Connected)
V2VNet [23]
59.6
53.8
50.5
46.9
Who2com [2]
52.9
44.3
40.5
37.0
When2com [22]
55.7
45.8
40.8
40.4
Intermediate Collaboration Where2comm [14]
57.5
51.4
48.6
44.2
(Partially Connected)
UMC [27]
57.3
52.3
48.3
44.3
UMC (w/o GRU)
58.8
53.2
48.5
45.9
DHD (Ours)
61.4
54.0
50.4
46.2
4.4 Quantitative Evaluation
Benchmark Comparison in BEV Generation. We select the vanilla LSS [13] and the drone-specifts DVDET [35] as baselines. For fairness, their depth estimation range is set from 1 to 100 meters, divided into 100 intervals. As shown in Table 1, DHDD outperforms the vanilla LSS regarding downstream performance, showing a 9.06% improvement in loU and a 12.11% increase in VPQ within short-range observations. Furthermore, it demonstrates 5.06% improvement in IoU and 6.42% increase in VPQ for the long-range setting. While DVDET incorporates a deformable attention mechanism refine BEV representations, exhibits gains in the short-range setting but notable decline in the long-range setting in contrast to the vanilla LSS. Inaccurate depth estimation causes the projected objects shift from their correct positions, resulting in mismatches with the ground truth. Specifically, DHD achieves fewer mistaken and missing matches and better localization, with over 20% reduction in position deviation. Remarkably, the DHD without height estimation still achieves performance gains and reduces long-range localization errors solely by relying on ground priors.
Benchmark Comparison in Collaboration. Table 2 shows that DHD achieves performance comparable to early collaboration, particularly within the long perceptual range setting. It also significantly outperforms No-Collaboration, with increase of 42.61% in IoU and 45.90% in VPQ for the short-range setting, and 74.29% in IoU and 79.36% in VPQ for the long-range setting, respectively, revealing the effectiveness of collaboration. In contrast fully connected baselines, DHD can approach and even exceed similar performance with only a quarter of the transmission ratio. When compared to the partially connected baselines, DHD surpasses When2com, the upgraded version of Who2com, by almost 20% in both IoU and VPQ. Furthermore, DHD also beats the previous ofate-of-the-art method, Where2com, by a considerable margin, for example, 5.82% IoU and 3.85% VPQ in the long-range setting. Because our DHD) not only considers the foreground objects but also incorporates relevant environmental information for predictions. Notably, the recent saate-of-the-ane art method, UMC, with GRU-based feature fusion, exhibits 3~ 4% lower performance than ours. We find that its temporal fusion deteriorates original downstream tasks. Although this temporal prior has enhanced object detection, it may lead to perception aliasing for prediction tasks.
4.5 Qualitative Evaluation
Visualization of Prediction Results. Fig. 7 demonstrates that collaboration among drones, when compared to collaboration, facilitates the acquisition of the location and state of occluded and out-of-range objects through feature-level interactions. Furthermore, DHD correctly forecasts the trajectory of multiple objects in challenging intersections and achieves more accurate segmentation
8

---

and prediction results compared to the well-known baseline. Where2com. This is attributed to Where2comm's transmitted features to focus on foreground objects while neglecting the surroundings that contributed to downstream tasks. These findings are consistent with our quantitative evaluations.
Aerial Observations
No Collaboration
WhereZcomm
DHD (ours)
Ground Truth
Drone
branes
Drone
Dione
Drone
Drane
Drone
Figure 7: A comparative analysis of visualizations among various collaboration baselines. Each instance allocated distinct color, and its predicted trajectory represented with the same color and slight transparency. Red circles highlight the areas where other baselines make mistaken predictions.
Aerial Observations
Features
Drone
Sparse Drone Features
Interpolated
Features
Drone
Drane
Figure 8: The visualization of feature- level interactions. Green circles represent the primary comple- mentary area, including missing objects and crucial environmental cues.
Visualization of Sparse Interactions. examine the interaction during the collaboration, we analyze sparse features transmitted between Drone 1 and Drone 2 via the SISW module. Fig.8 shows that transmissions include not only object features (in dark gray) but also surrounding features (in light white), which is more prominent in column 5. Columns 2 and 3 highlight an obvious discrepancy the mean of features, which could affect subsequent fusion processes. We implement a learnable interpolation to fill in blank features to alleviate the gap. Above all, these visualizations are in agreement with our designs and expectations.
4.6 Ablation Studies
Effectiveness of Proposed Modules. Our DHD frame- work introduces two innovative components: the GBG and SISW modules. We evaluate these modules based on their ability to enhance prediction accuracy and optimize the performance-transmiise
Table 3: The ablation study of proposed modules. 'Ratio" in the table refers to the transmission ratio during collaborative interactions.
trade-off, depicted in Table 3. The GBG-only variant significantly enhances predic- tion accuracy with improvements of approximately 10% in short-range and 5% long-range predictions, primar- ily attributable to the more accurate BEV representations guided by the ground prior. The SISW-only variant re-
GBG
SISW
Short
(5))
Short
(P)
Lung
Ratio
43.5
43.1
61.4
50.4
46
duces transmission costs by 75% % with only marginal performance decrease of about 1% relative to the baseline. Overall, our DHD, integrated with both modules, can achieve a balance between prediction enhancement and transmission cost.
Performance Changes with the Number of Collaborative Drones. To explore how the num- ber of collaborating drones affects performance, conducted relevant experiments, shown

---

in Table 4. For the short-range area of 50m x 50m, three drones are sufficient to predict tra- jectories, with performance comparable to that of four drones. However, for the long-range area of 100m x 100m, predictive performance improves as the number of drones increases. This difference can be attributed to the positional layout of the drones in the dataset, Air-Co-Pred The drones are situated near intersections to monitor traffic flow, facilitat- ing comprehensive coverage of the short-range areas close to these intersections. In contrast, much of the informa- tion in the long-range areas extends along specific road branches, which may only be captured by a single drone. Therefore, increasing the number of drones results in more comprehensive coverage of the long-range areas.
Table 4: Performance metrics for differ- ent numbers of drones.
Num of Drones
IoU {%
VPQ (%) 1
Short
Long
Short
Long
41.5
31.1
33.5
25.6
38.4
61.3
50.2
50.4
43.4
61.4
54.0
50.4
46.2
4.7 Generalization to Collaborative 3D Object Detection
We also conduct generalization validation for collaborative 3D object detection in CoPerception- UAVs [14], a publicly available multi-drone collaborative dataset. We select several models for BEV generation baselines: BEVDet [36] (a modified LSS model for detection), its temporal version BEVDet4D [37]. and DVDET [35]. the official detector for CoPerceepionnUAVs All of these models adopt the SISW module for inter-drone feature-level interactions. The evaluation metrics[38] include mean Average Precision (mAP), mean Absolute Trajectory Error (mATE), mean Absolute Scale Error (mASE), and mean Absolute Orientation Error (mAOE), represent- ing detection accuracy, offset error, size error, and orientation error, respectively. As illustrated in Table. 5, DHD achieves the best performance in mAP, MATE, and mASE. Specifically, DHD shows a 25.2% improvement in mAP and reductions of 13.7% in mATE and 2.9% in mASE compared BEVDet. Although BEVDet4D refines depth estimation with temporal information, the results indicate that the ground prior is more critical for aerial depth estimation. Notably, DVDET outperforms DHD orientation error, probably attributed its deformable attention.
5 Conclusion and Limitation
Table 5: A comparative analysis of collaborative 3D object detection across different BEV genera- tion baselines.
This paper presents DHD, a collaborative frame- work for multi-drone object trajectory predic- tion. Its GBG module leverages the ground prior and simpler height estimation for more accurate BEV representations. Meanwhile, the SISW module adaptively selects regions for collabora- tive interactions, guided by the sliding window's information volume calculation. Additionally, we construct the first simulated dataset of multi-drone collaborative prediction, named "Air-Co-Pred" evaluate the effectiveness of DHD through quantitative and qualitative experiments.
Models
mAP 1
mATE J
mASE 1
mAOE
BEVDer [36]
0.349
1.011
0.171
1.601
REVDe1AD [37]
0.371
0.949
0.172
1,118
DVDET 1351
0.387
0.844
DHD (ours)
0.437
0.872
0.166
0.869
Limitation and Future Work. The current work only exploits the simulated setting, an idealized scenario, for multi-drone object trajectory prediction. For practical applicability, future efforts will extend to real-world environments, taking into account realistic challenges such as flight turbulence, rough terrain, camera extrinsic noise, latency, communication frequency etc. Specifically, flight tur- bulence and rough terrain can impact the derivation of depth upper-bound in the GBG module, while inaceurate camera parameters may compromise multi-view projection transformations. Additionally, effectively managing latency and optimizing communication frequency are crucial for collaboration within the SISW module. Addressing these issues is essential for deploying the proposed model in real-world scenarios.
Acknowledgement. This work was supported by the National Nature Science Foundation of China under Grant 62331022 and supported by the Strategie Priority Research Program of the Chinese Academy of Sciences, Grant No. XDA0360303.
10

---

References
[1] Zhihao Liu, Yuanyuan Shang, Timing Li, Guanlin Chen, Yu Wang, Qinghua Hu, and Pengfei Zhu. Robust multi-drone multi-target tracking resolve target occlusion: benchmark. IEEE Transactions on Multimedia, 2023.
[2] Pengfei Zhu, Jiayu Zheng, Dawei Du Longyin Wen, Yiming Sun, and Qinghua Hu. Multi- ttone-based single object tracking with agent sharing network. IEEE Transactions Circuits and Systems for Video Technology, 31((0):4058 4070, 2020.
[3] Guanlin Chen, Pengfei Zhu, Bing Cao, Xing Wang. and Qinghua Hu. Cross-drone transformer network for robust single object tracking. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
[4] Huu Phuoc Dai Nguyen and Dinh Dung Nguyen. Drone application in smart cities: The general overview of security vulnerabilities and countermeasures for data communication. Development and Future of Internet of Drones (IoD); Insights, Trends and Road Ahead, pages 185-210, 2021.
[5] Chenyou Fan, Junjie Hu, and Jianwei Huang. Few-shot multi-agent perception with ranking- based feature learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023.
[6] Xianyu Chang, Chaoqun Yang, Junfeng Wu, Xiufang Shi, and Zhiguo Shi. A surveillance system for drone localization and tracking using acoustic arrays. In 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), pages 573-577. IEEE, 2018.
[7] Chengyi Qu, Francesco Betti Sorbelli, Rounak Singh. Prasad Calyam, and Sajal Das. Environmentally-aww and energy energy-efficient multi-dron coordination and networking for disas- ter response. IEEE transactions on network and service management, 2023.
[8] Haotian Zhang, Gaoang Wang, Zhichao Lei, and Jenq-Neng Hwang. Eye in the sky: Drone- based object tracking and 3d localization. In Proceedings of the 27th ACM international conference on multimedia, pages 899-909999999 907, 2019.
[9] Zhuoyuan Wu, Yuping Wang, Hengbo Ma, Zhaowei Li, Hang Qiu, and Jiachen Li. Cmp: Coop- erative motion prediction with multi-agent communication. arXiv preprint arXiv:2403. 17916, 2024.
[10] Tianqi Wang, Sukmin Kim, Ji Wenxuan, Enze Xie, Chongjian Ge, Junsong Chen, Zhenguo Li, and Ping Luo. Deepaccident: motion and accident prediction benchmark for v2x autonomous driving. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2024.
[11] Jiaxun Cui, Hang Qiu. Dian Chen, Peter Stone, and Yuke Zhu. Coopernaut: End-to-end driving with cooperative perception for networked vehicles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17252-17262, 2022.
[12] Xue Yang. Junchi Yan, Ziming Feng, and Tao He. R3det: Refined single-stage detector with feature refinement for rotating object. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 3163 3163-3171, 2021.
[13] Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In Computer Vision-ECCS 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16, pages 194-210. Springer, 2020.
[14] Yue Hu, Shaoheng Fang, Zixing Lei, Yiqi Zhong, and Siheng Chen. Where2comm: Communication-effiiieen collaborative perception via spatial confidence maps. Advances in newral information processing systems, 35:4874-4886, 2022.
[15] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In Conference robot learning, pages 1-16. PMLR, 2017.

---

[16] Hongyang Li, Chonghao Sima, Jifeng Dai, Wenhai Wang, Lewei Lu, Huijie Wang, Jia Zeng, Zhiqi Li, Jiazhi Yang. Hanming Deng, al. Delving into the devils of bird's bird's-eye-view percep- tion: A review, evaluation and recipe. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2023.
[17] Thomas Roddick and Roberto Cipolla. Predicting semantic map representations from images using pyramid occupancy networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages I 11138-111 1147, 2020.
[18] Yinhao Li, Zheng Ge. Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1477-1485, 2023.
[19] Yanqin Jiang, Li Zhang, Zhenwei Miao, Xiatian Zhu, Jin Gao, Weiming Hu, and Yu-Gang Jiang. Polarformer: Multi-camera 3d object detection with polar transformer. In Proceedings of the AAAI conference on Artificial Intelligence, volume 37, pages 1042-1050, 2023.
[20] Yushan Han, Hui Zhang, Huifang Li, Yi Jin, Congyan Lang, and Yidong Li. Collabora- tive perception in autonomous driving: Methods, datasets, and challenges. IEEE Intelligent Transportation Systems Magazine, 2023.
[21] Yen-Cheng Liu, Junjiao Tian, Chih- Yao Ma. Nathan Glaser, Chia-Wer Kuo, and Zsolt Kira. Who2com: Collaborative perception via learnable handshake communication. In 2020 IEEE International Conference on Robolics and Automation (ICRA), pages 6876-6883. IEEE, 2020.
[22] Yen-Cheng Liu, Junjiao Tian, Nathaniel Glaser, and Zsolt Kira. When2com: Multi-agent perception via communication graph grouping. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 4 4106-4115, 2020.
[23] Tsun-Hsuaa Wang, Sivabalan Manivasagam, Ming Liang, Bin Yang, Wenyuan Zeng, and Raquel Urtasun. V2vnet: Vehicle-to-veed ve-velelle communication for joint perception and prediction. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Purt II 16, pages 605-621. Springer, 2020.
[24] Runsheng Xu, Hao Xiang, Zhengzhong Tu, Xin Xia, Ming-Hsuan Yang, and Jiaqi Ma. V2x-vit: Vehicle-to-everyyhinng cooperative perception with vision transformer. In European conference on computer vision, pages 107-124. Springer, 2022.
[25] Yiming Li, Shunli Ren, Pengxiang Wu, Siheng Chen, Chen Feng, and Wenjun Zhang. Learning distilled collaboration graph for multi-agent perception. Advances in Neural Information Processing Systems, 34:29541-299 -29552, 2021.
[26] Binglu Wang, Lei Zhang, Zhaozhong Wang. Yongqiang Zhao, and Tianfci Zhou. Core: Cooper- ative reconstruction for multi-agent perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8710-8720, 2023.
[27] Tianhang Wang, Guang Chen, Kai Chen, Zhengfa Liu, Bo Zhang, Alois Knoll, and Changjun Jiang. Ume: A unified bandwidth-efficient and multi-resolution based collaborative perception framework. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8187-8196, 2023.
[28] Kun Yang, Dingkang Yang, Jingyu Zhang, Mingcheng Li, Yang Liu, Jing Liu, Hanqi Wang. Peng Sun, and Liang Song. Spatio-temporal domain awareness for multi-agent collaborative perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 23383-23399 2023.
[29] Hongzhi Ruan, Haibao Yu, Wenxian Yang, Siqi Fan, Yingjuan Tang, and Zaiqing Nie. Learning cooperative trajectory representations for motion forecasting. arXiv preprint arXiv:2311.00371, 2023.
[30] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-6114, PMLR, 2019.
12

---

[31] Anthony Hu, Zak Murez, Nikhil Mohan, Sofía Dudas, Jeffrey Hawke, Vijay Badrinarayanan, Roberto Cipolla, and Alex Kendall. Fiery: Future instance prediction in bird's-ey s-ee view from surround monocular cameras. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15273-15282, 2021.
[32] Zhe Cao, Hang Gao, Karttikeya Mangalam, Qi-Zhi Cai, Minh Vo, and Jitendra Malik. Long- term human motion prediction with scene context. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part 16, pages 387-404. Springer, 2020.
[33] Prasenjit Ghorai, Azim Eskandarian, Young-Keun Kim, and Goodarz Mehr. State estimation and motion prediction of vehicles and vulnerable road users for cooperative autonomous driving: A survey. IEEE transactions on intelligent transportation systems, 139100:1693311700, 2022.
[34] Peizheng Li, Shuxiao Ding, Xieyuanli Chen, Niklas Hanselmann, Marius Cordts, and Juergen Gall. Powerbev: a powerful yet lightweight framework for instance prediction in bird's e-e view. arXiv preprint arXiv:2306. 10761, 2023.
[35] Yue Hu, Shaoheng Fang, Weidi Xie, and Siheng Chen. Aerial monocular 3d object detection. IEEE Robotics and Automation Letters, 1(4):1959-196) 2023.
[36] Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du. Bevdet: ferf-pperormance multi-camera 3d object detection in bird-eye-view. arXiv preprint 12.11:21 2021.
[37] Junjie Huang and Guan Huang. Bevdet4d: Exploit temporal cues in multi-camera 3d object detection. arXiv preprint arXiv:2203. 17054, 2022.
[38] Holger Caesar, Varun Bankiti, Alex Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 11621-11631, 2020.
[39] Changsheng Lv, Mengshi Qi. Xia Li, Zhengyuan Yang, and Huadong Ma. Sgformer: Semantic graph transformer for point cloud-based 3d scene graph generation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages
2024.
[40] Yue Hu, Yifan Lu, Runsheng Xu, Weidi Xie, Siheng Chen, and Yanfeng Wang. Collaboration helps camera overtake lidar in 3d detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9243-9252, 2023.
[41] Xian Sun, Dongshuo Yin, Fei Qin, Hongfeng Yu, Wanxuan Lu, Fanglong Yao, Qibin He, Xingliang Huang, Zhiyuan Yan, Peijin Wang, et al. Revealing influencing factors on global waste distribution via deep-learning based dumpsite detection from satellite imagery. Nature Communications, 14(1):1444 2023.
[42] Dongshuo Yin, Yiran Yang, Zhechao Wang, Hongfeng Yu, Kaiwen Wei, and Xian Sun. 1% vs 100%: Parameter-efficient low rank adapter for dense predictions. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2023.
[43] Dongshuo Yin, Leiyi Hu, Bin Li, Youqun Zhang, and Xue Yang. 5% 100%: Breaking perfor- mance shackles of full fine-tuning on visual recognition tasks. arXiv preprint wxXiv:2408.08345 2024.
[44] Pengju Tian, Peirui Cheng, Yuchao Wang. Zhechao Wang. Zhirui Wang. Menglong Yan, Xue Yang, and Xian Sun. Ucdnet: Multi-uav collaborative 3d object detection network by reliable feature mapping. arXiv preprint arXiv:2406. 04648, 2024.
13

---

6
Appendix
6.1 Air-Co-Pred Dataset Details
Map Creation. The simulation scenes, including the road layouts, static objects, and traffic flow, generated using the CARLA simulation platform. We utilize the "Town10" 10" map provided by CARLA the foundational road layout. This scenario is similar the intelligent urhan setting, with high-rise buildings and roadside trees leading to observation occlusions. These challenges can be effectively addressed through multi-drone collaboration.
Traffic Flow Creation. Moving vehicles in the scene are controlled via CARLA, with hundreds of vehicles spawned in each scene using the official script provided by CARLA. The map's road layout determines each vehicle's 's initial location and motion trajectory.
Sensor Setup. Each drone is equipped with a forward-facing RGB camera with 90 field of view and a resolution of 1600 x 900. The cameras are fixed, with their internal positions and rotation degrees remaining constant. During data collection, each camera's s translation (x, y. z) and rotation (w, x, y. z in quaternion) are recorded in global and ego coordinates. With this sensor configuration, four collaborative drones flying at height of 50 meters can effectively cover area of 100m x 100m.
Data Collection. Our proposed dataset is collected by the CARLA simulation platform under the MIT license. We utilize CARLA to create complex simulation scenes and traffic flow, Aerial observation samples are collected at frequency 2 Hz. We synchronously collect images from four drones, resulting in four images per sample. Additionally, camera intrinsics and extrinsics in global coordinates are provided support coordinate transformation. A total of 32.000 images have been collected to support our experiments. Our ground truth labels for collaborative prediction are derived from 3D bounding boxes of observed targets. Thus, we take advantage of lidar sensors to collect these 3D bounding boxes, which include location (x, y, z), rotation (represented with quaternion), and dimensions (length, width and height), amounting to nearly 430,000 3D bounding boxes.
Data Usage. We randomly split the samples into training and validation sets, yielding 25.6k images for training and 4.4k for validation. The dataset is structured similarly the widely used autonomous driving dataset, nuScenes, so that it can be directly used with the esllbestall nuScenes-devkit.
Drone & Dotection BT
Drons & Detection GT
Drane & Detection GT
Drone & Detection GT
BCV Segmentaion Motion GT
Figure 9: The visualization of our Air-Co-Pred dataset along with the detailed annotations for collaborative detection, segmentation and prediction tasks.
6.2 Comparison with Other Existing Datasets
To demonstrate the contributions of our dataset, Air-Co-pred, we conduct a comparative anal- ysis of existing datasets in the multi-drone collaboration domain. By the submission deadline, several datasets are available for multi-drone collaboration, including two real-world datasets (VisDrone-MDOT [2] and VDMM (1]) and two simulation datasets (Airsim-Map [22] and CoPereeption-UAV [14]).
14

---

Regarding the existing real-world datasets, the VisDrone series has collected substantial amount of real-world video data. However, these datasets are constructed solely from visual perspective and do not provide any information about the drones' poses or camera parameters. As result, they are limited to supporting 2D visual algorithms [39] such as RelD and object tracking, and cannot be used to evaluate our proposed collaborative prediction framework, which integrates both visual and spatial information.
The simulation dataset, Airsim-Mann is designed to demonstrate the effectiveness of who2com and when2com in mitigating image degradation. However, it only provides multi-view 2D semantic seg- mentation masks. The dataset most similar to ours is CoPerception- UAVs, proposed by where2comm. While this dataset focuses exclusively on multi-drone collaborative 3D object detection and has been used to validate frameworks like Where2comm [14] and CoCa3D [40], it falls short in addressing joint temporal tasks. Additionally, its large sampling intervals are inadequate for validating our DHD. To bridge this gap, we propose more comprehensive dataset that includes detailed annotations and an appropriate sampling frequency. This dataset supports wide range of tasks, including 2D/31) detection [41-43], BEV segmentation [31], multi-object tracking [44], and trajectory prediction [34], and facilitates the preliminary validation of multi-drone collaboration across various scenarios within simulation environment.
6.3 Theoretical Justification and Derivation of the GBG Module
The Advantage of Depth Estimations from a Near-Ground Perspective. In acrial observations, objects are generally closer to the ground rather than to the drones, which introduces particular considerations for depth estimation. The estimated range determined by the distance to the drone is
D ((u,e) upperboundl. where H denotes the altitude of the drone, 1max represents the maximum height of the objects, and D Dupperbound (u, e) corresponds to the depth upper-bound of
represented by [H Amaz:
pixel (u, v). In contrast, considering the distance to the ground, the required range is Mmazz sin(Ow) where 0N. denotes the lower-bound of the viewing angle. The discrepancy between these two ranges significant:
Aman
D (u.w) upperbourd
.H + Iemoer
khmma sin ((o.))
khhma + hmaz
le +
Pin ((av))
>
(8)
imazz.
Itmaz
hmax 040
hmmox
0s
Win
Here we substitute D(a,)) with H and set H = hmazs where >>>>> n-class problem becomes increasingly complex as the interval range expands. Therefore, it is more feasible to estimate the distance from objects to the point where their sight intersects the ground.
1. Depth estimation as an
Rationality for Height-Based Depth Estimation. As car of height h moves away from the camera, its viewing angle (m).)) diminishes, as depicted in Fig. A small angular change incurs depth variation Ad, given by:
h
h
h COS ((1,,)) 8.
Ad
(9)
sin(O(u..)) 8) sin (0,o) sin ((w.e)
Additionally, Ad is monotonically decreasing function of the angle 0. As A decreases, the corre- sponding depth significantly increases, which exacerbates the challenge of depth estimation due to less visual information available greater distances. In contrast, object's height remains constant regardless of its position, making it relatively easier to estimate. Furthermore, the task of height estimation even be simplified to an object classification task. For instance, the ground plane maintains an altitude of zero, while different types of vehicles are associated with specific height values.
6.4 Additional Qualitative Evaluation
Impact of Flight Turbulence and Rough Terrain on DHD Performance. We acknowledge that flight vibrations and uneven terrain can interfere with the drone's 's relative height to the ground, affecting the BEV generation from the GBG module. Therefore, we introduce perturbations to the drone's altitude simulate these conditions.
Specifically, we introduce Gaussian noise to the drone's altitude, with noise levels ranging from 0.002 to 0.01. At the highest level, this results in maximum altitude variation of 0.5 meters, which
15

---

Metrics With Altitude Noise In the Short Rangee
Metrics With Altiudde Noise the Long Range
85
Our
-0
VPQ
Metrics
55
45
50
40
0.002
0.004
0.005
0.008
0.01
0.002
0.004
0.006
0.008
0.01
Noise Ratio
Noise Ratio
Figure 10: Impact of introducing varying levels of altitude noise on collaborative prediction perfor- mance. The dashed lines represent the performance of the depth based baseline, LSS.
Metrics with Camera Extrinsic Noise the Short Range
Metrics with Camera Extrinsic Noise in the Long Range
60
VPQ
VPQ
40
Motics
40
30
20
0.30
3332
6:044
0.06
0.000
3.003
0.02
0.04
0.16
3.30
0.08
0003
0.00
0.34
2.00
5.08
0000
0,312
0.304
0.036
0.01
0.000
Nosse
Figure 11: Impact of varying noise levels in camera extrinsics on collaborative prediction perfor- mance.
significant for drone flight. As illustrated in Fig. 10, the results demonstrate that when the noise level exceeds 0.01, the depth estimation advantage conferred by the geometric prior in the GBG module diminishes. However, this limitation of terrain variations is not unique our GBG module. Mainstream BEV generation methods, such as LSS, BEVFormer, and PETR, also assume flat ground. Complex terrain requires further study and can be considered distinct research direction.
The primary goal of our GBG is to explore BEV generation specifically designed for drones. While this is initial attempt, recognize the need account for more complex real-world conditions in future deployments. To address this, we propose a simple yet potentially effective solution: developing a ground flatness estimation module to assess variations in the ground plane, allowing the estimated object height to be adaptively adjusted and thereby mitigating the impact of uneven terrain on subsequent BEV generation.
Impact of Sensor Noise on DHD Performance. Sensor noise primarily affects the accurate acquisi- tion of camera extrinsic parameters. To examine this effect, we introduce Gaussian noise simulate varying levels disturbance the extrinsic calibration.
As illustrated in Fig. 11, increasing noise results in gradual decline in both IoU and VPQ. In short-range settings, noise ratios below 0.003 cause negligible performance drops. However, IoU decreases by about 25% when the noise ratio is between 0.003 and 0.013. Between 0.013 and 0.020, the decline slows, with an additional reduction of approximately 10%. VPQ exhibits a similar trend. In long-range settings, noise ratios below 0.003 also result in acceptable performance declines. However, when the ratio reaches 0.005, noticcable performance degradation occurs, with IoU dropping by 21.3% and VPQ by 32.9%. Overall, noise has more pronounced impact VPQ than on IoU, indicating that camera extrinsic bias more severely affects the consistency of future trajectory predictions. Furthermore, the greater impact of extrinsic noise in long-range rangrenge observations can be attributed to objects at long distances often being observed from a single perspective, lacking the multi-view validation available in short-range scenarios.
These results demonstrate that our DHD can tolerate a small amount of sensor-based extrinsic noise. Besides, larger biases in extrinsic parameters can significantly impact collaborative prediction. Therefore, accurate estimation of these parameters is crucial for maintaining high performance in collaborative perception systems. This finding is equally applicable to real-world scenarios.
16

---

Ground Truth
No Collaboration
DHD
Back
Back
Back
Bottom
Bottom
Bottom
Front
Front
Front
Left
Left
Right
Right
Right
Figure 12: The visualizations illustrate the enhancement of collaborative 3D object detection in CoPerception-UAN UAVs. For clarity, we select one of the five collaborative drones in this dataset, each equipped with five cameras. The red circles highlight areas where single-drone perception fails to detect objects, but our DHD framework successfully detects them.
Visualization of Collaborative 3D Object Detection. As illustrated in Fig. 12, although our DHD framework is not specially designed for collaborative 3D object detection, it still effectively overcomes the limitations of single-drone perception, such as occlusion and out-of-range observations, through multi-drone collaboration.
6.5
Ablation Studies on Hyper Peer-Parreeterss
Efffet of Sliding Window Size in SISW Module The size of the window determines the contextual range around the central point. Smaller windows more sensitive capturing frequenyyy local feature variations, but they may also be susceptible to noise. Conversely, larger windows encompass a broader range of information but potentially neglect valuable local discrepancies. After investigating the effects of various window sizes, as shown in Fig. 13, we select 7x7 window size to assess the information volume in the SISW module for optimal performance.
Performance Enhancement Brought by Transmission Ratio. Fig. shows that each drone's effective BEV feature coverage is approximately 50%. Accordingly, our transmission ratio decreases from 50% to 0. 1% to explore the balance between performance and bandwidth. the curve shown in Fig. 13, as the ratio decreases from 50% to 25%, there is negligible performance variation, maintaining
17

---

54.00
46.5
55
53.75
46.0
50
54.00
46.4-
45
53.75
46.2
53.50
40
53.25
46.0
45.8
53.25
45.0
0.20
0.25
0.30
IoU
35
VPQ
IoU
53.00
44.5
VPQ
0.0
0.1
0.2
0.3
0.4
0.5
Window
Transmission Ratio
Figure 13: Collaborative prediction performance of DHD in long-range settings with the varied transmission ratio and sliding windows.
relatively high value. However, significant decline in performance occurs below 10%, with no discernible benefit below 0.25%. To consider comprehensively, we set the transmission ratio as 25% for DHD's SISW module.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction succinetly outline the challenges in the multi- drone object trajectory prediction and introduce the Drones Help Drones (DHD) framework address these issues.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations of this work in the Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer: [NA]
Justification:
This paper focuses on the experimental demonstration of the proposed frame- work, and does not include theoretical results proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The details regarding the simulated dataset and experimental procedures are thoroughly outlined in Section 4, ensuring the replication of the findings.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
20

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes].
Justification: The source codes of the core modules and a demo dataset are provided in the supplemental materials zip file. The complete source code will be released upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. cc/public/aeeeccccccccccccccecct habbic/ggiids/////e Ctte//oooeubbiiiic wwwiibbmsssiini swiissosisiyyoooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper comprehensively outlines all relevant details pertaining to training and testing procedures in Section 4, including the specific hyper-parameter settings, optimizer configurations, and other pertinent information.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We have not repeated experiments many times get error bars since experi- ments of multi-drone object trajectory prediction on large scale datasets is time-consuming,
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
21

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The specifics about compute resources are thoroughly detailed in Section 4.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: tttttps://eeurre/ nwwrips.....p// cc Cetpblic/Ethcsssiiiiiiee ines?
Answer: [Yes]
Justification:
This research has no potential harm, socictal impact, or harmful consequences.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The research presented in this paper remains purely at the theoretical and experimental levels and has not yet resulted in any societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
22

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper is unrelated to the mentioned high-risk data or models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes].
Justification: The license and terms of use for our simulated dataset, "QuadDrones200", thoroughly detailed in Section 4.1 and the supplemental material.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
23

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cow datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes].
Justification: See Section 4.1 and the supplemental material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA].
Justification: This paper does not involve any crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA].
Justification:
This paper does not involve researches with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "JgQORS2MpL", "reviewer": "Reviewer_CzWU", "summary": "This paper proposes a framework, Drones Help Drones (DHD), which tackles trajectory prediction of objects in the scene. DHD consists of a Ground Prior Based Bird’s Eye View (BEV) Generation (GBG) module, which provides depth estimation from the drone to an object using ground priors to create an accurate BEV representation of the features. It also utilises Sparse Interaction via Sliding Window (SISW) to minimize the data transmission cost between drones. In addition, the authors develop a new dataset for multi-drone collaboration prediction, \"Air-Co-Pred\". The paper is interesting in the field of collaborative AI.", "questions": "There are many questions to address as the following:\n- With the DHD framework, given objects to track/predict, what is the optimal number of drones that need to be “watching” the object in order to produce the best prediction? Is it always the case that the more drones that have the object in view, the better the trajectory prediction? Why four collaborative drones in Air-Co-Pred? Can we get away with just 2 drones? \n- Can you explain more about why CARLA is used to simulate and produce the dataset? Has anyone attempted trajectory prediction with multiple drones in real-world settings? Is it feasible? What needs to happen before DHD can be deployed in production in the real-world setting? Maybe it can be included in a discussion section.\nIs there an optimal drone height/altitude? Is 50 meters the best height value, so it’s used in the dataset?\n- How often should the drones be communicating/transmitting data to each other? Is it the same as the aerial observation samples being collected (frequency of 2 Hz)? Can it be reduced to further lower transmission data?\n- How does the DHD framework respond to noises, such as flight turbulence?\n- What will be the performance if we turn on/off multiple components, such as BEV or Sparse Interaction? This should be included in ablation studies.\n- How far are we from real-world experiments?\n- What will be the impacts of your study to the field?", "limitations": "The authors have raised concerns about the use of simulated settings, which limit the research's practicality. It is important to develop real-world scenarios for future validations of the research.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-structured, referencing a well-established research stream in computer vision and collaborative object trajectory prediction. The following are some strengths of the paper:\n- The research problem is clearly described with good visuals and diagrams, which aid in the explanation.\n- There are several contributions, including BEV generation, sliding windows for sparse interaction, and Air-Co-Pred simulated dataset\n- The paper provides both quantitative and qualitative assessments of their framework, compared to baseline and state-of-the-art, such as Who2com and Where2com. It demonstrates good improvements.\n- The provided appendices are useful for further details and ablation studies.", "weaknesses": "The paper has several weaknesses:\n- It investigates methods to overcome single-drone issues, such as occlusions and blurs; nevertheless, it is also important to discuss a bigger picture of their use cases, including accident prevention and path planning, in greater details with certain limitations. For example, in accident prevention, if multiple drones collaborate and predict an accident is about to happen, what can it do? Does it then communicate/inject commands over the air to the vehicle causing certain actions?\n- The DHD framework consists of feature extraction, BEV, Sparse Interaction via Sliding Windows, and Trajectory Prediction. The idea of having sliding windows for sparse interaction is quite interesting; nevertheless, relying on a local coordinate system and pixel-level weight fusion can be a weak spot when it comes to real-world settings. It is also important to examine how each of the modules contributes to the overall performance in ablation studies.\n- The development of Air-Co-Pred remains questionable, based on CARLA. There are indeed many CARLA-based datasets and it is important to compare your dataset with others to prevent overlappings/duplications.\n- It is vital to discuss the following papers, in relation to the work:\n1. Wei, S., Wei, Y., Hu, Y., Lu, Y., Zhong, Y., Chen, S., & Zhang, Y. (2024). Asynchrony-robust collaborative perception via bird's eye view flow. Advances in Neural Information Processing Systems, 36.\n2. Lu, Y., Hu, Y., Zhong, Y., Wang, D., Chen, S., & Wang, Y. (2024). An extensible framework for open heterogeneous collaborative perception. arXiv preprint arXiv:2401.13964.\n3. Liang, J., Jiang, L., & Hauptmann, A. (2020). Simaug: Learning robust representations from simulation for trajectory prediction. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16 (pp. 275-292). Springer International Publishing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### [W1]:\nWe envision a scenario where vehicles are centrally coordinated. In this setting, collaborative drones predict abnormal trajectories that may lead to accidents. Upon identifying the risks, the drones use wireless communication to alert nearby vehicles, prompting them to take evasive actions to prevent accidents. The system can also integrate with LLMs to provide solutions. Besides, it can be used to forecast traffic congestion, offering lane recommendations.\n### [W2]:\nAs to the concern about obtaining UAV coordinate systems in real-world scenarios, these parameters are considered readily available data sources in [1].\n\nRegarding the pixel-level weight fusion in SISW, the reviewer's concern may stem from the computational overhead. Our approach focuses on fine-grained feature fusion, which is indeed more complex than simple addition or concatenation. As shown in TABLE III of the PDF, we supplement the results with common fusion methods. As expected, they perform worse than ours.\n\n[1] Zhao M, Chen J, Song S, et al. Proposition of UAV multi-angle nap-of-the-object image acquisition framework based on a quality evaluation system for a 3D real scene model of a high-steep rock slope[J]. \n### [W3]:\nBelow is a description of other CARLA-based datasets.\n\n**For Autonomous Driving:**\n- **OPV2V**: Focuses V2V collaborative perception to enhance safety and efficiency in autonomous driving. \n- **OPV2V-Occupancy**: An extension of OPV2V, it introduces an occupancy grid map to represent the distribution of surrounding obstacles.\n- **V2X-Sim**: Simulates both V2V and V2I modes, covering a range of complex scenarios. However, it does not simulate noise environments and includes only a single type of road.\n- **V2XSet**: Simulates real-world noise in V2X collaboration.\n- **V2XSet-w**: An extended version of V2XSet, it includes more adverse weather conditions.\n- **DeepAccident**: Generates various traffic accident scenes and records relevant sensor data. This dataset is intended for traffic accident analysis and prevention.\n\n**For UAV Research:**\n- **CoPerception-UAVs**: Refer to Answer 3 in the overall rebuttal. \n\nIt is evident that the key difference between our Air-Co-Pred and other CARLA-based datasets is our aim to solve fundamentally different problems from the outset. Furthermore, in terms of dataset characteristics, unlike the horizontal perspective of autonomous driving scenes, our multi-drone dataset focuses on aerial observations with oblique views over wide ranges. Additionally, compared to CoPerceptionUAVs, our dataset supports additional temporal tasks such as prediction and tracking.\n### [W4]:\n- **CoBEVFlow**: It addresses the challenge of asynchrony in collaborative perception by focusing on robustness to temporal misalignments. Although our current work does not account for asynchrony, the proposed solutions in their study provide valuable insights.\n- **HEAL**: Lu et al. propose a framework for heterogeneous collaborative perception, focusing on collaboration among diverse data sources and perception models. Our work could benefit from their approach when dealing with heterogeneous members in collaboration.\n- **SimAug**: This research explores learning robust representations for trajectory prediction using rich simulated data from CARLA. Specifically, they identify hard examples from multi-view information and combine them with the original view through adversarial learning to better adapt to real-world environments.\n### [Q1]: \nRefer to Answer 2 of the overall rebuttal.\n### [Q2]: \n- **Why CARLA?** \n  CARLA offers the flexibility to create complex urban traffic scenarios, providing rich dynamic objects along with automated annotations, including the camera intrinsics, extrinsics, and coordinate information needed for BEV generation. It can generate extensive data for algorithm validation with minimal time and manpower investment.\n- **Is it feasible for multi-drone trajectory prediction?**\n  To the best of our knowledge, real-world trajectory prediction with multiple drones is still in an early stage. The most relevant multi-drone collaborative datasets, such as VisDrone-MDOT and VisDrone-MDMT, are primarily collected for tracking. However, their temporal continuity and ID information make them adaptable for trajectory prediction, indicating the feasibility of using multiple drones for joint prediction.\n- **Why 50m?**\nAt this altitude, four drones near intersections can collaboratively cover an area of around 100mx100m. This height also results in significant occlusions, such as vehicles being obscured by buildings or trees, which we aim to address through multi-drone collaboration.\n### [Q3]: \nWe set the interaction frequency to match the sampling frequency of 2 Hz. However, we can explore reducing this by leveraging recent multi-drone collaboration predictions, such as instance segmentation and offsets, to extrapolate trends and strategically fuse them with current single-view predictions.\n### [Q4]: \nRefer to Answer 1 of the overall rebuttal.\n### [W2.1 & Q5]: \nRefer to TABLE IV of the PDF.\n### [Q6 & Q2.4]: \nBefore DHD can be deployed in real-world applications, several steps are necessary:\n- **Robustness**: Can handle sensor noise and communication delays, which are common in real-world scenarios.\n- **Scalability and Flexibility**: Adapt to the varying number of drones and remain resilient to drone failures or disconnections.\n- **Environmental Adaptability:**: Account for weather conditions, lighting variations, and complex terrains.\n- **Real-Time Performance**: Optimize for lightweight deployment on edge devices with high FPS.\n### [Q7]:\nTo the best of our knowledge, we are the first to employ an end-to-end approach that combines multi-drone joint observations for trajectory prediction. Additionally, we provide a simulated multi-drone dataset to support this work, which can be used for various tasks and serves as a benchmark for future research in multi-drone collaboration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u79r58u14r", "reviewer": "Reviewer_nEkT", "summary": "The paper proposes a ground-prior-based BEV generation for drone trajectory prediction. The methods also propose an efficient information interaction via a Sliding Windows module. It accesses information volume across different areas through sliding windows. The authors provide experiments on the simulator-based Air-Co-Pred dataset and 3D object detection on CoPerception-UAVs.", "questions": "Can the authors provide a component ablation of only the SISW component, like three rows on a 1:1 ratio? Furthermore, the results of missing GBG and SISW on a 0.25 transition rate are not provided.", "limitations": "The paper relies on simulations using the CARLA platform. Although the simulated environment provides controlled and varied scenarios, it may not fully capture the complexities and unpredictabilities of real-world environments.\n\nIt seems like the sliding window module is not that significant, and the authors have not provided a complete ablation study to show its efficacy. \n\nThe component ablation is not thorough.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tThe flow of the paper is thorough and clear.\n2.\tThe proposed ground-prior-based BEV generation and Sliding Windows module are innovative.", "weaknesses": "The Ground-prior-based BEV Generation (GBG) module relies on certain geometric assumptions for depth estimation. In real-world scenarios, these assumptions might not hold due to variations in terrain, drone stability, and other environmental factors, potentially affecting the accuracy of BEV representations.\n\n\nAlthough the paper mentions using EfficientNet-B4 for feature extraction due to its low latency and optimized memory usage, it does not provide a detailed analysis of the computational efficiency and scalability of the overall framework.\n\n\nThe Air-Co-Pred dataset, while a valuable contribution, is still limited to simulated data. The paper could benefit from additional datasets or validation for the main task.\n\n\nThe proposed Sparse Interaction via Sliding Windows (SISW) module, although innovative, might oversimplify the information selection process. The method assumes that feature discrepancies are the primary indicators of valuable information, which may not always be the case, especially in highly dynamic or cluttered environments.\n\n\nThere is a lack of baseline in the main experiment; the authors have compared their method with only two other methods for BEV generation.\n\nThere is a lack of trajectory prediction evaluation; the authors should provide the average displacement error as well.\n\nTypo in Table 3 caption.\n\nSISW has shown little improvement, and the performance declines when the transition rate is 0.25. Additionally, the authors do not provide the individual result of SISW on 1 with GBG still present.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### [W1 & L1]: Consider variations in terrain, drone stability, and other environmental factors.\nPlease refer to Answer 1 in the overall rebuttal.\n### [W2]: Analysis of the computational efficiency and scalability of the overall framework\nAs seen in Table VI of the PDF, the differences in trainable parameters (which reflect model complexity) and model parameter sizes (which impact memory usage) among the compared methods are not significant, except for Who2com and When2com. Notably, the FLOPs for V2VNet and Late Collaboration are significantly higher than those of other baselines. This is due to the iterative processes involving multiple for-loops for information exchange, which generate multiple parallel paths during computation. Each branch performs independent calculations before merging the results, significantly increasing computational complexity. The same principle applies to the difference in FLOPs between our DHD framework and Where2comm.\n\nFor the investigation of stability, we explore the number of collaborative drones in Answer 2 of the overall rebuttal.\n### [W3]: Additional datasets for validation.\nPlease refer to Answer 3 in the overall rebuttal.\n### [W4]: SISW might oversimplify the information selection process. Can it handle highly dynamic or cluttered environments?\nOur proposed SISW module is specifically designed with the characteristics of aerial perspectives, where targets are often small and sparse, and surrounding environmental textures can guide trend prediction. Specifically, SISW employs a sliding window mechanism to assess and select critical information. This design is simple yet efficient, outperforming previous sparse interaction baselines.\n\nAlthough the SISW module is not primarily designed for highly dynamic or cluttered environments, our dataset, Air-Co-Pred, inherently includes such scenarios, thereby enhancing the module's adaptability and robustness.\nSpecifically, our dataset contains substantial moving vehicles, which aligns with the highly dynamic scenarios mentioned. \nAs illustrated in Figure 6 of the initial manuscript, the low proportion of targets within the field of view indicates a high presence of clutter. \nFurthermore, our adopted BEV representations help filter out much of the clutter in 2D vision during spatial projection by truncating the z-axis.\n\nTherefore, the SISW module proposed in this paper can be considered applicable to dynamic and cluttered environments to a certain extent.\n### [W5]: A lack of BEV baselines.\nThank you for your feedback. In our work, we focus on optimizing depth estimation from aerial perspectives within the BEV generation process. The improvements we've made in the GBG module are indeed applicable to other depth-based BEV generation methods.\nWe recognize the importance of comparing our method against a broader range of baselines.\n\nTo provide additional comparisons, we include methods like **BEVDet4D**, which considers temporal BEV representations, and **BEVerse**, which applies multi-task learning to improve BEV generation. As shown in Table V of the PDF, while multi-task learning offers minimal performance gains, incorporating temporal information does enhance BEV generation. However, our DHD framework with the GBG module, which leverages geometric priors, still outperforms these methods.\n### [W6]: Calculate the average displacement.\nAs illustrated in Table II of the PDF, a lower ADE does not necessarily indicate better prediction performance. For example, \"No Collaboration\", which relies on single-view observations, misses a significant number of objects. However, its calculated ADE is relatively low because this metric only considers matched trajectories. Similarly, methods that performed well in our initial experiments, such as Early Collaboration, V2VNet, V2XViT, and our proposed DHD, exhibit higher ADE compared to Where2comm and UMC. This is partly due to Where2comm and UMC having a higher missing ratio, which excludes some difficult trajectories from the ADE calculation. Therefore, when using ADE as a metric, it's important to also consider the ratio of missing trajectories.\nThis is why we adopt evaluation metrics such as IoU and VPQ, as suggested in the research of our adopted predictor ''PowerBEV''. These metrics provide a more comprehensive assessment by considering missed detections, false positives, and the deviation between predicted and ground truth positions.\n\n### [W7]: Typo errors.\nWe have noted the incorrect usage of quotation marks and will correct it in the revised version.\n### [W8 & Q1 & L2 & L3]: Additional component ablation study about SISW module.\nRegarding your comment that \"SISW has shown little improvement, and the performance declines when the transition rate is 0.25,\" the SISW module is specifically designed to preserve critical information for the prediction task during sparse collaborative interactions. This concept is also positively acknowledged in Strength 2 of Reviewer nhVA's feedback. Given that our method transmits only a subset of features, it is understandable that there may be a slight performance decline compared to the complete feature transmission.\n\nWe have enriched the ablation studies as you suggested in Table IV of the PDF. Notably, when the SISW module is absent, we resort to randomly selecting 25% of spatial features for transmission to achieve spatial sparsity.\nThe additional experiments show that when the transmission ratio is close to 1, our DHD framework, which includes both the GBG and SISW modules, provides a performance gain compared to the variant with only GBG, particularly in the VPQ metric. This improvement is attributed to the collaborative feature aggregation component in SISW, which enhances the weight of features that are beneficial for downstream prediction tasks.\n\nThe enhanced ablation experiments, which consider both module-level and transmission-level factors, clearly demonstrate the effectiveness of both the GBG and SISW modules.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DHnvlaHqA2", "reviewer": "Reviewer_nhVA", "summary": "The paper presents the \"Drones Help Drones\" (DHD) framework for improving collaborative trajectory prediction in multi-drone systems. It addresses challenges with aerial observations and communication bandwidth by: (i) Using ground priors from inclined drone observations to enhance Bird's Eye View (BEV) accuracy; (ii) Implementing a selective mechanism to prioritize essential information, reducing communication needs; (iii) Introducing the \"Air-Co-Pred\" dataset for multi-drone collaborative prediction.\nExperiments show that DHD improves BEV accuracy, reduces required transmission bandwidth, and maintains high prediction performance.", "questions": "See Weaknesses section for questions, especially I would appreciate if the authors highlighted the differences with the other mentioned methodologies.", "limitations": "The authors adequately addressed the limitations.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper presents an innovative approach to multi-drone collaborative trajectory prediction. Here are some observations:\n\nS1. The module for estimating depth from multi-view drone perspectives is convincing and well-formulated. The use of object height contributes significantly to research in this field.\n\nS2. The strategy requires only 25% of the transmission ratio without affecting performance. This might be due to the SISW module, which retains only essential information.\n\nS3. The three strategies (depth estimation, interaction, and prediction) are trained end-to-end. This is somewhat surprising given the architecture design.", "weaknesses": "W1. The strategy follows the classic collaborative framework composed of three modules: BEV estimation, construction of interaction, and prediction. The methodology utilizes well-known strategies from the literature for this task. Specifically, for BEV estimation, it uses part of the framework from [13], although it is well adapted with the inclusion of height in the depth estimation. The Selective Sparse Interaction used in the SISW module appears similar to the approach in [14], as well as the aggregation performed. Finally, the prediction mechanism is taken from [34], as mentioned by the authors in L206.\nAlthough the construction of a unified framework is not trivial, the overall strategy deviates little from existing ones.\n\nW2. The authors have contributed to the creation of a new dataset that could benefit future research, but this will require appropriate adjustments. There is no comparison with other datasets used in the same or related fields, making it impossible to evaluate the quality of the dataset.\n\nW3. The lack of additional datasets also makes the effectiveness of the framework less convincing, as there is no evidence of generalization on other established benchmarks. I understand the uniqueness of the task and wonder if it is possible to test the framework on established benchmarks (e.g., DAIR-V2X, V2XSet, and OPV2V datasets).\n\nW4. In Table 2, the reasoning for separating V2X-ViT and V2VNet from other methodologies is not clear. The distinction between fully/partially connected is not a convincing rationale. Could the authors clarify this point?\n\nW5. [Minor] Some findings in the appendix, such as Figure 9 and Section 6.4, could have been included in the main paper to strengthen certain claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### [W1 & Q1]: Differences with existing methodologies.\nOur DHD builds on existing solutions, but we make significant improvements to address unique challenges in multi-drone collaborative prediction. \nFor instance, long-range aerial observations can lead to substantial depth estimation errors, affecting BEV representations. Additionally, objects in view are typically small and sparse, with critical information making up only a minor portion. However, previous methods, primarily focusing on collaborative detection, have not adequately considered the selection and utilization of sparse information for joint prediction. Our DHD framework is specifically designed to overcome these challenges, distinguishing our approach from others.\n\nRegarding the drone-specific BEV generation module, GBG, while it is based on [13], we introduce innovations of geometric priors to refine depth estimation, as acknowledged by Reviewer CzWU rEnT vjyF. Moreover, the improvements we've made in the LSS method are also applicable to other depth-based BEV generation techniques.\nAs for the similarity with [14], our SISW module focuses on areas with significant feature variations for subsequent collaborative feature fusion, which are likely to contain objects or critical environmental information essential for prediction tasks. This solution is not considered by previous collaborative strategies, including [14]. In terms of implementation, the sliding window mechanism for information volume assessment is a major contribution of our SISW module, as noted by Reviewer CzWU nEkT. Additionally, for the selective sparse interaction component, we optimize the selection mechanism by replacing traditional threshold-based filtering with score ranking. This modification allows us to precisely control transmission volume based on predefined ratios, rather than relying on manually-tuned empirical thresholds.\n\nFor the feature fusion module, while not our primary innovation, our approach differs significantly from [14].  Instead of their attention-based fusion, we use a multi-layer convolutional method to calculate contribution weights for each collaborative feature guided by local features. Theoretically, attention mechanisms are better suited for global feature associations, while the convolutional paradigm is widely recognized for its strength in preserving local features. Since our instance-level prediction task benefits more from strong local feature representations, we opt for this fusion method and provide additional experiments to demonstrate performance differences.\n\n### [Weakness2]: A comparison with other datasets.  \nPlease refer to Answer 3 of the overall rebuttal. Additionally, we provide a detailed comparison of key attributes in the table below:\n\n| **Attribute**                | **VisDrone-MDOT** | **VisDrone-MDMT** | **Airsim-Map** | **CoPerception-UAVs** | **Air-Co-Pred**       |\n|------------------------------|-------------------|-------------------|---------------|-----------------------|-----------------------|\n| Source                       | TCSVT-21          | TMM-23            | CVPR-20       | Neurips-22            | Submit to Neurips-24  |\n| Real/Simulation              | Real              | Real              | Simulation    | Simulation            | Simulation            |\n| Support Tasks                | 2D SOT            | 2D MOT            | 2D Seg        | 2D/3D Det, BEV Seg    | 2D/3D Det, BEV Seg, Pred, MOT |\n| Number of Drones             | 2~3               | 2                 | 5             | 5                     | 4                     |\n| Number of Samples            | 105,584           | 19,839            | 4,000         | 5,276                 | 8,000                 |\n| Frequency                    | 10Hz              | 10Hz              | Unknown       | 0.25Hz                | 2Hz                   |\n| Cam. Params. & Coord. Info.  | x  | x  | x  | √ | √  |\n| Suitability & Justification  | x; w/o Cam. Params. & Coord. Info. | x; w/o Cam. Params. & Coord. Info. | x; w/o Cam. Params. & Coord. Info. & Temporal Annotations.| Partially; Long time intervals render it unsuitable for temporal tasks. | √ |\n\nCam. Params. & Coord. Info. refers to camera parameters and coordinate information, which are critical for BEV generation.\n### [W3]: Additional datasets. (e.g., DAIR-V2X, V2XSet and OPV2V datasets).\nThe benchmarks you mentioned are primarily designed for collaboration in autonomous driving scenarios, where the viewing angles of vehicles are horizontal. This contrasts sharply with the aerial oblique perspectives in our multi-drone collaboration research, making our drone-specific BEV generation module, GBG, less applicable to these benchmarks.\nBesides, in the initial manuscript, we provided experimental results on the publicly available CoPerception-UAVs dataset in Table 4 of the initial manuscript, to demonstrate the generalization capabilities of our framework for collaborative detection.\n\nWe understand your concern about the reliance on simulated multi-drone datasets. In fact, we are currently developing a real-world multi-drone collaboration dataset to further validate and extend our work in future studies.\n### [W4]: Reasons for separating V2X-ViT and V2VNet from others.\nBenefiting from the fully connected paradigm, V2X-ViT and V2VNet achieve complete feature-level interaction among collaborators and employ more complex techniques for collaborative feature fusion. As a result, it is reasonable for them to demonstrate superior performance compared to the partially connected paradigm.\n\nFrom the perspective of communication overhead, it is logical to distinguish V2X-ViT and V2VNet from other communication-efficient methods for a fair comparison. \nNotably, our proposed SISW outperforms current partially connected methods and is comparable to the fully connected paradigm.\n### [W5]: \nThank you for your suggestion regarding the content layout.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YbWHYKjvPf", "reviewer": "Reviewer_rEnT", "summary": "This paper studied the problem of collaborative multi-drone object trajectory prediction. The authors proposed a framework named \"Drones Help Drones\" (DHD) that can improve accuracy compared to existing methods while reducing the required communication bandwidth. To be more specific, the authors leveraged a Ground-prior-based BEV Generation module that can generate more accurate BEV representations, and used a Sparse Interaction via Sliding Windows module that enables efficient information sharing between drones. The authors also constructed a new dataset named \"Air-Co-Pred\" for multi-drone collaborative prediction tasks.", "questions": "see above", "limitations": "Limitations are discussed", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-structured and clearly written, with helpful visualizations that aid in understanding the proposed methods and results. The DHD framework introduces a solution to key challenges in multi-drone collaborative perception, particularly the GBG module for more accurate BEV generation and the SISW module for efficient information sharing. The introduction of the \"Air-Co-Pred\" dataset addresses a gap in existing resources for multi-drone collaborative prediction tasks, potentially benefiting the broader research community.", "weaknesses": "* The primary evaluation is conducted on simulated data. While the authors acknowledge this limitation, it raises questions about the framework's performance in more complex, real-world scenarios with sensor noise, communication delays, environmental variability, and other common difficulties in real-world experiments. \n* It would be interesting to see how the performance scales with the number of collaborating drones, which could provide valuable insights into the method's scalability. Is there an optimal or maximum number of drones for effective collaboration?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### [Weakness 1]: Questions about the framework's performance in more complex, real-world scenarios with sensor noise, communication delays, environmental variability, and other common difficulties in real-world experiments.\n\nTo address the reviewer’s concerns about the gap between real-world scenarios and simulated data, we have incorporated some of the mentioned challenges into our experimental setup to assess their impact on the performance of multi-drone collaborative prediction. Specifically, we examine the effects of sensor noise, which primarily affects the acquisition of camera extrinsics, as well as environmental factors such as drone vibrations and rough terrain, which can influence the drone's relative height to the ground plane. To evaluate the impact of these factors on performance, we conduct complementary experiments by introducing Gaussian noise into these aspects. The statistical results are shown in the attached PDF, and a detailed analysis is provided in Answer 1 of the overall rebuttal.\n\nAs to communication delays, we propose integrating a delay assessment module into the collaborative interaction stage of DHD. This module would compensate for lost or delayed frames by leveraging historical collaborative features. Moreover, the compensation process can be streamlined by prioritizing critical areas using the information volume assessment component in the SISW module. At the downstream decoder, missing predictions can also be filled by blending predictions from adjacent time frames and extrapolating trends accordingly.\n\n### [Weakness 2]: How does the performance scale with the number of collaborative drones? Is there an optimal or maximum number of drones for effective collaboration?\n\nTo explore how the number of collaborating drones affects performance, we conducted relevant experiments, as shown in the table below:\n\n| Num of Drones  | IoU (Short) | IoU (Long) | VPQ (Short) | VPQ (Long) |\n|------|-------------|------------|-------------|------------|\n| 1    | 41.5        | 31.1       | 33.5        | 25.6       |\n| 2    | 57.1        | 45.4       | 48.2        | 38.4       |\n| 3    | 61.3        | 50.2       | 50.4        | 43.4       |\n| 4    | 61.4        | 54.0       | 50.4        | 46.2       |\n\nFor the short-range area of 50m x 50m, three drones are sufficient to predict trajectories, with performance comparable to that of four drones. However, for the long-range area of 100m x 100m, predictive performance improves as the number of drones increases.\n\nThis difference can be attributed to the positional layout of the drones in the dataset, Air-Co-Pred. The drones are situated near intersections to monitor traffic flow, facilitating comprehensive coverage of the short-range areas close to these intersections. In contrast, much of the information in the long-range areas extends along specific road branches, which may only be captured by a single drone. Therefore, increasing the number of drones results in more comprehensive coverage of the long-range areas.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vCeWl0pFpw", "reviewer": "Reviewer_vjyF", "summary": "This paper presents a collaborative framework for multi-drone object trajectory prediction, named DHD, which consists of two specifically designed components: the GBG module, aimed at generating more accurate BEV representations in aerial scenes, and the SISW module, which adaptively selects regions for collaborative interactions. Additionally, a simulated multi-drone collaborative observation dataset is created to demonstrate the effectiveness of the proposed DHD framework.", "questions": "- Could you provide more details about Air-Co-Pred, including the drone attitude settings, flight patterns, the speed of objects, etc.? Why were four drones chosen to collect data? \n- In Section 3.3, the equation regarding the derivation of the depth upper-bound is presented directly without any intermediate steps in the main text or supplementary materials. Please provide the detailed derivation process to help readers understand it better.\n- The reviewer found that the proposed GBG module is highly relevant to the applications of camera intrinsics and extrinsic in UAVs. The reviewer encourages the authors to acknowledge the existence of related works and clarify the differences. \nFor reference, see:  \nShen H, Lin D, Yang X, et al. Vision-Based Multi-Object Tracking through UAV Swarm[J]. IEEE Geoscience and Remote Sensing Letters, 2023.  \nPan T, Dong H, Deng B, et al. Robust Cross-Drone Multi-Target Association Using 3D Spatial Consistency[J]. IEEE Signal Processing Letters, 2023.  \n- Considering the complexity of the proposed framework, could you provide additional details or code for better understanding and reproducibility?", "limitations": "The authors acknowledge that the current focus of this paper is on simulated environments to investigate multi-drone object trajectory prediction without considering latency and camera noise. These are reasonable limitations, and addressing them in future work would be beneficial", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- This paper successfully implements an end-to-end multi-drone object trajectory prediction based purely on visual inputs and extends it to the collaborative object detection task. This work fills in the blank of end-to-end multi-drone collaborative prediction, demonstrating notable originality.\n- The proposed GBG module innovatively leverages the unique inclined observations from drone perspectives as geometric priors and replaces traditional depth estimation with height estimation. These enhancements significantly improve the accuracy of BEV representations for long-range aerial observations compared to the listed baselines, such as LSS and DVDET.\n- This paper presents an efficient communication strategy called SISW, which considers both the limited inter-drone communication and the prediction task's dependency on foreground and contextual information. Experimental results indicate that this design outperforms previous sparse interaction strategies like where2comm.\n- A simulated dataset, \"Air-Co-Pred,\" is created to support multi-drone object trajectory prediction. The dataset enriches the available resources within the multi-drone field and can serve as a valuable benchmark for future research.", "weaknesses": "- The authors need to carefully review the paper for grammar mistakes, typos, and formatting errors. Specific issues include:\n    - Line 82: BEV's full-term appears redundantly.\n    - Line 119: The subscript in$ Y_{k}^{o} $ is incorrect and should be$ Y_{k}^{t_o} $.\n    - Line 152: Figure reference is incorrect.\n    - Line 221: \"2HZ\" is improperly formatted and should be \"2Hz.\"\n    - Equation 6: The subscript next to $ \\Sigma $ should be $ xy $ instead of $ hw $.  \nAddressing these issues will improve the paper's readability.\n\n- The transmission volumes for various collaboration strategies in Table 2 are unclear. It would be beneficial to supplement this information to better reflect the balance between the performance improvements in prediction and the cost of collaborative interactions. Additionally, it appears that the results of DHD in Table 2 are not the best in every column, so why are they bolded?\n- Please consider exploring scenarios where the number of drones varies from 1 to 4. This would help investigate the impact of drone quantity on perception enhancement and also reflect the influence of potential drone failures on performance metrics.\n- In reality, extrinsic parameters are derived through computation and approximation, leading to certain biases. Please explore how the extent of extrinsic bias affects the performance of collaborative prediction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### [Weakness 1]: Grammar mistakes\n We have carefully reviewed and corrected the grammar mistakes, typos, and formatting errors.\n### [Weakness 2]: Transmission volumes. Why the results of DHD in Table 2 bold?\nBased on the V2V communication protocol [1], data broadcasting can achieve a bandwidth of 27 Mbps at a range of 300 meters. Considering the capture frequency of 2 Hz, the bandwidth requirement should be less than 13.5 Mbps to avoid severe delay. The original BEV feature map is `64 x 200 x 200` in size, corresponding to 9.77 MB. The result-level fusion utilizes the output segmentation and offset maps, which are `2 x 200 x 200` in size, corresponding to 0.305 MB. We calculate the transmission expenses for each method, as shown in Table VI of the attached PDF.\n\nAll partially connected methods of intermediate collaboration meet the above bandwidth requirements. Notably, although  V2X-ViT and V2VNet achieve better performance, their high transmission expenses make them less feasible for practical applications. By the way, our proposed DHD has shown the best performance within the partially connected approaches for intermediate collaboration, so we highlight it in bold.\n\n[1] Arena F, Pau G. An overview of vehicular communications[J]. Future internet, 2019, 11(2): 27.\n### [Weakness 3]: Exploring the number of drones varies from 1 to 4?\nTo explore how the number of collaborating drones affects performance, we conduct relevant experiments, as shown in the overall rebuttal.\n\nFor the short-range area of 50mx50m, three drones are sufficient to predict trajectories, with performance comparable to that of four drones. However, for the long-range area of 100mx100m, predictive performance improves as the number of drones increases.\nThis difference arises because the drones in our dataset are positioned near the intersection to monitor traffic flow, making it easier to cover areas close to the intersection, which fall within the short-range area. In contrast, much of the information in the long-range area extends along specific road branches, which are only partially captured by the drones at the intersection. Therefore, having more drones results in more comprehensive coverage of the long-range area.\nConversely, the reduction in the number of drones does indeed reflect the influence of potential drone failures on performance.\n### [Weakness 4]:  Explore how the extent of extrinsics bias affects collaborative prediction?\nWe have conducted a series of experiments by introducing Gaussian noise to the extrinsic parameters of drone cameras, as shown in Fig. 2 of the attached PDF document and described in Answer 1 of the overall rebuttal.\n### [Question 1]:  More details about Air-Co-Pred? Why are totally four drones?\nThe dataset is gathered using four drones, each with a field of view set to 70 degrees by default. The drones are positioned at an altitude of 50 meters, monitoring traffic flow at urban intersections. The speed distribution of the observed objects ranges from 0 to 10 m/s, with low-speed states primarily concentrated around 1.5 m/s and high-speed states around 8 m/s. In the CARLA simulations, speed is influenced by several factors, including the default safe speed settings and the necessity of complying with traffic regulations and preventing collisions. \n\nFurthermore, the four drones are arranged in a square formation to achieve a square-shaped coverage area for comprehensive aerial observations. This configuration allows for overlapping fields of view to mitigate occlusions, and the unique observations from each drone enrich the out-of-sight perception capabilities of the entire system.\n### [Question 2]:  The detailed derivation process.\nAccording to the pinhole camera model, the transformation between  the world coordinate system and the pixel coordinate system can be described by :\n$$\n\\begin{bmatrix}\n\tx \\\\\\\\\n\ty \\\\\\\\\n\tz\n\\end{bmatrix} = {R}^{-1} \\\\left( {K}^{-1} \\begin{bmatrix}\n\tu \\\\\\\\\n\tv \\\\\\\\\n\t1\n\\end{bmatrix} {d} - {T} \\\\right).\n$$\n\nNext, we consider \\\\({M} = {R}^{-1}{K}^{-1} \\\\) and \\\\( {N} = {R}^{-1}(-{T}) \\\\)  for further simplification and derivation:\n$$\n\\begin{bmatrix}\n\tx \\\\\\\\\n\ty \\\\\\\\\n\tz\n\\end{bmatrix} = {M} \\begin{bmatrix}\n\tu \\\\\\\\\n\tv \\\\\\\\\n\t1\n\\end{bmatrix} d + {N} = \\begin{bmatrix}\n\tm_{11} & m_{12} & m_{13} \\\\\\\\\n\tm_{21} & m_{22} & m_{23} \\\\\\\\\n\tm_{31} & m_{32} & m_{33}\n\\end{bmatrix} \\begin{bmatrix}\n\tu \\\\\\\\\n\tv \\\\\\\\\n\t1\n\\end{bmatrix} d + \\begin{bmatrix}\n\tn_1 \\\\\\\\\n\tn_2 \\\\\\\\\n\tn_3\n\\end{bmatrix}.\n$$\nBy setting \\\\( y = -H \\\\), \nthe depth upper-bound can be derived as:\n$$\n{D_{(u,v)}}=\\frac{-H-n_2}{m_{21} u+m_{22} v+m_{23}}.\n$$\n\n### [Question 3]: Clarify the differences with related works. \nIndeed, these two works leverage the camera parameters of UAVs to enhance cross-view association. Specifically, they project 2D detection results from each perspective into a unified 3D space with camera parameters and associate targets based on feature similarity and spatial distance.\nIn contrast, our proposed GBG module uses UAV camera parameters along with the available flight altitude to derive the depth upper bound of each pixel. \nThis geometric prior helps to constrain depth estimation and calculate the viewing angle of each pixel, facilitating the subsequent BEV generation.\nBy acknowledging these works and clarifying our distinct approach, we highlight the unique contributions and applications of the GBG module in multi-drone collaboration.\n### [Question 4]: Additional details or codes.\nWe have already provided the core  code in the initial supplementary materials. Additionally, to ensure better understanding and reproducibility, we plan to release the Air-Co-Pred dataset, the source codes of the complete project, and the model checkpoints, which include our methods and previous baselines. This will contribute to further reproduction and improvement of the system.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Answer 1: Investigate the effects of sensor noise, flight turbulence and rough terrain on the performance of DHD.

### Flight Vibrations and Uneven Terrain.
We acknowledge that flight vibrations and uneven terrain can interfere with the drone's relative height to the ground, affecting the BEV generation from the GBG module. Therefore, we introduce perturbations to the drone's altitude to simulate these conditions.

Specifically, we introduce Gaussian noise to the drone's altitude, with noise levels ranging from 0.002 to 0.01. At the highest level, this results in a maximum altitude variation of 0.5 meters, which is significant for drone flight. As illustrated in Fig. 1 of the attached PDF, the results demonstrate that when the noise level exceeds 0.01, the depth estimation advantage conferred by the geometric prior in the GBG module diminishes.
However, this limitation of terrain variations is not unique to our GBG module. Mainstream BEV generation methods, such as LSS, BEVFormer, and PETR, also assume a flat ground. Complex terrain requires further study and can be considered a distinct research direction.

The primary goal of our GBG is to explore BEV generation specifically designed for drones. While this is an initial attempt, we recognize the need to account for more complex real-world conditions in future deployments. To address this, we propose a simple yet potentially effective solution: developing a ground flatness estimation module to assess variations in the ground plane, allowing the estimated object height to be adaptively adjusted and thereby mitigating the impact of uneven terrain on subsequent BEV generation.

### Sensor Noise, Particularly in Extrinsic Parameters.

As illustrated in Fig. 2 of the PDF, increasing noise results in a gradual decline in both IoU and VPQ. In short-range settings, noise ratios below 0.003 cause negligible performance drops. However, IoU decreases by about 25% when the noise ratio is between 0.003 and 0.013. Between 0.013 and 0.020, the decline slows, with an additional reduction of approximately 10%. VPQ exhibits a similar trend.

In long-range settings, noise ratios below 0.003 also result in acceptable performance declines. However, when the ratio reaches 0.005, noticeable performance degradation occurs, with IoU dropping by 21.3% and VPQ by 32.9%. Overall, noise has a more pronounced impact on VPQ than on IoU, indicating that camera extrinsic bias more severely affects the consistency of future trajectory predictions.
Furthermore, the greater impact of extrinsic noise in long-range observations can be attributed to objects at long distances often being observed from a single perspective, lacking the multi-view validation available in short-range scenarios.

These results demonstrate that our DHD can tolerate a small amount of sensor-based extrinsic noise. Besides, larger biases in extrinsic parameters can significantly impact collaborative prediction. Therefore, accurate estimation of these parameters is crucial for maintaining high performance in collaborative perception systems. This finding is equally applicable to real-world scenarios.

## Answer 2: Investigate the number of collaborative drones.
To explore how the number of collaborating drones affects performance, we conduct relevant experiments, as shown in TABLE I of the PDF.

For the short-range, three drones are sufficient to predict trajectories, with performance comparable to that of four drones. However, for the long-range, predictive performance improves as the number of drones increases.

This difference arises because the drones in our dataset are positioned near the intersection to monitor traffic flow, making it easier to cover areas close to the intersection, which fall within the short-range area. In contrast, much of the information in the long-range area extends along specific road branches, which are only partially captured by the drones at the intersection. Therefore, having more drones results in more comprehensive coverage of the long-range area.

## Answer 3: Lack of comparison with other existing datasets.
To demonstrate the contributions of our dataset, Air-Co-pred, we conduct a comparative analysis of existing datasets in the multi-drone collaboration domain. By the submission deadline, several datasets are available for multi-drone collaboration, including two real-world datasets (VisDrone-MDOT and VisDrone-MDMT) and two simulation datasets (Airsim-Map and CoPerception-UAVs).

Regarding the existing real-world datasets, the VisDrone series has collected a substantial amount of real-world video data. However, these datasets are constructed solely from a visual perspective and do not provide any information about the drones' poses or camera parameters. As a result, they are limited to supporting 2D visual algorithms such as ReID and object tracking, and cannot be used to evaluate our proposed collaborative prediction framework, which integrates both visual and spatial information.

The simulation dataset, Airsim-Map, is designed to demonstrate the effectiveness of who2com and when2com in mitigating image degradation. However, it only provides multi-view 2D semantic segmentation masks.
The dataset most similar to ours is CoPerception-UAVs, proposed by where2comm. While this dataset focuses exclusively on multi-drone collaborative 3D object detection and has been used to validate frameworks like Where2comm and CoCa3D, it falls short in addressing joint temporal tasks. Additionally, its large sampling intervals are inadequate for validating our DHD.
To bridge this gap, we propose a more comprehensive dataset that includes detailed annotations and an appropriate sampling frequency. This dataset supports a wide range of tasks, including 2D/3D detection, BEV segmentation, multi-object tracking, and trajectory prediction, and facilitates the preliminary validation of multi-drone collaboration across various scenarios within a simulation environment.

Meta-review: The paper has undergone thorough evaluation by the reviewers, most of whom have raised their scores after the authors addressed their initial concerns. The reviewers are now generally satisfied with the paper, acknowledging its strengths and contributions to the field. After considering the updated reviews and the paper's overall quality, we recommend accepting this submission.
Decision: Accept (poster)