Fleet Active Learning: A Submodular Maximization Approach
Authors: Oguzhan Akcin
Orhan Unuvar
Onat Ure
Sandeep P. Chinchali
Venue: CoRL 2023 Poster (2023)
Keywords: Active Learning
Cloud Robotics
Robotic Perception

Abstract: In multi-robot systems, robots often gather data to improve the performance of their deep neural networks (DNNs) for perception and planning. Ideally, these robots should select the most informative samples from their local data distributions by employing active learning approaches. However, when the data collection is distributed among multiple robots, redundancy becomes an issue as different robots may select similar data points. To overcome this challenge, we propose a  fleet active learning (FAL) framework in which robots collectively select informative data samples to enhance their DNN models. Our framework leverages submodular maximization techniques to prioritize the selection of samples with high information gain. Through an iterative algorithm, the robots coordinate their efforts to collectively select the most valuable samples while minimizing communication between robots. We provide a theoretical analysis of the performance of our proposed framework and show that it is able to approximate the NP-hard optimal solution. We demonstrate the effectiveness of our framework through experiments on real-world perception and classification datasets, which include autonomous driving datasets such as Berkeley DeepDrive. Our results show an improvement by up to $25.0 \%$ in classification accuracy, $9.2 \%$ in mean average precision and $48.5 \%$ in the submodular objective value compared to a completely distributed baseline.

Fleet Active Learning: A Submodular Maximization Approach
Oguzhan Akcin', Orhan Unuvar", Onat Ure! and Sandeep P. Chinchali"
Abstract: In multi-robot systems, robots often gather data to improve the perfor- mance of their deep neural networks (DNNs) for perception and planning. Ide- ally, these robots should select the most informative samples from their local data distributions by employing active learning approaches. However, when the data collection is distributed among multiple robots, redundancy becomes an issue as different robots may select similar data points. To overcome this challenge, we propose fleet active learning (FAL) framework in which robots collectively select informative data samples to enhance their DNN models. Our framework leverages submodular maximization techniques to prioritize the selection of samples with high information gain. Through an iterative algorithm, the robots coordinate their efforts to collectively select the most valuable samples while minimizing commu- nication between robots. We provide theoretical analysis of the performance of our proposed framework and show that it is able approximate the NP-hard optimal solution. We demonstrate the effectiveness of our framework through experiments real-world perception and classification datasets, which include autonomous driving datasets such as Berkeley DeepDrive. Our results show an improvement by up 25.0% in classification accuracy, 9.2% in mean average precision and 48.5% in the submodular objective value compared to a completely distributed baseline.
1 Introduction
Consider a scenario where a fleet of autonomous vehicles (AVs) operates in various environments, ranging from urban rural or highway settings. These AVs aim enhance their machine learning (ML) models employed in perception, prediction, and planning by collecting data and sharing with central server. An ideal strategy for these AVs is gather diverse data, enabling the trained models to generalize effectively across differrnnt environments. However, due to limitations such as bandwidth, computational resources, and storage capacity, the AVs can only transmit limited amount of data to the central server. Notably, one AV can generate a data stream of more than 20-30 Gigabytes (GB) per second, combining video and LiDAR data [1]. In comparison, a standard 5G wireless network offers bandwidth of only 10 Gbps, which needs to be shared among multiple users [2]. Consequently, it becomes crucial for the AVs to collaborate in selecting the most informative data points that optimize the performance of the trained models.
To illustrate, suppose one AV collects data from congested urban area during peak hours while another AV captures data from sparsely populated rural region. Instead of duplicating efforts in either environment, the AVs should collaborate ensure data diversity in the collected images. By strategically selecting complementary images, the AVs contribute to the creation of a diverse dataset that enhances the robustness of the trained models across various driving scenarios.
In our formulation (Fig. 1), AVs communicate with each other using minimal information and ex- clusively have access to their observed data, ensuring the AVs maintain data privacy and only share selected information. Moreover, we acknowledge that the collected AV data is not independent and identically distributed (i.i.d.), and the incremental value of selecting a particular data point relies on the data points already chosen. Additionally, we do not assume the presence of a perfect labeler capable of labeling data points on the fly; instead, we leverage the outputs of imperfect MIL models.
Department of Electrical and Computer Engineering (ECE), The University of Texas Austin, Austin, TX foguzhanakcin, orhan, unuvar Sutexas. edu, (ureonat, sandeepc) sandeep] @utexas. edu
7th Conference on Robot Learning (CoRI. 2023), Atlanta, USA.

---

Ab
f())
Cloud
9.emb
Action
ai
(=
Ai of 4
x1,,.. x
1 m diy
a
D
D
1=1
A3--1 f()
Retraining
8. emmb
Action a'y
A a4 A
X
Figure I: Fleet Active Learning Framework: Each robot i observes a stream of data points ,..,..,,..,, processes them using its neural network SINN 0.) ) to obtain predictions ý and embeddings emb. Then, robots utilize these embeddings and predictions to select action a, sample data points, maximizing submodular function f while considering the previous robots' actions Af-1. The aggregated action AT of AA-1 passed to the next robot, repeating the process until robots have taken an action, At the end of each round, the actions are shared with the cloud, which labels the newly acquired data points and updates the training dataset D. with newly acquired data points U af. resulting in the creation new dataset D The model retrained with the dataset, yielding 05+1, Finally, the cloud shares the updated model weights with all robots, and each robot updates model parameters accordingly.
. and
To create framework that can be applied to variety of data collection problems, we formulated this problem as a distributed submodular maximization problem. The reasoning behind this comes from the fact that the gain in collecting data points decreases over time, and the value of the data points is not independent but rather depends on the previously selected data points. The submodularity property captures the diminishing return in collecting data points and the dependence of the value of the data points on the previously selected data points. The main contributions our work can be summarized follows:
1. We present framework scale active learning in multi-robot setting by formulating the data collection problem networked AV system as distributed submodular maximiza- tion problem.
2. We propose an interactive algorithm that iteratively updates the actions of the AVs and provide rigorous proof that it achieves a solution within the 1/2 optimality bound.
3. We empirically show that our interactive algorithm performs similarly to centralized al- gorithm while outperforming fully decentralized algorithm. Our algorithm shows an improvement of up to 48.5% in the submodular objective, 25.0% in classification accuracy, and 9.2% mean average precision (mAP) compared to the fully decentralized approach in real-world datasets including the Berkeley DeepDrive autonomous driving dataset [3].
We also utilize embeddings generated by vision and language models such as CL.IP [4], which trained on large-scale datasets to further enhance the data point selection process. These embeddings aid in selecting data points to effectively cover the entire data distribution.
Related Work
Data collection well-studied problem in the robotics and machine learning literature, closely related to active learning [5-9] and cloud robotics [10-20]. However, existing approaches typically involve robots sharing all their data without considering other robots' input during the data collection process. The closest work to ours is [19], which addresses data collection in multi-robot setting with the interaction between robots. However, their focus on minimizing a convex loss function on dataset statistics, with an equal value assigned to each class point. In contrast, our framework

---

Rober Obseraatoon
Figure 2: Enhancing Information Coverage through Cooperation: We illustrate toy example involving 3 robots observing data mapped into space, Each point represents an embedding of data point. The objec- tive is to maximize the coverage of the observed data points through se- lected points. In the distributed setting (red), each robot selects subset of data points independently. leading overlaps and redundancy. However, when the robots cooperate using an iterative policy (blue), they consider previous actions, resulting in more diverse data collection that covers a larger portion of the data points.
Divtribttd action
provides more general approach, considering cases where data points are unequally valued and utilizing submodular maximization for selecting the most informative samples.
Another related topic is federated learning (FL) [21-29]. FL aims to train model using data dis- tributed among multiple robots, where robots train the model locally and share parameters with each other. However, our problem setting fundamentally differs as the robots lack access to perfectly la- beled data and need to transmit their data to the cloud to obtain labels. Additionally, the models we consider are considerably large, making local training infeasible.
Our work predominantly falls into the category of pool-based active learning [7, 30), where the objective is to select the most informative data points from an unlabeled pool. Various methods have been proposed for selecting these data points in general machine learning settings [31-37] and robotics [38-41], each capturing informativeness in different ways and also requiring access to all unlabeled data points. In our case, however, we consider multi-robot setting where robots only have access to their local data distribution. Our work extends these methods naturally to be applicable in multi-robot context.
The problem of active learning is closely related to submodular maximization [42, 43], as submodu- lar functions capture the diminishing returns property and the marginal gain of selecting data point depends on the previously selected points [34, 43-46). Numerous works address submodular max- imization in both decentralized [42, 47-50] and centralized settings [43, 51-57J. Specifically, the works such as [49, 50] investigate a similar problem to ours. However, they restrict robots to select actions with a single element, whereas our work broadens this to accommodate actions with multi- ple elements. Our extension is essential since AVs upload a batch of sampled data to labelers over henddwiith--iiiteee networks. Furthermore, the problems in [49, 50] address classical submodular optimization, like set cover and sensor placement. Conversely, our work focuses on active learning for ML models and distributed data collection.
3 Problem Formulation
We consider practical problem that involves data collection from a distributed fleet of robots for training a robust machine learning model in the cloud. Our goal is select the most informative samples from each robot in the fleet, such that when these data points are added to the training dataset, the accuracy of the trained model is maximized. This problem formulation can be seen as a generalized version of the active learning problem in the context of distributed fleet of robots.
Robot Perception Model: We assume fleet of Nrobot robots operating in diverse environments, each performing a computer vision task such as classification or object detection. Robot i observes a set of data points {xt, ,...........................,. xx{ X which represents the accumulated data available for sampling in round r. A round represents a data collection period, like a day, during which robots operate and gather data. In each round, the robots sample Neache data points from their accumulated data X and send them to the cloud to retrain an ML model. The perception model used by robot i is a DNN represented as fDNN (z; 07) D, emb, where 0 denotes the parameters of the perception model. The perception model provides predictions ÿ and embeddings emb for the input data x.
Embeddings are crucial in our approach as they capture the similarity between data points. Un- like raw data, which is high-dimensional and ineffective at summarizing information, embeddings provide a more efficient and accurate representation. By leveraging embeddings generated by foun- dation models like CLIP [4] or other active learning methods [32-34, 58), we enhance our ability to capture data similarity effectively.
3

---

Assumption 1 (Robots' Accumulated Data Points are Disjoint), Each robot stores its observed data x separately. resulting in disjoint sets of accumulated data points for selection in each round That is, X x = 0 for all +
This assumption reflects the fact that each robot maintains its storage of observed data points. As a result, the sets of accumulated data available for selection in each round are disjoint. Then for each round we can define the set of all observed images as X = Noobos X!.
The fleet of Noobot robots operate in diverse environments, observing different data distributions X in each round r. In each round, each robot i selects at most Nç data points from its observed images X and sends them to the cloud for retraining the ML model. The size of the selected samples ANçache is determined based on factors such communication bandwidth and labeling budget. The samples chosen by robot in round r are denoted as action af, where af C C and |a]] S Neaccee indicating that the number of selected samples smaller than or equal the cache size Neache, Additionally, we assume that the robotic fleet is connected via wireless network, enabling communication among the robots.
Robotic Fleet:
Definition 1 (Robots' Feasible Action Space). In round r, each robot can select at most N Ncache samples from the observed datapoints X3 such that:
Neobot
A = {a c CX. |af] S Neeache),
A' =
of af A. vi= 1,.... Nrobot
im1
Here, A represents the set of all feasible actions for robot i, and A represents the combined feasible actions of all robots.
The goal of the data collection process is to select the most informative samples from each robot in the fleet, such that when these samples are added the training set, the accuracy of the trained model is maximized. However, there is no proven function that reliably represents dataset quality, as it depends on various factors such as data diversity and image quality [59]. Instead, heuristic methods are used to measure dataset quality. To provide a general framework for the data collection problem, we will assume two properties about the dataset quality.
Data Collection Problem:
First, we will assume that the quality of the dataset always increases with the addition of new samples [43-46, 60]. This is reasonable assumption since, unless the training data points intentionally manipulated, adding more data points improves the coverage of the training set of the real data distribution. Thus, we can safely make the following assumption:
Assumption 2 (Dataset Quality Function is Monotone). The dataset quality function f 2 I a monotone function, meaning that for all A C C S(A) < J(B).
Secondly, we assume that the marginal gain of adding a new sample to the training dataset exhibits a diminishing returns [43-46]. This is also a reasonable assumption because if the training dataset already covers a significant portion of the real data distribution, adding more samples may not in- crease the accuracy of the trained model significantly. We verified this on four real-world datasets, confirming a diminishing returns relationship between test accuracy and the percentage of training data (App. A.6). This property, known as submodularity, is exhibited by many problems, such as set cover, facility location, and sensor placement [51]. Formally, we state the following assumption: Assumption 3 (Dataset Quality Function is Submodular). The dutaset quality fanction f 2 + IR submodular, i.e., for all A c B C X and x E X X B it holds that f(A (x)) - S(A) Z J(BU (a:)) J(B).
Finally, we define the data collection problem: given fleet of Nrohot robots sharing common dataset D. in the cloud, the objective is to select a maximum of Neache samples from each robot i, maximizing the quality of the dataset. Formally, we can define the data collection problem Problem 1 (The Data Collection Problem).
Nrobos
max
S(D. U U af)
(I)
subject to:
af X
Vi 1,..., ,...,,., N Noobot
|a7|l S Nouche
Vi 1 1,..., Nrobot

---

This problem is combinatorial optimization problem and, without any further assumptions, solv- ing for the optimal solution is NP-hard [61]. Thus, we use algorithms that provide approximation guarantees to the optimal solution. For such problems, the greedy algorithm is shown to provide good approximation the optimal solution [53].
There are various submodular functions that can be used; such examples are facility location, mutual information, the cover problem [51]. In our experiments, we used facility location function, which assesses how well a subset A represents the entire dataset X [62]. It accomplishes this by calculating the sum of similarities Ma.a between each element x E X and its closest element E A. The facility location function is known to be submodular for non non-negative similarity values Mz.a 2 [63]. Fig. shows an example of how our facility location submodular objective encourages covering the data distribution. In formal terms, the facility location function is defined as follows:
f(A) = Σ max Mg,0- x acA
Model Retraining and Updating Weights: After each data collection round r, the cloud dataset DE is expanded with selected samples, and the perception model is retrained the cloud using the updated dataset D D U U Naator a to obtain new model weights 05+1, These weights are then shared with the robots. The robots can simply use the updated cloud model 0 or optionally fine-tune it on their local datasets yield models 0
4 Baselines
Centralized Action Policy: We now introduce centralized action policy, named CENTRALIZED, described Alg. In this policy, a centralized cloud has access to all the data points observed by the robots X (line 4). Starting from an empty solution set (line 3), the central cloud iteratively evaluates all combined data points X", selects the data point within the feasible action space that maximizes the submodular function f (line 6) and adds it to the action (line 7). This process continues until no more points can be added to the action (lines 5 8). The action the CENTRALIZED policy, denoted as a,, is then appended the cloud dataset D. (line 9), and this algorithm is repeated for each round r.
This algorithm has been proven to provide 1/2 optimality bound to the optimal solution [53]. Ad-
ditionally, randomized approximations exist that can achieve a 1 - 1/e optimality bound [52]. We have chosen this version of the centralized policy because of its simplicity and generality. Since solving the combinatorial optimization problem is NP-hard, we will use this centralized policy as target benchmark to compare the performance of our framework.
Input: D. XT.
1 Input: DE,
2 Output: D for = 1 to Nyobot do
Output:
D Initialize 0² 0
4 Initialize of =
X" = Niohot X
for j = to Naache do
5
for j to Noobat Aeache do
2 argmax f(D" ua u (z));
Tj = argmax f(D Ua u (+))
6
Pu())A
za =(CU(=)
af a (a;);
end
8 end
end
9 DE+L
DE U return D
D
D. UU of:
return D Algorithm 2: DISTRIBUTED Policy
Algorithm 1: CENTRALIZED Policy
In another scenario, we can solve this problem without considering any communication between robots, referred to as DISTRIBUTED This policy is described in Alg. the DISTRIBUTED policy, each robot independently runs the greedy selection in its observed images X in parallel (lines 3 9). Starting with an empty solution (line 4), each robot adds data points that maximize the submodular function f from its observed data points (line 5). At the end of round r, the selected data points af are shared with the cloud, which adds them to the cloud dataset
Distributed Action Policy:
5

---

Dr (line 11). Although the DISTRIBUTED policy can be run in parallel and preserves the distributed nature of the problem. it does not consider other robots' actions, potentially resulting in repeatedly selecting similar elements. Consequently, the solutions obtained through the DISTRIBUTED policy can be an 52 (((rrbbot) factor worse than the centralized solution [42].
These algorithms offer different approaches for sample selection from distributed robots, with the CENTRALIZED policy utilizing global information and the DISTRIBUTED policy relying on local observations of each robot. The CENTRALIZED policy needs Vy Neache) iterations over all robots and their samples, which may be impractical for large number of robots. On the other hand, the DISTRIBUTED policy can be implemented in parallel, making more scalable. Our method balances these two policies using distributed policy with iterative component.
5 An Interactive Approach to Fleet Active Learning
This section introduces our proposed policy, called INTERACTIVE, for the data collection problem. The INTERACTIVE policy begins with an empty solution (line 1) and sequentially determines the actions of the robots (lines -)
7). Each robot i takes into account the previous actions of the robots 1 to i - 1, denoted as 1 and runs the greedy algorithm given in Alg.3 to determine its action of and includes in the previous actions A! (line 6). This process is repeated until all robots have determined their actions. Finally, the actions of all the robots are combined (line 8), resulting in the combined action a², and the cloud dataset is updated (line 9).
Input: D. X.
Input: D., X7. f A
2 Output: D
2 Output: a!
3 Ai = 0
3 af = 0:
4 for i = 1 to Nrobot do
for j = to N Nosche do
Get action af using algorithm
5
2j argmax f(D. A/_, ud!
A = A Ua!
7 end
af = (
8 a' = U. af
7 end
D D. Ua
return of
return D
Algorithm 3: Action per Robot
Algorithm 4: INTERACTIVE Policy
Our proposed INTERACTIVE policy represents an improvement over both the CENTRALIZED and DISTRIBUTED policies. It achieves a solution with the same optimality bound as the CENTRALIZED policy while allowing for distributed execution. Moreover, the selection of points occurs only once between robots, leading to a reduced number of messages passings between robots. To minimize data exchange. we can share the embeddings of the points instead of the actual points themselves [15). This significantly reduces the amount of data transmitted. Furthermore, for specific functions like the facility location function, additional optimizations can be employed. For example, rather than sharing the points embeddings, robots update and share the maximum value of the similarity metric M.... We also theoretically show that the given algorithm will achieve solution that is at least 1/2 good as the optimal solution for submodular monotone objective functions.
Theorem 1 (Optimality of INTERACTIVE Policy). The INTERACTIVE policy described in Alg. 4 achieves a solution that is at least 1/2 as good as the optimal solution for submodular monotone objective functions, that is:
Joy (aa)) (a')2 (o:((aPTT)
Here we denote the improvement on the objective function as for (z) = f Ux) f(D)). a OPT is the optimal solution to the optimization problem given in Eq. 1 and a' is the solution obtained by the INTERACTIVE policy.
The proof of the theorem is given in Appendix A.2. As demonstrated in the theorem, the optimality bound of the INTERECTIVE ACTIVE policy is independent of the number or ordering of the robots. Further- more, the number of messages exchanged between robots is O O(Nrooo ). This subtle, yet significant, difference between the CENTRALIZZD policy and the INTERACTIVE policy enables the INTERAC- TIVE policy to be scalable to large number of robots. Morcover, our INTERACTIVE policy is

---

MNIST
CIFAR10
Adverse-Weather
DeepDrive
Accuracy
0:24
Round
Round
-4- Distributed
Contralized
Interactive
Random
Entropy
-
Least Confidence
-*
BvSB
Figure 3: Our INTERACTIVE policy achieves similar performance with the CENTRALIZED policy and outperforms other benchmarks. This figure presents the accuracy of the retrained neural networks using the datasct DE in each round r. with each column representing a diffcrent dataset. It can be clearly seen that INTERACTIVE and CENTRALIZED policies consistently outperform other benchmark policies across all datasets, underscoring the practical advantages of our INTERACTIVE policy.
computationally efficient compared to the CENTRALIZED policy and shares the same complexity as the DISTRIBUTED policy. In terms of function evaluations, both INTERACTIVE and DISTRIBUTED policies have computational complexity of O Nrobot
i= [X]] Nosche), while the CENTRALIZED
policy has a complexity of O Noobot [ x Σ
Noobot Naacho). Further analysis and a numerical i==
example are provided in the Appendix.
6 Experiments
We evaluate our method by comparing against several benchmarks using four diverse datasets. The first two datasets, MNIST [64] and CIFAR-10 [65], serve foundational benchmarks, while the ddterse-Weathee dataset [66] and the DeepDrive dataset [3] provide realistic scenarios with data captured by autonomous vehicles. In our simulations, under heterogeneous environments and diverse data distributions. For classification tasks, we em- ploy state-of-ther sate-offtheeart vision models such as the Vision Transformer [67] and ResNet [68], while for object detection tasks, we utilize YOLOv8 [69]. Detailed information regarding our experimental setup and the models is given in the Appendix.
simulate the data observation processes
In addition to the baselines described Section 4, we provide a comparison between our method and classical active learning approaches. These approaches include: i) Random: Selecting data points uniformly at random from the observations. ii) Least Confidence: Choosing samples with the smallest predicted class probabilities [70]. iii) Best versus Second Best (BvSB): Pick- ing samples with the smallest difference between the two most probable classes [71]. iv) En-
tropy: Selecting samples with the highest entropy in predictive class probabilities [70]. The latter three methods fall within the category of imcertainty-basee active learning techniques [7].
Comparison Metries: For classification tasks, we compare the accuracy of the trained model on newly acquired datasets D using the held-out test dataset. We compare the mean Average Precision (mAP) for object detection tasks among different data sampling policies. We also compare the submodular objective value of selected data points in each round. All experiments are repeated with 25 different seeds and the results are averaged.
Policy
mAP
INITIAL
33.2
DISTRIBUTED
37.2
CENTRALIZED
46.5
INTERACTIVE
46.4
Table 1: Results of DeepDrive object detection.
Submodular Objective Function: We use the facility location
function, given in Eq. 2, as the objective function. measure the similarity between points x and employ the similarity metric Mr I+3d(u,aj Here, the distance d(2, a) is computed the La distance between the embeddings of the data points, given by | - erab, 2|2. The embeddings are obtained by the models JDNN and we set the hyperparameter 3 = 0.01 to control the significance of the distance metric. our simulations, we employ the lazy greedy algorithm [54], a variant of the greedy algorithm that achieves the same solution while being more efficient in practice.
7

---

The experimental results (Fig. 3) show that our INTERACTIVE policy performs well as the CENTRALIZED policy, as we have shown in the Theorem 1. Moreover, our INTERACTIVE policy consistently outperforms the DISTRIBUTED policy in all simulations.
Results:
What is the accuracy of the vision model after each round?
In Fig.3, we show the perception model's accu- racy after each retraining round on the dataset Dr Clearly, our INTERACTIVE policy consis- tently achieves higher accuracies compared to the DISTRIBUTED policy, outperforming it by 9.7%, 10.1%, 25. 55.0% 8.8% and achieves simi- lar accuracy values the CENTRALIZED case. Furthermore, our INTERACTIVE policy exhibits significant accuracy improvements, with gains of up to 21.0%, 17.6%, 16.7%. and 14.3% in comparison the Random, Entropy, Least Confidence, and BvSB active learning methods, respectively.
Training Data Distributed Centralized
Interactive
Figure 4: Visualization of Policy Coverage in Fea- ture Space: We employed t-SNE [62] to project the embeddings of data points into a 2D space, facilitating the comparison of coverage
What is the object detection performance?
In Table 1, we present the mean average pre- cision (mAP) of the initial DNN, denoted as INITIAL and at the end of all data collec- tion rounds for each policy. Our INTERACTIVE policy and CENTRALIZED policy show signifi- cantly better performance compared the DIS- TRIBUTED policy, surpassing it by 9.2%.
different policies. Data points are color-coded according to the policy that selected them, with INTERACTIVG (green), CENTRAL- IZED (orange), and DISTRIBUTED (blue) policies rep- resented. The visualization demonstrates that our pro- posed INTERACTIVE method performs on par with CENTRALIZER haseline and achieves superior cover- of the training data (gray) in comparison to the Dis- TRIBUTED policy. Although the DISTRIBUTED policy might seem to have fewer points, this can be attributed overlapping repetition similar data points that are closely mapped together in the visualization.
Our formulation, while effective, has several assumptions that limit its general- ity. Firstly, our formulation assumes that the data collection rounds are synchronous among all robots and that robots have communication capabilities with each other. Finally, we assume that the robots trying maximize the objec- tive function cooperatively, whereas, real-world setting, the robots may need to be incentivized based on their contributions.
Limitations:
7 Conclusion and Future Work
We propose a framework to scale active learning algorithms to the multi-robot setting utilizing sub- modular maximization. Our framework enables robots to determine their actions sequentially, taking into account the actions of the previous robots. We show that when the objective function is submod- ular and monotone, the proposed framework achieves 1/2 optimality bound to the optimal solution, which is NP-hard to compute. Through experiments on real-world datasets. we confirm that such an iterative algorithm will result increased accuracy and objective function when compared to the distributed setting. Our work is especially useful when the robots are operating in resource- constrained environment, where data sharing is costly, and collective actions need be optimized to reduce redundancy. We believe that our work is step towards the development of a general framework for active learning in multi-robot systems.
Future research directions include incorporating more realistic constraints. This involves incorpo- rating asynchronous data collection rounds, where the robots can collect data at different times, and incorporating privacy-preserye mechanisms to ensure confidentiality. Furthermore, we aim extend our formulation where robots collect data for multiple tasks, each with varying value and importance.

---

Acknowledgements
This material is based upon work supported in part by Lockheed Martin Corporation and gift from Cisco Systems, Inc. Additionally, this material received support from the National Science Founda- tion under grant no. 2148186 and is further supported by funding provided by federal agencies and industry partners as specified in the Resilient & Intelligent NextG Systems (RINGS) program. This article solely reflects the opinions and conclusions of its authors and does represent the views of any sponsor.
References
[1] Data the new oil in the future of automated driving. https::/newsroon. intel. com/e hitorials/krzan [Online; accessed 10-June-2021).
2016.
[2] 10gbps 5g data speeds qualcomm and keysight achieve industry-first milestone. https: //datacenternews.... hoapp//tttttttttoopppp-ee--ee -oo-
2021. [Online; accessed 14-June-2022].
[3] Fisher Yu, Haofeng Chen, Xin Wang. Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogeneous mul- titask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2636 2645, 2020.
[4] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
[5] David A. Cohn, Zoubin Ghahramani, and Michael I. Jordan. Active learning with statistical models. JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, 4:129-145, 1996.
[6] Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with im- age data. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1183-1192. PMLR, 06-11 Aug 2017. URL http://prrcceedinga mmr. press/v7/ gal17a. html.
[7] Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin MMadison, 2009.
[8] Simon Tong. Active learning: theory and applications, volume 1. Stanford University USA, 2001.
[9] Kai Yang, Jie Ren, Yanqiao Zhu, and Weiyi Zhang. Active learning for wireless iot intrusion detection. IEEE Wireless Communications, 19(6):19-25, 2018, 10:1001109//WW.2011..1111 0079.
[10] Cloud robotics and automation. http:///gabbggg..azaraggggaaaeer
bew.olldeerrr..beeeeeee...ee.eee edu/clouu-roootice./ 2019. [Online; accessed 02-Jan-2019],
[11] Ben Kehoe. Sachin Patil, Pieter Abbeel, and Ken Goldberg. A survey of research on cloud robotics and automation. IEEE Trans. Automation Science and Engineering, 12(2):398 409, 2015.
[12] J Kuffner.
Cloud-enabled robots in: Icce-ras international conference on humanoid robots. Piscataway, NJ: IEEE, 2010.
[13] Sandcep Chinchali, E. Pergament, M. Nakanoya, E. Cidon, E. Zhang, D. Bharadia, M. Pavone, and S. Katti. Sampling training data for continual learning between robots and the cloud. In
2020 International Symposium on Experimental Robotics (ISER), Valetta, Malta, 2020.
[14] Ken Goldberg and Ben Kehoe. Cloud robotics and automation: A survey of related work. EECS Department, University of California. Berkeley, Tech. Rep. SCB/EECS-20133 2013.
[15] Sandeep Chinchali, Apoorva Sharma, James Harrison, Amine Elhafsi, Daniel Kang, Evgenya Pergament, Eyal Cidon, Sachin Katti, and Marco Pavone. Network offloading policies for cloud robotics: learning- based approach. Autonomous Robots, 45(7) 197-1129 2021. doi: 140007//00111-021-000900 URL https: : /doi. org/10 1007

---

[16] Ajay Kumar Tanwani, Nitesh Mor, John D. Kubiatowicz, Joseph E. Gonzalez, and Ken Gold- berg. A fog robotics approach deep robot learning: Application to object recognition and grasp planning in surface decluttering. 2019 International Conference on Robotics and Automation (ICRA), pages 4559-4566, 2019.
[17] Yuchong Geng, Dongyue Zhang, Po-han Li, Oguzhan Akein, Ao Tang, and Sandeep Chin- chali. Decentralized sharing and valuation of fleet robotic data. In 5th Annual Conference on Robot Learning, Blue Sky Submission Track, 2021.
[18] V. Casamayor Pujol and S. Dustdar. Fog robotics undiersaddd the research challenges. IEEE Internet Computing, 25(05):10-17, sep 2021. ISSN 1941-0131. .3000933.
[19] Oguzhan Akcin, Po-han Li, Shubhankar Agarwal, and Sandeep P. Chinchali. Decentralized data collection for robotic fleet learning: A game-theoretic approach. In Karen Liu, Dana Kulic, and Jeff Ichnowski, editors, Proceedings of The 6th Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, pages 978-988. PMLR, 14-18 mlr press / /2200/220///// weteee////5////iinnn/eee law.iiina.aaam...........
Dec 2023. URL https:
[20] Ryan Hoque, Lawrence Yunliang Chen, Satvik Sharma, K Dharmarajan, Brijen Thananjeyan, Abbeel, and Ken Goldberg. Fleet-dagger: Interactive robot fleet learning with scalable human supervision. In Conference on Robot Learning. 2022.
[21] Federated learning: Collaborative machine learning without centralized training data. https: www.:nnnnnnollllaatiiiiiioa .tt.laatiiieeeeeee.....mmm.. 2017.
//ai www.gllloologgoooooooooooooooooo ogg com
[Online;
accessed 15-June-2021J.
[22] Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. CoRR. abs/1610.05492, 2016. URL http: //atpi//arxiv.... org/ ab////as....././//////////////1/ 05492.
[23] Yu Xianjia, Jorge Peña Queralta, Jukka Heikkonen, and Tomi Westerlund. Federated learn- ing in robotic and autonomous systems. Prucedia Computer Science, 191:135-142, 2021. URL https: 1437X. The 1 18th International Conference Mobile Systems and Pervasive Computing (MobiSPC), The 16th International Conference on Future Networks and Communications (FNC), The 11th In- ternational Conference on Sustainable Energy Information Technology.
ISSN 1877-0509.
/ // www..sciint.. www.ssineeedeercee..........ccee
[24] Elsa Rizk, Stefan Vlaski, and Ali H. Sayed. Optimal importance sampling for federated learn- ing. In ICASSP 2021 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3095-3099, 2021.
2021 202.00111111000 9653.
[25] Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic, and Soummya Kar. Personalized federated learning via convex clustering. arXiv preprint arXiv:2202.00715 2022.
[26] Hong Xing. Osvaldo Simeone, and Suzhi Bi. Decentralized federated learning via sgd over wireless d2d networks. 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), pages 1-5, 2020. doi: 109/SPPW44S 57.2020.9154333
[27] Bin Gu, An Xu, Zhouyuan Huo, Cheng Deng, and Heng Huang. trivac--reeeevvnn asyn- chronous vertical federated learning algorithms for multiparty collaborative learning. IEEE Transactions on Neural Networks and Learning Systems, pages 1-13, 2021. doi:
NLS.2021.3072238.
[28] Solmaz Niknam, Harpreet S. Dhillon, and Jeffrey H. Reed. Federated learning for wire- less communications: Motivation, opportunities, and challenges. IEEE Communications Magazine, 58(6):46-11, 2020. do /09//C/..00..
[29] Sebastian Caldas, Jakub Konečny, Brendan McMahan, and Ameet Talwalkar. Expanding the reach of federated learning by reducing client resource requirements, 2018.
[30] Xueying Zhan, Huan Liu, Qing Li. and Antoni Chan. A comparative survey: Benchmarking for base-based active learning. pages 4679-4686, 08 2021. doi: 10.24963/jjai 2001//44.
[31] Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning, 2017.
10

---

[32] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core- set approach. In International Conference on Learning Representations, 2018. URL https: //openreview. net/forum? hett//forumndd-ilaalk-----
id-HHaluk-RRi
[33] Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agar- wal. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations, 2020. URL https: httt:s:///operrview..ooo nat.eenune/froi d=ryghZ.JBKPS.
[34] Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Buse E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https:/t///pproedaaaang neurip
s.cc/paper files r.pdf.
[35] Ajay Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for im- age classification. pages 2372-2379, 2009..
[36] Nicholas Roy and Andrew McCallum. Toward optimal active learning through monte carlo estimation of error reduction. ICML, Williamstown, pages 441-448, 2001.
[37] Jiaxi Wu, Jiaxin Chen, and Huang. Entropy-based active learning for object detection with progressive diversity constraint. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9387-9396, 2022.
[38] Jingyun Yang, Junwu Zhang, Connor Settle, Akshara Rai, Rika Antonova, and Jeannette Bohg. Learning periodic tasks from human demonstrations, In 2022 International Conference on Roboties and Automation (ICRA), pages 8658-8665, 2022. 1o::10.1109/ICCE 10T.1///////44433.220000 812402.
[39] Maya Cakmak, Crystal Chao, and Andrea L. Thomaz. Designing interactions for robot active learners. IEEE Transactions on Autonomous Mental Development, 2(2): 2010. doi: A.1109/AAMD..200...0. 1030.
[40] Crystal Chao, Maya Cakmak, and Andrea L. Thomaz. Transparent active learning for robots. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 317-324, 2010. doi: 100.100//...200...000
[41] Annalisa T. Taylor, Thomas A. Berrueta, and Todd D. Murphey. Active learning in roboties: A review of control principles. Mechatronics, 77:102576, 2021. ISSN 0957-4158. doichttps: //lo:.arg/.....ll/..eeehrroorooo ociaaree/artcce///1///9991111110
URL https: http///wwwwcinccdrrett.t.cc.
[42] Baharan Mirzasoleiman, Amin Karbasi, Rik Sarkar, and Andreas Krause. Distributed sub- modular maximization. Journal of Machine Learning Research, 17(235):1-44, 2016. URL
htjl:
[43] Kai Wei, Rishabh Iyer, and Jeff Bilmes. Submodularity in data subset selection and active learning. In Proceedings of the 32nd International Conference on International Conference on Machine Learning- Volume 37, ICML' 15, page 1954-1963. JMLR.org. 2015.
[44] Yuxin Chen and Andreas Krause. Near-optimal batch mode active learning and adaptive sub- modular optimization. In Sanjoy Dasgupta and David McAllester, editors, Proceedings of the 30th International Conference Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 160-168, Atlanta, Georgia, USA, 17-19 Jun 2013. PMLR. URL https: httpp:://proceedings mlr pptps//2///cealallec html.
[45] Jeffrey Bilmes. Submodularity In Machine Learning and Artificial Intelligence. abs/2202.00132. Oct 2022. URL https: //tpp://////xx//////eaeeaa org.abs.//./../.....o.o 22tp/2222.0000
Arxiv,
[46] Katrin Kirchhoff and Jeff A. Bilmes. Submodularity for data selection in machine translation.
In Conference on Empirical Methods Natural Language Processing, 2014. URL https:
// //api. sapi..semantcccchollaa org/ org/CorpusID:7555900 7585940.
[47] Bahman Gharesifard and Stephen L. Smith. Distributed submodular maximization with limited information. IEEE Transactions on Control of Network Systems, 5(4): 2018. doi: 10. 10.1109TCNS.220..770032.

---

[48] Vahab Mirrokni and Morteza Zadimoghaddam. Randomized composable core-sets for dis- tributed submodular maximization. In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing. STOC '15, page 153-162, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781 450335362. db::10.11445227453392222
4. URL https: org/10. 1145/2746539. 2746624.
[49] David Grimsman, Mohd. Shabbir Ali, João P. Hespanha, and Jason R. Marden. The impact of information in distributed submodular maximization. IEEE Transactions on Control of Network Systems, 6(4):1334-1343, 2019.
[50] Micah Corah and Nathan Michael. Distributed submodular maximization on partition matroids for planning on large sensor networks. In 2018 IEEE Conference on Decision and Control (CDC), pages 6792-6799 2018. 1011001119/CDDC22118.881111
[51] Andreas Krause and Daniel Golovin. Submodular function maximization. In Tractability, 2014.
[52] Gruia Calinescu, Chandra Chekuri, Martin Pál, and Jan Vondrák. Maximizing submodular set function subject matroid constraint (extended abstract). Matteo Fischetti and David P. Williamson, editors, Integer Programming and Combinatorial Optimization, pages 182-196, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg. ISBN 978-3-540-72792-7.
[53] G. L.. Nemhauser, L. A. Wolsey, and M. L.. Fisher. An analysis of approximations for maximiz- ing submodular set functions-i. i. Math. Program., (4(1):2655224 dec 1978. ISSN 0025-5610. URL httts:://datt
wrtg 1077/
[54] Michel Minoux. Accelerated greedy algorithms for maximizing submodular set functions, volume 7, pages 234-243, 01 2006. ISBN 3-540-08708-7. doi: 10t.10..100////000552222
[55] M. L. Fisher, G. L. Nemhauser, and L. A. Wolsey. An analysis of approximations for maximizing submodular set functions II. pages 73-87. Springer Berlin Heidelberg, Berlin, Heidelberg, 1978. ISBN 978-3 3-4---0009- dot:10.1007BBF6002119 URL https: //doi.org//1 1007 10011000777110 BF000229562
[56] Bin Du, Kun Qian, Christian Claudel, and Dengfeng Sun. Jacobi-style iteration for distributed submodular maximization. IEEE Transactions Automatic Control, 67(9):4687-4702, 2022.
100:1001109/TAA.2222..33 316066.
[57] Nived Rajaraman and Rahul Vaze. Submodular maximization under A matroid constraint: Asking more from an old friend, the greedy algorithm. CoRR, http://arxiv.. org /a//a//s///...../// //1001110..111 12861.
2018. URL
[58] Angel Andres Daruna, Weiyu Liu, Zsolt Kira, and Sonia Cherova. Robocse: Robot common sense embedding. CoRR, 0bs/19933.0042.. 2019. URL
wttp:/www..rg/b/////33/// 004
12.
[59] Youdi Gong, Guangzhen Liu, Yunzhi Xue, Rui Li, and Lingzhong Meng. A survey on dataset quality machine learning. Information and Software Technology, page 107268, 2023. ISSN 0950-5849. doithttps://doioorg/// http://....1111...fff22222222... 107268. URL edirect. www.///ci/////eeiiiiioooooooocce http:///.....c..c...cc.oo//////a
scienc
[60] Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kia- ninejad, Md. Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. 12 2017.
[61] Uriel Feige. A threshold of In n for approximating set cover. J. ACM, 45(46634-553 jul 1998. ISSN 0004-5411, doi: 145/285055.285052 URL htttp::/ddoi.org/1.... 10g//0..114//85505 .285059.
[62] Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning models. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 6950 6960. PMLR, 13-18 Jul 2020. URL https://proceeddng
s. s.mlr. press htt./////mmiesoeeeimaaaaa htmll.
[63]
A. M. Frieze. A cost function property for plant location problems. Math. Program., 7(1): 245-248, dec 1974. ISSN 0025-5610. 001:10.1007/BF0155555 URL. htt.::ttpp///e //doi..d./... org/ 10. 1007 /FFO 1585221..
12

---

[64]
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://aaanlluu..co//xxdmmmmmm 2. 2010.
[65]
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
[66]
Adverse weather dataset.
retaaddvers-e--eeteer dataset/.
[67]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. URL http://paaaaaieee..eeerrrrmracaa
[68] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
[69] Dillon Reis, Jordan Kupec, Jacqueline Hong, and Ahmad Daoudi. Real-time flying object detection with yolov8. 2023.
[70] Dan Wang and Shang. A new active laheling method for deep learning. In 2014 International Joint Conference on Neural Networks (JCCNN), pages 112-119, 2014. doi: 10.1 109/UCNN.20
14.6889457.
[71] Dan Roth and Kevin Small. Margin-based active learning for structured output spaces. In Jo- hannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou, editors, Machine Learning: ECML 2006, pages
Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-
46056-5.
[72] Thibaut Horel. Notes on greedy algorithms for submodular maximization thibaut. 2015.
[73] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication--fffiicee Learning of Deep Networks from Decentralized Data. In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages PMLR, 20-22 Apr 2017. URL https: https::/ httpps://pprceee http://poccceeng..///e mlr. wttpinn....r....r.....ee press//54/nccn
han17a. html.
13

---

A
Appendix
Code Availability: The code and related materials can be found in the following code repository:
The organization of the appendix is follows:
1. Subsection A. provides the necessary background and preliminary information required for the proofs.
2. Subsection A.2 presents the proof of the optimality bound for the INTERACTIVE policy,
3. Subsection A.3 describes the details of the experimental setup.
4. Subsection A.4 contains additional results related object detection experiments.
5. Subsection A.5 provides submodular objective results for experiments.
6. Subsection A.6 presents experimental analysis focusing the diminishing return property.
7. Subsection A.7 reports the results of the experiments with non-submodular and monotone objectives.
8. Subsection A.8 includes the function evaluation and runtime complexity analysis of the algorithms.
9. Subsection A.9 presents numerical evaluations the number of message exchanges and the number of function evaluations for the DeepDrive dataset experiment.
10. Subsection A.10 provides additional results for federated learning experiments.
A.1 Preliminaries
Here, we provide basic definitions of submodular functions, monotone functions, and matroids that we use in our proof.
Definition 2 (Submodular Function). A set function f: xx - IR is submodular for all A A B X and x X B, we have
f(AUfz)) f
f(B).
Definition (Monotone Function). A set function f f I is monotone for all A C B C X, we have
f(A) < (B).
Based on the definitions of submodular and monotone functions, we can directly write the following:
Corollary 1. Let f be a monotone submodular function. Then, also subadditive, and we have that:
BCCX,BXX: f(B) f(A)+
Σ fa(Z).
BEBYA
Here, we denote the marginal gain of adding an element to set A as fA(x) = (1) - which is also monotone and submodular.
Proof. This standard result in submodularity. See Corollary in [72].
Definition 4 (Matroid). A matroid is a pair M = (E, I) where E' is a finite set (called the ground set) and is nonempty set of subsets of (called the set of independent sets) with the following conditions:
1. VBEI: ACB
2. VA,BE1,
B
Definition 5 (Basis of a Matroid), A basis of a matroid an independent set of the matroid which is not contained in any other independent set.
14

---

Corollary 2. If B1 and B2 are two bases of matroid M, then there exists bijection Φ B1 B2 +
B2 \ B, such that:
V/x E B1 \ B2: B1 BIUO((=)) (x) E.
Proof. This standard result in matroid theory. See Proposition 11 in [72].
Definition 6 (Partition Matroid). A partition matroid is matroid where the ground set E is parti- tioned into disjoint subsets E1, Eg,... Et and the set of independent sets is defined as follows:
I [B S E |B E.l S k, vi (1,
A.2 Proof of Optimality Bound for the Interactive Policy
First, we show that the robots' feasible action spaces A' and the union set of all observed data X form partition matroid.
Lemma (Robots' feasible action space and set of all observed images form Partition Matroid). The robots' feasible action space A and the union of all observed data X form a partition matroid
M -X",,'' where X" = U Nichot X and X; nxj 0 Vi + j. The action space is defined
as A" = -UUNeaooo of :a;; E EA vi=1.... Nrobot
Proof. The proof follows directly from the definition of action and observed data points. For all E A", if s C a, then we know that for all j. is 7 nxy] S la 7 Xx] S N Ngache meaning E AA. And we know that for all o, A", if |s|< |al, then there exists subset X3 C Xx such that I XyI < lan Xy1 s Nzache and X3 O + 0. Then, for any element X3 na (s holds that s (x) EA.
Now we prove the main theorem of our paper. Our proof is similar to the proof given in [72], while proof involves sequential optimization methods, such the shown in Alg. 4.
Theorem. The algorithm given in Alg. 4 achieves solution greater than 1/2 of the optimal solu- tion.
Proof. Assume that a OPT is the optimal solution for the problem First, we show that a OPT and a' bases of the matroid (X" ,,A). Since assume that the objective function is monotone (Assmp. 2), it is trivial to see that OF n XTI Neaahhe for all
(1....,Nroboo) {1...., Nrobot J. making
a OPT basis of matroid (X", .")). a' is basis as well. since in Alg. 4 we construct it such that la' n 'x, = Neache for all i. For matroids, there exists a bijection ø a OPT - a', which maps the optimal solution to the solution of the INTERACTIVE policy. We express these solution sets as follows:
OPT {-1,1 I12
OPT Nrobor. NN Nesache Nyohot
} and a' = (x1,11 21,,, ................................ Nrobot .. Nencine }. Nrabot
Here Fijj ((OPT) for all í.j. Let af {-1,1,---- Fir) and af { .........0. -z 2-1 denote the sets of actions taken up to the i-th robot and the j-th cache and actions taken up to the i-th robot repectively. Then for fx)(x) =f U( {x}) - fDD)). we write the following:
Neobol Merahe
(a Spz((a))
Irguat (x (PT)
j=1
Nechot Nenche
fpguat,-.1 (1T
Nrahot Nonche
foguat (Ri.s)
1=1
Nrahot Nocuchi
(a for (o. for (a').
j=1
15

---

The first inequality is result of Corollary 1, while the second inequality stems from the submodu- larity of the function fDy. In the third inequality, we utilize the fact that in each iteration of Alg. 4, we select the element with the maximum marginal gain. Next, in the first equality, we use the fact that a U { = o The last equality follows from the fact that for fDD (0) O, the sum of the marginal gains equals to the value of (D:(a').
Therefore we have:
1 fDy (a') 2 fo:
This concludes the proof, showing that the INTERACTIVE policy achieves least half of the value of the optimal solution.
A.3 Experiments
To demonstrate the effectiveness of our proposed policy, we conducted simulations in scenarios in- volving multiple robots engaged in data collection from heterogeneous observation distributions. To create these heterogenous environments, we sampled incoming class distributions from the Dirichlet distribution, incorporating skewness parameter denoted a. This way, ensure that environ- ments have nonidentical incoming class distributions (non-i.i.d). Then, within each environment, simulated robots that observed the same data points.
Initially, an initial dataset denoted as D was chosen from the training set to train the initial model fDNN(.: .; P.). The initial dataset generated with uniform class distribution, employing the Dirich- let distribution with a skewness parameter value of a = 5. In each subsequent round, the vision model was retrained from the pre-trained vision model to ensure fair evaluation of the performance using the selected training set. To prevent overfitting, when identical data points were selected from multiple devices, the redundant instances were filtered out, and only single data point was added to the training set.
A.3.1 Embedding Functions:
To generate embeddings for the data points, we utilized multiple vision and language models de- pending on the datasets. Initially, we made use of the embeddings generated by the CLIP model [4], which is trained to create outputs in the same embedding space for both language and vision model inputs. However, we observed that when the embeddings generated by the CLIP model start to perform poorly the datasets when there is mismatch of the targets of the CLIP model with our classification output or the images are out-of-distribution for the CLIP model. For these datasets, we instead employ the embeddings generated by BADGE [33]. BADGE embeddings essentially correspond to the gradients of the final layer of the network with respect to the input.
A.3.2 Classification Experiments
In all classification experiments, we used the Adam optimizer with learning rate of 0.001 with batch size of 1000. Additionally, the learning rate scheduler is used with a decay rate of 0.99. We trained the DNNs in each round for 300 epochs. We did not apply any data augmentation to the training set. To ensure robustness, we conducted these experiments for 25 different seeds. Now, we provide additional explanations regarding the details and clttase--specffe parameters used in the simulations.
MNIST:
In our paper, we used the MNIST dataset to show the efficacy of our algorithm simple setting. The MNIST dataset is a collection of handwritten digits that contains 60,000 training images and 10,000 test images. Each image
x 28 grayscale image.
Simulation Parameters: In MNIST simulations, we used heterogeneous environments, each containing 4 robots that observe identical samples, resulting in total system of 20 robots. To create heterogenous incoming class distributions, we set the skewness parameter of the Dirichlet distribution to a 1.3. In each round, robots are observing 1000 data samples and collect Neache
3 data samples from their observations. We started with an initial dataset of size 16 and collected the data for 10 rounds. The final training dataset consists of 616 data points.
16

---

DNN and Embedding Function: We used a simple DNN with 6 layers, with 4 convolutional layers and 2 fully connected layers. Between each convolutional layer, we used the ReLU activation function and applied dropout with a probability of 0.3. To create embeddings, we utilized BADGE [33].
The CIFAR-10 10 dataset consists of 60,000 32 x 32 x 3 RGB images with ten different classes. The dataset is split into training and testing datasets of size 50,000 and 10,000, respectively. The classes in the dataset are truck, ship, horse, frog, dog, deer, cat, bird, automobile, and airplane.
CIFAR-10:
Simulation Parameters: In CIFAR-10 simulations, we used 6 heterogeneous environments, each containing 4 robots that observe identical samples, resulting in total system of 24 robots. To create heterogenous incoming class distributions, we set the skewness parameter of the Dirichlet distribution to a = 1.6. In each round, robots are observing 1000 data samples and collect Neaahhe = 1 data samples from observations. We started with initial dataset of size 10 and collected the data for 10 rounds. The final training dataset consists of 250 data points.
DNN and Embedding Function: We leveraged a pre-trained ResNet-50 model [68] as the back- bone for our vision model. To adapt it for our task, we replaced the final layer of the ResNet-50 with two fully connected layers, incorporating ReLU activation, and applied dropout with probability of 0.3 to mitigate overfitting. Only these replaced layers were retrained, following the transfer learning approach. This strategy significantly reduces training time while mitigating overfitting risks. To create embeddings, we utilized embeddings created by the CLIP model [4].
Adverse-Weather Dataset The Adverse- dataset comprises numerous RGB image se- quences, each with dimensions of 720 1280 x 3. These sequences were collected from moving vehicles around the University of Michigan campus, capturing diverse weather conditions. While most sequences exhibit dynamic scenes, some of them include static recordings. The dataset in- cludes two metadata classes: weather and time of day. The weather class consists of labels such as rain, fog, snow, sleet, overcast, sunny, and cloudy. The time of day class includes labels for Sunset, Afternoon, and Dusk. By combining these weather and time of day labels, we established total of 11 classes to train our model on. To avoid redundancy, we subsampled the images from the video sequences, selecting one image every 10 frames. Consequently, we constructed a dataset comprising 36,230 images, which we divided into a training set of 31,701 images and a test set of 4,529 images.
Simulation Parameters: In Weather simulations, we used 6 heterogeneous environ- ments, each containing 4 robots that observe identical samples, resulting in total system of 24 robots. To create heterogenous incoming class distributions, we set the skewness parameter of the Dirichlet distribution to a 1.2. In each round, robots are observing 1000 data samples and col- lect N /eeache = I data samples from observations. We started with an initial dataset of size 10 and collected the data for 10 rounds. The final training dataset consists of 250 data points.
DNN and Embedding Function: We adopted a pre-trained Vit-H14 model [67] as the backbone for our vision model. To adapt it to specific task, we replaced the final layer of the Vit-H14 with two fully connected layers, employing ReLU activation, and incorporated dropout with a prob- ability of 0.3 mitigate overfitting. Only these replaced layers were subjected to retraining. This methodology significantly reduces training time while effectively preventing overfitting. To create embeddings, have utilized BADGE [33].
DeepDrive Dataset: The DeepDrive data following the transfer learning approach, only encom- passes 100,000 images taken from driving videos in diverse cities and weather conditions. It com- prises 70,000 training images, 10,000 validation images, and 20,000 testing images. However, the testing images not publicly accessible, validation datasets. The classification model was designed to predict weather labels, including rainy, snowy, clear, overcast, partly cloudy, and foggy. However, due to limited number of foggy images (only 181), the foggy class was excluded from the simulations. Consequently, the classification model was trained subset of five classes.
analysis focused solely the original training and
Simulation Parameters: DeepDrive simulations, we used heterogeneous environments, each containing 6 robots that observe identical samples, resulting in total system of 30 robots. To create heterogenous incoming class distributions, we set the skewness parameter of the Dirichlet distribution to = 3.3. In each round, robots observe 1000 data samples and collect Neache
20
17

---

data samples from observations. We started with an initial dataset of size 50 and collected the data for 20 rounds. The final training dataset consists of 12,050 data points.
DNN and Embedding Function: We used a pre-trained Vit-H11[67]] model and replaced the final layer of the vision model with 2 fully connected layers with the ReLU activation layer and applied dropout with probability of 0.3. We only retrained the replaced layers of the network. This approach, known transfer learning, reduces training time and helps to prevent overfitting. To create embeddings, we utilized BADGE [33].
A.3.3 Object Detection Experiments
We ran object detection experiments on the DeepDrive dataset. As in the classification case, we used the training and validation images to train our model and report the final mean average precision (mAP) scores. We used an SGD optimizer with a learning rate of 0.01 in this experiment. We trained the DNNs at the start and end of the data collection rounds. We applied all data augmentations detailed [69].
Simulation Parameters: In object detection experiments. we used 5 heterogeneous environments, each containing 5 robots that observe identical samples, resulting total system of 25 robots. We set skewness parameter = 1. In each round, robots observe 1000 data samples and collect Neache = 50 data samples from observations. We started with an initial dataset of size 5000 and collected the data for 20 rounds. The final training dataset consists of 30,000 data points.
DNN and Embedding Function: We used pretrained YOLOv8-smal model [69] and retrained all model weights in each training. Since we are dealing with the objects, we utilized the CLIP model [4] to create embeddings.
A.4 Object Detection Experiments Metrics
In the object detection experiments, we used several metrics to evaluate the performance of the poli- cies. These metries include mAP50, representing the mean of average precision the intersection over union with a threshold of 50%; mAP50-95, denoting the mean of the average precision at the in- tersection over union for thresholds ranging from 50% 95%; precision, and recall. We conducted the experiments with 25 different seeds and averaged the metrics across the seeds. We present re- sults in Table 2. Our INTERACTIVE policy shows similar performance the CENTRALIZED policy while outperforming the DISTRIBUTED policy. In mAP50 results, the INTERACTIVE policy shows an improvement of 9.2%, whereas, in mAP50-95 results, the INTERACTIVE policy demonstrates development of 6.1%. Furthermore, our INTERACTIVE policy surpasses the DISTRIBUTED policy in precision and recall results by 11.6% and 6.5%, respectively.
Method
mAP@50
mAP@50-95
Precision
Recall
INITIAL
17.7 + 0.007
51.5 + 0.048
$2.7+0.00
DISTRIBUTED
37.2 1 0.009
20.2 0.006
56.3
0.046
35.5 0.008
CENTRALIZED
0.003
26.3 1 0.007
67.6
0.007
42.4 0.004
INTERACTIVE
0.002
26.3 0.000
67.9 77.900.00 0.007
42.0 0.003
Table 2: Additional metries for object detection experiments
A.5
Comparison of Submodular Objective Function Values among Policies
In addition the accuracy values presented in Fig. 3, we show the values of the submodular objec- tive function across different data collection rounds for the same experiments in Fig. 5. Consistent with the trends observed in accuracy values, proposed INTERACTIVE policy outperforms the DISTRIBUTED policy and achieves similar performance to the CENTRALIZED policy on all four datasets.
The performance gains of 29.8%, 19.7%, 41.2%, and 48.5% for each dataset are achieved because, in the INTERACTIVE policy, robot makes selections while considering all other robots before it, effectively reducing redundancy., as shown in Fig. 4. These emprical results align with the theoretical analysis presented for the lower-bound in Theorem 1.
18

---

MNIST
CIFAR10
Adwthse-Weather
DeepDrive
Submodular Objective (tZ
3730
3390
3000
Round
Round
Round
Round
Distributed
Centralized
Interactive
Entropy
Figure 5: Evaluating the submodular objective performance of INTERACTIVE, DISTRIBUTED, and CENTRALIZED policies. This figure illustrates submodular objective values for the cloud dataset across multiple rounds. Notably, both the CENTRALIZED and INTER ACTIVE policies achieve similar objective values, while the DISTRIBUTED policy fails to reach the same level of performance.
1.0
1.0
Accuracy
Accuracy
0.6
0.6
MNIST CIFAR-10 Adverse-Weather
MNIST
0.2
0.2
CIFAR-10 Adverse-Weather
DeepDrive
DeepDrive
0.0
0.1
0.2
0.3
0.4
0.5
20
40
60
80
100
% of Total Training Dataset
% of Total Training Dataset
Figure 6: Accuracy shows diminishing returns to training dataset size. Each line represents the accuracy classification models on different dataset. The x-axis represents the percentage datasets used for training the model, while the y-axis represents accuracy. The left figure provides a closer view of the right figure, specifically focusing consistent improvement in accuracy the percentage of training data increases. However, the dataset size grows, the slope of the curve gradually decreases, indicating diminishing returns where the incremental contributions new data samples hecome smaller. This observation supports our assumption dataset quality function being submodular and monotone.
training percentages ranging from 0 to 0.5. Across all datasets, we observe
A.6 Diminishing Returns between Accuracy and the Percentage of Training Data
To support our claim about dataset quality being submodular Assumption 3, and monotone 2. We conducted 5 experiments training classification models on four datasets of varying sizes. In Fig. 6, we show the accuracy of the classification models accuracy across the different percentages of the datasets. We can see in the general plot (Fig. 6-right) and zoomed-in version to smaller percentages (Fig. 6-left) accuracy shows diminishing returns property in terms of accuracy as the training dataset size increases. This indicates that the incremental improvement gained from adding new samples decreases with larger dataset. Additionally, the accuracy of the models consistently increases as the dataset size grows, providing evidence for the monotonicity of dataset quality. These observations align with our assumption about dataset quality being submodular (Assmp. 3) and monotone (Assmp. 2).
19

---

MNIST
CIFAR10
Adverse-Weather
DeepDrive
6.05
Accuracy
6.50
0.5
01
0.5
64
610
Round
Round
Round
Round
Distributed
Centralized
Interactive
Figure 7: Comparison of performance under non-monotone and non-submodular objective. We con- ducted the experiments in Fig. 3 but with nou-submodular and non-monotone objective function, deseribed A.7. Similar to Fig. 3, observe that both the INTERACTIVE and CENTRALIZED policies out- perform the DISTRIBUTED policy by a significant margin. This demonstrates the robustness and effectiveness of the INTERACTIVE and CENTRALIZED policies under non-submodular and non-monotone objectives.
A.7 Accuracy When Objective Function is Non-submodular, Non-monotone
In this section, we investigate the performance of our INTERACTIVE policy when the objective function f is not monotone or submodular through experiments. Instead of the submodular facility location function, we employed a variant called the maximin facility location function. This function aims maximize the minimum distance between any two points in set. Formally, the maximin facility location function as follows:
define
S maximmin (S) max min ||s:
syll
s.ES (s.))
where s; is the hthh point in the set S and || - is distance function. We used the same simulation parameters and models as we did with the facility location objective function. The accuracy results of the simulations are presented in Fig. 7. Similar the previous case, our INTERACTIVE policy exhibits equivalent performance to the CENTRALIZED policy and outperforms the DISTRIBUTED policy in all simulations.
A.8 Complexity Analysis
For all algorithms, suppose that there are Nrobot robots, each robot i observes samples X in each data collection round, and in each round at most Areace samples have to be chosen from samples X for each robot
is a known result that the total number of function evaluations of the greedy algorithm that used for submodular maximization is O(IV])), where V is the ground set for observations and k is the number data points that need to be selected [54].
The CENTRALIZED Algorithm: In the CENTRALIZES algorithm, a central server carries out all the computation required data collection round to choose the samples that are going to be added to the cloud dataset. That is, the central server has access to all the observations made by the robots and has to select Mecache samples from the observation set X; of each robot i. Therefore, our ground set is Unnghot XÇ, and the number points that need to be selected is Nroboo Arcache, Because the observed sets X of the robots are disjoint, the total number of elements in our ground is Neahot X = Eistoo
Noobot [XT].
This makes the computational complexity of the CENTRALIZED N..bbr Neache).
algorithm in terms of total number of function evaluations O 1
Nrobot
[X]]
The DISTRIBUTED Algorithm: In the DISTRIBUTED algorithm, all robots carry out the data selec- tion process themselves. Therefore, for each robot i, the ground set is X7 and the number of points that have to be selected is Ncachee Summing the number of function evaluations over all robots, the Noobat total complexity of the DISTRIBUTED algorithm is O i=1
[XT] x Neache).
20

---

The INTERACTIVE Algorithm: In our INTERACTIVE algorithm, just like the DISTRIBUTED al- gorithm, the robots select data points in the confines of their observation sets. Thus, the ground sets and the number of points to be selected from each ground set is again Xς and Neache, re- spectively. Summing again over all robots, the total complexity of our INTERACTIVE algorithm
O Neohot
[ x Ncache), which is the same as the DISTRIBUTED algorithm.
1=1
Number of Message Exchanges: Since our problem is distributed in nature, we should also ana- lyze the total number of messages passed between robots. The CENTRALIZED algorithm requires ENuabos Neache iterations over all robots, resulting in total of O(Nroatt ENrobot N(ache) message exchanges. On the other hand, our INTERACTIVE policy requires only one iteration over all robots, leading to a significantly lower number of message exchanges, specifically O(Nrobot). ). Lastly, since the DISTRIBUTED policy is executed without any interaction between robots, there are no message exchanges involved.
Method
Number of Message Exchanges
Number of Function Evaluations
O [X,"| x Arcache
DISTRIBUTED
CENTRALIZED
O Noohot Nreache)
O
Nrohot
X
x Σ Lisl
Niobot Nreache)
INTERACTIVE
(Nrobott
Nrahot
X," x Nreache)
Table 3: Computational Complexities of Different Policies
A.9 Numerical Evaluation of Policy Performance Metrics
To provide a quantitative example of the analysis conducted in Section A.8, we offer numerical instances from experiments carried out on the DeepDrive dataset, detailed in Table 4. As evidenced in the table, there are noteworthy distinctions in terms of computation and communication metrics among the compared policies.
Specifically, in terms of message exchanges, our proposed INTERACTIVE policy showcases excep- tional efficiency, outperforming the CENTRALIZED policy by factor of 600 Moreover, in relation to function evaluations, both the INTERACTIVE ACTIVE and DISTRIBUTED policies demonstrate efficiency improvements of x compared the CENTRALIZED policy.
With regards to accuracy, an improvement of 9.4% is achieved by both the CENTRALIZED and IN- TERACTIVE policies. This signifies that our proposed INTERACTIVE policy attains accuracy levels akin to those of the CENTRALIZED policy, with the addition of only 30 additional message ex- changes among robots, compared to the DISTRIBUTED policy. This, in turn, positions our proposed policy as significantly more scalable than the CENTRALIZED policy.
Method
No. of Message Exchanges
No. of Function Evaluations
Accuracy
DISTRIBUTED
600,000
0.65
CENTRALIZED
18,000
12,000,000
0.75
INTERACTIVE
30
600,000
0.75
Table 4: Performance Metrics for DeepDriive experiments
A.10 Federated Learning Experiments and Comparative Analysis
Federated learning (FL) is a commonly used method to train machine learning models in decentral- ized settings [73]. In FL, models are trained in distributed fashion, where individual robots train models locally and transmit only gradient updates to a central cloud. These gradient updates are then aggregated, producing averaged gradient that, in turn, updates the global model. FL preserves the privacy of local data, as only gradient updates are shared with the cloud, followed by their averaging.
To compare our method against FL methods, we trained FL models using the local data from each robot. We simulated scenarios where each robot had labels for randomly selected subsets of local observations, corresponding 1x, 2x, 5x, 10x, 20x, and 50x the cache size in the active learning counterpart. all experiments, each robot independently trained its local model for 20 epochs using the same optimization parameters outlined in Section A.3. Subsequently, each robot shared
21

---

MNIST
CIFAR10
Adverse-Weather
DeepDrive
Accuracy
Round
FL
2x FL
5x 10x 20x 50x FL
Interactive
Figure 8: Comparison our INTERACTIVE policy versus Federated Learning. In addition to the active learning benchmarks, we have compared our INTERACTIVE policy against federated learning scheme. The different shades of purple illustrate the FL experiments, where the local training of the ML model employs 1x,2 2x, 5x, 10x, 20x, and 50x, the amount of data used in active learning scheme. Darker shades indi- cate larger data amounts the FL experiments. Across experiments, the results consistently demonstrate the superior performance of our proposed INTERACTIVE policy when the same number of data points used in FL. Remarkably, the FL approach requires up to 50x, x and 50x more data to attain performance compa- rable to our policy on the MNIST, CIFAR-10, 10, and Adverse-Weatee Weather datasets, respectively. This underscores the remarkable data efficiency of our approach.
gradients with the cloud. At the end of each round, the cloud averaged these gradients to update the global model. This updated global model was then sent back each robot in the network.
The results for the FL experiments and our proposed policy are presented in Fig. 8. Across all the experiments, when our proposed policy and FL have access the same amount of data, our policy consistently outperforms the FL. counterparts by 42.1 %6, 43.9%, 41.0% and 18.9%. In addition, to achieve the same accuracy as policy, FL needs 50x, x0x, and 50x more labeled data samples datasets, respectively. However, in the more
for the MNIST, CIFAR-10, and Reverse-Weatthee challenging DeepDrive dataset, FL is able achieve comparable result even when x0x more labeled data samples all of the local observations are used in the training. This observation further underlines the efficiency of active learning-based approach relative FL.
Several factors contribute to this performance divergence. In the context active learning, the selection of data points for each robot is limited maximum of 20 across all simulations. As result, when FI. is trained on the same number of samples, the local models tend to overfit, thereby compromising the performance of the global model. Moreover, for more challenging MI tasks, the local dataset size within FL. falls short of achieving high accuracy, even when all observed data proposed INTERACTIVE policy mitigates these issues by pooling
points are utilized. In contrast, the data in a centralized cloud, enhancing the training dataset size of the global model. As rounds progress, the shared data pool expands, leading improved overall model performance. In this light, our INTERACTIVE policy complements traditional FL methods.
22


Reviews:
[{"review_id": "_cbrEOnHT8", "reviewer": "Reviewer_25hL", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper tackles the problem of intelligently selecting useful data collected by a multi-robot system under constraints. Each robot collects data, but only some of them are useful as other robots might have collected similar data already. The collected data is sent to a cloud server, where the underlying machine learning model is updated. \nThe paper proposes to allow communication between the robots to effectively select a subset of their newly collected data sets based on a submodular objective that involves similarity measures between data points. The whole data set can not be sent to the cloud due to bandwidth constraints. \nThe proposed method is shown to outperform the baseline and achieve the same performance as a centralized method which assumes to have access to the whole data set.", "summary_of_recommendation": "This paper presents an interesting approach to Fleet robotics with promising results. I believe it is relevant to the research community.\n\n", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "- Were experiments with different similarity measures conducted? \n- How is the data labeling performed?\n- It would be interesting to see how federated learning (FL) performs if labeled data was provided. It might be that it outperforms the proposed method but a comparison would still be very interesting.  \n\n\n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Review:\n\nStrengths:\n- Nice overview of related work, especially explicitly mentioning the differences between the proposed method to related works is very valuable \n- Nice writing. The paper is easy to read such that the reader can easily follow the story. \n- Promising results\n\n\nWeaknesses:\n- No major weaknesses, however, additional comparisons would be nice (see Questions for Rebuttal)", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "UY1WOcGzK9", "reviewer": "Reviewer_xJp2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents an active learning framework for a distributed learning system, i.e., a fleet of autonomous vehicles, collecting data for a perception system on the roads. Ideally, the data collected is limited in size, but at the same time, the most informative and diverse for a neural network to learn from. The paper builds upon the previous work of submodularity for active learning, and provide a solution called INTERACTIVE, which minimizes communication between the agents, while distributed maximizing the information gain. Experimental results show that the proposed solution yields similar results to the completely centralized approach, while outperforming the purely distributed one.", "summary_of_recommendation": "I generally find the paper interesting and relevant for many problems in robotics. The paper is also well written, and has rigours for the presented method. I however would like to see some of the clarifications to the raised issues above, before making the final recommendations.\n\n------------------------------------ Post rebuttal ----------------------------------\n\nI have increased the score to weak accept, since the authors have addressed most of my comments through experiments.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "See the section above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Pros:\n- the paper grounds itself in practical scenario of data gathering for autonomous cars. Perhaps in control literature, there has been similar set ups with distributed systems/sensing, e.g., sensor network placement, etc, the paper brings these ideas for practical robotic scenarios.\n- the paper is generally well written in terms of explanations. Especially, the paper clearly outlines the assumptions and settings.\n\n\nCons:\n- in the experiments, the choice of baselines should be motivated.\nThe paper chooses a completely distributed approach and a centralized approach as baselines. Why not add a random baselines, which picks from the complete data pool (like centralized) and the data pool of each robots (like decentralized)? While experiments do validate the design of the approach to certain degrees, readers may appreciate more, when a thorough comparison to basic baselines (random uniform sampling) and other pool-based active learning methods for distributed systems, e.g, uncertainty sampling.  \n\n- the claimed contributions and related work require clarifications.\nThe paper states that one of its contribution is to frame the data collection problem of multiple robots as submodular maximization problem. Does not reference 46 and 47 also use submodular maximization approach for similar problem, where the goal is to maximize the information gain? It would help to comprehend the contributions of the paper. \n\nI also think that some of the statements in the abstract can be misleading, The statements on the performance in particular, is with respect to one baseline, which is completely decentralized, while the paper utilizes certain communication with some of the robots. I am not sure, if that would be a fair comparison, and I would appreciate for the clarifications.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "8Bp70cqN_K", "reviewer": "Reviewer_fsBm", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper studies how autonomous vehicles (AV) can improve their datasets for perception and planning by actively selecting new datapoints for training. When deploying a fleet of AVs it is impractical to collect and process all available data due to bandwidth constraints and computational cost. Active learning methods allow for choosing only the most informative datapoints, yet distributed execution on multiple vehicles can lead to redundant data and thus poor performance. The authors propose a hybrid method where all AVs iteratively select their next data set. Here, each vehicle builds on the data collected by all AVs in previous rounds. The problem is formulated as a submodular maximation, and the authors show that their approach is a 2-factor approximation. Simulation results suggest that the method performs equally well as a centralized approach and clearly outperforms a purely distributed approach where robot exchange no information.\n", "summary_of_recommendation": "Overall, this is a well-written and original paper with a solid contribution. However, my two main concerns are the assumptions about monotonicity and submodularity that need further discussion, and some clarification on the simulation results since it is quite surprising that there is no gap between centralized and the proposed.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "1.\tAssumptions 2 and 3: I understand the intuition behind these assumptions; however I think that they need to be discussed more thoroughly. Any ML system can run into overfitting. Thus, couln't adding more samples become detrimental to the dataset quality if the new data only represents a small part of the distribution? For instance, if we ‘spam’ the dataset with a large amount of data recorded under certain road conditions, I would suppose that this can lead to poorer performance for other conditions.\nSimilarly, the diminishing return might not hold: Suppose we already have some data, but it does not cover, a certain scenario yet. The first datapoint for that scenario might not be of much value since it is insufficient to train any new behaviour. Yet, adding additional data from that scenario might eventually lead to a better network output, which would invalidate submodularity.\nIt is hinted that “the dataset already covers a significant portion of the real data distribution” – this needs more elaboration to justify assumptions 2 and 3 (Also the simulation setup does not support this since the initial datasets are quite small). Lastly, the appendix includes an empirical study that the accuracy measure is showing a diminishing return – while this provides additional motivation for the approach, it does not substitute a more rigorous theoretical justification.\n\n2.\tNP hardness: The paper simply states that the problem is a combinatorical optimization problem and is NP hard. There are many combinatorial problems that are not NP-hard. Hence, this needs some more justification, e.g., a reduction (I realize that space is limited) or clearly relating it to a specific NP-hard problem, or giving a citation that considered this problem and showed hardness.\n\n3.\tSimulation results: It is surprising that there is no gap between the proposed iterative approach and centralized. The problem statement states that each robot has X_i^r datapoints available – yet from the experiment description and the appendix it is not clear if these are the same sets for all robots or if they differ. If they are the same, it is clear that iterative would perform the same as centralized, but this might not be the most interesting problem setup to consider. Furthermore, in 2 experiments, each robot selects only 1 sample per round, in another it is 3 samples; only for DeepDrive robots select 20 samples. I think that the small number of selected samples might also lead to the two algorithms being quite similar. \n\n4.\tSimulations: It would be good if the experiments could highlight the shortcomings of the centralized approach. While the bandwidth constraints and computational burden are mentioned in the introduction, the current results do not show any disadvantage of centralized and rather focus on showing that the proposed method is equally good w.r.t. the submodular objective and the accuracy. I noticed the complexity analysis in the appendix, but some numerical evaluation would bring additional practical insights.\n\nMinor comments:\n\n5.\tFig 1: Could the submodular function f be somehow included in the figure?\n\n6.\tDefinition 1: I think it is a bit unusual to use a lower case for a set (here a_i^r). \n\n7.\tStructure: Why are the baselines (Algorithms 1 and 2) part of the problem formulation? I think these should be in a new section. Also after Assumption 3, the main problem could be put in a theorem-style environment to make it stand out more.\n\n8.\tLine 253: Theorem 1 has shown that the proposed policy has the same worst case bound as the centralized one, but that does not imply that it would perform as good as centralized.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overall, the paper is well written, easy to follow and studies a relevant and active problem. The proposed approach is technically sound and comes with a thorough theoretical analysis. The simulations are using real-world data sets with a variety of different configurations and showcase advantages of the proposed method. However, I have a few comments that should be clarified, please see below.\n\n", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors study a fleet active learning approach where autonomous agents can select training data samples to improve local models. The chosen approach in particular utilizes submodular maximization in the sample selection process. The review process resulted in a uniform recommendation of "weak accept" among the reviewers. Among the strengths of the work highlighted by the reviewers were
in particular that the work is well written and readable, that the grounding of the problem was well motivated and potentially relevant to many problems in robotics. The questions raised by one reviewer regarding the submodulariy assumptions are I believe an area that may warrant additional future investigation in particular and a careful discussion of potential edge or failure-cases could further strengthen the work.
Decision: Accept (Poster)