Decentralized Data Collection for Robotic Fleet Learning: A Game-Theoretic Approach
Abstract: Fleets of networked autonomous vehicles (AVs) collect terabytes of sensory data, which is often transmitted to central servers (the “cloud”) for training machine learning (ML) models.
Ideally, these ﬂeets should upload all their data, especially from rare operating contexts, in order to train robust ML models.
However, this is infeasible due to prohibitive network bandwidth and data labeling costs.
Instead, we propose a cooperative data sampling strategy where geodistributed AVs collaborate to collect a diverse ML training dataset in the cloud.
Since the AVs have a shared objective but minimal information about each other’s local data distribution and perception model, we can naturally cast cooperative data collection as an N -player mathematical game.
We show that our cooperative sampling strategy uses minimal information to converge to a centralized oracle policy with complete information about all AVs.
Moreover, we theoretically characterize the performance beneﬁts of our game-theoretic strategy compared to greedy sampling.
Finally, we experimentally demonstrate that our method outperforms standard benchmarks by up to 21 .
9% on 4 perception datasets, including for autonomous driving in adverse weather conditions.
Crucially, our experimental results on real-world datasets closely align with our theoretical guarantees.
Introduction
Envision a ﬂeet of autonomous vehicles (AVs) that observes heterogeneous street scenery, weather conditions and rural/urban trafﬁc patterns.
To train robust ML models for perception or trajectory prediction, these AVs should share as much diverse ﬂeet data as possible in the cloud, while balancing network bandwidth, data storage and labeling costs.
2 Given these constraints, we argue that AVs must coordinate how to sample rare, out-of-distribution (OoD) data with common examples based on their unique local data distributions.
For example, if only a few AVs operate in heavy snow, they should specialize in sending snowy images to the cloud while others should send data from more common scenarios like sunny weather.
Since the AVs have a shared target data distribution (objective) but limited information on each other’s local data distribution and potentially private ML models, our key contribution is to cast data collection as a N-Player mathematical game .
In our game-theoretic formulation (Fig. 1), the AVs exchange minimal information to choose a data sampling strategy (what limited subset of data-points to upload).
Importantly, we prove that an AV ﬂeet will quickly converge to a Nash equilibrium (i.e., a ﬁxed point where each robot does not change its sampling strategy)
[3, 4] with bounded communication.
Morever, our practical formulation accounts for perceptual uncertainty from imperfect computer vision models and heterogenous local data distributions.
As such, to the best of our knowledge, we are the ﬁrst to cast data sampling from networked robots as a mathematical game.
In summary, our key contributions are:
We provide a novel formulation for distributed data collection as a potential game [5] since the robots attempt to minimize a common convex objective function that incentivizes them to reach a balanced target data distribution in the cloud.
We prove that our strategy converges to a centralized oracle policy and, under mild assumptions, converges in a single iteration.
[Figure]
[Figure] Action # [Figure]
[Figure]
Cloud + =
Device Dataset Cloud Dataset
[Figure] Retrain " $ [Figure] Network
[Figure]
We provide theoretical performance bounds characterizing the beneﬁts of our game-theoretic approach compared to greedy, individual behavior.
We show that our proposed strategy outperforms competing benchmarks by 21 .
9% on 4 datasets, including the challenging Berkeley DeepDrive autonomous driving dataset [6].
Related Work:
Data collection from networked robots is related to cloud robotics [7–15] and active learning [16–20].
In such prior works, robots either send all their data to the cloud or select samples individually without coordination.
In contrast, we exploit the fact that networked AVs can coordinate how to sample rare data to achieve a better outcome (i.e., balanced data distribution).
Federated learning (FL) [21–29] enables a ﬂeet of mobile devices to train ML models on local private data and only share anonymized gradient updates with the cloud.
However, our work is fundamentally different, and even complementary, to standard FL.
First, FL makes the restrictive assumption that each robot has perfectly labeled local data, which is infeasible for AVs that observe rare, OoD images.
Instead, we address a practical scenario where robots run local inference with only an imperfect vision model that guides data collection.
Moreover, FL does not statistically sample data but trains on all of it locally, while our approach only uploads a limited set of images to reduce network and data labeling costs.
Finally, we assume robots only receive ground-truth labels for the uploaded data in the cloud, which is required for training on rare classes.
Our setting is a non-cooperative game since the robots do not explicitly form coalitions and act with minimal information about each other [5, 30–34].
Speciﬁcally, our setting is a potential game since each robot attempts to maximize a shared concave objective function (the common potential function) that rewards progress towards a balanced target data distribution in the cloud.
As detailed in Sec. 2 and Appendix 5.1, changes in the common potential function directly translate to changes in each robot’s policy towards a Nash Equilibrium.
While concave games have been applied to problems such as wireless network resource allocation [35], ours is the ﬁrst work to contribute a game-theoretic formulation for distributed data collection from a ﬂeet of robots.
Problem Formulation
We now formulate a practical scenario, shown in Fig.
1, where distributed robots collect data to train a robust ML model in the cloud.
Our goal is to select an appropriate action for each robot, specifically the data-points it should upload, so that the overall collected cloud dataset closely matches a given target, such as an equal distribution over all classes.
Fig.
2 intuitively depicts data sampling.
Our general formulation applies to any robot constrained by network, storage, or labeling costs, ranging from Mars Rovers constrained by the Deep Space Network ( < 5 Mbps) [36] or AV ﬂeets.
Robot Perception Model:
For a simple exposition, we ﬁrst consider a general computer vision classiﬁcation task with N class classes.
The dataset used for training the model is stored in the cloud.
Each period of data collection, such as a day, is denoted by a round r and data is uploaded to the cloud at the end of a round r .
The cumulative dataset stored in the cloud at the end of round r is denoted by D rc , whose size is given by N D rc =
| D rc | .
N y j denotes the number of class j datapoints in the dataset D rc .
Therefore, the distribution of classes in the dataset D rc is denoted by r D rc = h
N y 0 , N y 1 , ··· , N y N class i .
Each robot i has a perception model, such as a deep neural network (DNN), where local inference is denoted by ˆ y = f
( x ; q ri ) .
Here, f ( · ; q ri ) is a model with parameters q ri at round r , ˆ y is the predicted label for input x , and y is the corresponding ground-truth label.
Importantly, the models can be imperfect – each model has a confusion matrix, C ri 2 R N class ⇥ N class (Eq. 4) that captures the probability of predicting class ˆ y j for an image with true class y j , denoted by p ( ˆ y j | y j ) .
In practice, one of the N class classes can represent an “unknown” category while the rest of the N class  1 classes can represent a mixture of rare and well-understood concepts.
Further details on the confusion matrix are provided in Appendix 5.2.
Finally, while we use a (likely imperfect) classiﬁcation model to sample images, the uploaded data can be used to train models for diverse tasks such as object detection, semantic segmentation etc.
Robot Fleet: We consider a ﬂeet of N robot robots, where each robot i collects a data-point x ti at time t from its local environment (i.e., camera image or LiDAR scan).
The distribution of true classes observed by a robot i in round r is denoted by p ri ( y ) 2 R N class + , which sums to one over the N class classes.
From this distribution, a robot i observes a large dataset of images on round r denoted by D ri of size | D ri | = N ri .
However, to limit network bandwidth and data labeling costs, each robot i can only upload N cache ⌧ N ri images to the cloud at the end of round r , which it stores in an onboard cache within the round.
The size of N cache can be ﬂexibly set by a roboticist based on data upload and labeling budgets.
The class predictions, ˆ y j , are generated by running local inference of the classiﬁcation model, ˆ y i =
f ( x ti ; q ri ) , for the collected data-points x ti .
Finally, p ri ( ˆ y ) denotes the distribution of predicted classes observed by robot i .
Assumption 1.
The number of data-points collected by a robot on any round r, N ri , is signiﬁcantly greater than the size of the local robot cache, N ri  N cache .
This is a valid assumption since each robot will collect much more data compared to the amount it can economically upload.
Our formulation is extremely general – each robot can have different (or the same) model parameters q ri and observe a different distribution p ri ( y ) of the N class classes.
Robot Statistical Sampling Action: At each round r , each robot i takes an action which determines how many data-points of each class to send to the cloud.
We deﬁne each robot
i ’s action at round r as a ri = [Equation] i , i.e. the number of data-points of each class j .
Our key technical innovation will be to illustrate how to cooperatively select an optimal action.
Importantly, since each robot i has an imperfect perception model with confusion matrix C ri , there is uncertainty in the effect of taking any action a ri .
As such, our natural next step is to deﬁne the set of feasible actions any robot can upload given its local data distribution and perceptual uncertainty.
Deﬁnition 1 (Feasible data matrices) .
A feasible data matrix, P ri 2 R N class ⇥ N class , of robot i in round r is the probability matrix deﬁned as:
[Equation]
where p ri , j = C ri , j ⇤ p ri ( y ) k C ri , j ⇤ p ri ( y ) k 1 = p
( y | ˆ y j ) 2 R N class , 8 j = 1 ,..., N class .
We use ⇤ as element-wise multiplication of vectors, k · k 1 as the L 1 norm, and the second subscript j to denote the j-th column of a matrix.
We assume P ri has linearly independent columns, so there exists a left inverse.
In other words, we assume the mapping from action to feasible action (Defs.
2, 4) is one-to-one.
This assumption is justiﬁed in the Appendix due to space limits.
Deﬁnition 2 (Feasible spaces of robots) .
A feasible space, H ri , of robot i in round r is the set of feasible data-points the robot can send to the cloud:
[Equation]
H ri is the convex hull of all columns of P ri and 0 .
To simplify notation, v ri =
P ri a ri represents a feasible action v ri , which is obtained by multiplying an intended action a ri by the feasible data matrix P ri .
Intuitively, the feasible space (see Fig. 2) represents the expected number of datapoints uploaded per class but not the exact number due to perceptual uncertainty.
Each robot uploads N cache data-points sampled from action a ri , which is pooled in the cloud.
We assume we only get ground-truth labels y in the cloud, since the limited cache of images can be scalably annotated by a human.
Then, we re-train a new perception model on the new dataset D r + 1 c .
Each robot then downloads the new model parameters q r + 1 i , along with the new confusion matrix and latest cloud dataset distribution, r D r + 1 c .
Our formulation is general – models and confusion matrices do not have to be updated every round r and we can, for example, simply re-train a model after M rounds of data collection.
Collective Goal: Achieving a Target Data Distribution Often, we want to achieve a balanced dataset in the cloud with ample representation of rare events in order to train a robust ML model.
As such, the shared goal of all the robots is to achieve any user-speciﬁed target dataset r D target , which deﬁnes the number of data-points of each class the robots want to collect in the cloud.
The ﬂeet’s goal is to choose actions a ri ; 8 i = 1 ,..., N robot at round r to collectively reduce a strictly convex distance metric, denoted by L ( r D rc , r D target ) , penalizing the difference between the current cloud dataset r D rc and the target dataset r D target .
Our general framework can handle any strictly convex distance metric, such as the L 2 norm or the Kullback-Leibler (KL) Divergence [37].
Since all robots have a common goal to maximize the negative loss  L ( r D rc , r D target ) , which is a concave potential function, our setting is a potential game with concave rewards (see Sec. 5.1).
Centralized Oracle Action Policy: We now provide a formal optimization problem for distributed data collection.
To provide key insight, we ﬁrst describe a centralized “oracle” solution that has perfect information about all robots i , namely their confusion matrix C ri and statistics of their data distribution p r
i ( y ) .
Then, we formalize a greedy, individualized approach and our interactive gametheoretic approach that matches the oracle policy’s performance.
An oracle action policy, denoted by O RACLE , has access to all robots’ data distributions and confusion matrices C ri .
The oracle calculates each robot’s action a ri by solving the convex optimization problem in Eq.
The constraint (Eq. 1c) ensures that the actions a ri do not exceed the cache limit
N cache .
Eq.
1d shows the update of the cloud dataset for round r + 1 based on the actions a ri taken in the feasible space, P ri a ri , by each robot for round r , which we now detail.
A key subtlety is to update the cloud dataset r D r + 1 c by merging the current cloud dataset r D rc and each robots’ uploaded dataset a ri .
However, each robot’s action is imperfect – it might think it is uploading class j but due to perceptual uncertainty it might actually upload another class j 0 .
Speciﬁcally, the robot’s transmitted dataset a ri is calculated from the predicted class labels ˆ y j and not the true class labels y j , which are not available on-robot.
However, we can use predicted class probabilities p ri ( ˆ y j ) to estimate true class probabilities p ri ( y j ) by: p ri ( y j ) =
Â N class k = 1 p ri ( ˆ y k ) · p ri ( y j | ˆ y k ) .
Note that each robot only receives a confusion matrix C ri from the cloud which consists of conditional probabilities p ri ( ˆ y j | y j ) and not p ri ( y j | ˆ y j ) .
Therefore, we still need to ﬁgure out a way to calculate p ri ( y j | ˆ y j ) .
Due to space limits, we present the Bayesian update of p ri ( y j | ˆ y j ) in the Appendix 5.3.
min
[Equation]
(1a) subject to: a ri  0; 8 i [Equation] ,..., N robot (1b)
[Equation]
) r
[Equation]
N robot Â i [Equation] [Equation] (1d) min a ri
L ( r D r + 1 c , r D target ) (2a) subject to: a ri (2b) 1 T · a ri  N cache (2c) r D r + 1 c = r D rc +( P ri a ri ) (2d)
Greedy Action Policy: A greedy action policy, referred to as G REEDY , will not have any information about other robots’ local data distribution, confusion matrix, or observed datasets.
Thus, the best the robot can do is to attempt to minimize the loss function L ( r D r + 1 c , r D target ) by only optimizing its own action a ri individually , as shown in Eq. 2.
The optimization program 2 is very similar to that of the O RACLE policy (Eq. 1), with the only difference being that the decision variables are reduced to one.
Since the O RACLE (Eq. 1) and G REEDY (Eq. 2) policy optimization programs have a convex objective with linear constraints, they are guaranteed to converge to an optimal solution.
A Cooperative Algorithm for Data Collection
We propose an I NTERACTIVE algorithm for generating actions for each robot, which only requires interaction between the robots and no cloud coordination.
Rather than the cloud calculating actions for each robot in one-shot, as shown in the O RACLE optimization program (1), each robot calculates its actions individually using shared information from other robots.
Importantly, each robot only shares its feasible action without divulging its confusion matrix or local data distribution to others.
Alg. 1 describes our I NTERACTIVE policy, which runs for each round r .
The inputs (line 1), which are visible to each robot, are the target dataset r D target and the current cloud dataset r D rc .
We initialize each robot’s action a ri in lines 2 - 4 using the G REEDY policy (Eq. 2) because the robots have not yet communicated any information about each others’ tentative actions.
In lines 5 - 10, we calculate optimal actions for each robot using the I NTERACTIVE message passing algorithm.
We start by sharing each robot’s product of feasible data matrix and initial action (line 3) with all other robots (line 5).
Then, we iterate over each robot (lines 7 - 10) and calculate its best action a ri using the optimization program Eq.
3 while considering the other robots’ actions ﬁxed (line 8).
The optimization program in Eq.
3 is similar to that of the O RACLE policy (Eq. 1); the difference lies in the calculation of the cloud dataset at round r + 1 in Eq.
3d and having one decision variable.
In line 9, each robot shares its product of the feasible data matrix and the optimal action calculated using Eq.
3 with the others.
This repeats until our system reaches a Nash equilibrium (i.e. a ﬁxed point, where no robot would change its action).
Finally, after convergence, we upload data from each robot sampled according to its ﬁnal calculated action a ri (line 13).
Since the I NTERACTIVE optimization program 3 is convex, it converges to an optimal solution (see Thm. 1).
1 Input: Target, Cloud Dataset
D rc do
4 end 5 Share P
6 while do do
Get Share end 11 end do
14 end
P ROBLEM 3: I NTERACTIVE
O PTIMIZATION
[Equation]
Theoretical Analysis: We ﬁrst show that the while loop (lines 6 - 11) in our proposed Alg. 1 will eventually converge.
Moreover, we provide easily-obtained conditions for when it converges in one iteration , which minimizes inter-robot communication.
Crucially, we also show that our interactive policy matches the omniscient oracle policy.
All proofs are in the Appendix 5.6 - 5.9.
Theorem 1 (Convergence) .
The while loop (lines 6 - 11) in Alg.
1 will eventually converge.
Next, we show one of the main technical contributions of this paper, which states that our proposed
I NTERACTIVE algorithm will reach the same optimal solution as the O RACLE upon termination.
Theorem 2 (I NTERACTIVE converges to O RACLE ) .
The while loop in Alg.
1 lines 6 - 11 is guaranteed to return an action (denoted by a rint , i ) that is equal to the O RACLE action denoted by a ro , i .
Next, we provide practical conditions for when our proposed I NTERACTIVE action policy will converge in one iteration of message passing, which bounds inter-robot communication.
Theorem 3 (Bounded Communication) .
When the total number of uploaded data-points is smaller than the difference between the size of target dataset D target and the current cloud dataset D rc , namely 1 > ( D target  D rc ) > N robot ⇥ N cache , the while loop in Alg.
1 lines 6 - 11 terminates in one iteration.
The condition in Thm.
3 holds for all rounds except for the last round that reaches the target distribution, upon which data collection terminates.
All our theory assumes that all actions a r i 2 R N class can realize any feasible real-valued vector.
However, in reality, we will only have an integer-valued action vector since we can only upload a discrete set of images, which becomes an integer programming problem.
However, for real-world datasets with thousands of images, we can just round the continuous solution to get a very close approximation to the (generally intractable) integer case.
Experiments and Conclusion
We now compare our Alg. 1 with benchmark methods on four diverse datasets.
The ﬁrst two datasets of MNIST [38] and CIFAR-10 [39] serve as proof-of-concepts for the domains of handwritten digit and common object classiﬁcation.
Then, we use the Adverse-Weather dataset [40], which contains tens of thousands of images to train self-driving vehicles to classify rain, fog, snow, sleet, overcast, sunny, and cloudy driving conditions.
To show the generality of our theory, we then extend to the state-of-the-art Berkeley Deep Drive ( DeepDrive ) dataset [6], which has 100K images of various weather conditions and road scenarios for self-driving cars.
Comparison Metric: To compare all methods, we use the L 2 -norm (the optimization objective) between the target r D target and the current cloud dataset r D rc .
For statistical conﬁdence, all experiments are repeated for more than 10 times with different random seeds that capture uncertainty in sampling from the confusion matrix C ri and observing different distributions of local data per robot.
Further experiment parameters are detailed in the Appendix 5.4.
We compare the following methods:
G REEDY solves the optimization program in Eq.
2 individually per robot by minimizing the L 2 -norm between the target and cloud distribution without information about other robots.
O RACLE solves the optimization program in Eq.
It perfectly knows all incoming class data distributions p ri ( y ) and confusion matrices C ri for all robots i and thus calculates the optimal action for each robot in one common optimization problem (Eq. 1).
U NIFORM is a deterministic policy which assigns the same probabilities to all classes for each robot, i.e. a ri = h 1 N class ... 1 N class i .
It represents a simple heuristic for equally sampling all classes.
L OWER -B OUND (derived in Lemma 6) is the lower bound of the objective function of the O RACLE policy for a given target dataset, r D target , current cloud dataset, r D rc , and local data distribution p ri ( y ) .
It represents how well can sample in the absence of perceptual uncertainty.
I NTERACTIVE runs our Alg. 1.
It is not an Oracle policy, as it only shares the action taken by other robots and not the actual class data distribution, p ri ( y ) , nor the confusion matrix.
Results: Our experimental results (Fig. 3) demonstrate that our proposed I NTERACTIVE policy performs as well as the O RACLE , as proved in Thm. 2.
Additionally, we demonstrate that our I NTERACTIVE policy is much better than the G REEDY and U NIFORM policies on all datasets.
Finally, we show that no action policy can perform better than the derived L OWER -B OUND action policy.
M N I
S T
C
I F
AR
A d ve r sa r i a l
W ea t h e r
D ee p D r i ve
Does cooperation minimize distance to the target data distribution?
Our optimization objective is to minimize the L 2 -norm distance between the cloud dataset and target data distribution, which we plot in the ﬁrst column.
Clearly, our I NTERACTIVE policy signiﬁcantly outperforms the G REEDY and U NIFORM policies on all datasets.
Speciﬁcally, we beat the G REEDY policy by 23 .
7% in L 2 -norm distance on the MNIST , CIFAR-10 ,
Adverse-Weather , and DeepDrive datasets respectively.
These beneﬁts arise because the I NTERACTIVE policy allows robots to coordinate the rare classes they upload, but the G REEDY policy might lead to uncoordinated uploading of redundant data.
Moreover, our I NTERACTIVE policy performs nearly identically as the O RACLE method, with small deviations due to imperfect vision models and randomness in local data distributions between trials.
This is natural since we proved that the I NTERACTIVE action policy reaches the same optimal value in expectation as the O RACLE policy in Thm. 2.
Finally, we observe that no action policy outperforms our L OWER -B OUND policy derived in Lemma 6.
Does cooperation achieve more balanced datasets?
In column two, we see that the initial data distribution among robots (gray) is highly imbalanced since they operate in diverse contexts.
However, we see that our I NTERACTIVE policy (green) achieves a much more balanced dataset distribution compared to G REEDY (orange), which is natural since the convex objective minimizes the distance to a uniform distribution.
What is the ﬁnal accuracy of trained models?
In column 3, we show the ﬁnal accuracy of re-training DNN classiﬁcation models on the datasets accrued by each method in the cloud.
Importantly, our proposed I NTERACTIVE action policy leads to better accuracy gains than the G REEDY and U NIFORM action policies.
We beat the G REEDY policy by 1 . 4% , 1 . 7% , 21 .
4% in accuracy on the MNIST , CIFAR-10 , Adverse-Weather , and
DeepDrive datasets respectively.
This is because the I NTERACTIVE action policy makes sure we collect classes lacking in the current cloud dataset, thus preventing class-imbalance issues in model training.
While our theory only addresses convex distances between dataset distributions (column 1 and 2), we show strong experimental results for re-training non-convex DNN classiﬁers.
The
I NTERACTIVE and O RACLE algorithms lead to slightly different ﬁnal accuracies since they can potentially upload a different set of images and there is not a closed form relationship between the number of images and accuracy of a non-convex DNN.
As detailed in the Appendix, I NTERACTIVE achieves very close to state-of-the-art accuracy for each dataset with only a limited set of uploaded datapoints.
DNN architectures are also detailed in the Appendix.
Collectively, these results closely align with our theory and show strong experimental beneﬁts on real-world data.
Limitations: Our work assumes each robot can interact, which does not scale for extremely large ﬂeets.
Moreover, we assume that we sample images according to a classiﬁcation model, even though we can train models for other tasks on the uploaded images.
In future work, we aim to extend our theoretical guarantees for sub-clusters of communicating robots and cluster a continuous data distribution based on similar embeddings that serve as virtual “classes”.
Such an ability to generalize beyond discrete classes may enable our algorithm to scale to learning data-driven control policies.
Conclusion: This paper presents a theoretically-grounded, cooperative data sampling policy for networked robotic ﬂeets, which converges to an oracle policy upon termination.
Additionally, it converges in a single iteration under a mild practical assumption, which allows communication efﬁciency on real-world AV datasets.
Our approach is a ﬁrst step towards an increasingly timely problem as today’s AV ﬂeets measure terabytes of heterogenous data in diverse operating contexts [1].
In future work, we plan to develop policies that approximate the oracle solution when only a subset of robots can form coalitions and certify their resilience to adversarial node failures.
Acknowledgements
This material is based upon work supported in part by Cisco Systems, Inc. under MRA
MAG00000005.
This material is also based upon work supported by the National Science Foundation under grant no. 2148186 and is supported in part by funds from federal agency and industry partners as speciﬁed in the Resilient & Intelligent NextG Systems (RINGS) program.