Active Learning for Multiple Target Models
Abstract
We describe and explore a novel setting of active learning (AL), where there are multiple target models to be learned simultaneously.
In many real applications, the machine learning system is required to be deployed on diverse devices with varying computational resources (e.g., workstation, mobile phone, edge devices), which leads to the demand of training multiple target models on the same labeled dataset.
However, it is generally believed that AL is model-dependent and untransferable, i.e., the data queried by one model may be less effective for training another model.
This phenomenon naturally raises a question “ Does there exist an AL method that is effective for multiple target models? ”.
In this paper, we answer this question by theoretically analyzing the label complexity of active and passive learning under the setting with multiple target models, and conclude that AL does have potential to achieve better label complexity under this novel setting.
Based on this insight, we further propose an agnostic AL sampling strategy to select the examples located in the joint disagreement regions of different target models.
The experimental results on the OCR benchmarks show that the proposed method can signiﬁcantly surpass the traditional active and passive learning methods under this challenging setting.
Introduction
Data labeling is expensive due to the involving of human annotator.
Active learning (AL) is one of the main approaches to reduce the labeling cost [28].
It evaluates the utility of the unlabeled data based on the target model, and actively queries the labels from the oracle for the examples that is the most beneﬁcial to the performance improvement of the target model.
Existing active learning methods assume that there is only one speciﬁc target model and try to ﬁt it with least queries.
However, in many real applications, the machine learning system is required to be deployed on multiple types of devices with different resource constraints [6].
For example, a speech recognition software needs to support diverse machines with varying hardware efﬁciency, ranging from high-performance workstation to the mobile-phone.
Due to the different computational resources, the applicable model architectures vary a lot, e.g., a deep model which is well-performed on the cloud server may not be deployed on the edge device.
It thus raises the demand of training multiple models with different complexity to accommodate these devices.
Given multiple target models, how to effectively improve them with least labeled data becomes a practical and challenging problem.
It is generally believed that AL is usually model-dependent and untransferable [22, 24, 38], i.e., the best query strategy for different target models varies a lot [39].
In other words, the data queried by one model may be less effective for training another model [22].
These observations imply that the existing active query strategies can hardly beneﬁt all target models simultaneously, and the design of AL algorithm for multi-models can be rather difﬁcult.
A natural question might be asked: “ Does there exist an active learning method which queries a set of labeled data, such that all the target models can be effectively trained with them? ”
In this paper, we formally deﬁne the active learning for multiple target models problem, and reveal the potential improvement of AL under this novel setting.
Based on this insight, we further propose an agnostic disagreement-based selection criterion.
Speciﬁcally, we ﬁrst deﬁne and analyze the label complexity for both active and passive learning under the setting with multiple target models.
This label complexity characterizes the number of labeled examples sufﬁcient to train an ε -good classiﬁer with probability at least 1 − δ for each target model.
Moreover, we ﬁnd that the label complexity of single model has a close relation to that of multiple models under the realizable case, e.g., the former provides an upper bound of the label complexity for multiple models, which also implies the potential improvement of AL under this setting.
To further explore the agnostic case, we propose an active selection method DIAM (i.e., DIsagreement-based AL for Multi-models) to effectively select the best examples that are beneﬁcial to all target models.
It prefers the data located in the joint disagreement regions of different models, which is expected to have higher potential to reduce the soft version space (i.e., the set of hypotheses with lower errors).
Experiments are conducted on the
OCR benchmarks to validate the necessity of designing active query method under this practical setting and the effectiveness of the proposed approach.
The results show that the DIAM method can signiﬁcantly reduce the number of queries to achieve a higher mean accuracy for multiple models compared to the traditional active and passive learning methods.
The rest of the paper is organized as follows.
related work is ﬁrst reviewed in the following section, then we formally deﬁne the AL for multiple target models problem and provide a general result to bridge the label complexity between single and multiple models.
Then, we reveal the potential improvement of AL under this novel setting.
After that, an agnostic active selection criterion is proposed and analyzed, followed by the empirical studies.
And at last, we conclude this work.
Related work
Active learning has received much attention in recent years due to the greatly increasing demands of labeling data to effectively train more complex models (e.g., deep models)
One of the cores of AL is how to evaluate the potential contribution to the performance improvement of the target model for each candidate query.
Most of existing criteria for active learning can be categorized into informativeness and representativeness.
The informativeness-based methods [13, 36, 17] prefer the data which is near the decision boundary, and the representativeness-based methods [27, 30, 21] impose the constraints to regularize the queried data to be dissimilar with each other or conform to the latent data distribution.
Many works also try to combine both criteria to obtain better performances [11, 37, 31].
Beyond these hand-crafted selection criteria, several meta-active-learning methods [18, 23, 35] are proposed to learn a generalizable query strategy across tasks.
Most of the existing active learning query strategies target on improving one speciﬁc target model.
They are less applicable to the multiple target models setting.
From the theoretical view, active learning theory has also been widely studied under certain conditions (e.g., binary classiﬁcation, ﬁnite VC dimension) [16].
One of the interested properties of an active learning algorithm is the label complexity, which characterizes the number of queries needed to obtain an ε -good classiﬁer with probability at least 1 − δ [14].
To bound this value, disagreement coefﬁcient [5, 4] and Shattering [15, 7] are two commonly used techniques.
While most works deal with the single model setting, Balcan et al.
[3] study the label complexity of the hypothesis space and its subclasses, which sheds light on this work.
However, they mainly focus on how to construct subclasses to achieve a certain label complexity, while we aim to ﬁnd an effective active learning algorithm on the given target models.
Recently, some AL studies tackle a related problem that the target model prior cannot be obtained.
In this setting, they will not only search the appropriate target model for the current task, but also avoid noneffective querying.
To this end, ALMS [1] either randomly labels data to calculate the unbiased validation error for model selection, or queries by the expected error reduction to improve the models.
Active-iNAS [12] considers the deep learning setting, the authors on one hand perform
Neural Architecture Search (NAS) to ﬁnd the appropriate model architecture, on the other hand query the examples by the searched network.
Recently, Tang and Huang [32] propose a uniﬁed framework
DUAL to solve this problem.
They query the data that is beneﬁcial to not only the winner model, but also the model search to identify the high potential model with least queries.
All these methods try to search effective model conﬁgurations, but not improve multiple target models, which are different from our work.
Label Complexity of Single Model and Multiple Models 3.1 Notations and Deﬁnitions
Suppose the data is sampled from an unknown distribution D XY over the feature space X and label space Y .
Denote by D X the marginal data distribution, and D Y the marginal label distribution.
Given a dataset with n instances, which includes a small labeled set L = { ( x i , y i ) } n l i =1 with n l instances, and a large unlabeled set U =
{ x i } n l + n u i = n l +1 with n u instances, where n l (cid:28) n u and n = n l + n u .
At each iteration, the active learning method will select a batch of b examples Q from U for querying. In the single model setting, we have a given set of hypothesis space C .
While in the multiple target models setting, there are k hypothesis spaces T
= {C i | i = 1 , 2 , . . .
, k } with ˜ T = (cid:83) ki =1 C i , our goal is to actively query a set of examples to output a well-performed hypothesis
ˆ h i from each C i , ∀ i
= 1 , ..., k . We deﬁne the true error of a hypothesis as er( h ) =
P x ∼D X ( h ( x ) (cid:54) =
h ∗ ( x )) , where h ∗ is the target concept, h ( x ) is the model prediction on the data x .
The empirical error of h on L is deﬁned by er L ( h ) = 1 |L| (cid:80) x ∈L I [ h ( x ) (cid:54)
=
h ∗ ( x )] , where I [ · ] is the indicator function. Let ν
i = min
h ∈C i er ( h ) , Log( a ) =
max { ln( a ) , 1 } , ∀ a > 0 .
Here we introduce the deﬁnition of the pseudo-metric between hypotheses, which is frequently used in the subsequent proof.
Deﬁnition
Pseudo-metric between Hypotheses: Given D X , the probability of disagreement between two classiﬁers h 1 and h 2 is deﬁned as d ( h 1 , h 2 ) =
P x ∼D X ( h 1 ( x ) (cid:54) =
h 2 ( x )) .
Then we introduce the label complexity of AL for single target model [16].
Deﬁnition 2.
Label Complexity of AL for Single Target Model:
For any active learning algorithm A , we say A achieves label complexity Λ on the hypothesis space C if, for every ε, δ ∈ (0 , 1) , every distribution D XY over X × Y , and every integer t ≥ Λ ( ε, δ, D XY ) , if h t,δ is the classiﬁer produced by running A with budget t , then with probability at least 1 − δ, er( h t,δ ) − min h ∈C er( h ) ≤ ε . Now we formally deﬁne the label complexity of active learning for multiple target models.
It is deﬁned on multiple hypothesis spaces, and the goal is to output an ε -good classiﬁer for each target model. Speciﬁcally, the label complexity for the AL with multiple target models is deﬁned as
Deﬁnition 3.
Label Complexity of AL for Multiple Target Models: Given a set of target models
T =
{C i | i = 1 , 2 , . . .
, k } .
For any active learning algorithm A , we say A achieves label complexity ˜Λ for multiple target models if, for every ε, δ ∈ (0 , 1) , every distribution D XY over X ×Y , and every integer t ≥ ˜Λ ( ε, δ, D XY , T ) , if { h t,δi ∈ C
i | i = 1 , . . .
, k } is the classiﬁers produced by running A with budget t , then with probability at least 1 − δ, er( h t,δ i )
− ν i ≤ ε, ∀ i = 1 , . . .
, k .
In the following, we take the passive learning (PL), i.e., random sampling, as a trivial case of active learning, and use the notations Λ AL , Λ PL and ˜Λ AL , ˜Λ PL to distinguish the label complexity of them, respectively.
We hide the superscript when the context is clear.
3.2 Translating the Label Complexity of Single Model to Multiple Models
Denote by Λ i the label complexity of the i -th target model C i .
Trivially, the AL label complexity for multiple models has ˜Λ AL ≤ (cid:80) i Λ ALi (applying the AL algorithm A on each of the target model i to get the result).
For the passive learning, however, its label complexity for multiple models has ˜Λ PL ≤ max i Λ PLi .
Because the data is randomly sampled, if t ≥ max i Λ PLi ( ε, δ, D XY ) examples are queried, according to the deﬁnition of label complexity, passive learning will output an ε -good classiﬁer with probability at least 1 − δ for each target model.
Such result implicitly indicates that the AL can hardly outperform PL under this setting.
To break this curse, the following theorem is provided to show that, we can expect a much better ˜Λ AL for AL under the realizable case (i.e., h ∗ is in the combined hypothesis space ˜ T ). It generally says that, given arbitrary set of target models T
= {C i | i = 1 , 2 , . . .
, k } , if a learning method has label complexity Λ AL on the combined hypothesis space ˜ T , it also has the ability to output good classiﬁers for each C i , i.e., after querying at most t examples to output ε -good classiﬁers with probability at least 1 − δ for each C i .
Theorem 1.
Considering binary classiﬁcation tasks and realizable case, given target models T = {C i | i = 1 , 2 , . . .
, k } , assume that active learning algorithm
A achieves label complexity Λ on ˜ T .
Then, there exists an active learning algorithm A (cid:48) which achieves the label complexity ˜Λ such that ˜Λ ( ε, δ, D XY , T )
=
Λ ( ε/ 2 , δ, D XY ) .
Proof.
Deﬁne an algorithm A (cid:48) that can output the required classiﬁer ˆ h i ∈
C i , ∀ i = 1 , ..., k as follows.
First, run the algorithm A on ( ˜ T , D XY ) to query t ≥ Λ( ε/ 2 , δ, D XY ) labels and output a classiﬁer h A . According to the deﬁnition, d ( h A , h ∗ ) is bounded by ε/ 2 with probability at least 1 − δ .
Then, for any C i , output the classiﬁer
ˆ h i ∈ C i such that
ˆ h
i = arg
min h i ∈C i d ( h i , h A ) .
Next, we prove that er(ˆ h i ) − ν i ≤ ε holds with probability at least 1 − δ .
To bound er(ˆ h i ) , it is equivalent to bound d (ˆ h
i , h ∗ ) by Deﬁnition 1.
Let h ∗ i =
arg min h i ∈C i er( h i ) .
It is easy to verify that, d ( · ) satisﬁes triangle inequality in binary classiﬁcation problems, i.e., d ( [Equation] ≤ d (ˆ h [Equation]
h
[Equation]
+ d (
[Equation]
A [Equation] [Equation] For the d (ˆ h i , h A ) , we know that ˆ h
i = arg
min h i ∈C i d ( h i , h A ) , which means
[Equation]
Again, by the triangle inequality, we have
[Equation]
Combining Eq.
(1)(2)(3), we have
[Equation]
Since d ( h A , h ∗ ) is bounded by ε/ 2 with probability at least 1 − δ , we can get er(ˆ h i ) − ν i ≤ ε holds with probability at least 1 − δ .
Theorem 1 says that if we can ﬁnd an active learning method to obtain a classiﬁer h A ∈ ˜ T such that er( h A ) ≤ ε/ 2 with probability 1 − δ , then we can obtain ε -good classiﬁer
ˆ h i with probability at least 1 − δ for C i , ∀ i = 1 , ..., k , where ˆ h i =
arg min h i ∈C i d ( h i , h A ) .
This result provides a general guarantee that if an algorithm can achieve a label complexity on the combined hypothesis space of different models, it also can achieve a bounded label complexity on these models (i.e., the label complexity for multiple models).
It can be served as a baseline of AL under multi-models setting.
Potential Improvements of Active over Passive
Although Theorem 1 bridges the traditional label complexity to that of multiple models setting, it does not reveal the improvement of active over passive learning.
Next, we will show the potential of AL under this setting in the realizable case.
According to the theoretical analysis of the passive learning algorithm empirical risk minimization (ERM) [16] for single hypothesis space C with VC dimension d [34], we know that
Lemma 1.
Considering the binary classiﬁcation, given the hypothesis space C with VC dimension d.
The passive learning algorithm ERM achieves a label complexity Λ PL such that, for any D XY in the realizable case, ∀ ε, δ ∈ (0 , 1) ,
[Equation] [Equation]
For the agnostic case, ERM achieves a label complexity Λ PL such that,
[Equation] [Equation]
where ν
= min h ∈C er ( h ) , and θ ( · ) is the disagreement coefﬁcient which is formally deﬁned as
Deﬁnition 4.
Disagreement Coefﬁcient: For any r 0 ≥ 0 and classiﬁer h , deﬁne the disagreement coefﬁcient of h with respect to C on D XY as
[Equation]
Where ∨ is the max operator.
For a set of hypotheses H , DIS( H ) = { x ∈ X | ∃ h, h (cid:48) ∈
H , s.t. h
( x ) (cid:54) =
h (cid:48)
( x ) } , and B H ( h, r ) = { g ∈ H | d ( h, g ) ≤ r } .
This value roughly characterizes the behavior of the size of disagreement region DIS( · ) as a function of the hypotheses within a radius r around the classiﬁer h .
As aforementioned, passive learning has the label complexity for multiple models ˜Λ PL ≤ max i Λ PLi .
We note that the target concept h ∗ will usually not be included by every hypothesis space C i , thus its label complexity ˜Λ PL will usually be the agnostic form in Lemma 1 under the realizable case.
To show the potential of AL under this setting, we take the CAL method [9] as an example, which is a representative and well-analyzed approach in the active learning literature [16].
CAL queries the examples from the disagreement region of a set of consistent hypotheses, i.e., DIS( V ) = { x ∈
X | ∃ h, h (cid:48) ∈
V s.t.
h ( x ) (cid:54) =
h (cid:48)
( x ) } , where V = { h ∈ C | h ( x )
= h ∗ ( x ) , ∀ x ∈ L} .
It achieves the label complexity
O ( θ ( ε ) log(1 /ε ) log( θ ( ε ) log(1 /ε ))) for the realizable case.
According to
Theorem 1, it will have the following label complexity for the multiple target models
Corollary 1.
Given target models T =
{C i | i = 1 , 2 , . . .
, k } .
Suppose ˜ T has VC dimension d < ∞ .
CAL achieves a label complexity ˜Λ AL for multiple target models such that, for D XY in the realizable case, for any ∀ ε, δ ∈ (0 , 1) ,
[Equation]
Proof.
By combining the label complexity of CAL for single model from [16] and Theorem 1, we can get the result.
To reveal the potential improvement, note that the label complexity for passive learning heavily depends on the property of the worst hypothesis space, i.e., the value of max i min h ∈C i er ( h ) .
Assume that max i min h ∈C i er ( h ) > ε .
Then according to Lemma 1 and Corollary 1, the label complexity of passive learning for multiple target models ˜Λ PL ( ε, δ, D XY , T ) is Ω(2 /ε ) .
On the other side, CAL has a label complexity Ω(Log(2 /ε )) , which implies the potential of the improvement of active learning under this setting.
We leave the guarantee of strict improvement of AL under this setting an interesting future work.
Next we further study the agnostic case (i.e., h ∗ / ∈ ˜ T ).
An Agnostic Disagreement-based AL method for Multiple Models
Deﬁne the set V
i for each C
i as { h ∈ C
i | h
( x ) =
h ∗ ( x ) , ∀ x ∈ L} , we propose to query the examples located in the joint disagreement regions for all C i , ∀ i = 1 , 2 , . . .
, k , i.e., DIS( V 1 ) ∩ DIS( V 2 ) ∩ . . . DIS( V k ) .
Intuitively, we know that V i must be a subset of V , if such data exists, we can expect it has higher potential to reduce V .
This statement can be simply implied by the Bayesian formula.
Proposition 1.
Considering binary classiﬁcation problem. Given hypothesis space C . Let V + ( x ) =
{ h ∈ V | h ( x ) = +1 } , V − ( x ) = { h ∈ V | h ( x ) = − 1 } , where V = { h ∈ C | h ( x ) = h ∗ ( x ) , ∀ x ∈
L} .
Denote λ ( x ) =
| V + ( x ) | | V − ( x ) | , where | · | is the number of elements of a set.
The ideal case is to query the x which has λ ( x ) = 1 .
Given any sequences of subset V 1 , V 2 , . . .
, V k randomly sampled from V , deﬁne the event E x that data x falls into DIS( V 1 )
∩ DIS( V 2 ) · · · ∩ DIS( V k ) .
According to the Bayesian formula, we have
[Equation]
Initialize: hyperparameter q , constants σ i ; m ← 0 , ˆ V
i ← C i , ∀ i = 1 , . . .
, k .
Output: Any h ∈
ˆ V i , ∀ i = 1 , . . .
, k .
1: while Labeling budget is not run out do 2: m ← m +3:
Request an unlabeled data x m 4: if (cid:80) i I [ x m ∈ DIS( ˆ V i )] ≥ q then 5:
Query: L ← L ∪ { ( x m , h ∗ ( x m )) } 6: end if 7: if log 2 m ∈ N then 8: ˆ V i ← { h ∈
ˆ V i | er L ( h )
− min g ∈
ˆ V
i er L ( g ) ≤ σ i } , ∀ i = 1 , . . .
, k .
9: end if 10: end while
Initialize: labeled set L , unlabeled set U , hyperparameters ˆ σ i , ˆ V
i ← C i , ∀ i = 1 , . . .
, k .
Output: { ˆ h i | i = 1 , . . . , k } .
1: while Labeling budget is not run out do 2:
x ∗ = arg max x ∈U (cid:80) i I [ x ∈ DIS( ˆ V i )] 3:
Query x ∗ from the oralce:
L ← L ∪ { ( x ∗ , h ∗ ( x ∗ ))) } 4:
U ← U \ { x ∗ } 5: for i = 1 , . . .
, k do 6: ˆ h i ← min g ∈
ˆ V
i er L ( g ) 7: ˆ V i ← { h ∈
ˆ V i | (er L ( h ) − ˆ h i ) ≤ ˆ σ i } 8: end for 9: end while Proof. Since each V i is randomly sampled from V , P ( E x ) will reach its maximum value when λ ( x ) = 1 , thus we have P ( E x | λ ( x ) = 1) / P ( E x ) ≥ 1 , which leads to the conclusion. Following this principle, we would like to query the examples located in the joint disagreement regions of C i , ∀ i = 1 , 2 , . . .
, k .
However, since we have multiple target models, the target concept h ∗ might not be included by every C i in practice, which turns the learning problem to the agnostic setting.
Inspired by the RobustCAL method [2], which is a disagreement-based AL algorithm for the agnostic setting, we propose DIAM (i.e., DIsagreement-based AL for Multi-models) query strategy for the multiple target models problem.
Note that we deﬁne a new form of V i as ˆ V i to tackle the noisy setting, i.e., ˆ V i
= { h ∈ C
i | er L ( h )
− min g ∈C i er L ( g ) ≤ σ i } , where σ i is a constant.
To simplify the theoretical analysis, we ﬁrst propose an online version of DIAM, then we deﬁne the DIAM method for the pool-based setting and empirically validate its effectiveness.
They are summarized at Algorithm 1 and 2, respectively.
The hyperparameter q controls the conservativeness of the algorithm.
With a larger q , it will reject more less informative unlabeled data in the online setting.
Now let us analyze the DIAM method.
Since we are considering the agnostic setting, it is necessary to model the noise.
Here we employ the commonly used Tsybakov noise condition [33].
Condition 1.
[33, Tsybakov noise]
For some a ∈ [1 , ∞ ) and α ∈ [0 , 1] , assume that f (cid:63) achieves inf h ∈C er( h ) , for every h ∈ C ,
[Equation]
We assume that there exists pair of a i and α i for each target model C i .
Considering a conservative situation that the hyperparameter q = 1 , by further taking the constants σ i in DIAM-online algorithm as the same form in the RobustCAL method [16], which relates to the properties of the noise, hypothesis space, and disagreement coefﬁcient, we can have the following result.
The proof is deferred to the appendix.
Theorem 2.
Considering agnostic setting and binary classiﬁcation tasks.
Given a set of target models T =
{C i | i = 1 , 2 , . . .
, k } , in which each C i has VC dimensions
d i < ∞ and meet Condition 1.
Let h ∗ i =
arg min h i ∈C i er( h i ) .
For any ε, δ ∈ (0 , 1) , if q = 1 , DIAM-online algorithm achieves a label complexity ˜Λ ( ε, δ, D XY , T ) for multiple target models such that, for a i and α i as in Condition 1, for any D XY , ˜Λ ( ε, δ, D XY , T ) is no larger than
[Equation]
and no larger than,
[Equation]
Theorem 2 provides an upper bound of the label complexity of the DIAM-online method when q = 1 .
It considers a general situation with arbitrary target models and data distributions, even the unlabeled data will never fall into the joint disagreement regions.
However, one may be more interested in the situation that if we can always query the x such that (cid:80) i I [ x ∈ DIS(
ˆ V i )]
=
k .
Next, we prove that if such ideal situation exists, DIAM-online will achieve a better label complexity than applying CAL on the multiple target models setting even under the realizable setting.
Theorem 3.
Considering binary classiﬁcation tasks and realizable case.
Given a set of target models T = {C i | i = 1 , 2 , . . .
, k } , in which each C i has VC dimensions
d i < ∞ and meet Condition 1, and ˜ T with VC dimension d < ∞ . Assume that, if a data point falls into the disagreement region of any C i , it also falls into the disagreement regions of the others {C j | j (cid:54)
=
i, j = 1 , 2 , . . .
, k } .
Assume the m -th target model achieves the highest label complexity.
Let h ∗ m = arg min h m ∈C m er( h m ) and ν m =
er( h ∗ m ) .
For any δ ∈ (0 , 1) , ε ∈ (0 , 1 /e ) , h ∗ ∈ ˜ T .
If ν m ≤ ln22 ε , DIAM-online achieves a better upper bound of label complexity for multiple target models than that of applying CAL method on ˜ T .
The key of the proof is comparing the disagreement coefﬁcients deﬁned on different functions and hypothesis spaces, i.e., θ C m h ∗ m and θ ˜ T h ∗ .
We defer the proof to the appendix.
Although Theoremholds with somewhat strict conditions, we note that Theorem 1 only works in the realizable case, while DIAM does not require this condition.
Next, we discuss how to implement DIAM in the real applications for deep models.
It is generally believed that ﬁnding the disagreed pair of classiﬁers from a set of hypotheses for a given x is non-trivial.
Most existing methods randomly sample functions from the hypothesis space for validation, or turn to select the data close to the decision boundary (e.g., uncertainty), which can be expensive or inaccurate.
This problem becomes more prohibitive to the deep models.
To efﬁciently estimate the disagreement regions for the neural networks, we propose to exploit the predictions of the unlabeled data during the later epochs in the training phase, typically after the network converging.
Recall the deﬁnition of disagreement region DIS( ˆ V i ) , we should ﬁrstly ﬁnd the hypotheses that are basically consistent with the labeled data, then validate whether there exists a pair of hypotheses that disagree on the given unlabeled data.
For the ﬁrst characteristic, the models on the later epochs, i.e., has smaller training errors, can represent the well-learned hypotheses.
For the second characteristic, if there exists models i, j from the later epochs such that the model trained at epoch i has inconsistent prediction with the model trained at epoch j on the unlabeled data x , we can say that the example x falls into DIS( ˆ V i ) .
We also note that the query batch size in deep learning is usually large, to avoid overmuch information redundancy, we heuristically sort the unlabeled data according to the active selection scores, and randomly select a batch of examples from the top-rated candidates. More concretely, according to the training loss curve, we empirically take the models in the latter half of the training epochs as the well-performed hypotheses set, and estimate the disagreement region with them. Since there may be multiple examples have the same value of (cid:80) i I [ x ∈ DIS( ˆ V i )] , we further calculate the vote entropy [10] of the well-performed hypotheses on the speciﬁc data, and take it as the secondary sort key in the data selection phase. To impose the diversity on data selection, we heuristically keep the top-rated unlabeled data with 5 times the size of the batch size, and randomly sample 20% from it for querying.
By applying the above heuristics to the deep models, DIAM method is quite efﬁcient.
It evaluates the unlabeled data with the models trained with later epochs, which roughly takes the size of the wellperformed hypotheses set times of that of the entropy method to make the data selection.
However, we also note that the DIAM method has some limitations.
It only considers the informativeness of the unlabeled data, which may be less effective for the batch-mode selection.
We leave it to the future work.
For the potential negative social impact, DIAM may reduce the cost of training multiple malicious machine learning models.
Nevertheless, we believe the positive contribution is more signiﬁcant.
[Figure]
Experiment 6.1 Empirical Settings
To construct the multiple target models scenario, we introduce the results of a recent NAS method
OFA [6], which tries to efﬁciently search model architectures for different devices by training only one super-net.
They report the searched effective architectures that meet the hardware constraints of various machines on the GitHub 2 , which is well suited to our problem setting.
Speciﬁcally, we take 12 specialized model architectures with different prediction accuracies and speeds that target on Samsung S7 Edge, Samsung Note8 and Samsung Note10 as our target models.
They are pruned from a MobileNetV3 (which is the super-net), but have very different prediction time and accuracies. Their
Multiply-Accumulate Operations (MACs) range from 66M to 237M, which denote the diversity of the architectures.
The model speciﬁcations are listed in the appendix.
We compare the following query strategies in our experiments.
• DIAM:
The proposed method of this paper, which queries the data located in the joint disagreement regions of multiple target models.
• CAL [9]: Query the data falls into the disagreement region of any target models.
It has a bounded label complexity for the multiple target models setting according to Theorem 1.
• Entropy [20]: Query the data with the highest prediction entropies.
We take the mean entropies calculated by all target models to support the novel problem setting.
• Least Conﬁdence [29]: Query the data with the least prediction conﬁdence.
We take the mean values calculated by all target models to support the novel problem setting.
• Margin [26]: Query the data with the minimum prediction margin.
We take the mean margin values calculated by all target models to support the novel problem setting.
• Coreset [27]: Query the most representative data.
The distance is calculated by the features extracted by a pretrained MobileNetV3, which is the super-net in OFA [6]. • Random: Query data randomly.
Note that this is a highly competitive baseline.
Since Optical Character Recognition (OCR) is one of the representative machine learning systems that are required to be deployed on diverse devices, two commonly used hand-writing characters classiﬁcation benchmarks are employed in our experiments, i.e., the MNIST [19] and KuzushijiMNIST [8] datasets.
They are under the CC BY-SA 3.0 and CC BY-SA 4.0 licenses, respectively.
Here we consider the prevalent pool-based active learning setting.
Speciﬁcally, we randomly take 3 , 000 training data as our initially labeled data, and the rest as the unlabeled pool.
At each iteration, the compared sampling methods will select 1 , 500 unlabeled examples for querying, then re-train the the [Table]
models. The mean and standard deviation of the accuracies of multiple target models are reported.
Note that more results can be found in the appendix.
For the model training, We mainly follow the training conﬁgs of OFA.
Speciﬁcally, the hyperparameters are set by the default values in the project.
For example, the learning rate is set by 7 . 5 e − 3 , batch size is 128 , SGD optimizer is employed with momentum 0 .
Since the initially labeled data is limited, a small number of training epochs is taken to avoid over-ﬁtting.
Speciﬁcally, we employ the pretrained weights on the image-net dataset for initialization, then ﬁnetune 20 epochs on the labeled data.
6.2 Results
We report the trend of mean accuracy of multiple target models with the number of queries increasing in Fig.
The error bars indicate the standard deviation of the performances of multiple target models.
First of all, the high deviation of the performances of the initial point shows the diversity of the target models, which symbolizes the practicability and difﬁculty of the experimental settings.
It is conceivable that different target models will have various preferences of training data due to the diverse architectures.
Under this challenging setting, it can be observed from the ﬁgure that our method can signiﬁcantly outperform the traditional active and passive learning methods.
It shows a great potential of improvements over the random sampling, which is a very competitive baseline in this novel setting.
This result sufﬁciently reveals the effectiveness of DIAM and the necessity of designing active query method under this practical setting.
The uncertainty-based methods, i.e., entropy, least conﬁdence and margin, achieve comparable performances with random sampling.
These results meet our expectation.
Because traditional AL methods are usually model-dependent, i.e., the data queried by one model may be less effective for training another model.
By taking the mean uncertainty scores of diverse target models, the data selection may tend to be non-informative.
The coreset method is less stable than random.
We note that coreset is still a model-based selection method in deep learning.
Because the features of the data will be optimized along with the training procedures.
Thus it may also suffer from the model dependence problem.
6.3 Study on Different Numbers of Target Models
We further explore the inﬂuence of the number of target models to the data selection methods.
Due to the space limitation, we report the mean of the learning curves and the mean of standard deviations in Table 1, and defer the whole learning curves to the appendix.
The results show that our method can consistently outperform the other compared methods, which demonstrate its robustness to the number of models.
This property also denotes that the DIAM method has the potential to tackle more challenging situations, i.e., improving sufﬁcient numbers of target models simultaneously.
It is essential to the machine learning systems which have a wide range of applications.
The performances of the other compared methods have similar trends with more target models setting.
It again veriﬁes that the traditional AL methods are usually model-dependent, and emphasizes the necessity of designing novel selection approaches under this practical setting.
Conclusion
In this paper, we propose to study active learning in a novel setting, where the task is to select and label the most useful examples that are beneﬁcial to multiple target models.
We ﬁrstly analyze the label complexity of active and passive learning to reveal the potential improvement of AL under this novel setting.
Based on this insight, we further propose an active selection criterion DIAM that prefers the data located in the joint disagreement regions of different target models.
Empirical studies on the OCR benchmarks, which is one of the representative applications that are required to accommodate different devices, show the effectiveness of the proposed method.
In the future, we will tackle more complex and important learning tasks (e.g., face recognition, object detection), and design effective query strategies which incorporate both informativeness and representativeness under the multiple target models setting. Acknowledgments and Disclosure of Funding This research was supported by the National Key R&D Program of China (2020AAA0107000), NSFC (62222605, 62076128), and Natural Science Foundation of Jiangsu Province of China (BK20211517, BK2022050029).
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope? [Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results? [Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] Some of them are deferred to the appendix. If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] cf. appendix 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes] (b) Did you mention the license of the assets?
[Yes]
[-In Page 
-](c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating? [Yes] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content? [Yes] 5. If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
Appendix
In the appendix, we will ﬁrst prove of the theorems in the paper, then we introduce more details and results of the experiments, which include more particular empirical settings, running time, computational resources, the signiﬁcance of the performance comparisons and extra experimental results. The notations in the following contents are consistent with the paper. A Proofs of the Theorems in the Paper A.1 Proof of Theorem The proof of Theorem 2 is based on the results of the RobustCAL method [14].
Thus we ﬁrst introduce the following results of the RobustCAL method for the single model. Lemma ii. [16, Theorem 5.4] Considering binary classiﬁcation problem. Suppose the hypothesis space C has VC dimension d. For any δ ∈ (0 , 1) , RobustCAL achieves a label complexity Λ such that, for any D XY , for a and α as in Condition 1, ∀ ε ∈ (0 , 1) ,
[Equation]
(cid:19)(cid:19)
Log
[Equation]
[Equation]
and furthermore,
[Equation]
With the above techniques, we begin to prove Theorem 2.
Proof.
The hyperparameter q = 1 in the DIAM-online method means querying the data falls into any disagreement regions of target models.
In other words, the data queried for a speciﬁc target model must be queried by the DIAM-online algorithm.
Denote the number of data queried for i -th target model by t i such that it is sufﬁcient to output an ε -good classiﬁer with probability at least 1 − δ for
C i .
By a union bound, we know that DIAM-online queries t ≥ (cid:80)
i t i examples sufﬁcient to output ε -good classiﬁers for each target model with probability at least 1 − δ .
Recall that DIAM-online method takes the same form of σ i with the RobustCAL method (step 8 in the Algorithm 1 in the paper), thus it is equivalent to applying RobustCAL on each target model.
By incorporating Lemma ii and a union bound, we can get the conclusion.
A.2 Proof of Theorem
To compare the upper bound of the label complexity between DIAM and CAL under the multiple target models setting, we ﬁrst note that, if the ideal situation described in the Theorem 3 exists, then
DIAM-online will achieve the label complexity ˜Λ = max i Λ i .
Because the queried data is useful for all hypothesis spaces C i , ∀ i = 1 , . .
k .
Thus, if t > max i Λ i examples are labeled, DIAM-online will output the desired classiﬁers for each C i .
Assume the m -th target model achieves the highest label complexity, by Lemma ii, we know that DIAM achieves the label complexity for multiple models with ε ∈ (0 , 1 /e ) in binary classiﬁcation problem, such that
[Equation]
For the CAL method, according to the Corollary 1 in the paper, we know that it achieves the label complexity for multiple models in binary classiﬁcation problems such that
[Equation]
To compare the right side of Eq.
(11) and (12), one challenge is to compare the disagreement coefﬁcients deﬁned on different functions and hypothesis spaces, i.e., θ C m h ∗ m and θ ˜ T
h ∗ .
To this end, we ﬁrst introduce the following properties of the disagreement coefﬁcient θ ( · ) , which are analyzed in [16].
Lemma iii.
[16, Theorem 7.1] Given h ∈ C , θ h ( r ) is nonincreasing w.r.t.
r ∈ [0 , + ∞ ) .
Lemma iv.
[16, Theorem 7.8] Let C 1 and C 2 be sets of classiﬁers such that C =
C 1 ∪ C 2 .
For all ε > 0 , let θ C h ( ε ) , θ C 1 h ( ε ) , and θ C 2 h ( ε ) denote the disagreement coefﬁcients of arbitrary h (not necessarily in C ) with respect to C , C 1 , C 2 , respectively, under D X .
Then ∀ ε > 0 ,
[Equation]
Lemma v. [16, Corollary 7.2] Let ε ∈ (0 , ∞ ) and a ∈ (1 , ∞ ) .
Then θ C h ( ε/a ) ≤ aθ C h ( ε ) and θ C h ( ε ) /a ≤ θ C h ( aε ) .
Lemma vi. ∀
h ∈ C , given a hypothesis g (not necessarily in C ), if d ( h, g ) ≤ γ , for any γ > 0 .
{+Then ∀ ε > 
+}
we have
[Equation]
Proof.
(Lemma vi) d ( h, g ) ≤ γ implies that ∀ r > 0 , B C ( g, r + γ ) ⊇ B C ( h, r ) and B C ( h, r + γ ) ⊇
B C ( g, r ) .
Then
[Equation]
[Equation]
Lemma iv and Lemma vi bridge the disagreement coefﬁcients deﬁned on different classiﬁers and hypothesis spaces.
With the above results, we can now begin to prove Theorem 3.
Proof.
(Theorem 3)
To compare the right side of Eq.
(11) and (12), we turn to compare each corresponding term, i.e., θ C m h ∗ m ( ν m + ε ) and θ ˜ T
{+h ∗ ( ε/ 2) ; ν 2 m ε 
+}
{++ ln (cid:0) 
+}
ε (cid:1) and ln (cid:0) 2 ε (cid:1) .
For the ﬁrst group of terms (i.e., disagreement coefﬁcients), by Lemma vi, we have
[Equation]
Since ν m ≤ ln22 ε < 12 ε , we can get
[Equation]
According to Lemma v, we can get
[Equation]
Combining Lemma iii and Lemma iv, we have
[Equation]
For the {+second group of terms (i.e., ν 2 m ε 
 +ln (cid:0) 
 ε (cid:1) and ln (cid:0) 2 ε (cid:1) ), according to the assumptions ν m ≤ ln22 ε , we have [Equation]+}
To further compare d m ln( θ C m h ∗ m ( ν m + ε )) and
d ln( θ ˜ T
h ∗ ( ε/ 2)) , we know that C m is a subset of ˜ T , thus d m ≤ d .
By combining Eq.
(18), we can directly have
[Equation]
and
[Equation]
Combining the above results
[Equation]
we have the following deductions
[Equation]
which lead to the conclusion. [Equation] # by Eq. (18)(19)(21) ≤ θ ˜ T
h ∗ ( ε/ 2) ln(2 /ε )
(cid:18) d ln( θ ˜ T h ∗ ( ε/ 2))
+ ln (cid:18)
ln(2 /ε )
δ (cid:19)(cid:19) # by Lemma iv
B Experimental Details and Additional Results B.1 Empirical Settings
Speciﬁcations of Multiple Target Models We take the following 12 speciﬁcations from a recent
NAS work OFA [6] as our target models:
• s7edge_lat@88ms_top1@76.3_ﬁnetune@ • s7edge_lat@58ms_top1@74.7_ﬁnetune@ • s7edge_lat@41ms_top1@73.1_ﬁnetune@ • s7edge_lat@29ms_top1@70.5_ﬁnetune@ • note8_lat@65ms_top1@76.1_ﬁnetune@ • note8_lat@49ms_top1@74.9_ﬁnetune@ • note8_lat@31ms_top1@72.8_ﬁnetune@ • note8_lat@22ms_top1@70.4_ﬁnetune@ • note10_lat@22ms_top1@76.6_ﬁnetune@ • note10_lat@16ms_top1@75.5_ﬁnetune@ • note10_lat@11ms_top1@73.6_ﬁnetune@ • note10_lat@8ms_top1@71.4_ﬁnetune@
In the experiment with different number of target models (cf.
Sec. 6.3), we empirically take the ﬁrst 2 , 4 , 6 , 8 speciﬁcations from the above model conﬁguration list as the target models set.
B.2 Computational Resources and Running Time
We run our experiments on 3 cloud servers, each of them has 128GB memory and 4 RTXgraphic cards.
The CPU is Intel Xeon Silver 4110 @ 2.10GHz with 8 cores.
Since we run each of the compared method on one graphic card, respectively, we report the resource occupation of each individual process.
The minimum requirement to train and validate the model is 10GB memory and 11GB CUDA memory with 128 batch size, respectively.
If running the coreset query method, 10GB extra memory is needed to store the distance matrix.
For the running time, since there are multiple target models with varying complexities, they have different training and inference speed.
The real time (calculated by gettimeofday() function) of sequentially training 12 target models on the initially labeled dataset, i.e., 3 , 000 examples, is
[Table]
Win
Win
Win
Win
Win
Win
Win
Win 10/0/
W./T./L.
6/0/ 6/0/ 6/0/ 6/0/ 6/0/ 6/0/ 6/0/ 6/0/ 6/0/ 6/0/ 60/0/ 00:26:42 (hh:mm:ss).
For the data selection phase, i.e., select 1 , 500 examples from 40 , 000 unlabeled data, the real time of different methods are reported as following: random takes 1.73 seconds, least conﬁdence takes 00:08:29 (need to evaluate the unlabeled data with each of the target model), margin takes 00:08:26, entropy takes 00:08:11, coreset takes 00:09:37.
For the proposed DIAM method and
CAL method, they need to evaluate the unlabeled data with the models trained with later epochs, which roughly take the size of the well-performed hypotheses set times of that of the entropy method to make the data selection.
B.3 Additional Experimental Results
Signiﬁcance of Performance Comparison We further report the signiﬁcance results of the performance comparisons (cf.
Figure 1 in the paper).
Speciﬁcally, the win/tie/lose results of the performances of 12 target models after each query based on paired t-test at 0.05 signiﬁcance level are reported in Table ii.
The results show that our method can usually outperform the other compared methods signiﬁcantly, which demonstrate that DIAM can improve all the target models simultaneously, but not paying too much attention to the speciﬁc models.
This property is essential to the multiple target models applications.
Because the target models are usually of equal importance, even though they have different prediction accuracies.
Learning Curves of the Study on Different Numbers of Target Models We plot the entire learning curves of the compared methods with different numbers of target models in Fig.
ii (cf.
Sec. 6.3).
It can be observed that the proposed DIAM method can surpass the traditional active and passive learning methods under different numbers of target models in most cases These results reveal that our method is robust to the number of target models , i.e., the data in the joint disagreement region is beneﬁcial to all the target models. [Figure]
cc
[Figure]
n
[Figure]
cc u
[Figure]
c y [Figure] [Figure] m
[Figure]
n
[Figure]
cc u [Figure] c y [Figure]