Teaching an Active Learner with Contrastive Examples

1 Introduction . Active learning characterizes a natural learning paradigm where a learner actively engages in the learning process , and has demonstrated great success in both the educational domain [ Prince , 2004 ] and the machine learning literature [ Settles , 2012 ] . In the context of machine learning , active learning has traditionally been studied as the active instance labeling problem , where the goal of an active learner is to learn a concept by selectively querying the labels of a sequence of data points [ Kosaraju et al. , 1999 ; Dasgupta , 2004 ; Nowak , 2008 ; Balcan et al. , 2006 ; Gonen et al. , 2013 ; Hanneke and Yang , 2014 ; Yan and Zhang , 2017 ] . Recently , active learning has been investigated under richer interaction protocols , including learning from feature feedback [ Poulis and Dasgupta , 2017 ; Dasgupta and Sabato , 2020 ] or from demonstrations [ Chernova and Thomaz , 2014 ; Silver et al. , 2012 ] . Typically , these works assume that the rich feedback obtained from the teacher is adversarial [ Dasgupta and Sabato , 2020 ] or stochastic [ Poulis and Dasgupta , 2017 ] . While these results have shown promising improvements in sample complexity by leveraging more expressive feedback , the assumptions of an adversarial or stochastic teacher could fall short in characterizing many real-world interactive learning systems ( e.g . automated tutoring or multi-agent learning systems ) where the teacher cooperates with the learner to achieve the learning objective [ Zilles et al. , 2011 ; Zhu et al. , 2018 ] . Motivated by real-world cooperative learning and teaching scenarios [ Chen et al. , 2018a ; Teso and Kersting , 2019 ] , we consider an optimistic yet natural interaction protocol between an active learner and a helpful teacher ( see Figure 1 ( c ) ) : At each round , the learner proposes a query asking for the label of an instance xq ; the teacher then provides the requested label { xq , yq } along with explanatory 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . information to guide the learning process . In this paper , we consider this information in the form of an additional contrastive example ( { xc , yc } ) where xc is picked from a set constrained by xq ( e.g. , dissimilar instances with the same label , or similar instances with opposite labels ) . In contrast to existing work in active learning with contrastive examples [ Abe et al. , 2006 ; Liang et al. , 2020 ; Ma et al. , 2021 ] or with comparison queries [ Jamieson and Nowak , 2011 ; Kane et al. , 2017 ] which focus on developing active learning algorithms , we assume that the learner ’ s querying and learning strategy is fixed and known to the teacher , and focus on designing a teaching algorithm that can provide an informative sequence of contrastive examples to the learner to speed up the learning process . Note that in the absence of the learner ’ s query , this problem reduces to the classical machine teaching problem [ Goldman and Kearns , 1995 ] . When teaching a ( passive ) version space learner [ Goldman and Kearns , 1995 ] , classical machine teaching can be viewed as a set cover problem , and the greedy algorithm is known to compute a cover that is within a logarithmic factor of the minimum cover [ Chvatal , 1979 ] . However , as we show in section 3.3 , an active learner actively trying to reduce the label cost could actually hurt the performance of a greedy teacher . Given an active version space learner specified by an acquisition function , we show that the design of an optimal teaching algorithm induces a challenging sequence optimization problem , where the teacher ’ s available choices of constrastive examples at a given round depend on the history of interactions . We study the greedy teacher that adaptively picks contrastive examples in response to the learner ’ s queries . Specifically , we introduce several natural problem-dependent parameters to characterize ( 1 ) the constraints imposed by the active learner and ( 2 ) the structure of the coverage function resulting from the hypothesis space . Based on these characterizations , we explore the theoretical properties for the greedy teacher , and demonstrate the power and limitations of the interactive teaching framework . Our main technical contributions are summarized are : I . We consider a novel interactive protocol for teaching an active learner , and derive a general performance guarantee for the greedy teacher , regardless of the choice of acquisition function by the active learner . Our result generalizes existing near-optimality guarantees for the greedy teacher under the classical teaching [ Goldman and Kearns , 1995 ] to the more challenging sequence optimization problem ( section 3 ) . II . We then show that , perhaps surprisingly , a greedy teacher may fail badly when interacting with an active learner compared to the optimal teacher , due to the constrained choices of contrastive examples imposed by the learner ’ s query ( section 3 ) . In fact , there exist pessimistic scenarios where an active learner alone is better off without the help of a greedy teacher ( section 4 ) . III . When interacting with a generalized binary search ( GBS ) learner [ Dasgupta , 2004 ; Nowak , 2008 ] , we provide a strong performance guarantee for the greedy teacher , and show that it can not perform arbitrarily badly ( section 4 ) . Our bound establishes a rigorous connection between the problem-dependent parameters introduced in section 3 to the classical k-neighborly condition and coherence parameter which are widely used in the active learning literature [ Nowak , 2008 , 2011 ; Mussmann and Liang , 2018 ] . IV . We empirically illustrate our bounds via two experiments . These examples illustrate the intuitive sequential teaching process and demonstrate the effectiveness of our teaching framework . 1.1 Related Work . Teaching a sequential learner Machine teaching has shown significant development in the past two decades in both theoretical and practical domains . In a quest to achieve rich teacher-learner interactions , various different models for machine teaching have been proposed under the sequential teaching setting ( e.g. , local preference-based model for version space learners [ Chen et al. , 2018b ] , models for gradient learners [ Liu et al. , 2017 , 2018 ; Kamalaruban et al. , 2019 ] , models inspired by control theory [ Zhu , 2018 ; Lessard et al. , 2019 ] , models for sequential tasks [ Cakmak and Lopes , 2012 ; Haug et al. , 2018 ; Tschiatschek et al. , 2019 ] , and models for human-centered applications that require adaptivity [ Singla et al. , 2013 ; Hunziker et al. , 2019 ] ) . Compared to our setting , in these works , the teacher is responsible for designing all of the learning data . Recently , Peltola et al . [ 2019 ] investigated the task of teaching sequential learners such as multi-armed bandits or active learners where the learners also pose query , but they formulate the problem as a variant of reward shaping , and allow the teacher to provide teaching examples that are inconsistent with the true data distribution . While our work falls into the category of teaching sequential learners , it could be viewed as a form of explanation-based machine teaching , and hence inherits different teaching constraints and objectives from the above settings . Explanation-based teaching and learning Explanation-based teaching captures a rich class of teaching protocols where labels are coupled with additional information ( such as highlighting regions or features on an image ) . Having an additional mode in the feedback allows the learner to perceive more intuitive explanations of the target hypothesis , and hence dramatically improves the learner ’ s ability to learn a new concept . Explanations were shown to be effective in helping a human student to improve classification performance by steering the student ’ s attention [ Grant and Spivey , 2003 ; Roads et al. , 2016 ; Chen et al. , 2018a ; Aodha et al. , 2018 ] ; likewise , they also play an important role in helping machine learners to efficiently learn some concept classes that would otherwise form intractable learning problems [ Poulis and Dasgupta , 2017 ; Dasgupta and Sabato , 2020 ] . Our work is motivated by the success of explanation-based learning , and consider contrastive examples as a simple yet effective form of explanations . In comparison with existing work on explanation-based teaching , our work has an additional twist featured by the interaction with an active learner . Submodular set and sequence optimization Our theoretical framework in section 3 is inspired by recent results on sequence submodular function maximization [ Zhang et al. , 2015 ; Tschiatschek et al. , 2017 ; Hunziker et al. , 2019 ] . In particular , Zhang et al . [ 2015 ] introduced the notion of string submodular functions , which , analogous to the classical notion of submodular set functions [ Krause and Golovin , 2014 ] , enjoy similar performance guarantees for maximization of deterministic sequence functions . Our setting has three key differences in that ( 1 ) we focus on a constrained planning problem where the set of available actions is dynamic ( i.e . depending on the learner ’ s query ) , ( 2 ) our objective function no longer exhibits diminishing returns properties and ( 3 ) we consider a min-cost coverage problem as opposed to the budgeted maximization problem . 2 Active Learning with a Teacher : Problem Formulation . In this section , we formally introduce the new problem of active learning with a helpful teacher along with the corresponding interaction protocol . Then , we formulate the problem of teaching an active learner as a constrained sequential optimization problem . Throughout the entire paper , we use X to denote the ground set of unlabeled instances . We consider a finite class of hypothesisH , where each hypothesis h ∈ H is a binary classifier with its output being either +1 or −1 ( i.e. , h ( x ) ∈ { +1 , −1 } ) . 2.1 Learner-Teacher Interaction Protocol . We study the problem of teaching an active version space learner with a teacher . In contrast to the standard active learning protocol , our framework not only allows the learner to ask questions but also enables the teacher to provide additional explanatory information to guide the learning process . Specifically , we model the explanatory information using a contrastive example picked from a set constrained by the learner ’ s query ( e.g. , dissimilar instances with the same label , or similar instances with opposite labels ) . The full interaction protocol is presented in Protocol 1 . At each round t , the learner asks a query xqt = q ( x c 1 : t−1 , x q 1 : t−1 , H q t−1 ) with query function q conditioning on the received Protocol 1 An interaction protocol between the teacher and the active learner 1 : An active learner with a query function q ; Initial version space Hq0 = H ; Constraint function ξ for the contrastive example ; Target hypothesis h ? . 2 : for t = 1 , 2 , 3 , . . . do 3 : learner asks a query xqt = q ( x c 1 : t−1 , x q 1 : t−1 , H q t−1 ) 4 : teacher provides the label yqt for the query x q t 5 : teacher provides a contrastive example ( xct , y c t ) from a constrained set ξ ( x q t ) . 6 : learner computes the updated version space Hqt . 7 : if Hqt = { h ? } then teaching process terminates . contrastive examples ( xc1 : t−1 ) and version space H q t−1 . Then the teacher returns the ground-truth label yqt for x q t and also provides a contrastive example ( x c t , y c t ) . Under this setting , the version space at iteration t with teaching sequence xc1 : t = { xci } ti=1 , can be computed by ( we omitted y for clarity ) Hqt = H q ( xc1 : t ) = H \ ( t⋃ i=1 S ( xqi ) ∪ S ( x c i ) ) , ( 1 ) whereH is the initial version space of the active learner and S ( x ) = { h ∈ H|h ( x ) 6= h ? ( x ) } denotes the hypotheses covered by x ( i.e. , the hypotheses that predict different labels on x than h ? ) .

Summary: The paper proposes a teaching algorithm that uses contrastive examples to teach an active learner. Some performance guarantees are given for some special cases. Some empirical evidences are presented to support the results.