Teaching via Best-Case Counterexamples in the Learning-with-Equivalence-Queries Paradigm

1 Introduction . Learning-with-queries paradigm involves a learner who asks structured queries to a teacher in order to locate a target hypothesis . This paradigm has been extensively studied in machine learning and formal methods research , including automata learning [ 1 , 2 , 3 , 4 ] , model checking [ 5 ] , oracle-guided synthesis ( OGIS ) [ 6 ] , model learning [ 7 ] , among others . Classical literature involves different kinds of queries that a learner could ask [ 1 , 8 , 9 ] , such as membership queries ( i.e. , “ is the queried instance consistent with the target hypothesis ? ” ) and equivalence queries ( i.e. , “ is the queried hypothesis the target hypothesis ? ” ) . In this paper , we consider the learning-with-equivalence-queries ( LwEQ ) paradigm where the learner is only asking equivalence queries and the teacher responds to a query either “ yes '' or “ no '' along with a counterexample on which the current hypothesis disagrees with the target hypothesis . LwEQ paradigm captures a variety of important problem settings , such as counterexample-guided synthesis ( CEGIS ) [ 10 ] , data augmentation ( CEGAR ) [ 11 ] , and learning regular languages from counterexamples [ 12 ] . The focus of our work is to understand the query complexity for the LwEQ paradigm , i.e. , the number of equivalence queries needed by the learner for the exact identification of a target hypothesis [ 9 ] . The query complexity clearly depends on the learner model ( i.e. , the query function deciding the next hypothesis to query ) , as well as the informativeness of the counterexamples provided by the teacher . In the literature , the query complexity for the LwEQ paradigm has been extensively studied when the learner receives worst-case or random counterexamples [ 1 , 9 , 13 , 14 , 15 ] . To show worst-case bounds [ 1 , 13 ] , classical works have studied a teacher who responds with worst-case counterexamples 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . to maximize the number of equivalence queries asked by the learner . Recent work [ 14 ] has studied this query complexity when facing a more benign teacher who provides counterexamples selected at random from a known probability distribution . In particular , [ 14 ] proposed a learning algorithm ( i.e. , a query function for the learner ) that achieves O ( log n ) query complexity on the expected number of random counterexamples for any hypothesis class of size n. When contrasting this bound with worst-case bounds , it shows that random counterexamples could lead to exponential improvement in the query complexity when compared with worst-case counterexamples [ 13 , 14 ] . Along these lines , an important research question is to understand the query complexity where the counterexamples are picked by a more informed and helpful teacher , instead of random or worst-case counterexamples . In this paper , we consider a more powerful teaching setting , where the optimal teacher picks best-case counterexamples to steer the learner towards a target hypothesis . Our goal is to characterize the query complexity for this optimal teacher ( also , referred to as teaching complexity in the paper ) , for the LwEQ paradigm . This teaching complexity has been extensively studied for binary classification in the learning-from-samples ( LfS ) paradigm [ 16 , 17 ] and is termed as “ teaching dimension ” ( TD ) [ 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 ] . Beyond the LfS paradigm , various notions of teaching complexity have also been studied in other learning paradigms ( see Section 1.1 for further details ) . We introduce a new notion of teaching complexity for the LwEQ paradigm , namely “ LwEQ teaching dimension ” ( LwEQ-TD ) , capturing the number of queries needed by the learner when best-case counterexamples are provided by the optimal teacher . Our study of LwEQ-TD for some prominent hypothesis classes reveals the power of teaching via best-case counterexamples—we show a significant reduction in query complexity when compared to that for worst-case and random counterexamples . Furthermore , we establish several new connections of LwEQ-TD to the existing notions of TD in the LfS paradigm.1 Table 1 provides a summary of these different teaching settings and learning paradigms ; our main results and contributions are summarized below : I . We characterize the query complexity for the optimal teacher in the LwEQ paradigm , termed as learning-with-equivalence-queries teaching dimension ( LwEQ-TD ) . ( see Section 3 ) II . We study the query complexity in the LwEQ paradigm under different teaching settings : worst-case , random , and best-case , distinguished by the informativeness of counterexamples . We showcase the power of best-case counterexamples picked by the optimal teacher , in contrast to worst-case or random counterexamples , for different hypothesis classes , including Axes-aligned hyperplanes , Monotone monomials , and Orthogonal rectangles . ( see Section 4 ) III . We establish new connections between LwEQ-TD and LfS-TD by studying LwEQ-TD for different learner models based on the richness of their query functions . We show that LwEQ-TD is the same as wc-TD [ 18 ] , RTD [ 22 , 24 ] , and NCTD [ 27 ] for a hypothesis class when restricting query functions to specific families . In general , LwEQ-TD is weaker than LfS-TD , e.g. , LwEQ-TD is lower-bounded by local-PBTD [ 26 , 29 ] of the hypothesis class when the learner ’ s next query depends on the previous query . ( see Section 5 ) 1.1 Background and Related Work . Learning-with-queries paradigm and equivalence queries . Learning-with-queries paradigm was introduced in [ 1 ] which proposed L∗ algorithm for exact identification of DFAs ( deterministic finite automaton ) when the learner is allowed to ask membership queries and equivalence queries . Classical work has studied different types of queries ( subset , membership , equivalence , correction , among others ) [ 1 , 8 , 9 ] . Among these works , learning with membership queries has been explored in a variety of problem settings , such as PAC learning [ 16 ] , active learning [ 33 , 34 , 35 ] , and agnostic 1We will collectively refer to these different notions of TD in the LfS paradigm as LfS-TD . learning [ 36 ] . In the learning-with-equivalence-queries ( LwEQ ) paradigm , the learner can ask only equivalence queries ; furthermore , in our work , we consider proper queries , i.e. , the queried hypothesis is within the hypothesis class ( see Section 6 for a discussion on improper queries ) . In this LwEQ paradigm , [ 13 ] studied query complexity when the teacher picks worst-case counterexamples for some key hypothesis classes ( e.g. , DFA , NFA , Context-free Grammars ) , and showed an exponential lower bound . [ 14 ] studied random counterexamples , and proposed a learning algorithm , namely Max-Min , which achieves a substantially improved bound on query complexity . We will investigate the query complexity of Max-Min learner for the hypothesis class of Axes-aligned hyperplanes in Section 4 . Teaching in the LfS paradigm for binary classification . Algorithmic machine teaching , first introduced by [ 18 , 37 ] , studies the interaction between a teacher and a learner where the teacher ’ s goal is to find an optimal sequence of training samples to teach a target hypothesis . [ 18 ] introduced a measure of teaching complexity , named teaching dimension ( TD ) of the hypothesis class , in the learning-from-samples ( LfS ) paradigm . The classical notion of TD in [ 18 ] characterized the minimum number of samples ( i.e. , examples ) needed to teach a target hypothesis to a version-space learner who picks hypothesis within the version space arbitrarily ( in an adversarial way ) . In the past two decades , several new teaching settings have been studied , driven by the motivation to lower teaching complexity and to find settings for which TD has better connections with Vapnik–Chervonenkis dimension ( VCD ) [ 38 ] . In particular , several new teaching models and complexity measures have been proposed for both the batch teaching settings ( e.g. , worst-case [ 18 ] , recursive [ 22 , 24 ] , preference-based [ 25 ] , and non-clashing models [ 27 ] ) and the sequential settings ( e.g. , local preference-based model [ 26 , 29 ] ) . These teaching settings , in turn , lead to different notions of TD , that we collectively refer to as LfS-TD . In recent work , [ 29 ] has characterized these different notions of TD through a unified framework of modeling learners with preference/ranking functions . In Section 5 , we will build on this framework to model the learner ’ s query functions in the LwEQ paradigm through ranking functions , allowing us to connect LwEQ-TD with LfS-TD . Teaching in other learning settings . Within binary classification setting , teaching complexity results have been extended beyond version space learners , including models for gradient learners [ 39 , 40 ] , models inspired by control theory [ 41 , 42 ] , and models for human-centered applications [ 43 , 44 , 45 , 46 ] . Furthermore , a recent line of research has studied robust notions of teaching in settings where the teacher has limited information about the learner ’ s dynamics [ 47 , 48 , 49 ] . Given the importance of teacher-learner interactions in many real-world applications , teaching has also been studied in richer domains . In particular , teaching complexity has been investigated for imitation learning settings where the teacher provides demonstrations [ 50 , 51 , 52 , 53 , 54 ] , and for reinforcement learning settings where the teacher provides reward feedback [ 55 , 56 ] . We see these works as complementary to ours , and we refer the reader to see [ 28 ] for an overview . 2 Problem Setup . Teaching framework . Let X be a ground set of unlabeled instances and Y the set of labels . LetH be a finite class of hypotheses ; each element h ∈ H is a function h : X → Y . Here , we only consider boolean functions , and hence Y = { 0 , 1 } . Let Z ⊆ X × Y be the ground set of labeled examples . Each element z = ( xz , yz ) ∈ Z represents a labeled example where the label is given by the target hypothesis h∗ , i.e. , yz = h∗ ( xz ) . Furthermore , for any Z ⊆ Z , we define version space induced by the examples Z as the subset of hypothesesH ( Z ) ⊆ H that are consistent with the labels of all the examples , i.e. , H ( Z ) : = { h ∈ H | ∀z = ( xz , yz ) ∈ Z , h ( xz ) = yz } . ( 1 ) Equivalence queries . We consider the LwEQ paradigm where a learner seeks to identify a target hypothesis from the hypothesis class via equivalence queries . In an equivalence query , the learner asks if the current hypothesis , say h′ ∈ H , is equivalent to the target hypothesis h∗ or not . The teacher provides a response r where r is either “ yes ” if h′ ≡ h∗ or “ no ” along with a counterexample z : = ( xz , yz ) ∈ Z , such that h′ ( xz ) 6= yz . Learner model and query protocol . We consider a generic model of the learner that captures our assumptions about how the learner conjectures its hypothesis for equivalence queries based on the responses received from the teacher . A key aspect of this model is the learner ’ s query function ` over the hypotheses . Based on the information encoded in the inputs of this query function ( i.e. , the current hypothesis and the history of counterexamples ) , the learner will choose one hypothesis inH . In the beginning , the learner starts with an initial hypothesis h0 ∈ H , the history is Z0 = ∅ , and the version space is H0 = H. At a time step t ≥ 1 , the learner first picks the hypothesis ht as follows : ` ( Zt−1 , ht−1 ) −→ ht ∈ H ( Zt−1 ) , ( 2 ) where Zt−1 is the history of counterexamples seen up until time t andH ( Zt−1 ) is the corresponding version space . The learner then queries ht for equivalence to h∗ and receives a response rt . Then , the query protocol proceeds as follows : ( i ) if rt is “ yes ” , the learner has identified h∗ and stops ; ( ii ) otherwise rt is “ no ” along with a counterexample zt using which the learner updates Zt = Zt−1 ∪ { zt } , and continues . We summarize this query protocol in Algorithm 1 . Algorithm 1 : Query protocol between the learner and the teacher 1 Learner ’ s initial hypothesis is h0 ∈ H , history is Z0 = ∅ , and version space isH0 = H ; 2 for t = 1 , 2 , 3 , · · · do 3 learner picks ht ∈ H ( Zt−1 ) based on Zt−1 and ht−1 as per Eq . ( 2 ) ; 4 learner performs an equivalence query with ht ; 5 teacher provides a response rt that is either “ yes ” or ‘ no ” along with a counterexample zt ; 6 if rt is “ yes ” then learner has identified h∗ and stops ; 7 else learner updates Zt = Zt−1 ∪ { zt } ; We assume that both the learner and the teacher have full knowledge of X , Y , and H ; in addition , the teacher knows the target h∗ as well as the learner ’ s query function ` . In this work , we consider learner models which could be characterized by a specific query function ` as discussed above . These include well-known learners studied in the literature , such as a constant query learner ( denoted as ` const ) who picks the next hypothesis ht arbitrarily inH ( Zt−1 ) without any preference [ 18 , 1 , 29 ] , a global query learner ( denoted as ` global ) who uses a global ranking overH to pick the next hypothesis ht in H ( Zt−1 ) as per Eq . ( 2 ) [ 25 , 29 ] , and the Max-Min learning algorithm ( denoted as ` Max-Min ) introduced in a recent work on LwEQ paradigm [ 14 ] . Complexity of teaching ( i.e. , the number of queries made ) . In this paper , we study the number of equivalence queries asked by the learner to the teacher to identify a target hypothesis in Algorithm 1 , and we call it the query complexity or teaching complexity for LwEQ paradigm interchangeably . Clearly , this query complexity depends on the learner ’ s query function ` and the choice of counterexamples by the teacher . In the following sections , we study this complexity for different teacher types depending on the informativeness of the provided counterexamples , as well as for different families of learner types . 3 The Query Complexity with Best-Case Teacher : LwEQ-TD In this section , we consider the optimal teacher who picks best-case counterexamples with the objective of minimizing the learner ’ s queries for identifying h∗ . For this optimal teacher , we provide a formal characterization of teaching complexity , namely learning-with-equivalence queries teaching dimension ( LwEQ-TD ) paradigm , inspired by different notions of teaching dimension for the learning-from-samples ( LfS-TD ) paradigm [ 18 , 24 , 26 , 29 ] . Notation . We denote by L a family of learner models—alternatively , we can think of them as a family of query functions . To begin , we fix a query function ` ∈ L that the learner uses to pick next hypotheses for equivalence queries . Our characterization below will be based on understanding the minimal “ cost ” ( i.e. , the number of queries needed ) in steering the learner from a hypothesis h with the history of counterexamples Z to the target hypothesis h∗—in Algorithm 1 , h refers to ht−1 and Z refers to Zt−1 at the beginning of time t. Minimal cost of steering . We begin by providing a recursive function that captures this cost of steering and will be key to formalize teaching complexity in the LwEQ paradigm . As is typically considered in the LfS paradigm [ 18 , 24 , 26 , 29 ] , we consider the “ adversarial ” perspective in how the learner breaks ties in picking the hypothesis w.r.t . its query function in Eq . ( 2 ) . The optimal cost for steering the learner from the current history ( Z , h ) to h∗ in the query protocol of Algorithm 1 is then given by : D ` ( Z , h , h ∗ ) = { 0 , if ` ( Z , h ) = { h∗ } 1 + max h′∈ ` ( Z , h ) min z : h′ ( xz ) 6=yz D ` ( Z ∪ { z } , h′ , h∗ ) , otherwise ( 3 ) Note that D ` in Eq . ( 3 ) has a max operator w.r.t . the learner ’ s choice and min operator with the teacher ’ s choice ( see [ 29 ] ) . Furthermore , in this function , we are not counting the last query when the learner ’ s queried hypothesis ht equals h∗ ( hence , the number of queries is same as the number of counterexamples received by the learner ) . LwEQ-TD . Given a fixed query function ` ∈ L , an initial hypothesis h0 , and a target hypothesis h∗ , LwEQ-TD w.r.t . ` and h∗ is the optimal cost for teaching the target hypothesis h∗ : LwEQ-TDX , H , h0 ( ` , h ∗ ) = D ` ( ∅ , h0 , h∗ ) . ( 4 ) To characterize the teaching complexity for the hypothesis class , we consider the worst-case target hypothesis inH , given by : LwEQ-TDX , H , h0 ( ` ) = maxh∗ D ` ( ∅ , h0 , h∗ ) . ( 5 ) Finally , to compare LwEQ-TD with existing notions of TD in the LfS paradigm ( i.e. , LfS-TD , see Footnote 1 ) , we define LwEQ-TD for a given family of query functions L. Based on [ 24 , 27 , 29 ] , we define LwEQ-TD w.r.t the family L as the teaching complexity w.r.t . the best ` in that family : LwEQ-TDX , H , h0 ( L ) = min ` ∈L maxh∗ D ` ( ∅ , h0 , h ∗ ) . ( 6 ) In the following two sections , we will investigate how LwEQ-TD ( the complexity of the optimal teacher in the LwEQ paradigm ) compares with various other complexity notions . In Section 4 , we investigate different teachers in the LwEQ paradigm , showcasing the power of the optimal teacher . In Section 5 , we establish new connections of LwEQ-TD with LfS-TD .

Summary: This work studies the sample complexity of machine teaching finite hypothesis classes with equivalence queries (EQ), a model in which the learner queries hypotheses in the concept class directly, and the “teacher” responds “yes” or “no” along with a counter-example with the goal of eventually teaching the learner some ground truth concept.   There are many formalizations of this model in the literature. This work mainly focuses on a natural regime of the problem in which the teacher is “helpful,” always choosing the best possible counter-example to help the learner, and the learner is adversarial, though generally restricted under some fixed “learning rule,” (e.g. they must always pick a hypothesis consistent with previous counter-examples).  The authors study a notion of query complexity they call “Learning with Equivalence Queries Teaching Dimension” (LwEQ-TD) that is closely related to other forms of TD used in machine teaching. They show that for a variety of fundamental concept classes (axis-aligned halfspaces, axis-aligned rectangles, and monotone monomials), a helpful teacher can always teach even the most adversarial learner in O(1) examples. On the other hand, they give lower bounds on the sample complexity when the teacher gives random or adversarial samples that scales with the size of the class.  Finally, the authors give an in depth comparison between LwEQ-TD and previous notions of machine teaching in the membership query model. Indeed, for a very natural class of adversarial learner, they prove that LwEQ-TD is in fact exactly the standard teaching dimension (along with a host of other connections).