Teaching via Best-Case Counterexamples in the Learning-with-Equivalence-Queries Paradigm

1 Introduction . Learning-with-queries paradigm involves a learner who asks structured queries to a teacher in order to locate a target hypothesis . This paradigm has been extensively studied in machine learning and formal methods research , including automata learning [ 1 , 2 , 3 , 4 ] , model checking [ 5 ] , oracle-guided synthesis ( OGIS ) [ 6 ] , model learning [ 7 ] , among others . Classical literature involves different kinds of queries that a learner could ask [ 1 , 8 , 9 ] , such as membership queries ( i.e. , “ is the queried instance consistent with the target hypothesis ? ” ) and equivalence queries ( i.e. , “ is the queried hypothesis the target hypothesis ? ” ) . In this paper , we consider the learning-with-equivalence-queries ( LwEQ ) paradigm where the learner is only asking equivalence queries and the teacher responds to a query either “ yes '' or “ no '' along with a counterexample on which the current hypothesis disagrees with the target hypothesis . LwEQ paradigm captures a variety of important problem settings , such as counterexample-guided synthesis ( CEGIS ) [ 10 ] , data augmentation ( CEGAR ) [ 11 ] , and learning regular languages from counterexamples [ 12 ] . The focus of our work is to understand the query complexity for the LwEQ paradigm , i.e. , the number of equivalence queries needed by the learner for the exact identification of a target hypothesis [ 9 ] . The query complexity clearly depends on the learner model ( i.e. , the query function deciding the next hypothesis to query ) , as well as the informativeness of the counterexamples provided by the teacher . In the literature , the query complexity for the LwEQ paradigm has been extensively studied when the learner receives worst-case or random counterexamples [ 1 , 9 , 13 , 14 , 15 ] . To show worst-case bounds [ 1 , 13 ] , classical works have studied a teacher who responds with worst-case counterexamples 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . to maximize the number of equivalence queries asked by the learner . Recent work [ 14 ] has studied this query complexity when facing a more benign teacher who provides counterexamples selected at random from a known probability distribution . In particular , [ 14 ] proposed a learning algorithm ( i.e. , a query function for the learner ) that achieves O ( log n ) query complexity on the expected number of random counterexamples for any hypothesis class of size n. When contrasting this bound with worst-case bounds , it shows that random counterexamples could lead to exponential improvement in the query complexity when compared with worst-case counterexamples [ 13 , 14 ] . Along these lines , an important research question is to understand the query complexity where the counterexamples are picked by a more informed and helpful teacher , instead of random or worst-case counterexamples . In this paper , we consider a more powerful teaching setting , where the optimal teacher picks best-case counterexamples to steer the learner towards a target hypothesis . Our goal is to characterize the query complexity for this optimal teacher ( also , referred to as teaching complexity in the paper ) , for the LwEQ paradigm . This teaching complexity has been extensively studied for binary classification in the learning-from-samples ( LfS ) paradigm [ 16 , 17 ] and is termed as “ teaching dimension ” ( TD ) [ 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 ] . Beyond the LfS paradigm , various notions of teaching complexity have also been studied in other learning paradigms ( see Section 1.1 for further details ) . We introduce a new notion of teaching complexity for the LwEQ paradigm , namely “ LwEQ teaching dimension ” ( LwEQ-TD ) , capturing the number of queries needed by the learner when best-case counterexamples are provided by the optimal teacher . Our study of LwEQ-TD for some prominent hypothesis classes reveals the power of teaching via best-case counterexamples—we show a significant reduction in query complexity when compared to that for worst-case and random counterexamples . Furthermore , we establish several new connections of LwEQ-TD to the existing notions of TD in the LfS paradigm.1 Table 1 provides a summary of these different teaching settings and learning paradigms ; our main results and contributions are summarized below : I . We characterize the query complexity for the optimal teacher in the LwEQ paradigm , termed as learning-with-equivalence-queries teaching dimension ( LwEQ-TD ) . ( see Section 3 ) II . We study the query complexity in the LwEQ paradigm under different teaching settings : worst-case , random , and best-case , distinguished by the informativeness of counterexamples . We showcase the power of best-case counterexamples picked by the optimal teacher , in contrast to worst-case or random counterexamples , for different hypothesis classes , including Axes-aligned hyperplanes , Monotone monomials , and Orthogonal rectangles . ( see Section 4 ) III . We establish new connections between LwEQ-TD and LfS-TD by studying LwEQ-TD for different learner models based on the richness of their query functions . We show that LwEQ-TD is the same as wc-TD [ 18 ] , RTD [ 22 , 24 ] , and NCTD [ 27 ] for a hypothesis class when restricting query functions to specific families . In general , LwEQ-TD is weaker than LfS-TD , e.g. , LwEQ-TD is lower-bounded by local-PBTD [ 26 , 29 ] of the hypothesis class when the learner ’ s next query depends on the previous query . ( see Section 5 ) 1.1 Background and Related Work . Learning-with-queries paradigm and equivalence queries . Learning-with-queries paradigm was introduced in [ 1 ] which proposed L∗ algorithm for exact identification of DFAs ( deterministic finite automaton ) when the learner is allowed to ask membership queries and equivalence queries . Classical work has studied different types of queries ( subset , membership , equivalence , correction , among others ) [ 1 , 8 , 9 ] . Among these works , learning with membership queries has been explored in a variety of problem settings , such as PAC learning [ 16 ] , active learning [ 33 , 34 , 35 ] , and agnostic 1We will collectively refer to these different notions of TD in the LfS paradigm as LfS-TD . learning [ 36 ] . In the learning-with-equivalence-queries ( LwEQ ) paradigm , the learner can ask only equivalence queries ; furthermore , in our work , we consider proper queries , i.e. , the queried hypothesis is within the hypothesis class ( see Section 6 for a discussion on improper queries ) . In this LwEQ paradigm , [ 13 ] studied query complexity when the teacher picks worst-case counterexamples for some key hypothesis classes ( e.g. , DFA , NFA , Context-free Grammars ) , and showed an exponential lower bound . [ 14 ] studied random counterexamples , and proposed a learning algorithm , namely Max-Min , which achieves a substantially improved bound on query complexity . We will investigate the query complexity of Max-Min learner for the hypothesis class of Axes-aligned hyperplanes in Section 4 . Teaching in the LfS paradigm for binary classification . Algorithmic machine teaching , first introduced by [ 18 , 37 ] , studies the interaction between a teacher and a learner where the teacher ’ s goal is to find an optimal sequence of training samples to teach a target hypothesis . [ 18 ] introduced a measure of teaching complexity , named teaching dimension ( TD ) of the hypothesis class , in the learning-from-samples ( LfS ) paradigm . The classical notion of TD in [ 18 ] characterized the minimum number of samples ( i.e. , examples ) needed to teach a target hypothesis to a version-space learner who picks hypothesis within the version space arbitrarily ( in an adversarial way ) . In the past two decades , several new teaching settings have been studied , driven by the motivation to lower teaching complexity and to find settings for which TD has better connections with Vapnik–Chervonenkis dimension ( VCD ) [ 38 ] . In particular , several new teaching models and complexity measures have been proposed for both the batch teaching settings ( e.g. , worst-case [ 18 ] , recursive [ 22 , 24 ] , preference-based [ 25 ] , and non-clashing models [ 27 ] ) and the sequential settings ( e.g. , local preference-based model [ 26 , 29 ] ) . These teaching settings , in turn , lead to different notions of TD , that we collectively refer to as LfS-TD . In recent work , [ 29 ] has characterized these different notions of TD through a unified framework of modeling learners with preference/ranking functions . In Section 5 , we will build on this framework to model the learner ’ s query functions in the LwEQ paradigm through ranking functions , allowing us to connect LwEQ-TD with LfS-TD . Teaching in other learning settings . Within binary classification setting , teaching complexity results have been extended beyond version space learners , including models for gradient learners [ 39 , 40 ] , models inspired by control theory [ 41 , 42 ] , and models for human-centered applications [ 43 , 44 , 45 , 46 ] . Furthermore , a recent line of research has studied robust notions of teaching in settings where the teacher has limited information about the learner ’ s dynamics [ 47 , 48 , 49 ] . Given the importance of teacher-learner interactions in many real-world applications , teaching has also been studied in richer domains . In particular , teaching complexity has been investigated for imitation learning settings where the teacher provides demonstrations [ 50 , 51 , 52 , 53 , 54 ] , and for reinforcement learning settings where the teacher provides reward feedback [ 55 , 56 ] . We see these works as complementary to ours , and we refer the reader to see [ 28 ] for an overview . 2 Problem Setup . Teaching framework . Let X be a ground set of unlabeled instances and Y the set of labels . LetH be a finite class of hypotheses ; each element h ∈ H is a function h : X → Y . Here , we only consider boolean functions , and hence Y = { 0 , 1 } . Let Z ⊆ X × Y be the ground set of labeled examples . Each element z = ( xz , yz ) ∈ Z represents a labeled example where the label is given by the target hypothesis h∗ , i.e. , yz = h∗ ( xz ) . Furthermore , for any Z ⊆ Z , we define version space induced by the examples Z as the subset of hypothesesH ( Z ) ⊆ H that are consistent with the labels of all the examples , i.e. , H ( Z ) : = { h ∈ H | ∀z = ( xz , yz ) ∈ Z , h ( xz ) = yz } . ( 1 ) Equivalence queries . We consider the LwEQ paradigm where a learner seeks to identify a target hypothesis from the hypothesis class via equivalence queries . In an equivalence query , the learner asks if the current hypothesis , say h′ ∈ H , is equivalent to the target hypothesis h∗ or not . The teacher provides a response r where r is either “ yes ” if h′ ≡ h∗ or “ no ” along with a counterexample z : = ( xz , yz ) ∈ Z , such that h′ ( xz ) 6= yz . Learner model and query protocol . We consider a generic model of the learner that captures our assumptions about how the learner conjectures its hypothesis for equivalence queries based on the responses received from the teacher . A key aspect of this model is the learner ’ s query function ` over the hypotheses . Based on the information encoded in the inputs of this query function ( i.e. , the current hypothesis and the history of counterexamples ) , the learner will choose one hypothesis inH . In the beginning , the learner starts with an initial hypothesis h0 ∈ H , the history is Z0 = ∅ , and the version space is H0 = H. At a time step t ≥ 1 , the learner first picks the hypothesis ht as follows : ` ( Zt−1 , ht−1 ) −→ ht ∈ H ( Zt−1 ) , ( 2 ) where Zt−1 is the history of counterexamples seen up until time t andH ( Zt−1 ) is the corresponding version space . The learner then queries ht for equivalence to h∗ and receives a response rt . Then , the query protocol proceeds as follows : ( i ) if rt is “ yes ” , the learner has identified h∗ and stops ; ( ii ) otherwise rt is “ no ” along with a counterexample zt using which the learner updates Zt = Zt−1 ∪ { zt } , and continues . We summarize this query protocol in Algorithm 1 . Algorithm 1 : Query protocol between the learner and the teacher 1 Learner ’ s initial hypothesis is h0 ∈ H , history is Z0 = ∅ , and version space isH0 = H ; 2 for t = 1 , 2 , 3 , · · · do 3 learner picks ht ∈ H ( Zt−1 ) based on Zt−1 and ht−1 as per Eq . ( 2 ) ; 4 learner performs an equivalence query with ht ; 5 teacher provides a response rt that is either “ yes ” or ‘ no ” along with a counterexample zt ; 6 if rt is “ yes ” then learner has identified h∗ and stops ; 7 else learner updates Zt = Zt−1 ∪ { zt } ; We assume that both the learner and the teacher have full knowledge of X , Y , and H ; in addition , the teacher knows the target h∗ as well as the learner ’ s query function ` . In this work , we consider learner models which could be characterized by a specific query function ` as discussed above . These include well-known learners studied in the literature , such as a constant query learner ( denoted as ` const ) who picks the next hypothesis ht arbitrarily inH ( Zt−1 ) without any preference [ 18 , 1 , 29 ] , a global query learner ( denoted as ` global ) who uses a global ranking overH to pick the next hypothesis ht in H ( Zt−1 ) as per Eq . ( 2 ) [ 25 , 29 ] , and the Max-Min learning algorithm ( denoted as ` Max-Min ) introduced in a recent work on LwEQ paradigm [ 14 ] . Complexity of teaching ( i.e. , the number of queries made ) . In this paper , we study the number of equivalence queries asked by the learner to the teacher to identify a target hypothesis in Algorithm 1 , and we call it the query complexity or teaching complexity for LwEQ paradigm interchangeably . Clearly , this query complexity depends on the learner ’ s query function ` and the choice of counterexamples by the teacher . In the following sections , we study this complexity for different teacher types depending on the informativeness of the provided counterexamples , as well as for different families of learner types . 3 The Query Complexity with Best-Case Teacher : LwEQ-TD In this section , we consider the optimal teacher who picks best-case counterexamples with the objective of minimizing the learner ’ s queries for identifying h∗ . For this optimal teacher , we provide a formal characterization of teaching complexity , namely learning-with-equivalence queries teaching dimension ( LwEQ-TD ) paradigm , inspired by different notions of teaching dimension for the learning-from-samples ( LfS-TD ) paradigm [ 18 , 24 , 26 , 29 ] . Notation . We denote by L a family of learner models—alternatively , we can think of them as a family of query functions . To begin , we fix a query function ` ∈ L that the learner uses to pick next hypotheses for equivalence queries . Our characterization below will be based on understanding the minimal “ cost ” ( i.e. , the number of queries needed ) in steering the learner from a hypothesis h with the history of counterexamples Z to the target hypothesis h∗—in Algorithm 1 , h refers to ht−1 and Z refers to Zt−1 at the beginning of time t. Minimal cost of steering . We begin by providing a recursive function that captures this cost of steering and will be key to formalize teaching complexity in the LwEQ paradigm . As is typically considered in the LfS paradigm [ 18 , 24 , 26 , 29 ] , we consider the “ adversarial ” perspective in how the learner breaks ties in picking the hypothesis w.r.t . its query function in Eq . ( 2 ) . The optimal cost for steering the learner from the current history ( Z , h ) to h∗ in the query protocol of Algorithm 1 is then given by : D ` ( Z , h , h ∗ ) = { 0 , if ` ( Z , h ) = { h∗ } 1 + max h′∈ ` ( Z , h ) min z : h′ ( xz ) 6=yz D ` ( Z ∪ { z } , h′ , h∗ ) , otherwise ( 3 ) Note that D ` in Eq . ( 3 ) has a max operator w.r.t . the learner ’ s choice and min operator with the teacher ’ s choice ( see [ 29 ] ) . Furthermore , in this function , we are not counting the last query when the learner ’ s queried hypothesis ht equals h∗ ( hence , the number of queries is same as the number of counterexamples received by the learner ) . LwEQ-TD . Given a fixed query function ` ∈ L , an initial hypothesis h0 , and a target hypothesis h∗ , LwEQ-TD w.r.t . ` and h∗ is the optimal cost for teaching the target hypothesis h∗ : LwEQ-TDX , H , h0 ( ` , h ∗ ) = D ` ( ∅ , h0 , h∗ ) . ( 4 ) To characterize the teaching complexity for the hypothesis class , we consider the worst-case target hypothesis inH , given by : LwEQ-TDX , H , h0 ( ` ) = maxh∗ D ` ( ∅ , h0 , h∗ ) . ( 5 ) Finally , to compare LwEQ-TD with existing notions of TD in the LfS paradigm ( i.e. , LfS-TD , see Footnote 1 ) , we define LwEQ-TD for a given family of query functions L. Based on [ 24 , 27 , 29 ] , we define LwEQ-TD w.r.t the family L as the teaching complexity w.r.t . the best ` in that family : LwEQ-TDX , H , h0 ( L ) = min ` ∈L maxh∗ D ` ( ∅ , h0 , h ∗ ) . ( 6 ) In the following two sections , we will investigate how LwEQ-TD ( the complexity of the optimal teacher in the LwEQ paradigm ) compares with various other complexity notions . In Section 4 , we investigate different teachers in the LwEQ paradigm , showcasing the power of the optimal teacher . In Section 5 , we establish new connections of LwEQ-TD with LfS-TD .

Summary: This paper studies the equivalence query learning model where the teacher presents counterexamples if the learner proposes an incorrect hypothesis.  Three types of teachers are considered: (1) the "worst-case teacher" that presents adversarial counterexamples, (2) the "random teacher" that presents random counterexamples and (3) the "best-case teacher" that presents counterexamples designed to guide the learner towards the correct hypothesis in as few learning steps as possible.  Further, the query learner is equipped with a preference function on which the choice of its query depends (if its current hypothesis is incorrect).