Subjective Learning for Open-Ended Data

Conventional supervised learning typically assumes that the learning task can be solved by learning a single function since the data is sampled from a fixed distribution . However , this assumption is invalid in open-ended environments where no task-level data partitioning is available . In this paper , we present a novel supervised learning framework of learning from open-ended data , which is modeled as data implicitly sampled from multiple domains with the data in each domain obeying a domain-specific target function . Since different domains may possess distinct target functions , open-ended data inherently requires multiple functions to capture all its input-output relations , rendering training a single global model problematic . To address this issue , we devise an Open-ended Supervised Learning ( OSL ) framework , of which the key component is a subjective function that allocates the data among multiple candidate models to resolve the “ conflict ” between the data from different domains , exhibiting a natural hierarchy . We theoretically analyze the learnability and the generalization error of OSL , and empirically validate its efficacy in both open-ended regression and classification tasks . 1 INTRODUCTION . A hallmark of general intelligence is the ability of handling open-ended environments , which roughly means complex , diverse environments with no manual task specification ( Adams et al. , 2012 ; Clune , 2019 ; Colas et al. , 2019 ; Wang et al. , 2020 ) . Conventional supervised learning typically assumes that the learning task can be solved by approximating a single ground-truth target function ( Vapnik , 2013 ) . However , this assumption is invalid in open-ended environments where the data may implicitly belong to multiple , disparate domains with potentially different target functions since no manual task-level data partitioning is available . For instance , when collecting image-label pairs from the Internet , an image of a red sphere can correlate with the label of both “ red ” and “ sphere ” , implicitly representing two distinct domains or “ metaconcepts ” ( Han et al. , 2019 ) : “ color ” and “ shape ” . While being generic , this setting also characterizes many practical scenarios where the data comes from multiple sources , contexts or groups : for example , in federated learning the data is distributed on multiple clients , and in algorithmic fairness research the data is from different populations . In these scenarios , the input-conditional label distribution may vary in different domains ( also referred to as concept shift ( Kairouz et al. , 2021 ) ) due to personal preferences or other latent factors , corresponding to different target functions . Since different domains may possess different target functions , training a single global model by running Empirical Risk Minimization ( ERM ) using all data is problematic due to the potential “ conflict ” between the data from different domains : in the “ red sphere ” example above , directly training on all data will lead to the unfavorable result of “ 50 % red , 50 % sphere ” . Similar phenomena have also been observed by prior works ( Finn et al. , 2019 ; Su et al. , 2020 ) , where a learner simultaneously regressing from multiple target functions trivially outputs their mean . This indicates that the data sampled from open-ended environments , which we refer to as open-ended data , exhibits a structural difference from conventional supervised data . Formally , we introduce a novel , dataset-level measure named mapping rank , which represents the minimal number of functions required to “ fully express ” all input-output relations in the data and can be used to expound such difference more clearly . Definition 1 ( Mapping rank ) . Let X be an input space , Y an output space , and Z = { ( xi , yi ) } li=1 a dataset with cardinality l. Let F ( r ) = { fi } ri=1 be a function set with cardinality r , where each element is a single-valued deterministic function fromX to Y . Then , the mapping rank ofZ , denoted by R : = R ( Z ) , is defined as the minimal positive integer r , satisfying that there exists a function set F ( r ) such that for every ( x , y ) ∈ Z , there exists f ∈ F ( r ) with f ( x ) = y . Note that here we assume that there exist deterministic relations between inputs and outputs in the same domain , which is generally a mild assumption and is satisfied in many practical applications . Under Definition 1 , conventional supervised data yields a mapping rank R = 1 as it assumes that the whole dataset can be expressed by a single function . In contrast , open-ended data has a mapping rank R > 1 since for the same input different outputs exist , manifested as the conflict between data samples that renders training a single model problematic . Hence , it is natural to consider allocating the data to multiple models , so that the data processed by each model has a mapping rank R = 1 and thus can be handled with ERM . Although in some scenarios , apart from data samples there also exists side-information or metadata that can be exploited to identify the domains , this information may be difficult to define or collect in practice ( Hanna et al. , 2020 ; Creager et al. , 2021 ) ; even when such side-information is available , in many cases it still remains unclear how to leverage such information to detect and resolve the potential conflict between domains . Therefore , the problem of how to properly allocate open-ended data among the models is highly non-trivial . To tackle the aforementioned challenge , we present an Open-ended Supervised Learning ( OSL ) framework to enable effective learning from open-ended data . Concretely , OSL maintains a set of low-level models and a high-level subjective function that automatically allocates the data among these models so that the data processed by each model exhibits no conflict . The term “ subjective ” is used since in conventional supervised learning such allocation is manually performed during the data collecting process and thus conforms to human subjectivity . The motivation of such process is that if the subjective function yields an inappropriate allocation , i.e. , assigning conflicting data samples to the same model , then it will hinder the global minimization of the training error due to the conflict , which in turn drives the subjective function to alter its allocation strategy . Using a probablistic reformulation of OSL , we establish the connection between the data allocation and the posterior maximization using a variant of Expectation-Maximization ( EM ) algorithm ( Dempster et al. , 1977 ) , and show that the optimal form of the subjective function can be explicitly derived . Theoretically , we respectively analyse the Probably Approximately Correct ( PAC ) learnability ( Valiant , 1984 ) and the generalization error of OSL . Using the tools from statistical learning theory ( Vapnik , 2013 ) , we show that the relation between the number of low-level models and the mapping rank of data plays a key role in the learnability of OSL , and the generalization error of OSL can be decomposed into terms that respectively reflect high-level data allocation and lowlevel prediction errors . Empirically , we conduct extensive experiments including simulated openended regression and classification tasks to verify the efficacy of OSL . Our results show that OSL can effectively allocate and learn from open-ended data without additional human intervention . In summary , our contributions are three-fold : Open-ended data and mapping rank . We formalize a new problem of learning from open-ended data , and introduce a novel measure termed as mapping rank to outline the structural difference between open-ended data and conventional supervised data . OSL framework with theoretical guarantee . We present an OSL framework to enable effective learning from open-ended data ( Section 2 ) , and theoretically justify its learnability ( Section 3.1 ) and generalizability ( Section 3.2 ) respectively . Empirical validation of efficacy . We conduct extensive experiments on both open-ended regression and classification tasks . Experimental results validate our theoretical claims and demonstrate the efficacy of OSL ( Section 4 ) . 2 OPEN-ENDED SUPERVISED LEARNING . In this section , we present the overall formulation and the algorithm of OSL . We adhere to the conventional terminology in supervised learning , and let X be an input space , Y an output space , H a hypothesis space where each hypothesis ( model ) is a function from X to Y , and ` : Y×Y → [ 0 , 1 ] a non-negative and bounded loss function without loss of generality . We use [ k ] = { 1 , 2 , · · · , k } for positive integers k , and denote by 1 ( · ) the indicator function . We use superscripts to denote sampling indices ( e.g. , di and xij ) and subscripts as element indices ( e.g. , di ) . 2.1 PROBLEM STATEMENT . We begin by introducing the notion of domain to formulate the generation process of open-ended data . Inspired by Ben-David et al . ( 2010 ) , we define a domain d as a pair 〈P , c〉 consisting of a distribution P on X and a deterministic target function c : X → Y , and assume that the open-ended data is generated by a domain set D = { di } Ni=1 = { 〈Pi , ci〉 } Ni=1 containing N ( agnostic to the learner ) domains . Each domain has its own sub-dataset Zi = { ( xij , yij ) } lij=1 with cardinality li , where xi1 , · · · , xili are i.i.d . drawn from Pi and yij = ci ( xij ) . The whole dataset is the union of these sub-datasets : Z = ⋃N i=1 Zi with cardinality l = ∑N i=1 li and mapping rank 1 < R ≤ N . We then consider a bilevel sampling procedure : first , m domain samples d1 , · · · , dm are i.i.d . drawn from a distribution Q defined on D ( the same domain may be sampled multiple times ) , resulting in m sampling episodes ; second , in each sampling episode n data samples are i.i.d . drawn from the sub-dataset corresponding to the sampled domain . Hence , the dataset or any sub-datasets may be sampled multiple times during training . This sampling regime is analagous to the bilevel sampling process adopted by meta-learning . However , meta-learning usually assumes a dense distribution of related domains to enable task-level generalization ( Pentina & Lampert , 2014 ; Amit & Meir , 2018 ) , while OSL is compatible with scarce and disparate domains and inter-domain transfer is orthogonal . As we have mentioned in Section 1 , a single model is not sufficient in this setting whenR > 1 . Thus , we equip the learner with a hypothesis set H = { hi } Ki=1 consisting ofK > 1 hypotheses , enhancing its expressive capability . Although both N and K are assumed to be unknown , we will show that in general K ≥ R suffices ( Section 3.1 ) , which eases the difficulty of setting the hyperparameter K. In the above setting , an episodic sample number parameter n is introduced to maintain the local consistency of data , implicitly assuming that we are able to sample a size-n data batch at a time from each domain . While this formulation subsumes the fully online case of n = 1 , we note that although sometimes setting n = 1 works in practice , it also tends to be risky since it may raise difficulties in controlling the generalization error , as we will both theoretically and empirically demonstrate in the following sections ( see Section 3.2 and Section 4.3 ) .

Summary: The authors present a novel framework for learning different target concepts. They argue that so far, a single classifier is supposed to learn mixed concepts but usually fail. Hence, they propose a framework for sampling instances according to only a single target concept and solving this problem with classical ERM and repeating for other concepts. They provide theoretical analyses and empirical insights.