Subjective Learning for Open-Ended Data

Conventional supervised learning typically assumes that the learning task can be solved by learning a single function since the data is sampled from a fixed distribution . However , this assumption is invalid in open-ended environments where no task-level data partitioning is available . In this paper , we present a novel supervised learning framework of learning from open-ended data , which is modeled as data implicitly sampled from multiple domains with the data in each domain obeying a domain-specific target function . Since different domains may possess distinct target functions , open-ended data inherently requires multiple functions to capture all its input-output relations , rendering training a single global model problematic . To address this issue , we devise an Open-ended Supervised Learning ( OSL ) framework , of which the key component is a subjective function that allocates the data among multiple candidate models to resolve the “ conflict ” between the data from different domains , exhibiting a natural hierarchy . We theoretically analyze the learnability and the generalization error of OSL , and empirically validate its efficacy in both open-ended regression and classification tasks . 1 INTRODUCTION . A hallmark of general intelligence is the ability of handling open-ended environments , which roughly means complex , diverse environments with no manual task specification ( Adams et al. , 2012 ; Clune , 2019 ; Colas et al. , 2019 ; Wang et al. , 2020 ) . Conventional supervised learning typically assumes that the learning task can be solved by approximating a single ground-truth target function ( Vapnik , 2013 ) . However , this assumption is invalid in open-ended environments where the data may implicitly belong to multiple , disparate domains with potentially different target functions since no manual task-level data partitioning is available . For instance , when collecting image-label pairs from the Internet , an image of a red sphere can correlate with the label of both “ red ” and “ sphere ” , implicitly representing two distinct domains or “ metaconcepts ” ( Han et al. , 2019 ) : “ color ” and “ shape ” . While being generic , this setting also characterizes many practical scenarios where the data comes from multiple sources , contexts or groups : for example , in federated learning the data is distributed on multiple clients , and in algorithmic fairness research the data is from different populations . In these scenarios , the input-conditional label distribution may vary in different domains ( also referred to as concept shift ( Kairouz et al. , 2021 ) ) due to personal preferences or other latent factors , corresponding to different target functions . Since different domains may possess different target functions , training a single global model by running Empirical Risk Minimization ( ERM ) using all data is problematic due to the potential “ conflict ” between the data from different domains : in the “ red sphere ” example above , directly training on all data will lead to the unfavorable result of “ 50 % red , 50 % sphere ” . Similar phenomena have also been observed by prior works ( Finn et al. , 2019 ; Su et al. , 2020 ) , where a learner simultaneously regressing from multiple target functions trivially outputs their mean . This indicates that the data sampled from open-ended environments , which we refer to as open-ended data , exhibits a structural difference from conventional supervised data . Formally , we introduce a novel , dataset-level measure named mapping rank , which represents the minimal number of functions required to “ fully express ” all input-output relations in the data and can be used to expound such difference more clearly . Definition 1 ( Mapping rank ) . Let X be an input space , Y an output space , and Z = { ( xi , yi ) } li=1 a dataset with cardinality l. Let F ( r ) = { fi } ri=1 be a function set with cardinality r , where each element is a single-valued deterministic function fromX to Y . Then , the mapping rank ofZ , denoted by R : = R ( Z ) , is defined as the minimal positive integer r , satisfying that there exists a function set F ( r ) such that for every ( x , y ) ∈ Z , there exists f ∈ F ( r ) with f ( x ) = y . Note that here we assume that there exist deterministic relations between inputs and outputs in the same domain , which is generally a mild assumption and is satisfied in many practical applications . Under Definition 1 , conventional supervised data yields a mapping rank R = 1 as it assumes that the whole dataset can be expressed by a single function . In contrast , open-ended data has a mapping rank R > 1 since for the same input different outputs exist , manifested as the conflict between data samples that renders training a single model problematic . Hence , it is natural to consider allocating the data to multiple models , so that the data processed by each model has a mapping rank R = 1 and thus can be handled with ERM . Although in some scenarios , apart from data samples there also exists side-information or metadata that can be exploited to identify the domains , this information may be difficult to define or collect in practice ( Hanna et al. , 2020 ; Creager et al. , 2021 ) ; even when such side-information is available , in many cases it still remains unclear how to leverage such information to detect and resolve the potential conflict between domains . Therefore , the problem of how to properly allocate open-ended data among the models is highly non-trivial . To tackle the aforementioned challenge , we present an Open-ended Supervised Learning ( OSL ) framework to enable effective learning from open-ended data . Concretely , OSL maintains a set of low-level models and a high-level subjective function that automatically allocates the data among these models so that the data processed by each model exhibits no conflict . The term “ subjective ” is used since in conventional supervised learning such allocation is manually performed during the data collecting process and thus conforms to human subjectivity . The motivation of such process is that if the subjective function yields an inappropriate allocation , i.e. , assigning conflicting data samples to the same model , then it will hinder the global minimization of the training error due to the conflict , which in turn drives the subjective function to alter its allocation strategy . Using a probablistic reformulation of OSL , we establish the connection between the data allocation and the posterior maximization using a variant of Expectation-Maximization ( EM ) algorithm ( Dempster et al. , 1977 ) , and show that the optimal form of the subjective function can be explicitly derived . Theoretically , we respectively analyse the Probably Approximately Correct ( PAC ) learnability ( Valiant , 1984 ) and the generalization error of OSL . Using the tools from statistical learning theory ( Vapnik , 2013 ) , we show that the relation between the number of low-level models and the mapping rank of data plays a key role in the learnability of OSL , and the generalization error of OSL can be decomposed into terms that respectively reflect high-level data allocation and lowlevel prediction errors . Empirically , we conduct extensive experiments including simulated openended regression and classification tasks to verify the efficacy of OSL . Our results show that OSL can effectively allocate and learn from open-ended data without additional human intervention . In summary , our contributions are three-fold : Open-ended data and mapping rank . We formalize a new problem of learning from open-ended data , and introduce a novel measure termed as mapping rank to outline the structural difference between open-ended data and conventional supervised data . OSL framework with theoretical guarantee . We present an OSL framework to enable effective learning from open-ended data ( Section 2 ) , and theoretically justify its learnability ( Section 3.1 ) and generalizability ( Section 3.2 ) respectively . Empirical validation of efficacy . We conduct extensive experiments on both open-ended regression and classification tasks . Experimental results validate our theoretical claims and demonstrate the efficacy of OSL ( Section 4 ) . 2 OPEN-ENDED SUPERVISED LEARNING . In this section , we present the overall formulation and the algorithm of OSL . We adhere to the conventional terminology in supervised learning , and let X be an input space , Y an output space , H a hypothesis space where each hypothesis ( model ) is a function from X to Y , and ` : Y×Y → [ 0 , 1 ] a non-negative and bounded loss function without loss of generality . We use [ k ] = { 1 , 2 , · · · , k } for positive integers k , and denote by 1 ( · ) the indicator function . We use superscripts to denote sampling indices ( e.g. , di and xij ) and subscripts as element indices ( e.g. , di ) . 2.1 PROBLEM STATEMENT . We begin by introducing the notion of domain to formulate the generation process of open-ended data . Inspired by Ben-David et al . ( 2010 ) , we define a domain d as a pair 〈P , c〉 consisting of a distribution P on X and a deterministic target function c : X → Y , and assume that the open-ended data is generated by a domain set D = { di } Ni=1 = { 〈Pi , ci〉 } Ni=1 containing N ( agnostic to the learner ) domains . Each domain has its own sub-dataset Zi = { ( xij , yij ) } lij=1 with cardinality li , where xi1 , · · · , xili are i.i.d . drawn from Pi and yij = ci ( xij ) . The whole dataset is the union of these sub-datasets : Z = ⋃N i=1 Zi with cardinality l = ∑N i=1 li and mapping rank 1 < R ≤ N . We then consider a bilevel sampling procedure : first , m domain samples d1 , · · · , dm are i.i.d . drawn from a distribution Q defined on D ( the same domain may be sampled multiple times ) , resulting in m sampling episodes ; second , in each sampling episode n data samples are i.i.d . drawn from the sub-dataset corresponding to the sampled domain . Hence , the dataset or any sub-datasets may be sampled multiple times during training . This sampling regime is analagous to the bilevel sampling process adopted by meta-learning . However , meta-learning usually assumes a dense distribution of related domains to enable task-level generalization ( Pentina & Lampert , 2014 ; Amit & Meir , 2018 ) , while OSL is compatible with scarce and disparate domains and inter-domain transfer is orthogonal . As we have mentioned in Section 1 , a single model is not sufficient in this setting whenR > 1 . Thus , we equip the learner with a hypothesis set H = { hi } Ki=1 consisting ofK > 1 hypotheses , enhancing its expressive capability . Although both N and K are assumed to be unknown , we will show that in general K ≥ R suffices ( Section 3.1 ) , which eases the difficulty of setting the hyperparameter K. In the above setting , an episodic sample number parameter n is introduced to maintain the local consistency of data , implicitly assuming that we are able to sample a size-n data batch at a time from each domain . While this formulation subsumes the fully online case of n = 1 , we note that although sometimes setting n = 1 works in practice , it also tends to be risky since it may raise difficulties in controlling the generalization error , as we will both theoretically and empirically demonstrate in the following sections ( see Section 3.2 and Section 4.3 ) .

Summary: The paper introduces "Open-ended Supervised Learning (OSL)", a method that handles data form multiple domains (open-ended data). OSL leverages an Expectation-Maximization (EM) algorithm to train the "subjective function" that allocates models to data domains. The paper presents the theoretical foundations and empirical results in which it outperforms existing baselines for (generated) regression and classification tasks.