Original: S UBJECTIVE L EARNING FOR C ONFLICTING D ATA
Revised: S UBJECTIVE L EARNING FOR C ONFLICTING D ATA

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Conventional supervised learning typically assumes that the learning task can be solved by approximating a single target function.
Revised: Conventional supervised learning typically assumes that the learning task can be solved by approximating a single target function.

Original: However, this assumption is often invalid in open-ended environments where no manual task-level data partitioning is available.
Revised: However, this assumption is often invalid in open-ended environments where no manual task-level data partitioning is available.

Original: In this paper, we investigate a more general setting where training data is sampled from multiple domains while the data in each domain conforms to a domain-speciﬁc target function.
Revised: In this paper, we investigate a more general setting where training data is sampled from multiple domains while the data in each domain conforms to a domain-speciﬁc target function.

Original: When different domains possess distinct target functions, training data exhibits inherent “conﬂict”, thus rendering single-model training problematic.
Revised: When different domains possess distinct target functions, training data exhibits inherent “conﬂict”, thus rendering single-model training problematic.

Original: To address this issue, we propose a framework termed subjective learning where the key component is a subjective function that automatically allocates the data among multiple candidate models to resolve the conﬂict in multi-domain data, and draw an intriguing connection between subjective learning and a variant of Expectation-Maximization.
Revised: To address this issue, we propose a framework termed subjective learning where the key component is a subjective function that automatically allocates the data among multiple candidate models to resolve the conﬂict in multi-domain data, and draw an intriguing connection between subjective learning and a variant of Expectation-Maximization.

Original: We present theoretical analysis on the learnability and the generalization error of our approach, and empirically show its efﬁcacy and potential applications in a range of regression and classiﬁcation tasks with synthetic data.
Revised: We present theoretical analysis on the learnability and the generalization error of our approach, and empirically show its efﬁcacy and potential applications in a range of regression and classiﬁcation tasks with synthetic data.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Conventional supervised learning typically assumes that the learning task can be solved by approximating a single ground-truth target function (Vapnik, 1999).
Revised: Conventional supervised learning typically assumes that the learning task can be solved by approximating a single ground-truth target function (Vapnik, 1999).

Original: However, this assumption is often violated in open-ended environments where the data may implicitly belong to multiple, disparate domains with potentially different target functions when no manual task-level data partitioning is available.
Revised: However, this assumption is often violated in open-ended environments where the data may implicitly belong to multiple, disparate domains with potentially different target functions when no manual task-level data partitioning is available.

Original: For instance, when curating a dataset using web images, an image of a red sphere may be labeled as both “red” and “sphere”, implicitly representing two distinct domains that respectively corresponds to two metaconcepts (Han et al., 2019): “color” and “shape”.
Revised: For instance, when curating a dataset using web images, an image of a red sphere may be labeled as both “red” and “sphere”, implicitly representing two distinct domains that respectively corresponds to two metaconcepts (Han et al., 2019): “color” and “shape”.

Original: Also, in some practical scenarios such as federated learning (McMahan et al., 2017) and algorithmic fairness (Mitchell et al., 2021), training data is usually collected from multiple sources (e.g., clients or populations) with concept shift (Kairouz et al., 2021), thus exhibiting different target functions (more generally, different input-conditional label distributions) due to personal preferences or other latent factors.
Revised: Also, in some practical scenarios such as federated learning (McMahan et al., 2017) and algorithmic fairness (Mitchell et al., 2021), training data is usually collected from multiple sources (e.g., clients or populations) with concept shift (Kairouz et al., 2021), thus exhibiting different target functions (more generally, different input-conditional label distributions) due to personal preferences or other latent factors.

Original: When different subsets of training data conform to different target functions, it is not hard to see that training a single model with standard Empirical Risk Minimization (ERM) is problematic due to the inherent “conﬂict” in data: in the above example, single-model training provably leads to the unfavorable result of “50% red, 50% sphere” (assuming the data is balanced).
Revised: When different subsets of training data conform to different target functions, it is not hard to see that training a single model with standard Empirical Risk Minimization (ERM) is problematic due to the inherent “conﬂict” in data: in the above example, single-model training provably leads to the unfavorable result of “50% red, 50% sphere” (assuming the data is balanced).

Original: Similar phenomena have also been demonstrated by prior works (Finn et al., 2019;
Revised: Similar phenomena have also been demonstrated by prior works (Finn et al., 2019;

Original: Su et al., 2020) in the context of regression, where a learner simultaneously regressing from multiple target functions trivially outputs their mean.
Revised: Su et al., 2020) in the context of regression, where a learner simultaneously regressing from multiple target functions trivially outputs their mean.

Original: This indicates that conﬂicting data exhibits a structural difference compared with conventional supervised data.
Revised: This indicates that conﬂicting data exhibits a structural difference compared with conventional supervised data.

Original: In the sequel, we introduce a novel, dataset-level measure termed mapping rank to explicitly formalize such difference:
Revised: In the sequel, we introduce a novel, dataset-level measure termed mapping rank to explicitly formalize such difference:

Original: Deﬁnition 1 (Mapping rank) .
Revised: Deﬁnition 1 (Mapping rank) .

Original: Let X be an input space, Y an output space, and Z = { ( x i , y i ) } li = a dataset with cardinality l .
Revised: Let X be an input space, Y an output space, and Z = { ( x i , y i ) } li = a dataset with cardinality l .

Original: Let F ( r ) =
Revised: Let F ( r ) =

Original: { f i } ri =1 be a function set with cardinality r , where each element is a deterministic function from X to Y .
Revised: { f i } ri =1 be a function set with cardinality r , where each element is a deterministic function from X to Y .

Original: Then, the mapping rank of Z , denoted by R or
Revised: Then, the mapping rank of Z , denoted by R or

Original: R ( Z ) , is deﬁned as the minimal positive integer r satisfying that there exists a function set F ( r ) such that for every ( x, y ) ∈ Z , there exists f ∈ F ( r ) with f ( x ) =
Revised: R ( Z ) , is deﬁned as the minimal positive integer r satisfying that there exists a function set F ( r ) such that for every ( x, y ) ∈ Z , there exists f ∈ F ( r ) with f ( x ) =

Original: y .
Revised: y .

Original: Note that we assume that the relation between inputs and outputs in the same domain is deterministic, and we will further discuss this assumption in Section ?? .
Revised: Note that we assume that the relation between inputs and outputs in the same domain is deterministic, and we will further discuss this assumption in Section 6.

Original: Under Deﬁnition 1, conventional supervised data yields a mapping rank R = 1 as it assumes that the whole dataset can be characterized by a single target function.
Revised: Under Deﬁnition 1, conventional supervised data yields a mapping rank R = 1 as it assumes that the whole dataset can be characterized by a single target function.

Original: In contrast, conﬂicting data has a mapping rank R > 1 since for the same input different outputs exist.
Revised: In contrast, conﬂicting data has a mapping rank R > 1 since for the same input different outputs exist.

Original: Hence, it is natural to consider allocating the data to multiple models, so that the data processed by each model has a mapping rank R = 1 and thus can be handled with
Revised: Hence, it is natural to consider allocating the data to multiple models, so that the data processed by each model has a mapping rank R = 1 and thus can be handled with

Original: ERM.
Revised: ERM.

Original: Although in some scenarios, apart from data samples there also exists side-information or metadata that can be exploited to identify the domains, this information may be difﬁcult to deﬁne or collect in practice (Hanna et al., 2020; Creager et al., 2021); even when such side-information is available, in many cases it still remains unclear how to leverage such information to detect and resolve the potential conﬂict between domains.
Revised: Although in some scenarios, apart from data samples there also exists side-information or metadata that can be exploited to identify the domains, this information may be difﬁcult to deﬁne or collect in practice (Hanna et al., 2020; Creager et al., 2021); even when such side-information is available, in many cases it still remains unclear how to leverage such information to detect and resolve the potential conﬂict between domains.

Original: Therefore, the problem of how to allocate conﬂicting data properly and automatically without additional human intervention is highly non-trivial.
Revised: Therefore, the problem of how to allocate conﬂicting data properly and automatically without additional human intervention is highly non-trivial.

Original: To tackle the aforementioned challenge, we present a subjective learning framework to enable effective learning from conﬂicting data.
Revised: To tackle the aforementioned challenge, we present a subjective learning framework to enable effective learning from conﬂicting data.

Original: Concretely, our framework maintains a set of low-level models and a high-level subjective function that automatically allocates the data among these models so that the data processed by each model exhibits no conﬂict.
Revised: Concretely, our framework maintains a set of low-level models and a high-level subjective function that automatically allocates the data among these models so that the data processed by each model exhibits no conﬂict.

Original: Here we use the term “subjective” because in conventional supervised learning such allocation is manually performed during the data cleaning process and thus complies with human subjectivity.
Revised: Here we use the term “subjective” because in conventional supervised learning such allocation is manually performed during the data cleaning process and thus complies with human subjectivity.

Original: The high-level motivation of our method is that if the subjective function yields an inappropriate allocation, i.e., assigning conﬂicting data to the same model, then it will hinder the global minimization of the training error due to the conﬂict, which itself may be harnessed to update the allocation strategy of the subjective function.
Revised: The high-level motivation of our method is that if the subjective function yields an inappropriate allocation, i.e., assigning conﬂicting data to the same model, then it will hinder the global minimization of the training error due to the conﬂict, which itself may be harnessed to update the allocation strategy of the subjective function.

Original: Using a probablistic reinterpretation of our framework, we establish the connection between subjective learning and a variant of the Expectation-Maximization (EM) algorithm
Revised: Using a probablistic reinterpretation of our framework, we establish the connection between subjective learning and a variant of the Expectation-Maximization (EM) algorithm

Original: (Dempster et al., 1977), and show that the form of the subjective function can be explicitly derived.
Revised: (Dempster et al., 1977), and show that the form of the subjective function can be explicitly derived.

Original: Theoretically, we respectively analyse the Probably Approximately Correct (PAC) learnability (Valiant, 1984) and the generalization error of subjective learning using the tools from statistical learning theory (Vapnik, 2013).
Revised: Theoretically, we respectively analyse the Probably Approximately Correct (PAC) learnability (Valiant, 1984) and the generalization error of subjective learning using the tools from statistical learning theory (Vapnik, 2013).

Original: We show that the relation between the number of low-level models and the mapping rank of data plays a key role in the learnability of subjective learning, and the generalization error of our method can be decomposed into terms that respectively reﬂect high-level data allocation and low-level prediction errors.
Revised: We show that the relation between the number of low-level models and the mapping rank of data plays a key role in the learnability of subjective learning, and the generalization error of our method can be decomposed into terms that respectively reﬂect high-level data allocation and low-level prediction errors.

Original: Empirically, we conduct extensive experiments that span regression and classiﬁcation tasks with synthetic data.
Revised: Empirically, we conduct extensive experiments that span regression and classiﬁcation tasks with synthetic data.

Original: Experimental results validate our theoretical claims, demonstrate the efﬁcacy of subjective learning, and showcase its potential applicability in several different scenarios.
Revised: Experimental results validate our theoretical claims, demonstrate the efﬁcacy of subjective learning, and showcase its potential applicability in several different scenarios.

Original:  S UBJECTIVE L EARNING F RAMEWORK
Revised:  S UBJECTIVE L EARNING F RAMEWORK

Original: In this section, we present the overall formulation and the algorithm of subjective learning.
Revised: In this section, we present the overall formulation and the algorithm of subjective learning.

Original: We adhere to the conventional terminology in supervised learning, and let X be an input space, Y an output space, H a hypothesis space where each hypothesis (model) is a function from X to Y , and (cid:96) :
Revised: We adhere to the conventional terminology in supervised learning, and let X be an input space, Y an output space, H a hypothesis space where each hypothesis (model) is a function from X to Y , and (cid:96) :

Original: Y × Y → [0 , 1] a non-negative and bounded loss function without loss of generality.
Revised: Y × Y → [0 , 1] a non-negative and bounded loss function without loss of generality.

Original: We use [ k ] = { 1 , 2 , · · · , k } for positive integers k , and denote by 1 ( · ) the indicator function.
Revised: We use [ k ] = { 1 , 2 , · · · , k } for positive integers k , and denote by 1 ( · ) the indicator function.

Original: We use superscripts to denote sampling indices (e.g., d i and x ij ) and subscripts as element indices (e.g., d i ).
Revised: We use superscripts to denote sampling indices (e.g., d i and x ij ) and subscripts as element indices (e.g., d i ).

Original: 2. P ROBLEM S TATEMENT
Revised: 2. P ROBLEM S TATEMENT

Original: We begin by introducing the notion of domain to formulate the sampling process of conﬂicting data that is considered throughout the paper.
Revised: We begin by introducing the notion of domain to formulate the sampling process of conﬂicting data that is considered throughout the paper.

Original: Inspired by the seminal work in domain adpatation (BenDavid et al., 2010), we deﬁne a domain d as a pair (cid:104) P, c (cid:105) consisting of a distribution P on X and a target function c : X → Y , and assume that the data is generated from a domain set D
Revised: Inspired by the seminal work in domain adpatation (BenDavid et al., 2010), we deﬁne a domain d as a pair (cid:104) P, c (cid:105) consisting of a distribution P on X and a target function c : X → Y , and assume that the data is generated from a domain set D

Original: = { d i } Ni =1 = {(cid:104) P
Revised: = { d i } Ni =1 = {(cid:104) P

Original: i , c i (cid:105)} Ni =1 containing N (agnostic to the learner) domains.
Revised: i , c i (cid:105)} Ni =1 containing N (agnostic to the learner) domains.

Original: Each domain has its own sub-dataset Z i = { ( x ij , y ij ) } l i j =1 with cardinality l i , where x i 1 , · · · , x il i are i.i.d.
Revised: Each domain has its own sub-dataset Z i = { ( x ij , y ij ) } l i j =1 with cardinality l i , where x i 1 , · · · , x il i are i.i.d.

Original: drawn from
Revised: drawn from

Original: P i and y ij =
Revised: P i and y ij =

Original: c i ( x ij ) .
Revised: c i ( x ij ) .

Original: The whole dataset is the union of these sub-datasets: Z = (cid:83) Ni
Revised: The whole dataset is the union of these sub-datasets: Z = (cid:83) Ni

Original: =

Revised: =

Original: Z i with cardinality l
Revised: Z i with cardinality l

Original: = (cid:80) Ni =
Revised: = (cid:80) Ni =

Original: 1 l i and mapping rank 1 < R ≤ N .
Revised: 1 l i and mapping rank 1 < R ≤ N .

Original: We consider a bilevel sampling procedure: ﬁrst, m domain samples are i.i.d.
Revised: We consider a bilevel sampling procedure: ﬁrst, m domain samples are i.i.d.

Original: drawn from a distribution Q deﬁned on D with replacement (thus the same domain may be sampled multiple times), resulting in m sampling episodes ; second, in each sampling episode n data samples are i.i.d.
Revised: drawn from a distribution Q deﬁned on D with replacement (thus the same domain may be sampled multiple times), resulting in m sampling episodes ; second, in each sampling episode n data samples are i.i.d.

Original: drawn from the sub-dataset corresponding to the sampled domain.
Revised: drawn from the sub-dataset corresponding to the sampled domain.

Original: This sampling regime is analagous to the bilevel sampling process adopted by meta-learning (Pentina & Lampert, 2014; Amit & Meir, 2018).
Revised: This sampling regime is analagous to the bilevel sampling process adopted by meta-learning (Pentina & Lampert, 2014; Amit & Meir, 2018).

Original: However, meta-learning usually assumes a dense distribution of related domains to enable task-level generalization, while our setting here is compatible with scarce and disparate domains and inter-domain transfer is orthogonal.
Revised: However, meta-learning usually assumes a dense distribution of related domains to enable task-level generalization, while our setting here is compatible with scarce and disparate domains and inter-domain transfer is orthogonal.

Original: As we have mentioned in Section 1, single-model training is insufﬁcient when R > 1 .
Revised: As we have mentioned in Section 1, single-model training is insufﬁcient when R > 1 .

Original: Thus, we equip the learner with a hypothesis set H
Revised: Thus, we equip the learner with a hypothesis set H

Original: = { h i } K i =
Revised: = { h i } Ki =

Original: 1 consisting of K > 1 hypotheses, enhancing its expressive capability.
Revised: 1 consisting of K > 1 hypotheses, enhancing its expressive capability.

Original: Although both N and K are assumed unknown, we will show that in general
Revised: Although both N and K are assumed unknown, we will show that in general

Original: K ≥ R sufﬁces (see Section 3.1), which eases the difﬁculty of setting the hyperparameter K .
Revised: K ≥ R sufﬁces (see Section 3.1), which eases the difﬁculty of setting the hyperparameter K .

Original: In the above setting, we introduce an episodic sample number parameter n , implicitly assuming that we are able to sample a size- n data batch at a time from each domain.
Revised: In the above setting, we introduce an episodic sample number parameter n , implicitly assuming that we are able to sample a size- n data batch at a time from each domain.

Original: While this formulation subsumes the fully online case of n = 1 , we note that although sometimes n = 1 works in practice, it also tends to be risky since it may raise difﬁculties in controlling the generalization error, as we will both theoretically and empirically demonstrate in the following sections (see Sections 3.2 and 4.3).
Revised: While this formulation subsumes the fully online case of n = 1 , we note that although sometimes n = 1 works in practice, it also tends to be risky since it may raise difﬁculties in controlling the generalization error, as we will both theoretically and empirically demonstrate in the following sections (see Sections 3.2 and 4.3).

Original: 2. G LOBAL E RROR
Revised: G LOBAL E RROR

Original: In this section, we present the global learning objective of subjective learning.
Revised: In this section, we present the global learning objective of subjective learning.

Original: Since conﬂicting data implicitly contains multiple input-output mappings, a primary start point can be the empirical multi-task loss with pre-deﬁned data-domain correspondences:
Revised: Since conﬂicting data implicitly contains multiple input-output mappings, a primary start point can be the empirical multi-task loss with pre-deﬁned data-domain correspondences:

Original: [Equation]
Revised: [Equation]

Original: where O RACLE : [ m ] → [ K ] is an oracle mapping function that determines which hypothesis the data batch Z i :=
Revised: where O RACLE : [ m ] → [ K ] is an oracle mapping function that determines which hypothesis the data batch Z i :=

Original: { ( x ij , y ij ) } nj =1 in each episode should be assigned to.
Revised: { ( x ij , y ij ) } nj =1 in each episode should be assigned to.

Original: However, in subjective learning the oracle mapping function is unavailable, imposing a fundamental discrepancy.
Revised: However, in subjective learning the oracle mapping function is unavailable, imposing a fundamental discrepancy.

Original: To tackle this difﬁculty, here we substitute the oracle mapping function with a learnable empirical subjective function ˆ g : H K × X n × Y n → H that aims to select a hypothesis h from the hypothesis set H for the data batch Z i .
Revised: To tackle this difﬁculty, here we substitute the oracle mapping function with a learnable empirical subjective function ˆ g : H K × X n × Y n → H that aims to select a hypothesis h from the hypothesis set H for the data batch Z i .

Original: This substitution yields the empirical global error of subjective learning:
Revised: This substitution yields the empirical global error of subjective learning:

Original: [Equation]
Revised: [Equation]

Original: Our insight is that the data batch itself can be harnessed to guide its suitable allocation in the presence of conﬂict: intuitively, a single model trained by conﬂicting data batches results in an inevitable global training error.
Revised: Our insight is that the data batch itself can be harnessed to guide its suitable allocation in the presence of conﬂict: intuitively, a single model trained by conﬂicting data batches results in an inevitable global training error.

Original: Therefore, minimizing the global error can in turn facilitates data allocation with less conﬂict.
Revised: Therefore, minimizing the global error can in turn facilitates data allocation with less conﬂict.

Original: Given the empirical error 2, we can give its expected counterpart:
Revised: Given the empirical error 2, we can give its expected counterpart:

Original: [Equation]
Revised: [Equation]

Original: where g :
Revised: where g :

Original: H K × D
Revised: H K × D

Original: → H is an expected subjective function , which can be viewed as the empirical subjective function with inﬁnite data from every single domain so that all available domain information can be fully reﬂected by the sampled data.
Revised: → H is an expected subjective function , which can be viewed as the empirical subjective function with inﬁnite data from every single domain so that all available domain information can be fully reﬂected by the sampled data.

Original: So far, our framework remains incomplete since the exact form of the subjective function is still undeﬁned.
Revised: So far, our framework remains incomplete since the exact form of the subjective function is still undeﬁned.

Original: In the next section, we will present our design of the subjective function and elucidate its rationale.
Revised: In the next section, we will present our design of the subjective function and elucidate its rationale.

Original: 2. D ERIVATION OF S UBJECTIVE F UNCTION
Revised: 2. D ERIVATION OF S UBJECTIVE F UNCTION

Original: To attain a reasonable choice of the subjective function, in this section we provide an alternative, probabilistic reinterpretation of subjective learning from the angle of maximum conditional likelihood, and draw an intriguing connection between the choice of the subjective function and the posterior maximization in a variant of the EM algorithm (Dempster et al., 1977) on conﬂicting data.
Revised: To attain a reasonable choice of the subjective function, in this section we provide an alternative, probabilistic reinterpretation of subjective learning from the angle of maximum conditional likelihood, and draw an intriguing connection between the choice of the subjective function and the posterior maximization in a variant of the EM algorithm (Dempster et al., 1977) on conﬂicting data.

Original: Concretely, let p ( Y | X ) represents the predictive conditional distribution of the hypothesis set H , where we use X and Y as the shorthand for ( x 11 , x 12 , · · · , x mn ) and ( y 11 , y 12 , · · · , y mn ) respectively.
Revised: Concretely, let p ( Y | X ) represents the predictive conditional distribution of the hypothesis set H , where we use X and Y as the shorthand for ( x 11 , x 12 , · · · , x mn ) and ( y 11 , y 12 , · · · , y mn ) respectively.

Original: We consider maximizing the empirical log-likelihood log p
Revised: We consider maximizing the empirical log-likelihood log p

Original: ( Y | [Equation]
Revised: ( Y | X )

Original: ) = (cid:80) [Equation] log (cid:80)Kk
Revised: = (cid:80) mi =1 (cid:80) nj =1 log (cid:80)

Original:  =1 p ( y ij , h = h k | x ij ) using EM, where h denotes the selected hypothesis.
Revised: Kk =1 p ( y ij , h = h k | x ij ) using EM, where h denotes the selected hypothesis.

Original: Accordingly, in the i -th sampling episode, in the E-step we aim to estimate the model posterior P ( h = h
Revised: Accordingly, in the i -th sampling episode, in the E-step we aim to estimate the model posterior P ( h = h

Original: k
Revised: k

Original: | Z i ) that represents the responsibility of the k -th hypothesis in the hypothesis set w.r.t.
Revised: | Z i ) that represents the responsibility of the k -th hypothesis in the hypothesis set w.r.t.

Original: the local data batch Z i = { ( x ij , y ij ) } nj =1 , while in the M-step we seek to maximize
Revised: the local data batch Z i = { ( x ij , y ij ) } nj =1 , while in the M-step we seek to maximize

Original: [Equation]
Revised: [Equation]

Original: where π k :=
Revised: where π k :=

Original: P ( h = h k ) > 0 denotes the prior of the k -th hypothesis in H .
Revised: P ( h = h k ) > 0 denotes the prior of the k -th hypothesis in H .

Original: This draws a direct connection between subjective learning and EM: the E-step corresponds to the functionality of the subjective function that chooses a hypothesis for a given data batch; the M-step corresponds to updating the selected hypothesis by minimizing the empirical prediction error in the episode.
Revised: This draws a direct connection between subjective learning and EM: the E-step corresponds to the functionality of the subjective function that chooses a hypothesis for a given data batch; the M-step corresponds to updating the selected hypothesis by minimizing the empirical prediction error in the episode.

Original: The main difference is that here we assume the subjective function to be deterministic , representing a “hard” assignment of the data to the model.
Revised: The main difference is that here we assume the subjective function to be deterministic , representing a “hard” assignment of the data to the model.

Original: This entails the usage of a variant known as hard
Revised: This entails the usage of a variant known as hard

Original: EM (Samdani et al., 2012), which considers the posterior to be a Dirac delta function.
Revised: EM (Samdani et al., 2012), which considers the posterior to be a Dirac delta function.

Original: Applying this constraint, the E-step of EM yields h = arg max h k ∈ H (cid:80) nj =1 log p ( y ij | x ij , h k ) under a uniform prior, motivating a principled choice of the subjective function:
Revised: Applying this constraint, the E-step of EM yields h = arg max h k ∈ H [Equation] nj =1 log p ( y ij | x ij , h k ) under a uniform prior, motivating a principled choice of the subjective function:

Original: [Equation]
Revised: [Equation]

Original: which can be interpreted as selecting the hypothesis that incurs the smallest (empirical or expected) error.
Revised: which can be interpreted as selecting the hypothesis that incurs the smallest (empirical or expected) error.

Original: While this connection is not rigorous in general, in some cases exact equivalence can be derived when certain types of loss functions and likelihood families are applied, which encompasses common regression and classiﬁcation settings.
Revised: While this connection is not rigorous in general, in some cases exact equivalence can be derived when certain types of loss functions and likelihood families are applied, which encompasses common regression and classiﬁcation settings.

Original: We provide a concrete analysis in Appendix A.
Revised: We provide a concrete analysis in Appendix A.

Original:  O VERALL A LGORITHM
Revised: 2. O VERALL A LGORITHM

Original: We assume that the hypothesis set H comprises K parameterized hypotheses with parameter vectors Θ
Revised: We assume that the hypothesis set H comprises K parameterized hypotheses with parameter vectors Θ

Original: = ( θ 1 , θ 2 , · · · , θ K ) respectively.
Revised: = ( θ 1 , θ 2 , · · · , θ K ) respectively.

Original: In the high level, with the choice of the empirical subjective function 5a, our algorithm consists of two phases in each sampling episode: (i) evaluating the error of each hypothesis in H w.r.t.
Revised: In the high level, with the choice of the empirical subjective function 5a, our algorithm consists of two phases in each sampling episode: (i) evaluating the error of each hypothesis in H w.r.t.

Original: the data in this episode, and (ii) training the hypothesis with the smallest error.
Revised: the data in this episode, and (ii) training the hypothesis with the smallest error.

Original: For brevity, we introduce a notion of empirical episodic error deﬁned as
Revised: For brevity, we introduce a notion of empirical episodic error deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where h ∈ H is a hypothesis parameterized by θ .
Revised: where h ∈ H is a hypothesis parameterized by θ .

Original: Then, phase (i) aims to ﬁnd a hypothesis that mininize 6.
Revised: Then, phase (i) aims to ﬁnd a hypothesis that mininize 6.

Original: Note that this selection process may induce a bias between empirical and expected objectives 2 and 3, since a hypothesis that minimizes the empirical loss on ﬁnite samples may not minimize the expected loss of this domain.
Revised: Note that this selection process may induce a bias between empirical and expected objectives 2 and 3, since a hypothesis that minimizes the empirical loss on ﬁnite samples may not minimize the expected loss of this domain.

Original: Hence, the global error of subjective learning can be intuitively decomposed into a high-level subjective error that measures the reliability of this selection process, and a low-level model error that measures the accuracy of models, on which we provide detailed theoretical analysis in Section 3.2.
Revised: Hence, the global error of subjective learning can be intuitively decomposed into a high-level subjective error that measures the reliability of this selection process, and a low-level model error that measures the accuracy of models, on which we provide detailed theoretical analysis in Section 3.2.

Original: In practice, we parameterize each hypothesis in the hypothesis set with a deep neural network (DNN), and apply stochastic gradient descent (SGD) for the optimization process.
Revised: In practice, we parameterize each hypothesis in the hypothesis set with a deep neural network (DNN), and apply stochastic gradient descent (SGD) for the optimization process.

Original: We provide the pseudo-code of subjective learning in Appendix B.
Revised: We provide the pseudo-code of subjective learning in Appendix B.

Original:  T HEORETICAL A NALYSIS
Revised:  T HEORETICAL A NALYSIS

Original: In this section, we analyze the learnability and the generalization error of subjective learning.
Revised: In this section, we analyze the learnability and the generalization error of subjective learning.

Original: 3. L EARNABILITY
Revised: 3. L EARNABILITY

Original: We ﬁrst analyze the learnability of subjective learning based on PAC learnability (Valiant, 1984).
Revised: We ﬁrst analyze the learnability of subjective learning based on PAC learnability (Valiant, 1984).

Original: Since our analysis directly applys to conventional supervised learning by setting K = 1 , we also verify the conﬂict phenomenon mentioned in Section 1 from a theoretical perspective.
Revised: Since our analysis directly applys to conventional supervised learning by setting K = 1 , we also verify the conﬂict phenomenon mentioned in Section 1 from a theoretical perspective.

Original: While the learnability in conventional PAC analysis mainly relates to the choice of the hypothesis space, conﬂicting data imposes a new source of complexity by its mapping rank, and we expect the cardinality of the proposed hypothesis set can compensate this complexity.
Revised: While the learnability in conventional PAC analysis mainly relates to the choice of the hypothesis space, conﬂicting data imposes a new source of complexity by its mapping rank, and we expect the cardinality of the proposed hypothesis set can compensate this complexity.

Original: We consider the realizable case where the hypothesis space covers the target functions in all domains, which helps to underline the core characteristic of our problem.
Revised: We consider the realizable case where the hypothesis space covers the target functions in all domains, which helps to underline the core characteristic of our problem.

Original: We begin by a result on the form of the optimal solution of subjective learning.
Revised: We begin by a result on the form of the optimal solution of subjective learning.

Original: The proofs of all theoretical results are deferred to Appendix C.
Revised: The proofs of all theoretical results are deferred to Appendix C.

Original: Proposition 1 (Form of the optimal solution) .
Revised: Proposition 1 (Form of the optimal solution) .

Original: Assume that the target functions in all domains are realizable.
Revised: Assume that the target functions in all domains are realizable.

Original: Then, the following two propositions are equivalent:
Revised: Then, the following two propositions are equivalent:

Original: (1) For all domain distributions Q and data distributions P 1 , P 2 , · · · , P N , er ( H )
Revised: (1) For all domain distributions Q and data distributions P 1 , P 2 , · · · , P N , er ( H )

Original: = 0 .
Revised: = 0 .

Original: (2) For each domain d = (cid:104) P, c (cid:105) in D , there exists h ∈ H such that E x ∼ P (cid:96)
Revised: (2) For each domain d = (cid:104) P, c (cid:105) in D , there exists h ∈ H such that E x ∼ P (cid:96)

Original: [ h ( x ) , c ( x )]
Revised: [ h ( x ) , c ( x )]

Original: = 0 .
Revised: = 0 .

Original: Proposition 1 suggests that minimizing the expected global error 3 with 5b elicits a global optimal solution where every target function is learned accurately.
Revised: Proposition 1 suggests that minimizing the expected global error 3 with 5b elicits a global optimal solution where every target function is learned accurately.

Original: Note that this does not require N hypotheses for N domains, since non-conﬂict domains can be incorporated into the same model.
Revised: Note that this does not require N hypotheses for N domains, since non-conﬂict domains can be incorporated into the same model.

Original: In other words, what determines the minimal cardinality of the hypothesis set is not the number of the domains, but the number of conﬂicting domains, which can be exactly characterized by the mapping rank.
Revised: In other words, what determines the minimal cardinality of the hypothesis set is not the number of the domains, but the number of conﬂicting domains, which can be exactly characterized by the mapping rank.

Original: Formally, we attain a necessary condition of the PAC learnability of subjective learning.
Revised: Formally, we attain a necessary condition of the PAC learnability of subjective learning.

Original: Theorem 1.
Revised: Theorem 1.

Original: A necessary condition of the PAC learnability of subjective learning is K ≥ R .
Revised: A necessary condition of the PAC learnability of subjective learning is K ≥ R .

Original: Theorem 1 indicates that the cardinality of the hypothesis set should be large enough to enable effective learning, and shows the impact of mapping rank on the learnability of subjective learning.
Revised: Theorem 1 indicates that the cardinality of the hypothesis set should be large enough to enable effective learning, and shows the impact of mapping rank on the learnability of subjective learning.

Original: Remark 1.
Revised: Remark 1.

Original: While it is generally hard to derive a necessary and sufﬁcient condition of PAC learnability theoretically (which requires a sample-efﬁcient optimization algorithm), we empirically ﬁnd that K ≥ R is indeed an essential condition for learnability with complex hypothesis spaces such as parameterized DNNs.
Revised: While it is generally hard to derive a necessary and sufﬁcient condition of PAC learnability theoretically (which requires a sample-efﬁcient optimization algorithm), we empirically ﬁnd that K ≥ R is indeed an essential condition for learnability with complex hypothesis spaces such as parameterized DNNs.

Original: We also note that several recent works (Allen-Zhu et al., 2019; Du et al., 2019) have proved that over-parameterized neural networks trained by SGD can achieve zero training error in polynomial time under non-convexity, which may also be used to enhance our analysis.
Revised: We also note that several recent works (Allen-Zhu et al., 2019; Du et al., 2019) have proved that over-parameterized neural networks trained by SGD can achieve zero training error in polynomial time under non-convexity, which may also be used to enhance our analysis.

Original: We leave a more rigorous study for future work.
Revised: We leave a more rigorous study for future work.

Original: 3. G ENERALIZATION E RROR
Revised: 3. G ENERALIZATION E RROR

Original: We have shown that minimizing the expected global error is sufﬁcient for effective learning from conﬂicting data.
Revised: We have shown that minimizing the expected global error is sufﬁcient for effective learning from conﬂicting data.

Original: However, in practice, since we only have access to the empirical global error, how to control the discrepancy between these two errors, i.e., the generalization error, remains crucial.
Revised: However, in practice, since we only have access to the empirical global error, how to control the discrepancy between these two errors, i.e., the generalization error, remains crucial.

Original: In this section, we identify the terms in the generalization error that respectively correspond to the high-level subjective error and the low-level prediction error of subjective learning, and discuss their controlling strategies.
Revised: In this section, we identify the terms in the generalization error that respectively correspond to the high-level subjective error and the low-level prediction error of subjective learning, and discuss their controlling strategies.

Original: The key results are (i) the number of episodes and episodic samples can compensate each other in controlling the low-level prediction error, and (ii) the number of episodic samples is critical for controlling the high-level subjective error.
Revised: The key results are (i) the number of episodes and episodic samples can compensate each other in controlling the low-level prediction error, and (ii) the number of episodic samples is critical for controlling the high-level subjective error.

Original: We have the following theorem:
Revised: We have the following theorem:

Original: Theorem 2 (Generalization error upper bound) .
Revised: Theorem 2 (Generalization error upper bound) .

Original: For any δ ∈ (0 , 1] , the following inequality holds uniformly for all hypothesis sets H ∈ H K with probability at least 1 − δ :
Revised: For any δ ∈ (0 , 1] , the following inequality holds uniformly for all hypothesis sets H ∈ H K with probability at least 1 − δ :

Original: [Equation]
Revised: [Equation]

Original: where ¯ S :
Revised: where ¯ S :

Original: = {(cid:104) P, c (cid:105) (cid:55)→ E x ∼ P (cid:96)
Revised: = {(cid:104) P, c (cid:105) (cid:55)→ E x ∼ P (cid:96)

Original: [ h ( x ; θ ) , c ( x )] } , θ ∈ Θ is the function set of the domain-wise expected error, S :
Revised: [ h ( x ; θ ) , c ( x )] } , θ ∈ Θ is the function set of the domain-wise expected error, S :

Original: = { ( x, y ) (cid:55)→ (cid:96) [ h ( x ; θ ) , y ] } , θ ∈ Θ is the function set of the sample-wise error, m k := (cid:80) mi =1 1 (cid:0) c
Revised: = { ( x, y ) (cid:55)→ (cid:96) [ h ( x ; θ ) , y ] } , θ ∈ Θ is the function set of the sample-wise error, m k := (cid:80) mi =1 1 (cid:0) c

Original: i =
Revised: i =

Original: c k (cid:1) is the sampling count of the target function from the k -th domain d k ( k ∈ [ N ]) , and VC ( · ) the Vapnik-Chervonenkis (VC) dimension (Vapnik & Chervonenkis, 1971). 
Revised: c k (cid:1) is the sampling count of the target function from the k -th domain d k ( k ∈ [ N ]) , and VC ( · ) the Vapnik-Chervonenkis (VC) dimension (Vapnik & Chervonenkis, 1971). [Figure]

Original: Theorem 2 indicates that the expected global error is bounded by the empirical global error plus three terms.
Revised: Theorem 2 indicates that the expected global error is bounded by the empirical global error plus three terms.

Original: The subjective estimation error term 7c is derived by bounding the discrepancy between the empirical and the expected subjective functions due to the limitation of ﬁnite episodic samples.
Revised: The subjective estimation error term 7c is derived by bounding the discrepancy between the empirical and the expected subjective functions due to the limitation of ﬁnite episodic samples.

Original: This term can be controlled by sample-level complexity VC ( S ) and the number of episodic samples n .
Revised: This term can be controlled by sample-level complexity VC ( S ) and the number of episodic samples n .

Original: Although in theory this error term converges to zero only if n → ∞ , in practice we ﬁnd that usually a very small n (e.g., n = 2 ) sufﬁces (see Section 4).
Revised: Although in theory this error term converges to zero only if n → ∞ , in practice we ﬁnd that usually a very small n (e.g., n = 2 ) sufﬁces (see Section 4).

Original: We posit that this is because the domains in our experiments are relatively diverse, thus reducing the difﬁculty of discriminating between different domains.
Revised: We posit that this is because the domains in our experiments are relatively diverse, thus reducing the difﬁculty of discriminating between different domains.

Original: The domain estimation error term 7a contains domain-level complexity VC ( ¯ S ) , and converges to zero if the number of episodes m → ∞ ; the instance estimation error term 7b contains sample-level complexity VC ( S ) , and converges to zero if the sample number in each episode or the number of episodes reaches inﬁnity ( n → ∞ or m → ∞ ), showing the synergy between high-level domain samples and low-level data samples in controlling the model-wise generalization error.
Revised: The domain estimation error term 7a contains domain-level complexity VC ( ¯ S ) , and converges to zero if the number of episodes m → ∞ ; the instance estimation error term 7b contains sample-level complexity VC ( S ) , and converges to zero if the sample number in each episode or the number of episodes reaches inﬁnity ( n → ∞ or m → ∞ ), showing the synergy between high-level domain samples and low-level data samples in controlling the model-wise generalization error.

Original: Comparison with existing bounds.
Revised: Comparison with existing bounds.

Original: We compare our bound 7 with existing bounds of conventional supervised learning (Vapnik, 2013; McAllester, 1999) and meta-learning (Pentina & Lampert, 2014;[Figure]
Revised: We compare our bound 7 with existing bounds of conventional supervised learning (Vapnik, 2013; McAllester, 1999) and meta-learning (Pentina & Lampert, 2014;

Original: Amit & Meir, 2018).
Revised: Amit & Meir, 2018).

Original: Typically, supervised learning bounds contain a instance-level complexity term as 7b, and meta-learning bounds further contain a task-level complexity term as 7a.
Revised: Typically, supervised learning bounds contain a instance-level complexity term as 7b, and meta-learning bounds further contain a task-level complexity term as 7a.

Original: Yet, conventional supervised learning only considers a single domain or multiple known domains, while metalearning treats each episode as a new domain rather than domains that may have been encountered as in subjective learning.
Revised: Yet, conventional supervised learning only considers a single domain or multiple known domains, while metalearning treats each episode as a new domain rather than domains that may have been encountered as in subjective learning.

Original: Thus, none of these bounds contain an explicit inference term as 7c.
Revised: Thus, none of these bounds contain an explicit inference term as 7c.

Original: Remark 2.
Revised: Remark 2.

Original: While our bound applies VC dimension as the complexity measure, extensions to other data-dependant complexity measures such as Rademacher and Gaussian complexities (Bartlett &
Revised: While our bound applies VC dimension as the complexity measure, extensions to other data-dependant complexity measures such as Rademacher and Gaussian complexities (Bartlett &

Original: Mendelson, 2002; Koltchinskii & Panchenko, 2000) is straightforward.
Revised: Mendelson, 2002; Koltchinskii & Panchenko, 2000) is straightforward.

Original: It is worth noting that the bounds based on these measures share the same asymptotic property w.r.t.
Revised: It is worth noting that the bounds based on these measures share the same asymptotic property w.r.t.

Original: m and n as in bound 7.
Revised: m and n as in bound 7.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we report experimental results on two basic supervised learning tasks with conﬂicting data: regression and classiﬁcation.
Revised: In this section, we report experimental results on two basic supervised learning tasks with conﬂicting data: regression and classiﬁcation.

Original: Our experiments are designed to (i) validate our theoretical claims, (ii) assess the effectiveness of subjective learning and show its potential applicability in different settings, and (iii) compare subjective learning with task-speciﬁc baselines.
Revised: Our experiments are designed to (i) validate our theoretical claims, (ii) assess the effectiveness of subjective learning and show its potential applicability in different settings, and (iii) compare subjective learning with task-speciﬁc baselines.

Original: 4. S ETUP AND B ASELINES
Revised: 4. S ETUP AND B ASELINES

Original: Regression.
Revised: Regression.

Original: We consider a regression task in which data points are simultaneously sampled from three heterogeneous functions, as shown in Figure 2a (solid lines).
Revised: We consider a regression task in which data points are simultaneously sampled from three heterogeneous functions, as shown in Figure 2a (solid lines).

Original: We compare subjective learning with three baselines: (1) Vanilla : a conventional ERM-based learner.
Revised: We compare subjective learning with three baselines: (1) Vanilla : a conventional ERM-based learner.

Original: (2) MAML (Finn et al., 2017): a popular gradient-based meta-learning approach.
Revised: (2) MAML (Finn et al., 2017): a popular gradient-based meta-learning approach.

Original: (3) Modular (Alet et al., 2018): a modular metalearning approach that extends MAML using multiple modules.
Revised: (3) Modular (Alet et al., 2018): a modular metalearning approach that extends MAML using multiple modules.

Original: We set the hyperparameters of subjective learning to K = 3 , m = 250 and n = 2 .
Revised: We set the hyperparameters of subjective learning to K = 3 , m = 250 and n = 2 .

Original: To verify our theoretical results, we also run subjective learning with different number of hypotheses ( K = 2 and K = 4 ) and different sampling hyperparameters ( m = 50 , n = 2 and m = 250 , n = 1 ).
Revised: To verify our theoretical results, we also run subjective learning with different number of hypotheses ( K = 2 and K = 4 ) and different sampling hyperparameters ( m = 50 , n = 2 and m = 250 , n = 1 ).

Original: More details on the task and the baselines are in Appendix D.
Revised: More details on the task and the baselines are in Appendix D.

Original: In addition, we demonstrate the effectiveness of subjective learning on a real-world multi-dimensional regression task; details and results are in Appendix E.1.
Revised: In addition, we demonstrate the effectiveness of subjective learning on a real-world multidimensional regression task; details and results are in Appendix E.1.

Original: Classiﬁcation.
Revised: Classiﬁcation.

Original: We consider two types of image recognition tasks where the same image may correspond to different labels in different sample pairs.
Revised: We consider two types of image recognition tasks where the same image may correspond to different labels in different sample pairs.

Original: We refer to these tasks according to the structure of their label spaces, namely parallel and hierarchical tasks.
Revised: We refer to these tasks according to the structure of their label spaces, namely parallel and hierarchical tasks.

Original: For parallel tasks, we derive the data  respectively from Colored MNIST , a variant of MNIST where each digit is assigned with a digit label and a color label, and Fashion Product Images (Aggarwal, 2019), a multi-attribute clothes dataset that involves 3 main parallel tasks including gender, category and color classiﬁcation, as shown in
Revised: For parallel tasks, we derive the data [Figure] [Figure] respectively from Colored MNIST , a variant of MNIST where each digit is assigned with a digit label and a color label, and Fashion Product Images (Aggarwal, 2019), a multi-attribute clothes dataset that involves 3 main parallel tasks including gender, category and color classiﬁcation, as shown in

Original: Figure 1a; we construct conﬂicting datasets by randomly choosing one label from the label set for each image.
Revised: Figure 1a; we construct conﬂicting datasets by randomly choosing one label from the label set for each image.

Original: For the hierarchical task, we derive the data from CIFAR-100 (Krizhevsky & Hinton, 2009), a widely-used image recognition dataset comprising 100 classes with “ﬁne” labels subsumed by 20 superclasses with “coarse” labels, as shown in Figure 1b; we construct the conﬂicting dataset by randomly using the ﬁne or the coarse label for each image.
Revised: For the hierarchical task, we derive the data from CIFAR-100 (Krizhevsky & Hinton, 2009), a widely-used image recognition dataset comprising 100 classes with “ﬁne” labels subsumed by 20 superclasses with “coarse” labels, as shown in Figure 1b; we construct the conﬂicting dataset by randomly using the ﬁne or the coarse label for each image.

Original: In classiﬁcation, the most relevant problem setting to subjective learning is multi-label learning (Zhang & Zhou, 2014), which considers the scenario where an input x is related with a label set y =
Revised: In classiﬁcation, the most relevant problem setting to subjective learning is multi-label learning (Zhang & Zhou, 2014), which considers the scenario where an input x is related with a label set y =

Original: { y i } N i =1 .
Revised: { y i } Ni =1 .

Original: The key difference is that multi-label learning requires that all labels in the label set are provided simaltaneously , while in subjective learning each data sample only contains one label y i ∈
Revised: The key difference is that multi-label learning requires that all labels in the label set are provided simaltaneously , while in subjective learning each data sample only contains one label y i ∈

Original: y .
Revised: y .

Original: Therefore, the setting of subjective learning can be alternatively modeled as “multi-label learning with missing labels”, i.e., in each sample N − 1 labels in y are missing and only one label remains (note that this is a very extreme setting).
Revised: Therefore, the setting of subjective learning can be alternatively modeled as “multi-label learning with missing labels”, i.e., in each sample N − 1 labels in y are missing and only one label remains (note that this is a very extreme setting).

Original: Therefore, we compare subjective learning with the following baselines.
Revised: Therefore, we compare subjective learning with the following baselines.

Original: Probabilistic concepts (ProbCon) (Devroye et al., 1996): this baseline directly models the relation[Figure] [Figure] between inputs and outputs using an conditional probability distribution and choose top- N labels as ﬁnal prediction results.
Revised: Probabilistic concepts (ProbCon) (Devroye et al., 1996): this baseline directly models the relation between inputs and outputs using an conditional probability distribution and choose top- N labels as ﬁnal prediction results.

Original: (2) Semi-supervised multi-label learning : this class of methods model classiﬁcation as the problem of “multi-label learning with missing labels” as discussed above and we compare subjective learning with two representative methods: Pseudo-label (Pseudo-L) (Lee, 2013) and Label propagation (LabelProp) (Iscen et al., 2019).
Revised: (2) Semi-supervised multi-label learning : this class of methods model classiﬁcation as the problem of “multi-label learning with missing labels” as discussed above, and we compare subjective learning with two representative methods: Pseudo-label (Pseudo-L) (Lee, 2013) and Label propagation (LabelProp) (Iscen et al., 2019).

Original: In addition, we introduce two oracle baselines as ablations: (3) Full labels (Full-L) : a standard multi-label learning method where we provide the full label set for each image, hence there is no missing labels.
Revised: In addition, we introduce two oracle baselines as ablations: (3) Full labels (Full-L) : a standard multi-label learning method where we provide the full label set for each image, hence there is no missing labels.

Original: (4) Full tasks (Full-T) : a standard multi-task learning method where the “task” of each image is designated by human experts in advance to ensure that there is no conﬂict within each task.
Revised: (4) Full tasks (Full-T) : a standard multi-task learning method where the “task” of each image is designated by human experts in advance to ensure that there is no conﬂict within each task.

Original: More details on baselines can be found in Appendix D. To further demonstrate the applicability of subjective learning, we also conducted an experiment on Fashion Product Images with simulated concept shift between domains with the same label space; details and results are in Appendix E.
Revised: More details on baselines can be found in Appendix D. To further demonstrate the applicability of subjective learning, we also conducted an experiment on Fashion Product Images with simulated concept shift between domains with the same label space; details and results are in Appendix E.

Original: 4. E VALUATION M ETRICS
Revised: 4. E VALUATION M ETRICS

Original: Since the error of subjective learning is related to the error of both the high-level subjective function and the low-level models, we respectively adopt two metrics to quantitatively estimate these errors.
Revised: Since the error of subjective learning is related to the error of both the high-level subjective function and the low-level models, we respectively adopt two metrics to quantitatively estimate these errors.

Original: Subjective error.
Revised: Subjective error.

Original: This metric measures the learner’s ability to perform appropriate data allocation.
Revised: This metric measures the learner’s ability to perform appropriate data allocation.

Original: Given a domain d , a good subjective function should yield stable allocations for all data batches  sampled from this domain.
Revised: Given a domain d , a good subjective function should yield stable allocations for all data batches [Table] sampled from this domain.

Original: Thus, we measure the error of the subjective function using the rate of inconsistent data allocations , which we deﬁne as
Revised: Thus, we measure the error of the subjective function using the rate of inconsistent data allocations , which we deﬁne as

Original: [Equation]
Revised: [Equation]

Original: where l d denotes the total number of samples in domain d (the same below).
Revised: where l d denotes the total number of samples in domain d (the same below).

Original: Model error.
Revised: Model error.

Original: This metric measures the learner’s ability to make accurate in-domain predictions, which is analagous to the traditional single-task error.
Revised: This metric measures the learner’s ability to make accurate in-domain predictions, which is analagous to the traditional single-task error.

Original: Given a domain d , it is deﬁned as
Revised: Given a domain d , it is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: (9) where we apply e ( y, y (cid:48) )
Revised: where we apply e ( y, y (cid:48) )

Original: = ( y − y (cid:48) ) 2 for regression and e ( y, y (cid:48) )
Revised: = ( y − y (cid:48) ) 2 for regression and e ( y, y (cid:48) )

Original: =
Revised: =

Original: 1 ( y (cid:54) =
Revised: 1 ( y (cid:54) =

Original: y (cid:48) ) for classiﬁcation.[Table] [Table]
Revised: y (cid:48) ) for classiﬁcation.

Original:  E MPIRICAL R ESULTS AND A NALYSES
Revised: 4. E MPIRICAL R ESULTS AND A NALYSES

Original: We compare the performance of subjective learning and the baselines in Figure 2.
Revised: We compare the performance of subjective learning and the baselines in Figure 2.

Original: Unsurprisingly, the vanilla baseline converges to a trivial mean function (dashed curve in Figure 2a).
Revised: Unsurprisingly, the vanilla baseline converges to a trivial mean function (dashed curve in Figure 2a).

Original: MAML successfully predicts the left part of all target functions by ﬁne-tuning from episodic samples [Table] but fails in the right part where functions exhibit larger difference.
Revised: MAML successfully predicts the left part of all target functions by ﬁne-tuning from episodic samples, but fails in the right part where functions exhibit larger difference.

Original: We hypothesis that it is because metalearning typically requires tasks to be in sufﬁcient numbers, and with more similarity.
Revised: We hypothesis that it is because metalearning typically requires tasks to be in sufﬁcient numbers, and with more similarity.

Original: Although
Revised: Although

Original: Modular currectly predicts the general trend of the curves, its predictions are still inaccurate in ﬁne-grained details.
Revised: Modular currectly predicts the general trend of the curves, its predictions are still inaccurate in ﬁne-grained details.

Original: Note that both meta-learning methods use more episodic data samples than subjective learning (see Appendix D.3.1).
Revised: Note that both meta-learning methods use more episodic data samples than subjective learning (see Appendix D.3.1).

Original: Meanwhile, subjective learning with K ≥ R ( K = 3 or 4 ) successfully distinguishes different functions and recover each of them accurately while subjective learning with K < R
Revised: Meanwhile, subjective learning with K ≥ R ( K = 3 or 4 ) successfully distinguishes different functions and recover each of them accurately while subjective learning with K < R

Original: ( K = 2 ) fails, which matches our analysis in Section 3.1.
Revised: ( K = 2 ) fails, which matches our analysis in Section 3.1.

Original: In particular, subjective learning with K = 4 automatically leaves one network to be redundant (dashed curve in
Revised: In particular, subjective learning with K = 4 automatically leaves one network to be redundant (dashed curve in

Original: Figure 2f), demonstrating the robustness of our framework.
Revised: Figure 2f), demonstrating the robustness of our framework.

Original: Figure 3 illustrates the impact of sampling hyperparameters on subjective learning.
Revised: Figure 3 illustrates the impact of sampling hyperparameters on subjective learning.

Original: Concretely, fewer sampling episodes m = 50 induces a large model error 9, which corresponds to the sample-wise estimation term in the generalization error 7b since the product mn is not sufﬁciently large.
Revised: Concretely, fewer sampling episodes m = 50 induces a large model error 9, which corresponds to the sample-wise estimation term in the generalization error 7b since the product mn is not sufﬁciently large.

Original: On the other hand, subjective learning with fewer episodic samples n = 1 induces a large subjective error 8, which corresponds to the subjective estimation term in the generalization error 7c.
Revised: On the other hand, subjective learning with fewer episodic samples n = 1 induces a large subjective error 8, which corresponds to the subjective estimation term in the generalization error 7c.

Revised: Another interesting phenomenon is that the curves in

Revised: Figure 3b are partially swapped compared with the ground truth when n = 1 , indicating wrong data allocation, which also corroborates our theory.

Original: Classiﬁcation.
Revised: Classiﬁcation.

Original: Table 1 shows the results of subjective learning and the baselines on classiﬁcation tasks.
Revised: Table 1 shows the results of subjective learning and the baselines on classiﬁcation tasks.

Original: On all tasks, subjective learning outperforms all baselines on both the subjective error and the model error.
Revised: On all tasks, subjective learning outperforms all baselines on both the subjective error and the model error.

Original: It is also worth noting that compared to the oracle Full-L with full label annotations, subjective learning still induces smaller subjective error, showing a strong capability of domain-level cognition that resembles the “ground truth” annotated by humans (Full-T).
Revised: It is also worth noting that compared to the oracle Full-L with full label annotations, subjective learning still induces smaller subjective error, showing a strong capability of domain-level cognition that resembles the “ground truth” annotated by humans (Full-T).

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Apart from the formulation in this paper, conﬂicting data may also be formulated using the framework of multi-label learning with partial labels (Zhang & Zhou, 2014) or probabilistic density estimation such as probabilistic concepts (Kearns & Schapire, 1994; Devroye et al., 1996) and the energy-based learning framework (LeCun et al., 2006).
Revised: Apart from the formulation in this paper, conﬂicting data may also be formulated using the framework of multi-label learning with partial labels (Zhang & Zhou, 2014) or probabilistic density estimation such as probabilistic concepts (Kearns & Schapire, 1994; Devroye et al., 1996) and the energy-based learning framework (LeCun et al., 2006).

Original: A fundamental difference between these formulations and subjective learning is that these methods learn a uniﬁed model, while our method explicitly encourages the learner to perform high-level data allocation and result in a set of independent models.
Revised: A fundamental difference between these formulations and subjective learning is that these methods learn a uniﬁed model, while our method explicitly encourages the learner to perform high-level data allocation and result in a set of independent models.

Original: Su et al.
Revised: Su et al.

Original: (2020) studies a similar problem as ours where the goal of the learner is to learn from online multi-task samples without task annotation.
Revised: (2020) studies a similar problem as ours where the goal of the learner is to learn from online multi-task samples without task annotation.

Original: However, our work employs a different objective function and presents more formal theoretical justiﬁcation.
Revised: However, our work employs a different objective function and presents more formal theoretical justiﬁcation.

Original: Extensive literature has explored the collaboration of multiple models or modules in completing one or multiple tasks (Doya et al., 2002;
Revised: Extensive literature has explored the collaboration of multiple models or modules in completing one or multiple tasks (Doya et al., 2002;

Original: Andreas et al., 2016;
Revised: Andreas et al., 2016;

Original: Alet et al., 2018; Meyerson & Miikkulainen, 2019;
Revised: Alet et al., 2018; Meyerson & Miikkulainen, 2019;

Original: Yang et al., 2020; Gao et al., 2020).
Revised: Yang et al., 2020; Gao et al., 2020).

Original: A crucial difference between our work and these works is that our multi-model architecture is driven by the inherent conﬂict in conﬂicting data itself.
Revised: A crucial difference between our work and these works is that our multi-model architecture is driven by the inherent conﬂict in conﬂicting data itself.

Original: Our approach can also be viewed as an implementation of mixture-of-experts methods (Yuksel et al., 2012;
Revised: Our approach can also be viewed as an implementation of mixture-of-experts methods (Yuksel et al., 2012;

Original: Shazeer et al., 2017), where our main innovation is the subjective function which serves as an effective model selection method for conﬂicting data, and we only allow a single low-level model to be invoked during a sampling episode.
Revised: Shazeer et al., 2017), where our main innovation is the subjective function which serves as an effective model selection method for conﬂicting data, and we only allow a single low-level model to be invoked during a sampling episode.

Original: More discussions on related work are in Appendix F.
Revised: More discussions on related work are in Appendix F.

Original: A  D ISCUSSIONON THE S UBJECTIVE F UNCTION
Revised:  D ISCUSSION

Revised: In this work, we investigate a novel learning scenario of learning from conﬂicting data, which generalizes the single target function assumption that has been widely adopted by conventional machine learning methods. We hope that our work can serve as a stepping stone in the pursuit of general learning paradigms with fewer assumptions on data distributions compared with conventional machine learning regimes. In the following we list two limitations of our current work for future research:

Revised: Noisy data.

Revised: As mentioned in Deﬁnition 1, our formulation is limited to fully-informative data where absolute predictions can be made given the inputs.

Revised: While this assumption is valid in a variety of applications, it is interesting to develop methods that can also handle data with noise.

Revised: However, this raises a fundamental question: how can we decide whether the stochasticity in data is caused by “pure” noise or some unobserved, semantically-meaningful factors (e g., different target functions)?

Revised: We believe that answering this question is crucial for devising algorithms that apply to more general learning scenarios.

Revised: Continual learning.

Revised: In open-ended environments, the data usually comes in a continual manner with no explicit train-test delineation.

Revised: Hence, developing continual learning agents that beneﬁt from the growing diversity (in terms of both data and domains) of conﬂicting data would be an exciting future research avenue.

Revised: A D ISCUSSION ON THE S UBJECTIVE F UNCTION

Original: In Section 2.3, we derive our design of the subjective function using an EM-based maximum likelihood formulation.
Revised: In Section 2.3, we derive our design of the subjective function using an EM-based maximum likelihood formulation.

Original: However, the exact equivalence between the E-step of hard EM and the subjective function has not been established, since it relies on the exact form of the loss function (cid:96) ( y, y (cid:48) ) and the conditional likelihood p ( y | x , h k ) , k ∈ [ K ] .
Revised: However, the exact equivalence between the E-step of hard EM and the subjective function has not been established, since it relies on the exact form of the loss function (cid:96) ( y, y (cid:48) ) and the conditional likelihood p ( y | x , h k ) , k ∈ [ K ] .

Original: As a complement, in the sequel we provide two examples where under the uniform prior, the exact equivalence between calculating the posterior
Revised: As a complement, in the sequel we provide two examples where under the uniform prior, the exact equivalence between calculating the posterior

Original: P ( h = h k | x , y ) and the expected subjective function 5b can be obtained (hence their empirical counterparts are also equivalent).
Revised: P ( h = h k | x , y ) and the expected subjective function 5b can be obtained (hence their empirical counterparts are also equivalent).

Original: Example 1 (Regression with isotropic Gaussian joint likelihood) .
Revised: Example 1 (Regression with isotropic Gaussian joint likelihood) .

Original: Consider a regression task where we assume that given the random variable ( x , y ) , the joint distribution of its conditional likelihoods conforms to an isotropic Gaussian N ( µ K , (cid:15) I K )
Revised: Consider a regression task where we assume that given the random variable ( x , y ) , the joint distribution of its conditional likelihoods conforms to an isotropic Gaussian N ( µ K , (cid:15) I K )

Original: where (cid:15) ∈
Revised: where (cid:15) ∈

Original: R
Revised: R

Original: + is a variance parameter
Revised: + is a variance parameter

Original: and I K is a
Revised: and I K is a

Original: K × K identity matrix.
Revised: K × K identity matrix.

Original: From p ( y | x , h k ) ∝ exp (cid:104) − ( y − µ k
Revised: From p ( y | x , h k ) ∝ exp (cid:104) − ( y − µ k

Original: ( x )) 2 2 (cid:15)(cid:105) and µ k ( x ) =
Revised: ( x )) 2 2 (cid:15)(cid:105) and µ k ( x ) =

Original: h k ( x ) we have that
Revised: h k ( x ) we have that

Original: [Equation]
Revised: [Equation]

Original: ( y
Revised: [Equation] arg

Original: [Equation]
Revised: [Equation]

Original: ( x )) 2 (cid:3) .
Revised: x )) 2 (cid:3) .

Original: This equals to the expected subjective function with squared loss (cid:96) ( y, y (cid:48) )
Revised: This equals to the expected subjective function with squared loss (cid:96) ( y, y (cid:48) )

Original: = ( y − y (cid:48) ) 2 .
Revised: = ( y − y (cid:48) ) 2 .

Original: Example 2 (Classiﬁcation with independent categorical likelihoods) .
Revised: Example 2 (Classiﬁcation with independent categorical likelihoods) .

Original: Consider a multi-class classiﬁcation task where we assume that given the random variable ( x , y ) , all conditional likelihoods conform to independent categorical distributions with parameters ( λ 1 , λ 2 , · · · , λ K ) , where λ k = ( λ k 1 , · · · , λ kL ) , (cid:80)
Revised: Consider a multi-class classiﬁcation task where we assume that given the random variable ( x , y ) , all conditional likelihoods conform to independent categorical distributions with parameters ( λ 1 , λ 2 , · · · , λ K ) , where λ k = ( λ k 1 , · · · , λ kL ) , (cid:80)

Original: Lj =

Revised: Lj =

Original: λ kj = 1 , k ∈
Revised: λ kj = 1 , k ∈

Original: [ K ] and L is the number of classes.
Revised: [ K ] and L is the number of classes.

Original: From p ( y | x , h k ) =
Revised: From p ( y | x , h k ) =

Original: (cid:81)
Revised: (cid:81)

Original: Lj =1 λ kj ( x ) y j , where y =
Revised: Lj =1 λ kj ( x ) y j , where y =

Original: ( y 1 , · · · , y L ) ∈ { 0 , 1 } L , (cid:80)
Revised: ( y 1 , · · · , y L ) ∈ { 0 , 1 } L , (cid:80)

Original: Lj =1 y j = 1 and λ k ( x ) =
Revised: Lj =1 y j = 1 and λ k ( x ) =

Original: h k ( x ) we have that arg max h k ∈ H log p ( y | x , h k ) ⇔ arg min h k ∈ H [ − (cid:80)
Revised: h k ( x ) we have that arg max h k ∈ H log p ( y | x , h k ) ⇔ arg min h k ∈ H [ − (cid:80)

Original: Lj =

Revised: Lj =

Original: y j log h kj ( x )] , where h kj ( x ) denotes the predicted probability of the j -th class and y j is the corresponding ground truth.
Revised: y j log h kj ( x )] , where h kj ( x ) denotes the predicted probability of the j -th class and y j is the corresponding ground truth.

Original: This equals to the expected subjective function with cross-entropy loss (cid:96) ( y, y (cid:48) )
Revised: This equals to the expected subjective function with cross-entropy loss (cid:96) ( y, y (cid:48) )

Original: = − (cid:80)
Revised: = − (cid:80)

Original: Li =1 y (cid:48)
Revised: Li =1 y (cid:48)

Original: i log y i .
Revised: i log y i .

Original: B A LGORITHM P SEUDO - CODE
Revised: B A LGORITHM P SEUDO - CODE

Original: We provide the pseudo-code of subjective learning in Algorithm 1.
Revised: We provide the pseudo-code of subjective learning in Algorithm 1.

Original: C P ROOFS OF T HEORETICAL R ESULTS
Revised: C P ROOFS OF T HEORETICAL R ESULTS

Original: In this section, we provide the proofs of our theoretical results.
Revised: In this section, we provide the proofs of our theoretical results.

Original: For better exposition, we restate each theorem before its proof.
Revised: For better exposition, we restate each theorem before its proof.

Original: C. P ROOF OF P ROPOSITION
Revised: C. P ROOF OF P ROPOSITION

Original: Proposition 1 (Form of the optimal solution) .
Revised: Proposition 1 (Form of the optimal solution) .

Original: Assume that the target functions in all domains are realizable.
Revised: Assume that the target functions in all domains are realizable.

Original: Then, the following two propositions are equivalent:
Revised: Then, the following two propositions are equivalent:

Original: (1) For all domain distributions Q and data distributions P 1 , P 2 , · · · , P N , er ( H )
Revised: (1) For all domain distributions Q and data distributions P 1 , P 2 , · · · , P N , er ( H )

Original: = 0 .
Revised: = 0 .

Original: For each d = (cid:104) P, c (cid:105) in D , there exists h ∈ H such that E x ∼ P (cid:96)
Revised: For each d = (cid:104) P, c (cid:105) in D , there exists h ∈ H such that E x ∼ P (cid:96)

Original: [ h ( x ) , c ( x )]
Revised: [ h ( x ) , c ( x )]

Original: = 0 .
Revised: = 0 .

Original: Proof.
Revised: Proof.

Original: The derivation of proposition (1) from proposition (2) is obvious.
Revised: The derivation of proposition (1) from proposition (2) is obvious.

Original: On the other hand, if proposition (2) is false, i.e., there exists k ∈ [ N ] such that for all j ∈ [ K ] , h j (cid:54) =
Revised: On the other hand, if proposition (2) is false, i.e., there exists k ∈ [ N ] such that for all j ∈ [ K ] , h j (cid:54) =

Original: c k .
Revised: c k .

Original: Then, we haveer ( H
Revised: Then, we have

Original: [Equation]
Revised: [Equation]

Original: From the above we know that c k / ∈ H , thus there exists P k such that er P k ( h, c k ) > 0 for every h ∈ H .
Revised: From the above we know that c k / ∈ H , thus there exists P k such that er P k ( h, c k ) > 0 for every h ∈ H .

Original: This indicates that er ( H ) > 0 , which is in contradiction with proposition (a).
Revised: This indicates that er ( H ) > 0 , which is in contradiction with proposition (a).

Original: Therefore, proposition (2) must hold if proposition (1) is true.
Revised: Therefore, proposition (2) must hold if proposition (1) is true.

Original: Require:
Revised: Require:

Original: Hypothesis set H =
Revised: Hypothesis set H =

Original: { h 1 , h 2 , · · · , h K } , sampling hyperparameters m and n .
Revised: { h 1 , h 2 , · · · , h K } , sampling hyperparameters m and n .

Original: 1: forh =1 , 2 , · · · , T epochs do 2: for i = 1 , 2 , · · · , m episodes do 3:
Revised: for =

Original: Sample data Z i = { ( x i 1 , y i 1 ) , ( x i 2 , y i 2 ) , · · · , ( x in , y in ) } . Select a hypothesis ˆ h i from the hypothesis set using the empirical subjective function 5a. Train the hypothesis ˆ h i by minimizing the empirical episodic error 6. 6: end for 7: end for
Revised: , m

Original: C. P ROOF OF T HEOREM
Revised: C. P ROOF OF T HEOREM

Original: We ﬁrst present a generalized deﬁnition of PAC learnability.
Revised: We ﬁrst present a generalized deﬁnition of PAC learnability.

Original: Deﬁnition 2 (PAC learnability) .
Revised: Deﬁnition 2 (PAC learnability) .

Original: A target function set class C is said to be PAC learnable if there exists an algorithm and a polynomial function poly( · , · , · ) such that for any (cid:15) > 0 and δ > 0 , for all distributions Q and distribution set P :=
Revised: A target function set class C is said to be PAC learnable if there exists an algorithm and a polynomial function poly( · , · , · ) such that for any (cid:15) > 0 and δ > 0 , for all distributions Q and distribution set P :=

Original: { P 1 , · · · , P n } , the following holds for any sample size mn ≥ poly(1 /(cid:15), [Equation] /δ, size( C )) : 
Revised: { P 1 , · · · , P n } , the following holds for any sample size mn ≥ poly(1 /(cid:15), 1 /δ, size( C )) : P

Original: [Equation]
Revised: [Equation]

Original: The above deﬁnition can be viewed as an extension of the single-task PAC learnability (Haussler, 1990) that considers the problem of learning a single target function.
Revised: The above deﬁnition can be viewed as an extension of the single-task PAC learnability (Haussler, 1990) that considers the problem of learning a single target function.

Original: Based on this deﬁnition, in the following we give the proof.
Revised: Based on this deﬁnition, in the following we give the proof.

Original: Theorem 1.
Revised: Theorem 1.

Original: A necessary condition of the PAC learnability of subjective learning is K ≥ R .
Revised: A necessary condition of the PAC learnability of subjective learning is K ≥ R .

Original: Proof.
Revised: Proof.

Original: According to Deﬁnition 2, if subjective learning is PAC learnable, there must exist an algorithm that outputs a hypothesis set H with zero error, i.e., er ( H ) = 0 for every Q and P (otherwise the inequality 10 will not hold for a small enough (cid:15) and δ < 1 ).
Revised: According to Deﬁnition 2, if subjective learning is PAC learnable, there must exist an algorithm that outputs a hypothesis set H with zero error, i.e., er ( H ) = 0 for every Q and P (otherwise the inequality 10 will not hold for a small enough (cid:15) and δ < 1 ).

Original: Proposition 1 indicates that this is equivalent to c
Revised: Proposition 1 indicates that this is equivalent to c

Original: i ∈ H, i ∈ [ N ] , which is impossible if K < R according to Deﬁnition 1 and the drawer principle.
Revised: i ∈ H, i ∈ [ N ] , which is impossible if K < R according to Deﬁnition 1 and the drawer principle.

Original: C. P ROOF OF T HEOREM
Revised: C. P ROOF OF T HEOREM

Original: We ﬁrst introduce several technical lemmas.
Revised: We ﬁrst introduce several technical lemmas.

Original: Lemma 1.
Revised: Lemma 1.

Original: Let { E i } ni
Revised: Let { E i } ni

Original: =1 be a set of events satisfying P ( E i ) ≥ 1 − δ i , with some δ i ≥ 0 , i = 1 , · · · , n .
Revised: =1 be a set of events satisfying P ( E i ) ≥ 1 − δ i , with some δ i ≥ 0 , i = 1 , · · · , n .

Original: Then, P ( (cid:84) n i =1 E i ) ≥ 1 − (cid:80) n i =1{+ni =
+} δ i .
Revised: Then, P ( (cid:84) ni =1 E i ) ≥ 1 − (cid:80) {+ni =
+} δ i .

Original: Lemma 2 (Single-task generalization error bound (Vapnik, 2013)) .
Revised: Lemma 2 (Single-task generalization error bound (Vapnik, 2013)) .

Original: Let A ≤ Q ( z, α ) ≤ B, α ∈ Λ be a measurable and bounded real-valued function set, of which the Vapnik-Chervonenkis (VC) dimension (Vapnik & Chervonenkis, 1971) is VC ( Q ) .
Revised: Let A ≤ Q ( z, α ) ≤ B, α ∈ Λ be a measurable and bounded real-valued function set, of which the Vapnik-Chervonenkis (VC) dimension (Vapnik & Chervonenkis, 1971) is VC ( Q ) .

Original: Let { z 1 , z 2 , · · · , z n } ni =1 be data samples sampled i.i.d.
Revised: Let { z 1 , z 2 , · · · , z n } ni =1 be data samples sampled i.i.d.

Original: from a distribution P with size n .
Revised: from a distribution P with size n .

Original: Then, for any δ ∈ (0 , 1] , the following inequality holds with probability at least 1 − δ : 
Revised: Then, for any δ ∈ (0 , 1] , the following inequality holds with probability at least 1 − δ : (cid:12)

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Then, we upper bound the error induced by the estimation of the expected subjective function 5b in each sampling episode of subjective learning, which is critical in bounding the generalization error.
Revised: Then, we upper bound the error induced by the estimation of the expected subjective function 5b in each sampling episode of subjective learning, which is critical in bounding the generalization error.

Original: Recall that we use superscripts to denote the sampling index, e.g., d i = (cid:104) P i , c i (cid:105) denotes the i -th domain sample, which can be any domains in the domain set D .
Revised: Recall that we use superscripts to denote the sampling index, e.g., d i = (cid:104) P i , c i (cid:105) denotes the i -th domain sample, which can be any domains in the domain set D .

Original: We deﬁne two shorthands as follows:
Revised: We deﬁne two shorthands as follows

Original: [Equation]
Revised: [Equation]

Original: where i denotes the i -th sampling episode of subjective learning, H is the hypothesis set.
Revised: where i denotes the i -th sampling episode of subjective learning, H is the hypothesis set.

Original: Lemma 3 (Subjective estimation error bound) .
Revised: Lemma 3 (Subjective estimation error bound) .

Original: Let (cid:8)(cid:0) x
Revised: Let (cid:8)(cid:0) x

Original: i 1 , y i 1 (cid:1) , · · · , (cid:0) x
Revised: i 1 , y i 1 (cid:1) , · · · , (cid:0) x

Original: in , y in (cid:1)(cid:9) be episodic samples in the i -th ( i ∈ [ m ]) sampling episode i.i.d.
Revised: in , y in (cid:1)(cid:9) be episodic samples in the i -th ( i ∈ [ m ]) sampling episode i.i.d.

Original: drawn from domain d i with size n .
Revised: drawn from domain d i with size n .

Original: Then, for any δ ∈ (0 , 1] , the following inequality holds uniformly for all hypothesis h ∈ H with probability at least 1 − δ :
Revised: Then, for any δ ∈ (0 , 1] , the following inequality holds uniformly for all hypothesis h ∈ H with probability at least 1 − δ :

Original: [Equation]
Revised: [Equation]

Original: where S :
Revised: where S :

Original: = { ( x, y ) (cid:55)→ (cid:96) [ h ( x ; θ ) , y ] } , θ ∈ Θ is the function set of the sample-wise error.
Revised: = { ( x, y ) (cid:55)→ (cid:96) [ h ( x ; θ ) , y ] } , θ ∈ Θ is the function set of the sample-wise error.

Original: Proof.
Revised: Proof.

Original: We have the following decomposition:
Revised: We have the following decomposition:

Original: [Equation]
Revised: [Equation]

Original: in which the original difference is decomposed into three terms.
Revised: in which the original difference is decomposed into three terms.

Original: By deﬁnition 13a and 13b the middle term is non-positive.
Revised: By deﬁnition 13a and 13b the middle term is non-positive.

Original: Using Lemma 2 by substituting Q with S
Revised: Using Lemma 2 by substituting Q with S

Original: = { ( x, y ) (cid:55)→ (cid:96)
Revised: = { ( x, y ) (cid:55)→ (cid:96)

Original: [ h ( x ; θ ) , y ] } and replacing δ with δ / 2 m , the ﬁrst term and the last term can both be bounded by 
Revised: [ h ( x ; θ ) , y ] } and replacing δ with δ / 2 m , the ﬁrst term and the last term can both be bounded by (cid:114) [Equation] n

Original: [Equation]
Revised: [Equation]

Original: n with probability at least 1 − δ / 2 (Lemma 1).
Revised: n with probability at least 1 − δ / 2 (Lemma 1).

Original: Combining these two bounds using Lemma 1 completes the proof.
Revised: Combining these two bounds using Lemma 1 completes the proof.

Original: Now we can give the proof of the main theorem.
Revised: Now we can give the proof of the main theorem.

Original: Theorem 2 (Generalization error upper bound) .
Revised: Theorem 2 (Generalization error upper bound) .

Original: For any δ ∈ (0 , 1] , the following inequality holds uniformly for all hypothesis sets H ∈ H K with probability at least 1 − δ :
Revised: For any δ ∈ (0 , 1] , the following inequality holds uniformly for all hypothesis sets H ∈ H K with probability at least 1 − δ :

Original: [Equation]
Revised: [Equation]

Original: where ¯ S :
Revised: where ¯ S :

Original: = {(cid:104) P, c (cid:105) (cid:55)→ E x ∼ P (cid:96)
Revised: = {(cid:104) P, c (cid:105) (cid:55)→ E x ∼ P (cid:96)

Original: [ h ( x ; θ ) , c ( x )] } , θ ∈ Θ is the function set of the domain-wise expected error, S :
Revised: [ h ( x ; θ ) , c ( x )] } , θ ∈ Θ is the function set of the domain-wise expected error, S :

Original: = { ( x, y ) (cid:55)→ (cid:96) [ h ( x ; θ ) , y ] } , θ ∈ Θ is the function set of the sample-wise error, m k : = [Equation] mi = [Equation]
Revised: = { ( x, y ) (cid:55)→ (cid:96) [ h ( x ; θ ) , y ] } , θ ∈ Θ is the function set of the sample-wise error, m k := (cid:80) mi =1 1 (cid:0) c

Original: i =
Revised: i =

Original: c k (cid:1) is the sampling count of the target function from the k -th domain d k ( k ∈ [ N ]) , and VC ( · ) the Vapnik-Chervonenkis (VC) dimension (Vapnik & Chervonenkis, 1971).
Revised: c k (cid:1) is the sampling count of the target function from the k -th domain d k ( k ∈ [ N ]) , and VC ( · ) the Vapnik-Chervonenkis (VC) dimension (Vapnik & Chervonenkis, 1971).

Original: Proof.
Revised: Proof.

Original: Combining the objectives 2 3 with the subjective function 5a 5b, we have the following decomposition:
Revised: Combining the objectives 2 3 with the subjective function 5a 5b, we have the following decomposition:

Original: [Equation]
Revised: [Equation]

Original: By deﬁnition 13a and 13b we rewrite the equation above:
Revised: By deﬁnition 13a and 13b we rewrite the equation above:

Revised: [Equation]

Revised: 

Original: [Equation]
Revised: [Equation]

Original: in which the generalization error of subjective learning is decomposed into three terms.
Revised: in which the generalization error of subjective learning is decomposed into three terms.

Original: By substituting Q in Lemma 2 by ¯ S = {(cid:104) P, c (cid:105) [Equation] E x ∼ P [Equation]
Revised: By substituting Q in Lemma 2 by ¯ S = {(cid:104) P, c (cid:105) (cid:55)→ E x ∼ P (cid:96)

Original: [ h ( x ; θ ) , c ( x )] } and replacing δ with δ / 3 , the ﬁrst term can be bounded by
Revised: [ h ( x ; θ ) , c ( x )] } and replacing δ with δ / 3 , the ﬁrst term can be bounded by

Original: [Equation]
Revised: [Equation]

Original: m with probability at least [Equation] δ / 3 .
Revised: m with probability at least 1 − δ / 3 .

Original: By replacing δ with δ / 3 in Lemma 3 we bound the last term by[Equation] n
Revised: By replacing δ with δ / 3 in Lemma 3 we bound the last term by

Original: [Equation]
Revised: [Equation]

Original: n with probability at least 1 − δ / 3 .
Revised: n with probability at least 1 − δ / 3 .

Original: There remains the middle term for which we havem
Revised: There remains the middle term for which we have

Original: [Equation]
Revised: [Equation]

Original: n

Original: [Equation]

Original: =m

Original: [Equation]

Original: i =   

Original: E

Original: [Equation]

Original: (cid:2) h

Original: [Equation]

Original: ( x

Original: [Equation]

Original: c i ( x [Equation] (cid:3) − 1 n n (cid:88) j = (cid:96) (cid:2) h ∗
Revised: 

Original: i (cid:0) x ij (cid:1)

Original: [Equation]
Revised: [Equation]

Original: y ij (cid:3)    =m

Original: N (cid:88) k

Original: =   m

Original: [Equation]

Original: E

Original: [Equation]

Original: ∼ P

Original: [Equation]

Original: (cid:96) [ h ∗ k

Original: ( x )

Original: [Equation]

Original: c k ( x )] − 1 n nm k (cid:88)

Original: j = (cid:96) [ h ∗ k ( x kj , y kj )]

Original:  

Original: =m

Original: N (cid:88) k

Original: = m k  

Original: E x ∼ P k (cid:96) [ h ∗ k

Original: ( x ) , c k ( x )] − nm k nm k (cid:88)

Original: j = (cid:96) [ h ∗ k ( x kj , y kj )]

Original:   .

Original: Recall that m k := (cid:80) mi =1 1 (cid:0) c i =
Revised: Recall that m k := (cid:80) mi =1 1 (cid:0) c i =

Original: c k (cid:1) .
Revised: c k (cid:1) .

Original: In the above we re-arrange the total m data batches according to the domains they belong to.
Revised: In the above we re-arrange the total m data batches according to the domains they belong to.

Original: With a little abuse of notation, in the third and fourth rows we use h ∗ k to denote the hypothesis that yields the smallest expected error in the k -th domain in
Revised: With a little abuse of notation, in the third and fourth rows we use h ∗ k to denote the hypothesis that yields the smallest expected error in the k -th domain in

Original: D , i.e., h ∗ k = arg min h ∈ H E
Revised: D , i.e., h ∗ k = arg min h ∈ H E

Original: x ∼ P k (cid:96)
Revised: x ∼ P k (cid:96)

Original: [ h ( x ) , c k ( x )] , k ∈ [ N ] .
Revised: [ h ( x ) , c k ( x )] , k ∈ [ N ] .

Original: Note that this is different from the deﬁnition 13a, where h ∗ i is deﬁned upon the i -th domain sample .
Revised: Note that this is different from the deﬁnition 13a, where h ∗ i is deﬁned upon the i -th domain sample .

Original: The above transformation aggregates the domain samples so that data samples from the same domains that emerge multiple times can be accumulated and jointly considered, which leads to a tighter and more realistic error bound.
Revised: The above transformation aggregates the domain samples so that data samples from the same domains that emerge multiple times can be accumulated and jointly considered, which leads to a tighter and more realistic error bound.

Original: By Lemma 2, for every k ∈ [ N ] and δ ∈ (0 , 1] , each inside term E x ∼ P k (cid:96) [ h ∗ k
Revised: By Lemma 2, for every k ∈ [ N ] and δ ∈ (0 , 1] , each inside term E x ∼ P k (cid:96) [ h ∗ k

Original: ( x ) , c k ( x )] − [Equation]
Revised: ( x ) , c k ( x )] − nm k k

Original: [Equation]
Revised: (cid:96) [

Original: y kj )] can be bounded by
Revised: )] can be bounded by

Original: [Equation]
Revised: [Equation]

Original: n with probability at least 1 − δ .
Revised: with probability at least 1 − δ .

Original: By replacing δ with δ / 3 N we bound the whole second term by(cid:80)
Revised: By replacing δ with δ / 3 N we bound the whole second term by

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − δ /(Lemma 1).
Revised: with probability at least 1 − δ /(Lemma 1).

Original: Finally, combining the above bounds for all three terms using Lemma 1 gives the result.
Revised: Finally, combining the above bounds for all three terms using Lemma 1 gives the result.

Original: D E XPERIMENT D ETAILS
Revised: D E XPERIMENT D ETAILS

Original: In this section, additional details on the setup and settings of the experiment are provided.
Revised: In this section, additional details on the setup and settings of the experiment are provided.

Original: All experiments were conducted based on PyTorch (Paszke et al., 2019) using a NVIDIA 2080ti GPU.
Revised: All experiments were conducted based on PyTorch (Paszke et al., 2019) using a NVIDIA 2080ti GPU.

Original: All data we used does not contain personally identiﬁable information or offensive content, and can be obtained from public data sources.
Revised: All data we used does not contain personally identiﬁable information or offensive content, and can be obtained from public data sources.

Original: D. R EGRESSION
Revised: D. R EGRESSION

Original: We consider three heterogeneous mapping functions in absolute, sinusoidal and logarithmic function families as below:
Revised: We consider three heterogeneous mapping functions in absolute, sinusoidal and logarithmic function families as below:

Original: [Equation]
Revised: [Equation]

Original: where the range of x is [ − 2 , 2] for all three functions, yielding the conﬂicting data with R = 3 .
Revised: where the range of x is [ − 2 , 2] for all three functions, yielding the conﬂicting data with R = 3 .

Original: The sample hyperparameters are m = 250 , n = 2 , i.e., a total number of 500 data points are collected in 250 episodes, each with 2 samples in the data batch, and their underlying generation functions are randomly chosen from the three mapping functions.
Revised: The sample hyperparameters are m = 250 , n = 2 , i.e., a total number of 500 data points are collected in 250 episodes, each with 2 samples in the data batch, and their underlying generation functions are randomly chosen from the three mapping functions.

Original: Network and optimizer.
Revised: Network and optimizer.

Original: A simple network architecture with 5 fully-connected linear layers, each with 32 hidden units, is adopted and trained with SGD with step size α = 0 .
Revised: A simple network architecture with 5 fully-connected linear layers, each with 32 hidden units, is adopted and trained with SGD with step size α = 0 .

Original: 05 , momentum = 0 .
Revised: 05 , momentum = 0 .

Original: 9 , and (cid:96) 2 weight decay =
Revised: 9 , and (cid:96) 2 weight decay =

Original: 10 − 4 .
Revised: 10 − 4 .

Original: D. C LASSIFICATION
Revised: D. C LASSIFICATION

Original: Colored MNIST is an extended version of the well-known character recognition dataset MNIST, in which each gray-scale digital images are randomly colored with 8 different colors.
Revised: Colored MNIST is an extended version of the well-known character recognition dataset MNIST, in which each gray-scale digital images are randomly colored with 8 different colors.

Original: The dataset contains two parallel tasks of color and number classiﬁcation ( R = 2 ) with a total number ofcolored digits.
Revised: The dataset contains two parallel tasks of color and number classiﬁcation ( R = 2 ) with a total number ofcolored digits.

Original: The sample hyperparameters are m = 60000 , n = 1 .
Revised: The sample hyperparameters are m = 60000 , n = 1 .

Original: Fashion Product Images (Aggarwal, 2019) is a dataset for automatic attribute completion and Q&A of clothing product images with multiple category labels in different domains.
Revised: Fashion Product Images (Aggarwal, 2019) is a dataset for automatic attribute completion and Q&A of clothing product images with multiple category labels in different domains.

Original: We choose 3 main parallel tasks (color, gender, category) with 8 main labels from the original dataset ( R = 3 ), with 15000 images in total.
Revised: We choose 3 main parallel tasks (color, gender, category) with 8 main labels from the original dataset ( R = 3 ), with 15000 images in total.

Original: The sampling hyperparameters are m = 15000 , n = 1 .
Revised: The sampling hyperparameters are m = 15000 , n = 1 .

Original: CIFAR-100 (Krizhevsky & Hinton, 2009) is a classical benchmark for general image classiﬁcation.
Revised: CIFAR-100 (Krizhevsky & Hinton, 2009) is a classical benchmark for general image classiﬁcation.

Original: It has a hierarchical structure with 20 superclasses and 100 classes, with 60000 images in total.
Revised: It has a hierarchical structure with 20 superclasses and 100 classes, with 60000 images in total.

Original: Each superclass is from an upper-level “coarse” task, consisting of 5 “ﬁne” classes, e.g., “insects” includes “bee, beetle, butterﬂy, caterpillar, cockroach”.
Revised: Each superclass is from an upper-level “coarse” task, consisting of 5 “ﬁne” classes, e.g., “insects” includes “bee, beetle, butterﬂy, caterpillar, cockroach”.

Original: In our experiments, these two different kinds of labels are randomly provided given an image ( R = 2 ).
Revised: In our experiments, these two different kinds of labels are randomly provided given an image ( R = 2 ).

Original: The sampling hyperparameters are m = 30000 , n = 2 .
Revised: The sampling hyperparameters are m = 30000 , n = 2 .

Original: Network and optimizer.
Revised: Network and optimizer.

Original: In Colored MNIST , each network consists of 3 convolutional layers andfully-connected layer.
Revised: In Colored MNIST , each network consists of 3 convolutional layers andfully-connected layer.

Original: In Fashion Product Images , each network consists of 5 convolutional layers and 3 fully-connected layers.
Revised: In Fashion Product Images , each network consists of 5 convolutional layers and 3 fully-connected layers.

Original: We use Adam optimizer (Kingma & Ba, 2015) with step size α = 0 .
Revised: We use Adam optimizer (Kingma & Ba, 2015) with step size α = 0 .

Original: 002 and betas = (0 . 5 , 0 . 999)
Revised: 002 and betas = (0 . 5 , 0 . 999)

Original: for these two datasets.
Revised: for these two datasets.

Original: In CIFAR-100 , for subjective learning and all baselines, we use a pre-trained DenseNet (Huang et al., 2017) backbone of DenseNet-L190-k40 for feature extraction to ensure a fair comparison, and add 2 fully-connected layers after the DenseNet backbone.
Revised: In CIFAR-100 , for subjective learning and all baselines, we use a pre-trained DenseNet (Huang et al., 2017) backbone of DenseNet-L190-k40 for feature extraction to ensure a fair comparison, and add 2 fully-connected layers after the DenseNet backbone.

Original: We use SGD as the optimizer with step size α = 0 . 1 and momentum = 0 . 9 .
Revised: We use SGD as the optimizer with step size α = 0 . 1 and momentum = 0 . 9 .

Original: D. B ASELINE D ETAILS
Revised: D. B ASELINE D ETAILS

Original: In this section, we provide more details on the baselines.
Revised: In this section, we provide more details on the baselines.

Original: D.3. R EGRESSION
Revised: D.3. R EGRESSION

Original: MAML.
Revised: MAML.

Original: For MAML, we adapt a standard PyTorch implememtation from GitHub 1 with the same network with ours, and use the following hyperparameters:
Revised: For MAML, we adapt a standard PyTorch implememtation from GitHub 1 with the same network with ours, and use the following hyperparameters:

Original: Shot = 2 , Evalation = 100 , Outer step size = 0 .
Revised: Shot = 2 , Evalation = 100 , Outer step size = 0 .

Original: 05 , Inner step size = 0 .
Revised: 05 , Inner step size = 0 .

Original: Inner grad steps = 2 , Eval grad steps = 5 , Eval iters = 10 , Iterations = 20000 .
Revised: Inner grad steps = 2 , Eval grad steps = 5 , Eval iters = 10 , Iterations = 20000 .

Original: Modular.
Revised: Modular.

Original: For modular meta-learning, we use the ofﬁcial PyTorch implementation from GitHub 2 , and use the following hyperparameters:
Revised: For modular meta-learning, we use the ofﬁcial PyTorch implementation from GitHub 2 , and use the following hyperparameters:

Original: Shot = 2 , Support = 2 , Network =
Revised: Shot = 2 , Support = 2 , Network =

Original: Linear 1 − 16 − 16 − 1 , Num modules = 5 ,
Revised: Linear 1 − 16 − 16 − 1 , Num modules = 5 ,

Original: Composer =
Revised: Composer =

Original: Sum, meta lr = 0 .
Revised: Sum, meta lr = 0 .

Original: 003 , Steps = 3000 .
Revised: 003 , Steps = 3000 .

Original: Note that for Modular Meta-Learning a larger episodic sample number n = 4 (consisting ofsupport samples and 2 query samples) is adopted.
Revised: Note that for Modular Meta-Learning a larger episodic sample number n = 4 (consisting ofsupport samples and 2 query samples) is adopted.

Original: Nevertheless, subjective learning still outperforms this approach using a smaller episodic sample number ( n = 2 ).
Revised: Nevertheless, subjective learning still outperforms this approach using a smaller episodic sample number ( n = 2 ).

Original: D.3. C LASSIFICATION
Revised: D.3. C LASSIFICATION

Original: Probablistic concepts.
Revised: Probablistic concepts.

Original: From the perspective of probability modeling, the relation between input x and output y is subject to a probability distribution p ( y | x ) , which is the learning target.
Revised: From the perspective of probability modeling, the relation between input x and output y is subject to a probability distribution p ( y | x ) , which is the learning target.

Original: In classiﬁcation, when different domains share similar frequency, such a distribution can be approximated with the total probability formula:
Revised: In classiﬁcation, when different domains share similar frequency, such a distribution can be approximated with the total probability formula:

Original: [Equation]
Revised: [Equation]

Original: For classiﬁcation problems, in each domain d , the corresponding p ( y | x, h ) is unimodal.
Revised: For classiﬁcation problems, in each domain d , the corresponding p ( y | x, h ) is unimodal.

Original: Thus, their sum p ( y | x ) is a multi-modal distribution, and a network trained with cross-entropy loss is still an unbiased estimation for it.
Revised: Thus, their sum p ( y | x ) is a multi-modal distribution, and a network trained with cross-entropy loss is still an unbiased estimation for it.

Original: The ﬁnal prediction is the labels with top- N conditional probabilities p ( y | x ) .
Revised: The ﬁnal prediction is the labels with top- N conditional probabilities p ( y | x ) .

Original: Semi-supervised multi-label learning.
Revised: Semi-supervised multi-label learning.

Original: From the perspective of semi-supervised multi-label learning, the classiﬁcation problem can be modeled as “multi-label learning with missing labels”: consider the fully labeled data x → y = { y i } Ni =1 , then, conﬂicting data provides only one label y ∈ Y for each x , i.e., all other labels are missing.
Revised: From the perspective of semi-supervised multi-label learning, the classiﬁcation problem can be modeled as “multi-label learning with missing labels”: consider the fully labeled data x → y = { y i } Ni =1 , then, conﬂicting data provides only one label y ∈ Y for each x , i.e., all other labels are missing.

Original: Therefore, existing semi-supervised learning approaches may be modiﬁed to handle such problems.
Revised: Therefore, existing semi-supervised learning approaches may be modiﬁed to handle such problems.

Original: We consider two representative techniques, including
Revised: We consider two representative techniques, including

Original: Pseudo-label and Label propagation .
Revised: Pseudo-label and Label propagation .

Original: Both implementations are slightly modiﬁed to ﬁt our tasks.
Revised: Both implementations are slightly modiﬁed to ﬁt our tasks.

Original: Pseudo-label randomly allocates additional ”pseudo” labels to each input to compensate the missing labels.
Revised: Pseudo-label randomly allocates additional ”pseudo” labels to each input to compensate the missing labels.

Original: The learning machine is trained on the augmented dataset and then reevaluate the conﬁdence of all pseudo labels according to its predictions, and all pseudo labels will iteratively be modiﬁed during training until convergence.
Revised: The learning machine is trained on the augmented dataset and then reevaluate the conﬁdence of all pseudo labels according to its predictions, and all pseudo labels will iteratively be modiﬁed during training until convergence.

Original: Label propagation builds a graph over the samples, where each node on the graph represents a data sample, and each edge represents the distance of two nodes it collects in the feature space.
Revised: Label propagation builds a graph over the samples, where each node on the graph represents a data sample, and each edge represents the distance of two nodes it collects in the feature space.

Original: The labels are propagated on adjacent nodes until all samples are fully labeled.
Revised: The labels are propagated on adjacent nodes until all samples are fully labeled.

Original: The feature space is also iteratively adjusted along during training.
Revised: The feature space is also iteratively adjusted along during training.

Original: Full labels & full tasks.
Revised: Full labels & full tasks.

Original: Both these oracle baselines utilize manual annotations to tranform the conﬂicting data with R > 1 to conventional supervised data with R = 1 .
Revised: Both these oracle baselines utilize manual annotations to tranform the conﬂicting data with R > 1 to conventional supervised data with R = 1 .

Original: More concretely, for Full labels , label annotations from all domains are provided simultaneously as a “multi-hot” label vector[Table] for each input sample, resulting in a standard multi-label learning problem.
Revised: More concretely, for Full labels , label annotations from all domains are provided simultaneously as a “multi-hot” label vector for each input sample, resulting in a standard multi-label learning problem.

Original: For Full tasks , raw data is separated into single-class classiﬁcation tasks according to additional manual task annotation, resulting in a standard multi-task learning problem.
Revised: For Full tasks , raw data is separated into single-class classiﬁcation tasks according to additional manual task annotation, resulting in a standard multi-task learning problem.

Original: D.3. M EASURING THE S UBJECTIVE E RROR OF B ASELINES IN C LASSIFICATION
Revised: [Table]

Original: Since the baselines in classiﬁcation experiments do not explicitly assign a separate model to each domain, the subjective error metric 8 does not directly apply to these baselines.
Revised: Since the baselines in classiﬁcation experiments do not explicitly assign a separate model to each domain, the subjective error metric 8 does not directly apply to these baselines.

Original: Hence, we estimate their subjective errors using another method: we directly select top- N predictions of these methods and compare them with the label spaces of different domains.
Revised: Hence, we estimate their subjective errors using another method: we directly select top- N predictions of these methods and compare them with the label spaces of different domains.

Original: We deﬁne the “coverage” of the selected top- N labels over domain d as
Revised: We deﬁne the “coverage” of the selected top- N labels over domain d as

Original: [Equation]
Revised: [Equation]

Original:  ( ∃ top- N labels for input x
Revised: 1 ( ∃ top- N labels for input x

Original: i that is in the label space of d ) , where l d denotes the total number of samples in domain d .
Revised: i that is in the label space of d ) , where l d denotes the total number of samples in domain d .

Original: The subjective error on d is then calculated as 1 − C OVERAGE ( d ) .
Revised: The subjective error on d is then calculated as 1 − C OVERAGE ( d ) .

Original: E A DDITIONAL E XPERIMENTAL R ESULTS AND V ISUALIZATIONS
Revised: E A DDITIONAL E XPERIMENTAL R ESULTS AND V ISUALIZATIONS

Original: In this section, we provide additional experimental results and visualizations.
Revised: In this section, we provide additional experimental results and visualizations.

Original: E. M ULTI -D IMENSIONAL O PEN - ENDED R EGRESSION
Revised: E. M ULTI -D IMENSIONAL O PEN - ENDED R EGRESSION

Original: To further demonstrate the efﬁcacy of subjective learning in regression problems, we conducted experiments on a real-world multidimensional regression dataset from UCI machine learning repository: Gas sensor array under dynamic gas mixtures dataset (Fonollosa et al., 2015).
Revised: To further demonstrate the efﬁcacy of subjective learning in regression problems, we conducted experiments on a real-world multidimensional regression dataset from UCI machine learning repository: Gas sensor array under dynamic gas mixtures dataset (Fonollosa et al., 2015).

Original: This dataset contains the recordings of 16 chemical sensors exposed to two dynamic gas mixtures and the aim is to predict the concentrations of gases, with 417,8504 instances and 16-dimensional attributes.
Revised: This dataset contains the recordings of 16 chemical sensors exposed to two dynamic gas mixtures and the aim is to predict the concentrations of gases, with 417,8504 instances and 16-dimensional attributes.

Original: We treat each gas mixture as one domain, respectively representing Ethylene & Methane (domain 1) and
Revised: We treat each gas mixture as one domain, respectively representing Ethylene & Methane (domain 1) and

Original: Ethylene & CO (domain 2) gas mixtures, and randomly split both domains into training (90%) and test sets (10%).
Revised: Ethylene & CO (domain 2) gas mixtures, and randomly split both domains into training (90%) and test sets (10%).

Original: In this task, we compare subjective learning (two models, trained on the union of both domains) with a vanilla single model regressor (trained on the union of both domains) and an oracle regressor with two models separately trained on each domain.
Revised: In this task, we compare subjective learning (two models, trained on the union of both domains) with a vanilla single model regressor (trained on the union of both domains) and an oracle regressor with two models separately trained on each domain.

Original: We report root mean square error (RMSE) on each domain and the macro-average RMSE over both domains in Table 2.
Revised: We report root mean square error (RMSE) on each domain and the macro-average RMSE over both domains in Table 2.

Original: The results indicate that subjective learning beneﬁts from its automatic data allocation process, surpassing the vanilla baseline that trains a single global model by a large margin and performs similarly with the oracle.
Revised: The results indicate that subjective learning beneﬁts from its automatic data allocation process, surpassing the vanilla baseline that trains a single global model by a large margin and performs similarly with the oracle.

Original: E. S IMULATED C ONCEPT S HIFT C LASSIFICATION
Revised: E. S IMULATED C ONCEPT S HIFT C LASSIFICATION

Original: To further demonstrate the applicability of subjective learning, we conducted an experiment on
Revised: To further demonstrate the applicability of subjective learning, we conducted an experiment on

Original: Fashion Product Images with simulated concept shift between domains with the same label spaces.
Revised: Fashion Product Images with simulated concept shift between domains with the same label spaces.

Original: Concretely, we randomly split the dataset into two domains according to the “gender” attribute: 50% of samples labeled as “Male” and 50% samples labeled as “Female” are assigned to domain[Figure] [Table] 1 with “Male” relabeled as “1” and “Female” relabeled as “0”, and other samples are assigned to domain 2 with “Male” relabeled as “0” and “Female” relabeled as “1”.
Revised: Concretely, we randomly split the dataset into two domains according to the “gender” attribute: 50% of samples labeled as “Male” and 50% samples labeled as “Female” are assigned to domain 1 with “Male” relabeled as “1” and “Female” relabeled as “0”, and other samples are assigned to domain 2 with “Male” relabeled as “0” and “Female” relabeled as “1”.

Original: As shown in Figure 4, this setting resembles a simple scenario of concept shift caused by human preference: the label “1” can be interpreted as an indicator of “interested” or “recommended”, while the label “0” can be interpreted as an indicator of “not interested” or “not recommended”.
Revised: As shown in Figure 4, this setting resembles a simple scenario of concept shift caused by human preference: the label “1” can be interpreted as an indicator of “interested” or “recommended”, while the label “0” can be interpreted as an indicator of “not interested” or “not recommended”.

Original: Since different people may  have different preferences (in this simulated experiment this is caused by gender – domain 1 and domain 2 respectively represents appropriate recommendations to male and female users), the data samples from different domains may possess different input-label conditional probability P ( y | x ) , albeit with the same label space.
Revised: Since different people may [Figure] [Table] have different preferences (in this simulated experiment this is caused by gender – domain 1 and domain 2 respectively represents appropriate recommendations to male and female users), the data samples from different domains may possess different input-label conditional probability P ( y | x ) , albeit with the same label space.

Original: Since the label space is binary and completely shared by both domains, multi-label baselines are inapplicable in this setting since they will always produce both labels “0” and “1” for every input and do not discriminate between different domains, which is unmeaningful.
Revised: Since the label space is binary and completely shared by both domains, multi-label baselines are inapplicable in this setting since they will always produce both labels “0” and “1” for every input and do not discriminate between different domains, which is unmeaningful.

Original: Therefore, we compare subjective learning with the single-model baseline (vanilla) and an oracle (Full-T) which separately train two models on both domains explicitly with data-domain correspondences known in advance.
Revised: Therefore, we compare subjective learning with the single-model baseline (vanilla) and an oracle (Full-T) which separately train two models on both domains explicitly with data-domain correspondences known in advance.

Original: We report the model error and the subjective error of subjective learning in Table 3.
Revised: We report the model error and the subjective error of subjective learning in Table 3.

Original: Results show that the vanilla single-model baseline cannot learn meaningful prediction results since it lacks the mechanism to distinguish conﬂicting samples from different domains, while subjective learning achieves relatively small subjective errors and similar model errors as the Full-T oracle, which demonstrate that subjective learning is effective even in the context where different domains possess exactly the same label spaces yet different input-label relations.
Revised: Results show that the vanilla single-model baseline cannot learn meaningful prediction results since it lacks the mechanism to distinguish conﬂicting samples from different domains, while subjective learning achieves relatively small subjective errors and similar model errors as the Full-T oracle, which demonstrate that subjective learning is effective even in the context where different domains possess exactly the same label spaces yet different input-label relations.

Original: E. T RAINING AND I NFERENCE T IME ON F ASHION P RODUCT I MAGES
Revised: E. T RAINING AND I NFERENCE T IME ON F ASHION P RODUCT I MAGES

Original: In Table 4, we compare the computational cost of subjective learning and other baselines in terms of training and inference time on the Fashion Product Images dataset.
Revised: In Table 4, we compare the computational cost of subjective learning and other baselines in terms of training and inference time on the Fashion Product Images dataset.

Original: We measure the required wall-clock time (in seconds) for each method to reach convergence during training as well as the averaged wall-clock time for each method to predict all labels of one given input (in milliseconds).
Revised: We measure the required wall-clock time (in seconds) for each method to reach convergence during training as well as the averaged wall-clock time for each method to predict all labels of one given input (in milliseconds).

Original: Concretely, for subjective learning and all baselines except for two oracles (Full-L and Full-T), we train the models for 50 epochs with about 15,000 images in every epoch; for Full-L and Full-T, we train for 10 epochs since these methods generally converge faster.
Revised: Concretely, for subjective learning and all baselines except for two oracles (Full-L and Full-T), we train the models for 50 epochs with about 15,000 images in every epoch; for Full-L and Full-T, we train for 10 epochs since these methods generally converge faster.

Original: For each method, we test its total inference time on the same 3,000 test samples randomly sampled from the test set and report the[Table] mean inference time on each test sample.
Revised: For each method, we test its total inference time on the same 3,000 test samples randomly sampled from the test set and report the mean inference time on each test sample.

Original: All results are obtained with PyTorch using a NVIDIA 2080ti GPU.
Revised: All results are obtained with PyTorch using a NVIDIA 2080ti GPU.

Original: As shown in the table, the time cost of subjective learning is generally on par with or lower than baselines that also involve iterative training (Pseudo-L and LabelProp).
Revised: As shown in the table, the time cost of subjective learning is generally on par with or lower than baselines that also involve iterative training (Pseudo-L and LabelProp).

Original: Although the ProbCon baseline trains faster, it learns a global model without the mechanism of data allocation and thus performs signiﬁcantly worse than subjective learning as we have shown.
Revised: Although the ProbCon baseline trains faster, it learns a global model without the mechanism of data allocation and thus performs signiﬁcantly worse than subjective learning as we have shown.

Original: Meanwhile, compared  with the Full-T oracle that knows the data-domain correspondences in advance, subjective learning only exhibits a little additional computational overhead.
Revised: Meanwhile, compared [Table] with the Full-T oracle that knows the data-domain correspondences in advance, subjective learning only exhibits a little additional computational overhead.

Original: This indicates that although subjective learning incorporates an extra data allocation process implemented by the subjective function, this process only induces a limited computational cost since it only requires additional network forward process without loss backpropagation, which is in general efﬁcient.
Revised: This indicates that although subjective learning incorporates an extra data allocation process implemented by the subjective function, this process only induces a limited computational cost since it only requires additional network forward process without loss backpropagation, which is in general efﬁcient.

Original: E. I TERATION P ROCESS ON THE R EGRESSION T ASK
Revised: E. I TERATION P ROCESS ON THE R EGRESSION T ASK

Original: In subjective learning, since the performance of the low-level models will impact the data allocation process of the high-level subjective function, the training of the subjective function and low-level networks can be regarded as an iterative process, as shown by Figure 5 with an example in regression, in which the different colored lines are designated to different subjects.
Revised: In subjective learning, since the performance of the low-level models will impact the data allocation process of the high-level subjective function, the training of the subjective function and low-level networks can be regarded as an iterative process, as shown by Figure 5 with an example in regression, in which the different colored lines are designated to different subjects.

Original: All networks are randomly initialized, and in each iteration, each sample may be reallocated by the subjective function and used to further train the low-level networks.
Revised: All networks are randomly initialized, and in each iteration, each sample may be reallocated by the subjective function and used to further train the low-level networks.

Original: With the increasing of iterations, both subjective and model errors will reduce and converge along with the global loss.
Revised: With the increasing of iterations, both subjective and model errors will reduce and converge along with the global loss.

Original: The last subﬁgure displays the ﬁnal decision boundary of subjective function.
Revised: The last subﬁgure displays the ﬁnal decision boundary of subjective function.

Original: E. F EATURE V ISUALIZATION ON THE C LASSIFICATION T ASK
Revised: E. F EATURE V ISUALIZATION ON THE C LASSIFICATION T ASK

Original: The subjective learning approach can extract different semantics from the same input sample, and map them to different feature spaces.
Revised: The subjective learning approach can extract different semantics from the same input sample, and map them to different feature spaces.

Original: Figure 6 displays the features output under all subjectives, where each color represents a subjective and each point corresponds to an image in the dataset.
Revised: Figure 6 displays the features output under all subjectives, where each color represents a subjective and each point corresponds to an image in the dataset.

Original: F M ORE D ISCUSSIONS ON R ELATED W ORK
Revised: F M ORE D ISCUSSIONS ON R ELATED W ORK

Original: In this section, we provide more discussion on related work.
Revised: In this section, we provide more discussion on related work.

Original: Ensemble learning.
Revised: Ensemble learning.

Original: Ensemble learning approaches typically employ multiple models to cooperatively solve a given task (Dietterich, 2002; Zhang & Ma, 2012; Sagi & Rokach, 2018; Zhou, 2021).
Revised: Ensemble learning approaches typically employ multiple models to cooperatively solve a given task (Dietterich, 2002; Zhang & Ma, 2012; Sagi & Rokach, 2018; Zhou, 2021).

Original: The prediction of each model is combined by weighting (boosting), majority voting (bagging) or learning a second-level meta-learner (stacking).
Revised: The prediction of each model is combined by weighting (boosting), majority voting (bagging) or learning a second-level meta-learner (stacking).

Original: Since different models process the same set of data (although sometimes with different sample weights), in ensemble learning there is typically no explicit “hard” allocation process between the data and models.
Revised: Since different models process the same set of data (although sometimes with different sample weights), in ensemble learning there is typically no explicit “hard” allocation process between the data and models.

Original: In contrast, the multi-model architecture of subjective learning is driven by the inherent conﬂict in conﬂicting data, and each model only handles a proportion of the whole dataset without overlapping.
Revised: In contrast, the multi-model architecture of subjective learning is driven by the inherent conﬂict in conﬂicting data, and each model only handles a proportion of the whole dataset without overlapping.

Original: Domain adaptation and domain generalization.
Revised: Domain adaptation and domain generalization.

Original: Domain adaptation (Ben-David et al., 2010;
Revised: Domain adaptation (Ben-David et al., 2010;

Original: Pan & Yang, 2010; Tan et al., 2018; Wang & Deng, 2018; Hoffman et al., 2018) and domain generalization (Blanchard et al., 2011; Muandet et al., 2013; Zhou et al., 2021) consider the scenario where the learner trained on one or multiple source domain(s) is transferred to one or multiple new target domain(s).
Revised: Pan & Yang, 2010; Tan et al., 2018; Wang & Deng, 2018; Hoffman et al., 2018) and domain generalization (Blanchard et al., 2011; Muandet et al., 2013; Zhou et al., 2021) consider the scenario where the learner trained on one or multiple source domain(s) is transferred to one or multiple new target domain(s).

Original: Typically, domain adaptation focuses on the problem where there exist some labeled or unlabeled instances in the new domain, while domain generalization considers the setting where there the information of the new domains is inaccessible during training (i.e., zero-shot generalization).
Revised: Typically, domain adaptation focuses on the problem where there exist some labeled or unlabeled instances in the new domain, while domain generalization considers the setting where there the information of the new domains is inaccessible during training (i.e., zero-shot generalization).

Original: In other words, these formulations focus on the adaptation or generalization capability of the model on target domain(s).
Revised: In other words, these formulations focus on the adaptation or generalization capability of the model on target domain(s).

Original: In contrast, subjective learning focuses on the multi-domain training process and considers the scenario where directly training a global model using the data from multiple conﬂicting domains is problematic, and aims to resolve this training issue by performing automatic data allocation.
Revised: In contrast, subjective learning focuses on the multi-domain training process and considers the scenario where directly training a global model using the data from multiple conﬂicting domains is problematic, and aims to resolve this training issue by performing automatic data allocation.

Original: Another important difference is that in domain adaptation and domain generalization the data-domain correspondences are available, while subjective learning weakens this assumption by only assuming that the data from each sampling episode is obtained from the same domain.
Revised: Another important difference is that in domain adaptation and domain generalization the data-domain correspondences are available, while subjective learning weakens this assumption by only assuming that the data from each sampling episode is obtained from the same domain.