Title: Minimax Forward and Backward Learning of Evolving Tasks with Performance Guarantees
Venue: NeurIPS 2023
Accepted: yes
Keywords: Concept drift

Continual learning

Minimax classification

Performance guarantees
Abstract: For a sequence of classification tasks that arrive over time, it is common that tasks are evolving in the sense that consecutive tasks often have a higher similarity. The incremental learning of a growing sequence of tasks holds promise to enable accurate classification even with few samples per task by leveraging information from all the tasks in the sequence (forward and backward learning). However, existing techniques developed for continual learning and concept drift adaptation are either designed for tasks with time-independent similarities or only aim to learn the last task in the sequence. This paper presents incremental minimax risk classifiers (IMRCs) that effectively exploit forward and backward learning and account for evolving tasks. In addition, we analytically characterize the performance improvement provided by forward and backward learning in terms of the tasks’ expected quadratic change and the number of tasks. The experimental evaluation shows that IMRCs can result in a significant performance improvement, especially for reduced sample sizes.

Minimax Forward and Backward Learning of Evolving Tasks with Performance Guarantees

Abstract

For a sequence of classification tasks that arrive over time, it is common that tasks are evolving in the sense that consecutive tasks often have a higher similarity. The incremental learning of a growing sequence of tasks holds promise to enable accurate classification even with few samples per task by leveraging information from all the tasks in the sequence (forward and backward learning). However, existing techniques developed for continual learning and concept drift adaptation are either designed for tasks with time-independent similarities or only aim to learn the last task in the sequence. This paper presents incremental minimax risk classifiers (IMRCs) that effectively exploit forward and backward learning and account for evolving tasks. In addition, we analytically characterize the performance improvement provided by forward and backward learning in terms of the tasks' expected quadratic change and the number of tasks. The experimental evaluation shows that IMRCs can result in a significant performance improvement, especially for reduced sample sizes.

1 Introduction

In practical scenarios, it is often of interest to incrementally learn a growing sequence of classification problems (tasks) that arrive over time. In such a sequence, it is common that tasks are evolving in the sense that consecutive tasks often have a higher similarity. Examples of evolving tasks are the classification of portraits from different time periods [1] and the classification of spam emails over time [2]; in these problems, the similarity between consecutive tasks (portraits of consecutive time periods and emails from consecutive years) is significantly higher (see Figure 1). The incremental learning of a growing sequence of tasks holds promise to significantly improve performance by leveraging information from different tasks. Specifically, at each time step, information from preceding tasks can be used to improve the performance of the last task (forward learning) and, reciprocally, the information from the last task can be used to improve the performance of the preceding tasks (backward learning) [3-5]. Such transfer of information can enable accurate classification even in cases with reduced sample sizes, thus significantly increasing the effective sample size (ESS) of each task. However, exploiting the benefits of forward and backward learning is challenging due to the continuous arrival of samples from tasks characterized by different underlying distributions [6-8].

Techniques developed for concept drift adaptation (aka learning in a dynamic scenario) [9-16] are designed for evolving tasks but only aim to learn the last task in the sequence. In particular, methods based on learning rates learn the last task by slightly updating the classification rule for the preceding task [12, 13]; and methods based on sliding windows learn the last task by using a set of stored samples from the most recent preceding tasks [14, 15]. In concept drift adaptation, at each time

Forward learning

Single-task learning

Forward and backward learning

2000

1990

1970

1980

p

1

p

2

p

3

p

4

U

1

U

2

U

3

U

4

step only the last task is considered of interest, and is learned leveraging information from the most recent preceding tasks since they are the most similar to the last task.

Techniques developed for continual learning (aka lifelong learning) [4-7, 17-20] aim to learn the whole sequence of tasks but existing methods are designed for situations where tasks' similarities do not depend on the time steps when tasks are observed. In particular, methods based on dynamic architectures learn shared parameters using samples from all the tasks together with task-specific parameters using samples from the corresponding task [6, 20]; and methods based on replay learn parameters using a pool of stored samples from all the preceding tasks together with the samples from the last task [5, 7]. Existing methods for continual learning are not designed for evolving tasks and consider scenarios in which the order of the tasks in the sequence is not relevant. In the current literature of continual learning, only [21] considers scenarios with evolving tasks but focus on the theoretical analysis of transferring information from the preceding tasks.

This paper presents incremental minimax risk classifiers (IMRCs) that determine classification rules minimizing worst-case error probabilities over uncertainty sets that can contain the sequence of evolving underlying distributions. The proposed techniques can effectively exploit forward and backward learning by obtaining uncertainty sets that get smaller using information from all the tasks (see Figure 1). Specifically, the main contributions presented in the paper are as follows.

We propose forward learning techniques that recursively use the information from preceding tasks to reduce the uncertainty set of the last task.

We propose forward and backward learning techniques that use the information from the last task to further reduce the sequence of uncertainty sets obtained with forward learning.

We analytically characterize the increase in ESS provided by the presented methods in terms of the expected quadratic change between consecutive tasks and the number of tasks.

We numerically quantify the performance improvement provided by IMRCs in comparison with existing techniques using multiple datasets, sample sizes, and number of tasks.

Notations Calligraphic letters represent sets; ‖ · ‖ 1 and ‖ · ‖ ∞ denote the 1 -norm and the infinity norm of its argument, respectively; /precedesequal and /followsequal denote vector inequalities; I {·} denotes the indicator function; and E p { · } and V ar p {·} denote the expectation and the variance of its argument with respect to distribution p . For a vector v , v ( i ) and v T denote its i-th component and transpose. Non-linear operators acting on vectors denote component-wise operations. For instance, | v | and v 2 denote the vector formed by the absolute value and the square of each component, respectively. For the reader's convenience, we also provide in Table 2 in Appendix A a list with the main notions used in the paper and their corresponding notations.

2 Preliminaries

This section first formulates the problem of incrementally learning a growing sequence of tasks, and describes evolving tasks in comparison with the time-independent assumption common in continual learning. Then, we briefly describe MRCs that determine classification rules by minimizing the worst-case error probability over an uncertainty set.

2.1 Problem formulation

In the following, we denote by X the set of instances or attributes, Y the set of labels or classes, ∆( X ×Y ) the set of probability distributions over X ×Y , and T ( X , Y ) the set of classification rules. A classification task is characterized by an underlying distribution p ∗ ∈ ∆( X × Y ) , and standard supervised classification methods use a sample set D = { ( x i , y i ) } n i =1 formed by n i.i.d samples from distribution p ∗ to find a classification rule h ∈ T ( X , Y ) with small expected loss /lscript (h , p ∗ ) .

Evolving tasks are considered by methods for concept drift adaptation, but existing continual learning methods are designed for scenarios where tasks' similarities do not depend on the time steps when tasks are observed. These scenarios are usually mathematically modeled assuming that the tasks' distributions p 1 , p 2 , . . . are independent and identically distributed (i.i.d.) [18, 19]. In this paper, we develop techniques designed for evolving tasks that can be mathematically modeled assuming that the changes between consecutive distributions p 2 -p 1 , p 3 -p 2 , . . . are independent and zero-mean (evolving task assumption). The assumption in this paper is not stronger than the usual i.i.d. assumption and can better describe evolving tasks; their main difference is considering independent changes between consecutive distributions instead of independent distributions. Note that with independent distributions the difference between the distributions of the i -th and the ( i + t ) -th tasks has zero mean and variance V ar { p i + t -p i } = V ar { p i +1 -p i } = 2 V ar { p 1 } that does not depend on t . On the other hand, with independent changes between consecutive distributions, the difference between the distributions of the i -th and the ( i + t ) -th tasks has zero mean and variance V ar { p i + t -p i } = ∑ t j =1 V ar { p i + j -p i + j -1 } that increases with t . Appendix H further describes how the assumption used in the paper is more appropriate for evolving tasks than the conventional i.i.d. assumption using real datasets.

In the addressed settings, sample sets D 1 , D 2 , . . . arrive over time steps 1 , 2 , . . . corresponding with different classification tasks characterized by underlying distributions p 1 , p 2 , . . . . Incremental learning aims to continually learn over time the growing sequence of tasks exploiting information acquired from all the tasks. At each time step k , learning methods obtain classification rules h 1 , h 2 , . . . , h k for the current sequence of k tasks using the new sample set D k and certain information retained from step k -1 . The performance of learning methods is assessed at each time step k by quantifying the performance in the current k tasks. For instance, overall performance can be assessed by the averaged error 1 k ∑ k j =1 /lscript (h j , p j ) with /lscript (h j , p j ) the expected loss of the classification rule h j for distribution p j .

2.2 Minimax risk classifiers

The methods presented in the paper are based on robust risk minimization [22, 23] instead of empirical risk minimization since training samples corresponding to different tasks follow different distributions. In particular, we utilize MRCs [24, 25] that learn classification rules by minimizing the worst-case expected loss against an uncertainty set given by constraints on the expectation of a feature mapping Φ : X × Y → R m as

where τ denotes a mean vector of expectation estimates and λ denotes a confidence vector. Feature mappings are vector-valued functions over X × Y , e.g., one-hot encodings of labels [26] with instances represented by values from the last layers in a neural network [8, 27] or by random Fourier features (RFF) [28, 29].

Given the uncertainty set U , MRC rules are solutions of the optimization problem

/negationslash where R ( U ) denotes the minimax risk and /lscript (h , p) denotes the expected loss of classification rule h for distribution p . In the following, we utilize the 0-1-loss so that /lscript (h , p) = E p { I { h( x ) = y }} and the expected loss with respect to the underlying distribution becomes the error probability of the classification rule. Deterministic MRCs assign each instance x ∈ X with the label h( x ) ∈ arg max y ∈Y Φ( x, y ) T µ ∗ where the parameter µ ∗ is the solution of the convex optimization problem

given by the Lagrange dual of (2) [24, 25].

The baseline approach of single-task learning obtains a classification rule h j for each j -th task leveraging information only from the sample set D j = { ( x j,i , y j,i ) } n j i =1 given by n j samples from distribution p j . In that case, IMRCs coincide with MRCs for standard supervised classification that obtain the mean and confidence vectors as

with σ 2 j an estimate of V ar p j { Φ( x, y ) } , e.g., the sample variance of the n j samples. The vector s j describes the mean squared errors (MSEs) of the mean vector components and directly gives the confidence vector λ j as shown in (4).

MRCs provide bounds for the minimax risk R ( U j ) in terms of the smallest minimax risk as described in [24, 25, 30]. The smallest minimax risk is that corresponding to the ideal case of knowing mean vectors exactly, that is, the minimax risk corresponding with the uncertainty set U ∞ j = { p ∈ ∆( X × Y ) : E p { Φ( x, y ) } = τ ∞ j } given by the expectation τ ∞ j = E p j { Φ( x, y ) } . In the baseline approach of single-task learning, if R ∞ j and µ ∞ j denote the smallest minimax risk and the MRC parameter corresponding with U ∞ j , with probability at least 1 -δ we have that

where M and κ are such that ‖ Φ( x, y ) ‖ ∞ ≤ M for any ( x, y ) ∈ X × Y and subG (Φ j ) /precedesequal κ σ j , subG ( · ) denotes the sub-Gaussian parameter of the argument components, and Φ j denotes the random variable given by the feature mapping of samples from the j -th task. Inequality (5) is obtained using the bounds in [25] together with the Chernoff bound [31] for sub-Gaussian variables.

In the following sections, we describe techniques that obtain the mean and MSE vectors using forward and backward learning. Once such vectors are obtained, IMRC methods obtain the classifier parameter µ j for each j -th task solving the convex optimization problem in (3) that can be efficiently addressed using conventional methods [32, 33].

3 Forward learning with performance guarantees

This section presents the recursions that allow to obtain mean and MSE vectors for each task retaining information from preceding tasks. In addition, it characterizes the increase in ESS provided by forward learning in terms of the tasks' expected quadratic change and the number of tasks.

3.1 Forward learning

Let τ j ⇀ and s j ⇀ denote the mean and MSE vectors for forward learning corresponding to the j -th task. The following recursions allow to obtain τ j ⇀ and s j ⇀ for each j -th task using the vectors for the preceding task τ j ⇀ -1 , s j ⇀ -1 as

with τ j and s j given by (4) and τ ⇀ 1 = τ 1 and s ⇀ 1 = s 1 .

The vector d 2 i assesses the expected quadratic change between consecutive tasks described by w i = τ ∞ i -τ ∞ i -1 . Taking d 2 i = E { w 2 i } and σ 2 i = V ar p i { Φ( x, y ) } for any i , the first recursion in (6) provides the unbiased linear estimator of the mean vector τ ∞ j based on D 1 , D 2 , . . . , D j that has the minimum MSE, while the second recursion in (6) provides its MSE (see Appendix B for a detailed derivation). Vectors σ 2 i and d 2 i can be estimated online using the sample sets. In particular, σ 2 i can be estimated as the sample variance, while d 2 i can be estimated using sample averages as

Recursions in (6) obtain mean and MSE vectors for the j -th task by using information from preceding tasks and the j -th sample set D j . Specifically, the first recursion in (6) obtains the mean vector τ j ⇀ by adding a correction to the sample average τ j . This correction is proportional to the difference between τ j and τ j ⇀ -1 with a proportionality constant that depends on the MSE vectors s j , s j ⇀ -1 and the expected quadratic change d 2 j . In particular, if s j /lessmuch s j ⇀ -1 + d 2 j , the mean vector is given by the sample average as in single-task learning, and if s j /greatermuch s j ⇀ -1 + d 2 j , the mean vector is given by that of the preceding task. Note that for forward learning, at each step k , only the vectors for the last task τ k ⇀ and s k ⇀ need to be obtained from those of the ( k -1) -th task, the vectors for the remaining tasks stay the same as at step k -1 (see also Fig. 2 and Alg. 1 below).

3.2 Performance guarantees and effective sample sizes with forward learning

The following result provides bounds for the minimax risk for each task using forward learning.

Theorem 1. Let U j ⇀ be the uncertainty set given by (1) using the mean and confidence vectors τ j ⇀ and λ j ⇀ = √ s j ⇀ provided by (6), and let κ be such that subG (Φ j ) /precedesequal κ σ j and subG ( w j ) /precedesequal κ d j for j = 1 , 2 , . . . , k . Then, under the evolving task assumption of Section 2, we have with probability at least 1 -δ that with n ⇀ 1 = n 1 and n j ⇀ ≥ n j + n j ⇀ -1 ‖ σ 2 j ‖ ∞ ‖ σ 2 j ‖ ∞ + n j ⇀ -1 ‖ d 2 j ‖ ∞ for j ≥ 2 .

The value n j ⇀ in (8) is the ESS of the proposed IMRC method with forward learning since the bound in (8) coincides with that of single-task learning in (5) if the sample size for the j -th task is n j ⇀ . The ESS of each task is obtained by adding a fraction of the ESS for the preceding task to the sample size. In particular, if d 2 j is large, the ESS is given by the sample size, while if d 2 j is small, the ESS is given by the sum of the sample size and the ESS of the preceding task.

The bound in (8) shows that recursions in (6) do not need to use very accurate values for σ j and d j . Specifically, the coefficienty κ in (8) can be taken to be small as long as the values used for σ j and d j are not much lower than the sub-Gaussian parameters of Φ j and w j , respectively. In particular, κ is smaller than the maximum of M/ min j,i { σ ( i ) j } and 2 M/ min j,i { d ( i ) j } due to the bound for the sub-Gaussian parameter of bounded random variables (see e.g., Section 2.1.2 in [31]).

Theorem 1 shows the increase in ESS in terms of the ESS of the preceding task. The following result allows to directly quantify the ESS in terms of the tasks' expected quadratic change and the number of tasks.

Theorem 2. If ‖ d 2 j ‖ ∞ ≤ d 2 , M ≤ 1 , and n j ≥ n for j = 1 , 2 , . . . , k , then for any j ∈ { 1 , 2 , . . . , k } , we have that the ESS in (8) can be taken so that it satisfies

In particular, if nd 2 < 1 /j 2 , we have that n j ⇀ ≥ n ( 1 + j -1 3 ) . Proof. See Appendix D.

The above theorem characterizes the increase in ESS provided by forward learning in terms of the tasks' expected quadratic change. Such increase grows monotonically with the number of preceding tasks j as shown in (9) and becomes proportional to j when the expected quadratic change is smaller than 1 / ( j 2 n ) . Figure 3 below further illustrates the increase in ESS with respect to the sample size ( n j ⇀ /n ) due forward learning in comparison with forward and backward learning.

4 Forward and backward learning with performance guarantees

This section presents the recursions that allow to obtain mean and MSE vectors for each task leveraging information from all the tasks. In addition, it characterizes the increase in ESS provided by forward and backward learning in terms of the tasks' expected quadratic change and the number of tasks.

Backward learning is more challenging than forward learning since the new task provides additional information for preceding tasks at each time step, while the information from preceding tasks is always the same. The techniques proposed below for backward learning effectively increase the ESS over time by carefully accounting for the new information at each step.

4.1 Forward and backward learning

At each step k , the proposed techniques learn to classify each j -th task leveraging information obtained from the j preceding tasks (tasks { 1 , 2 , . . . , j } ) and from the k -j succeeding tasks (tasks { j +1 , j +2 , . . . , k } ). From preceding tasks, we obtain the forward mean and MSE vectors τ j ⇀ , s j ⇀ using recursions in (6), while from succeeding tasks, we obtain the backward mean and MSE vectors τ ↽k j , s ↽k j using recursions in (6) in retrodiction. Specifically, vectors τ ↽k j and s ↽k j are obtained using the same recursion as for τ j ⇀ and s j ⇀ in (6) with s ↽k j +1 , d 2 j +1 , and τ ↽k j +1 instead of s j ⇀ -1 , d 2 j , and τ j ⇀ -1 .

Let τ /harpoonleftright k j and s /harpoonleftright k j denote the mean and MSE vectors for forward and backward learning corresponding to the j -th task for j ∈ { 1 , 2 , . . . , k } . The following recursions allow to obtain the mean and MSE vectors τ /harpoonleftright k j and s /harpoonleftright k j for each j -th task using those vectors for forward learning τ j ⇀ , s j ⇀ and backward learning τ ↽k j +1 , s ↽k j +1 as

with τ ↽k k = τ k , s ↽k k = s k and τ /harpoonleftright k k = τ k ⇀ , s /harpoonleftright k k = s k ⇀ . Analogously to the case of forward learning in Section 3.1, taking d 2 i = E { w 2 i } and σ 2 i = V ar p i { Φ( x, y ) } for any i , the first recursion in (10) provides the unbiased linear estimator of the mean vector τ ∞ j based on D 1 , D 2 , . . . , D j and D j +1 , D j +2 , . . . , D k that has the minimum MSE, while the second recursion in (10) provides its MSE (see Appendix B for a detailed derivation).

Recursions in (10) obtain at step k the mean and MSE vectors for the j -th task by retaining information from preceding tasks and acquiring information from the new task. Specifically, the first recursion in (10) obtains the mean vector τ /harpoonleftright k j by adding a correction to the mean vector of the corresponding task τ j ⇀ obtained for forward learning. This correction is proportional to the difference between τ j ⇀ and τ ↽k j +1 with a proportionality constant that depends on the MSE vectors s j ⇀ , s ↽k j +1 and the expected quadratic change d 2 j +1 . In particular, if s j ⇀ /lessmuch s ↽k j +1 + d 2 j +1 , the mean vector is given by that of the corresponding task for forward learning, and if s j ⇀ /greatermuch s ↽k j +1 + d 2 j +1 , the mean vector is given by that of the next task for backward learning.

4.2 Implementation

This section describes the implementation of the proposed IMRCs with forward and backward learning and its computational and memory complexities.

Figure 2 depicts the flow diagram for the proposed IMRC methodology. The presented techniques carefully avoid the repeated usage of the same information from the sequence

of tasks. At each step k , the IMRC method obtains forward mean vector τ k ⇀ for the k -th task leveraging information from preceding tasks using the forward mean vector τ k ⇀ -1 and the sample average τ k . Reciprocally, backward mean vectors τ ↽k j +1 for each j -th task are obtained leveraging information from the k -th task through the sample average τ k .

τ

j

-

1

τ

j

τ

j

+1

τ

/harpoonleftright

k

j

τ

j ⇀

-

1

τ

j ⇀

τ

j ⇀

+1

τ

j

+1

τ

j

-

1

τ

j

τ

/harpoonleftright

k

j

-

1

Then, the forward and backward mean vectors τ /harpoonleftright k j are obtained from the forward mean vectors τ j ⇀ and the backward mean vectors τ ↽k j +1 . In particular, τ j ⇀ provides the information from the preceding tasks 1 , 2 , . . . , j , while τ ↽k j +1 provides the information from the succeeding tasks j + 1 , j + 2 , . . . , k . At each step k , the IMRC method obtains forward and backward mean vectors τ /harpoonleftright k j for j = k -b, k -b + 1 , . . . , k with b the number of backward steps. In particular, if b = 0 , IMRC carries out only forward learning. Note that, at each step k , the proposed IMRC methods only need to retain the forward mean vectors τ j ⇀ and sample averages τ j for j = k -b, k -b +1 , . . . , k .

Algorithm 1 IMRC at step k

Algorithm 1 details the implementation of the proposed IMRCs at each step. For k steps, IMRCs have computational complexity O (( b +1) Kmk ) and memory complexity O (( b + k ) m ) where K is the number of iterations used for the convex optimization problem (3), m is the length of the feature vector, and b is the number of backward steps. The complexity of forward and backward learning increases proportionally to the number of backward steps that can be taken to be rather small, as shown in the following. Even more efficient implementations can be obtained using Rauch-TungStriebel recursions (see e.g., Section 7.2 in [34]) that can be used to obtain τ /harpoonleftright k j from τ /harpoonleftright k j +1 , as shown in Appendix E.

4.3 Performance guarantees and effective sample sizes with forward and backward learning

The following result provides bounds for the minimax risk for each task using forward and backward learning

Theorem 3. Let U /harpoonleftright k j be the uncertainty set given by (1) using the mean and confidence vector τ /harpoonleftright k j and λ /harpoonleftright k j = √ s /harpoonleftright k j provided by (10), and let κ and n j ⇀ be as in Theorem 1. Then, under the evolving task assumption of Section 2, we have with probability at least 1 -δ that

Theorem 3 shows that the methods proposed can increase ESS of each task by leveraging information from all the tasks. In particular, the bounds for forward and backward learning provided by inequality (11) are significantly lower than those for forward learning in Theorem 1. The ESS of each task is obtained by adding a fraction of the ESS for the next task to the ESS of the corresponding task using forward learning. In particular, if d 2 j is large, the ESS is given by that with forward learning, while if d 2 j is small, the ESS is given by the sum of the ESS using forward learning and the ESS of the next task.

Theorem 3 shows the increase in ESS in terms of the ESS with forward learning and the ESS of the next task. The following result allows to directly quantify the ESS in terms of the tasks' expected quadratic change and the number of tasks.

Theorem 4. If ‖ d 2 j ‖ ∞ ≤ d 2 , M ≤ 1 , and n j ≥ n for j = 1 , 2 , . . . , k , then for any j ∈ { 1 , 2 , . . . , k } , we have that the ESS in (11) can be taken so that it satisfies

In particular, for j ≥ 2 , we have that

The above theorem characterizes the increase in ESS provided by forward and backward learning. This increase grows monotonically with the number of preceding tasks j and with the number of succeeding tasks k -j as shown in (12).

ESS/

n

100

10

-

4

10

5

2

20

1

10

-

2

n

⇀

10

/n

n

/harpoonleftright

13

10

/n

n

/harpoonleftright

20

10

/n

n

⇀

100

/n

n

/harpoonleftright

103

100

/n

n

/harpoonleftright

110

100

/n

nd

2

In addition, it becomes proportional to the total number of tasks k when the expected quadratic change is smaller than 1 / ( j 2 n ) and j ≥ k/ 2 . Figure 3 further illustrates the increase in ESS with respect to the sample size ( n /harpoonleftright k j /n ) due to forward and backward learning in comparison with forward learning. As the figure shows, the increase in ESS can be classified into three regimes depending on the sample size n and the expected quadratic change d 2 . The ESS is only marginaly larger than the sample size for sizeble values of nd 2 (large samples sizes or drastic changes in the distribution); the ESS quickly increases when nd 2 becomes small (reduced sample sizes and moderate changes in the distribution); and the ESS becomes proportional to the total number of tasks k if nd 2 is rather small (very small sample sizes and very slow changes in the distribution).

5 Numerical results

This section first compares the classification performance of IMRCs with existing techniques using multiple datasets, then we show the performance improvement of the presented IMRCs due to forward and backward learning. In Appendix H, we provide additional implementation details and numerical results. The implementation of the proposed IMRCs is available on the web https:// github.com/MachineLearningBCAM/IMRCs-for-incremental-learning-NeurIPS-2023 .

The proposed method is evaluated using 12 public datasets composed by evolving tasks [10, 1, 35-41, 2]. Six datasets are formed by images with characteristics/quality/realism that change over time; while the rest are formed by tabular data that are segmented by time (see further details in AppendixH). For instance, each task in the 'Yearbook' dataset corresponds to portrait's photographs from one year from 1905 to 2013 and the goal is to predict males and females; and each task in the 'CLEAR' dataset corresponds to images with a natural temporal evolution of visual concepts per year from 2004 to 2014 and the goal is to predict if an image is soccer, hockey, or racing. For the image datasets, instances are represented by pixel values in "Rotated MNIST" dataset, and by the last layer of the ResNet18 pre-trained network [42] in the remaining datasets; while for the tabular datasets, instances are represented by 200 RFFs [28, 29] with scaling parameter σ 2 = 10 .

The proposed IMRC method is compared with 7 state-of-the-art-techniques: 2 techniques of continual learning based on experience replay [5, 17], a technique of continual learning based on regularization [6], a technique of continual learning based on dynamic architectures [4], 2 techniques of concept drift adaptation based on weight factors [9, 16], and a technique of concept drift adaptation based on sliding windows [10]. The hyper-parameters in all methods are set to the default values provided by the authors. IMRCs are implemented using b = 3 backward steps and the expected quadratic change d 2 j is estimated using W = 2 in (7). In Appendix H, among other additional results, we study the change in classification error and processing time achieved by varying the number b of backward steps.

In the first set of numerical results, we compare the performance of the proposed IMRCs with existing techniques using n = 10 samples per task. These numerical results are obtained computing the average classification error over all the time steps and tasks in 50 random instantiations of data samples. As can be observed in Table 1, IMRCs can significantly improve performance in evolving tasks with respect to existing methods. Certain techniques of continual learning or concept drift adaptation achieve top performance in specific datasets, but the performance improvement of the methods presented are realized over general datasets with evolving tasks. The results for n = 10 samples per task in Table 1 are complemented with results for n = 50 , 100 , and 150 samples per task in Tables 4, 5, and 6 in Appendix H. These results show that the improved performance of IMRCs compared to the state-of-the-art techniques is similar for different sample sizes. In addition, Table 1 shows that the average running time per task of IMRC methods is similar to other state-of-the-art methods.

Forward

n

= 100

Forward and backward

n

= 10

Forward

n

= 10

Forward and backward

n

= 100

Classification error/single-task

10

30

50

70

0.8

0.6

1

Number of tasks

k

Classification error

Forward

k

= 10

Single-task

Forward and backward

k

= 10

Forward

k

= 100

Forward and backward

k

= 100

Sample size

n

10

30

50

70

90

0.1

0.2

0.3

In the second set of numerical results, we analyze the contribution of forward and backward learning to the final performance of IMRCs. In particular, we show the relationship among classification error, number of tasks, and sample size for single-task, forward, and forward and backward learning. These numerical results are obtained averaging, for each number of tasks and sample size, the classification errors achieved with 10 random instantiations of data samples in "Yearbook" dataset (see Appendix H for further details). Figure 4a shows the classification error of IMRC method divided by the classification error of single-task learning for different number of tasks with n = 10 and n = 100 sample sizes. Such figure shows that forward and backward learning can significantly improve the performance of IMRCs as tasks arrive. In addition, Figure 4b shows the classification error of IMRC method for different sample sizes with k = 10 and k = 100 tasks. Such figure shows that IMRCs with forward and backward learning for k = 100 tasks and n = 10 samples achieve significantly better results than single-task learning using n = 100 samples. In particular, the experiments show that the methods proposed can effectively exploit backward learning that results in enhanced classification error in all the experimental results.

6 Conclusion

The paper proposes IMRCs that effectively exploit forward and backward learning and account for evolving tasks by using a sequence of uncertainty sets that can contain the true underlying distributions. In addition, the paper analytically characterizes the increase in ESS achieved by the proposed techniques in terms of the tasks' expected quadratic change and number of tasks. The numerical results assess the performance improvement of IMRCs with respect to existing methods using multiple datasets, sample sizes, and number of tasks. The proposed methodology for incremental learning with evolving tasks can lead to techniques that further approach the humans' ability to learn from few examples and to continuously improve on tasks that arrive over time.

Acknowledgments

Funding in direct support of this work has been provided by projects PID2022-137063NBI00, PID2022-137442NB-I00, CNS2022-135203, and CEX2021-001142-S funded by MCIN/AEI/10.13039/501100011033 and the European Union 'NextGenerationEU'/PRTR, BCAM Severo Ochoa accreditation CEX2021-001142-S / MICIN / AEI/ 10.13039/501100011033 funded by the Ministry of Science and Innovation, and programes ELKARTEK, IT1504-22, and BERC-2022-2025 funded by the Basque Government.

References

Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A Efros. A century of portraits: A visual historical record of american high school yearbooks. In IEEE International Conference on Computer Vision Workshops , 2015.

Tegjyot Singh Sethi and Mehmed Kantardzic. On the reliable detection of concept drift from streaming unlabeled data. Expert Systems with Applications , 82, 2017.

Zhiyuan Chen and Bing Liu. Lifelong machine learning . Morgan & Claypool Publishers, second edition, 2018.

Paul Ruvolo and Eric Eaton. ELLA: An efficient lifelong learning algorithm. In International Conference on Machine Learning , volume 30, 2013.

David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems , volume 30, 2017.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences , (13), 2017.

Christian Henning, Maria Cervera, Francesco D'Angelo, Johannes Von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F Grewe, and João Sacramento. Posterior meta-replay for continual learning. In Advances in Neural Information Processing Systems , volume 34, 2021.

Julio Hurtado, Alain Raymond, and Alvaro Soto. Optimizing reusable knowledge for continual learning via metalearning. In Advances in Neural Information Processing Systems , volume 34, 2021.

Peng Zhao, Le-Wen Cai, and Zhi-Hua Zhou. Handling concept drift via model reuse. Machine Learning , 109(3), 2020.

Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, and Phillip B Gibbons. DriftSurf: stable-state/reactive-state learning under concept drift. In International Conference on Machine Learning , volume 38, 2021.

Verónica Álvarez, Santiago Mazuelas, and Jose A Lozano. Minimax classification under concept drift with multidimensional adaptation and performance guarantees. In International Conference on Machine Learning , volume 39, 2022.

Jyrki Kivinen, Alexander J Smola, and Robert C Williamson. Online learning with kernels. IEEE Transactions on Signal Processing , 52(8), 2004.

Francesco Orabona, Joseph Keshet, and Barbara Caputo. The projectron: a bounded kernelbased perceptron. In International Conference on Machine Learning , volume 25, 2008.

Albert Bifet and Ricard Gavalda. Learning from time-changing data with adaptive windowing. In SIAM International Conference on Data Mining , 2007.

Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. Variational continual learning. In International Conference on Learning Representations , 2018.

Dariusz Brzezinski and Jerzy Stefanowski. Reacting to different types of concept drift: The accuracy updated ensemble algorithm. IEEE Transactions on Neural Networks and Learning Systems , 25(1), 2013.

Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations , 2018.

Andreas Maurer, Massimiliano Pontil, and Bernardino Romera-Paredes. The benefit of multitask representation learning. Journal of Machine Learning Research , 17(81), 2016.

Giulia Denevi, Dimitris Stamos, Carlo Ciliberto, and Massimiliano Pontil. Online-withinonline meta learning. In Advances in Neural Information Processing Systems , 2019.

Ju Xu and Zhanxing Zhu. Reinforced continual learning. In Advances in Neural Information Processing Systems , volume 31, 2018.

Anastasia Pentina and Christoph H Lampert. Lifelong learning with non-iid tasks. In Advances in Neural Information Processing Systems , volume 28, 2015.

Farzan Farnia and David Tse. A minimax approach to supervised learning. In Advances in Neural Information Processing Systems , volume 29, 2016.

Rizal Fathony, Anqi Liu, Kaiser Asif, and Brian D. Ziebart. Adversarial multiclass classification: A risk minimization perspective. In Advances in Neural Information Processing Systems , volume 29, 2016.

Santiago Mazuelas, Andrea Zanoni, and Aritz Perez. Minimax classification with 0-1 loss and performance guarantees. In Advances in Neural Information Processing Systems , volume 33, 2020.

Santiago Mazuelas, Mauricio Romero, and Peter Grünwald. Minimax risk classifiers with 0-1 loss. Journal of Machine Learning Research , 24(208):1-48, 2023.

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning . MIT press, 2018.

Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence , 35(8), 2013.

Jing Lu, Steven CH Hoi, Jialei Wang, Peilin Zhao, and Zhi-Yong Liu. Large scale online kernel learning. Journal of Machine Learning Research , 17(1), 2016.

Yanning Shen, Tianyi Chen, and Georgios B Giannakis. Random feature-based online multikernel learning in environments with unknown dynamics. The Journal of Machine Learning Research , 20(1), 2019.

Santiago Mazuelas, Yuan Shen, and Aritz Pérez. Generalized maximum entropy for supervised classification. IEEE Transactions on Information Theory , 68(4), 2022.

Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint . Cambridge University Press, 2019.

Yu Nesterov and Vladimir Shikhman. Quasi-monotone subgradient methods for nonsmooth convex minimization. Journal of Optimization Theory and Applications , 165(3), 2015.

Wei Tao, Zhisong Pan, Gaowei Wu, and Qing Tao. The strength of Nesterov's extrapolation in the individual convergence of nonsmooth optimization. IEEE Transactions on Neural Networks and Learning Systems , 31(7), 2019.

Brian DO Anderson and John B Moore. Optimal Filtering . Prentice-Hall, 1979.

Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing , 469, 2022.

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In IEEE International Conference on Computer Vision , 2019.

Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/regression by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition , 2017.

Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren. Gradient-based editing of memory examples for online task-free continual learning. In Advances in Neural Information Processing Systems , volume 34, 2021.

Zhiqiu Lin, Jia Shi, Deepak Pathak, and Deva Ramanan. The CLEAR benchmark: Continual learning on real-world imagery. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021.

Ioannis Katakis, Grigorios Tsoumakas, and Ioannis Vlahavas. An ensemble of classifiers for coping with recurring contexts in data streams, 2008.

Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition , 2016.

A Main notations used in the paper.

In this section we summarize the notations used in the paper. Table 2 shows the notation used for the mean vector, the confidence vector, the MSE vector, the ESS, the uncertainty set, and the minimax risk using single task, forward, backward, and forward and backward learning.

U

U

U

U

B Derivation of recursions in (6) for forward learning and recursions in (10) for forward and backward learning

This section shows how recursions in (6) and recursions in (10) are obtained using those for filtering and smoothing in linear dynamical systems.

The mean vectors evolve over time steps through the linear dynamical system

where vectors w j for j ∈ { 2 , 3 , . . . , k } are independent and zero-mean because p j -p j -1 are independent and zero-mean. In addition, each state variable τ ∞ j is observed at each step j through τ j that is the sample average of i.i.d. samples from p j , so that we have

where vectors v j for j ∈ { 1 , 2 , . . . , k } are independent and zero mean, and independent of w j for j ∈ { 1 , 2 , . . . , k } . Therefore, equations (13) and (14) above describe a linear dynamical system (state-space model with white noise processes) [34]. For such systems, the Kalman filter recursions provide the unbiased linear estimator with minimum MSE based on samples corresponding to preceding steps D 1 , D 2 , . . . , D j , and fixed-lag smoother recursions provide the unbiased linear estimator with minimum MSE based on samples corresponding to preceding and succeeding steps D 1 , D 2 , . . . , D k [34]. Then, equations in (6) and equations in (10) are obtained after some algebra from the Kalman filter recursions and fixed-lag smoother recursions, respectively.

C Proof of Theorem 1

Proof. To obtain bound in (8) we first prove that the mean vector estimate and the MSE vector given by (6) satisfy

for any component i = 1 , 2 , . . . , m . Then, we prove that ‖ √ s j ⇀ ‖ ∞ ≤ M/ √ n j ⇀ for j ∈ { 1 , 2 , . . . , k } , where the ESSs satisfy n ⇀ 1 = n 1 and n j ⇀ ≥ n j + n j ⇀ -1 ‖ σ 2 j ‖ ∞ ‖ σ 2 j ‖ ∞ + n j ⇀ -1 ‖ d 2 j ‖ ∞ for j ≥ 2 .

To obtain inequality (15), we prove by induction that each component i = 1 , 2 , . . . , m of the error in the mean vector estimate z j ⇀ ( i ) = τ ∞ j ( i ) -τ j ⇀ ( i ) is sub-Gaussian with parameter η j ⇀ ( i ) ≤ κ √ s j ⇀ ( i ) . Firstly, for j = 1 , we have that

Since the bounded random variable Φ ( i ) 1 is sub-Gaussian with parameter σ (Φ ( i ) 1 ) , then the error in the mean vector estimate z ⇀ 1 ( i ) is sub-Gaussian with parameter that satisfies

If z j ⇀ -1 ( i ) = τ ∞ j -1 ( i ) -τ j ⇀ -1 ( i ) is sub-Gaussian with parameter η j ⇀ -1 ( i ) ≤ κ √ s j ⇀ -1 ( i ) for any i = 1 , 2 , . . . , m , then using the recursions in (6) we have that

since w j = τ ∞ j -τ ∞ j -1 . If v j = τ j -τ ∞ j , the error in the mean vector estimate is given by

where w ( i ) j and v ( i ) j are sub-Gaussian with parameter σ ( w ( i ) j ) and σ ( Φ ( i ) j ) / √ n j , respectively. Therefore, we have that z j ⇀ ( i ) is sub-Gaussian with parameter η j ⇀ ( i ) that satisfies

since z j ⇀ -1 , w j , and v j are independent. Using that η j ⇀ -1 ( i ) ≤ κ √ s j ⇀ -1 ( i ) and the definition of κ , we have that

where (16) is obtained using the second recursion in (6).

The inequality in (15) is obtained using the union bound together with the Chernoff bound (concentration inequality) [31] for the random variables z j ⇀ ( i ) that are sub-Gaussian with parameter η j ⇀ ( i ) .

Now, we prove by induction that, for any j , ‖ √ s j ⇀ ‖ ∞ ≤ M/ √ n j ⇀ where the ESSs satisfy n ⇀ 1 = n 1 and n j ⇀ ≥ n j + n j ⇀ -1 ‖ σ 2 j ‖ ∞ ‖ σ 2 j ‖ ∞ + n j ⇀ -1 ‖ d 2 j ‖ ∞ for j ≥ 2 . For j = 1 , using the definition of s j ⇀ in the

second recursion in (6), we have that for any component i

Then, vector s ⇀ 1 satisfies

by using the second recursion in (6) and the induction hypothesis. Then, vector s j ⇀ satisfies

The inequality in (8) is obtained because the minimax risk is bounded by the smallest minimax risk as shown in [24, 25, 30] so that

D Proof of Theorem 2

Proof. To obtain bound in (9), we proceed by induction. For j = 1 , using the expression for the ESS in (8), we have that

If (9) holds for the ( j -1) -task, then for the j -th task, we have that where the second inequality is obtained because n j ≥ n , ‖ σ 2 j ‖ ∞ ≤ 1 , and ‖ d 2 j ‖ ∞ ≤ d 2 . Using that n j ⇀ -1 ≥ n ( 1 + (1+ α ) 2 j -3 -1 -α α (1+ α ) 2 j -3 + α ) , the ESS of the j -th task satisfies

Now, we obtain bounds for the ESS if nd 2 < 1 j 2 . In the following, the constant φ represents the golden ratio φ = 1 . 618 . . . .

If nd 2 < 1 j 2 ⇒ √ nd 2 ≤ α ≤ √ nd 2 φ ≤ φ j ≤ 1 because α = nd 2 2 (√ 1 + 4 nd 2 +1 ) = √ nd 2 √ nd 2 +4+ √ nd 2 2 , then we have that n j ⇀ satisfies n j ⇀ ≥ n ( 1 + 1 α α (2 j -2) 2 + α (2 j -1) ) = n ( 1 + 2 j -2 2 + α (2 j -1) )

where the first inequality follows because (1 + α ) 2 j -2 ≥ 1+ α (2 j -2) . Using α ≤ φ j , we have that

E More efficient recursions for forward and backward learning

The Rauch-Tung-Striebel smoother recursions [34] allow to obtain forward and backward mean and MSE vectors directly from those vectors for the succeeding task. Specifically, for each j -th task, the mean vector τ /harpoonleftright k j together with the MSE vector s /harpoonleftright k j can be obtained using those vectors for the succeeding task τ /harpoonleftright k j +1 , s /harpoonleftright k j +1 as

The above recursions provide the same mean vector estimate as the recursions in (10) in the paper since they are obtained using the Rauch-Tung-Striebel smoother recursions instead of fixed-lag smoother recursions [34].

F Proof of Theorem 3

Proof. To obtain bound in (11) we first prove that the mean vector estimate and the MSE vector given by (10) satisfy

for any component i = 1 , 2 , . . . , m . Then, we prove that ‖ s /harpoonleftright k j ‖ ∞ ≤ M/ √ n /harpoonleftright k j for j ∈ { 1 , 2 , . . . , k } , where the ESSs satisfy n /harpoonleftright k k = n k ⇀ and n /harpoonleftright k j ≥ n j ⇀ + n ↽k j +1 ‖ σ 2 j ‖ ∞ ‖ σ 2 j ‖ ∞ + n ↽k j +1 ‖ d 2 j +1 ‖ ∞ for j ≥ 2 .

To obtain inequality (19), we prove that each component i = 1 , 2 , . . . , m of the error in the mean vector estimate z /harpoonleftright k j ( i ) = τ ∞ j ( i ) -τ /harpoonleftright k j ( i ) is sub-Gaussian with parameter η /harpoonleftright k j ( i ) ≤ κ √ s /harpoonleftright k j ( i ) . Analogously to the proof of Theorem 1, it is proven that each component in the error of the backward mean vector τ ↽k j +1 is sub-Gaussian with parameters satisfying η ↽k j +1 /precedesequal κ √ s ↽k j +1 . The error in the forward and backward mean vector estimate is given by

where the second equality is obtained using the recursion for τ /harpoonleftright k j ( i ) in (10). Adding and subtracting s j ⇀ ( i ) s j ⇀ ( i ) + s ↽k j +1 ( i ) + d 2 j +1 ( i ) τ ∞ j +1 ( i ) , we have that

since w j = τ ∞ j -τ ∞ j -1 and z j ⇀ ( i ) = τ ∞ j ( i ) -τ j ⇀ ( i ) . Then, we have that

where z j ⇀ ( i ) , z ↽k j +1 ( i ) , and w ( i ) j +1 are sub-Gaussian with parameters η j ⇀ ( i ) ≤ κ √ s j ⇀ ( i ) , η ↽k j +1 ( i ) ≤ κ √ s ↽k j +1 ( i ) , and σ ( w ( i ) j ) , respectively. Since z j ⇀ , z ↽k j +1 , and w j +1 are independent, we have that z /harpoonleftright k j ( i ) given by (20) is sub-Gaussian with parameter that satisfies

Using the second recursion in (10) we have that the sub-Gaussian parameter satisfies

The inequality in (19) is obtained using the union bound together with the Chernoff bound (concentration inequality) [31] for the random variables z /harpoonleftright k j ( i ) that are sub-Gaussian with parameter η /harpoonleftright k j ( i ) .

Now, we prove that, for any j , ‖ √ s /harpoonleftright k j ‖ ≤ M/ √ n /harpoonleftright k j where the ESSs satisfy n /harpoonleftright k k = n k ⇀ and n /harpoonleftright k j ≥ n j ⇀ + n ↽k j +1 ‖ σ 2 j ‖ ∞ ‖ σ 2 j ‖ ∞ + n ↽k j +1 ‖ d 2 j +1 ‖ ∞ for j ≥ 2 . Analogously to the proof of Theorem 1, we prove that the backward MSE vector s ↽k j +1 satisfies ‖ √ s ↽k j +1 ‖ ∞ ≤ M/ √ n ↽k j +1 . Then, using that s ↽k j +1 M/ n ↽k j +1 , we have that for every component i

The inequality in (11) is obtained because the minimax risk is bounded by the smallest minimax risk as shown in [24, 25, 30] so that

G Proof of Theorem 4

Proof. To obtain bound in (12), we use the ESS obtained with forward learning in Theorem 2 and obtained with backward learning. Analogously to the proof of Theorem 2, we prove that the ESS obtained at backward learning satisfies

Therefore, the ESS obtained with forward an backward learning satisfies

where the second equality follows because nd 2 = α 2 α +1 since α = nd 2 2 (√ 1 + 4 nd 2 +1 ) . Then, we have that

Now, we obtain bounds for the ESS depending on the value value of nd 2 . Such bounds are obtained similarly as in Theorem 2 and we also denote by φ the golden ratio φ = 1 . 618 . . . .

If nd 2 < 1 j 2 ⇒ √ nd 2 ≤ α ≤ √ nd 2 φ ≤ φ j ≤ 1 because

then we have that n /harpoonleftright k j satisfies

where the first inequality follows because (1 + α ) 2( k -j ) -1 ≥ 1 + α (2( k -j ) -1) and the second inequality is obtained using α ≤ φ j .

If 1 j 2 ≤ nd 2 < 1 ⇒ 1 j ≤ √ nd 2 ≤ α ≤ √ nd 2 φ because

then we have that n /harpoonleftright k j satisfies

where the second inequality follows because the ESS is monotonically increasing for α and α ≥ nd 2 . Since (1 + √ nd 2 ) 2( k -j ) ≥ 1 + 2 √ nd 2 ( k -j ) and k -j ≥ 1 , we have that because α ≤ √ nd 2 φ .

If nd 2 ≥ 1 ⇒ 1 ≤ nd 2 ≤ α ≤ nd 2 φ because α = nd 2 √ 1+ 4 nd 2 +1 2 , then we have that n /harpoonleftright k j satisfies

where the first inequality follows because the ESS is monotonically increasing for α and α ≥ 1 and the second inequality is obtained using k -j ≥ 1 and α ≤ nd 2 φ .

H Additional numerical results and implementation details

In this section we describe the datasets used for the numerical results in Section 5, we provide further details for the numerical experimentations carried out, and include several additional results. Specifically, in the first set of additional results, we evaluate the classification performance of the proposed method in comparison with state-of-the-art techniques for different sample sizes; in the second set of additional results, we further show the performance improvement leveraging information from all the tasks in the sequence with additional datasets; in the third set of additional results, we show the classification error and the running time of IMRCs for different hyper-parameter values; and in the fourth set of additional results, we evaluate the assumption of change between tasks being independent and zero-mean. In addition, in the folder Implementation_IMRC in the supplementary materials we provide the code of the proposed IMRCs with the setting used in the numerical results.

In Section 5, we use 12 publicly available datasets [1, 37, 36, 39, 10, 40, 41, 2], and http://yann. lecun.com/exdb/mnist/ . The summary of these datasets is provided in Table 3 that shows the number of classes, the number of samples, and the number of tasks. In the following, we further describe the tasks and the time-dependency of each dataset used.

The 'Yearbook' dataset contains portraits' photographs over time and the goal is to predict males and females. Each task corresponds to portraits from one year from 1905 to 2013.

The 'ImageNet noise' dataset contains images with increasing noise over tasks and the goal is to predict if an image is a bird or a snake. The sequence of tasks corresponds to the noise factors [0 . 0 , 0 . 4 , 0 . 8 , 1 . 2 , 1 . 6 , 2 . 0 , 2 . 4 , 2 . 8 , 3 . 2 , 3 . 6] [35].

The 'DomainNet' dataset contains six different domains with decreasing realism and the goal is to predict if an image is an airplane, bus, ambulance, or police car. The sequence of tasks corresponds to the six domains: real, painting, infograph, clipart, sketch, and quickdraw.

The 'UTKFaces' dataset contains face images in the wild with increasing age and the goal is to predict males and females. The sequence of tasks corresponds to face images with different ages from 0 to 116 years.

The 'Rotated MNIST' dataset contains rotated images with increasing angles over tasks and the goal is to predict if the number in an image is greater than 5 or not. Each j -th task corresponds to a rotation angle randomly selected from [ 180( j -1) k , 180 j k ] degrees where j ∈ { 1 , 2 , . . . , k } and k is the number of tasks.

The 'CLEAR' dataset contains images with a natural temporal evolution of visual concepts and the goal is to predict if an image is soccer, hockey, or racing. Each task corresponds to one year from 2004 to 2014.

The 'Power Supply' dataset contains three year power supply records from 1995 to 1998 and the goal is to predict which hour the current power supply belongs to. We relabel into binary classification according to pm. or am. as in [10].

The 'Usenet' dataset is splitted into Usenet1 and Usenet2 which both contains a stream of messages from different 20 newsgroups that are sequentially presented to a user and the goal is to predict the personal interests.

The 'German' dataset contains information about people who take a credit by a bank and the goal is to classify each person as good or bad credit risks.

The 'Spam' dataset contains emails and the task is to predict if an email is malicious spam email or legitimate email.

The 'Covertype' dataset contains cartographic variables of a forest area obtained from US Forest Service (USFS) and the goal is to predict the forest cover type.

The samples in each task are randomly splitted in 100 samples for test and the rest of the samples for training. The samples used for training in the numerical results are randomly sampled from each group of training samples in each repetition.

In Section 5, we compare the results of IMRC methods with 7 state-of-the-art-techniques [5, 17, 4, 6, 9, 10, 16]. In the following, we briefly describe each method used.

GEM method [5] is a technique developed for continual learning. The method provided by Lopez-Paz & Ranzato learns each new task using a stochastic gradient descent with inequality constraints given by the losses of preceding tasks. Such constraints avoid the increase of the loss of each preceding tasks.

MERmethod[17] is a technique developed for continual learning. The method provided by Riemer et al. learns each new task using sample sample sets that include random samples of preceding tasks. Such samples of preceding tasks are stored in a memory buffer.

ELLA method [4] is a techniques developed for continual learning. The method provided by Ruvolo & Eaton learns each new task transferring knowledge from a shared basis of task models.

EWCmethod [6] is a technique developed for continual learning. The method provided by Kirkpatrick et al. learns each new task regularizing the loss with regularization parameters given by the Fisher information.

Condor method [9] is a technique developed for concept drift adaptation. The method provided by Zhao et al. is an ensemble method that adapts to evolving tasks by learning weighting the models in the ensemble at each time step.

Forward

n

= 100

Forward and backward

n

= 10

Forward

n

= 10

Forward and backward

n

= 100

Classification error/single-task

2

3

4

5

0.9

0.8

1

1

6

Number of tasks

k

Classification error

Forward

k

= 2

Single-task

Forward and backward

k

= 2

Forward

k

= 3

Forward and backward

k

= 3

Sample size

n

10

30

50

70

90

0.3

0.36

0.42

(a) Classification error per number of tasks using 'DomainNet' dataset. (b) Classification error per sample size using 'DomainNet' dataset.

DriftSurf method [10] is a technique developed for concept drift adaptation. The method provided by Tahmasbi et al. adapts to evolving tasks by using a drift detection method. Such method allows to restart a new model when a change in the distribution is detected.

AUE method [16] is a technique developed for concept drift adaptation. The method provided by Brzezinski & Stefanowski is an ensemble method that adapts to evolving tasks by incrementally updating all classifiers in the ensemble and weighting them with non-linear error functions.

The classifier parameters in the numerical results are obtained using an accelerated subgradient method based on Nesterov approach [32, 33]. Such subgradient method applied to optimization (3) obtains at each step classifier parameters µ from the mean and confidence vectors τ , λ using the iterations for l = 1 , 2 , . . . , K

where sign ( · ) denotes the sign function, µ ( l ) is the l -th iterate for µ , θ l = 2 / ( l + 1) and a l = 1 / ( l + 1) 3 / 2 are the step sizes and ∂ϕ ( µ ( l )) denotes a subgradient of ϕ ( · ) at µ ( l ) with

In addition, the above subgradient method is implemented using K = 2000 iterations and a warmstart that initializes the classifier parameters in (22) with the solution obtained for the closest task.

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

Task

Step

1

1

3

3

5

5

7

7

9

9

11

11

(a) Single task.

Task

Step

1

1

3

3

5

5

7

7

9

9

11

11

(b) Forward.

Task

Classification

error

Step

1

1

3

3

5

5

7

7

9

9

11

0

.

1

0

0

.

2

0

.

3

0

.

4

(c) Forward and backward.

In the first set of additional results , we further compare the classification error of IMRCs with the state-of-the-art techniques. The results in Table 1 in the paper as well as Tables 4, 5, and 6 are obtained computing the classification error 50 times for each sample size. Table 1 in the paper shows classification errors for n = 10 samples, while Tables 4, 5, and 6 show the classification error for n = 50 , n = 100 , and n = 150 samples, respectively. As can be observed in Table 4, the performance improvement of IMRCs in comparison with the state-of-the-art techniques for n = 50 , n = 100 , and n = 150 is similar to that shown in the paper for n = 10 .

In the second set of additional results , we further illustrate the relationship among classification error, number of tasks, and sample size. Figure 4 in the paper as well as Figure 5 are obtained computing the classification error over all the sequences of consecutive tasks of length k in the dataset. Then, we repeat such experiment 10 times with randomly chosen training sets of size n . Figure 5 extends the results for IMRCs using 'DomainNet' dataset completing those in the main paper that show the results using 'Yearbook' dataset. Figure 5a shows the classification error of IMRC method divided by the classification error of single-task learning for different number of tasks with n = 10 and n = 100 sample sizes. In addition, Figure 5b shows the classification error of IMRCmethod for different sample sizes with k = 10 tasks. Figures 5a and 5b show similar behavior to Figures 4a and 4b in the paper, respectively. In addition, Figure 6 shows the classification error of IMRCs per step and task with single-task learning, forward learning, and forward and backward learning using the 'Yearbook' dataset. Such figure shows that forward and backward learning can improve performance of preceding tasks, while forward learning and single task learning maintain the same performance over time.

In the third set of additional results , we further assess the change in classification error and the running time of IMRCs varying the hyper-parameters. Table 7 shows the classification error of IMRCs varying the values of hyper-parameter for the window size W and the number of backward steps b , completing those in the paper that show the results for W = 2 and b = 3 . As shown in the table, the proposed IMRCs do not require a careful fine-tuning of hyper-parameters and similar performances are obtained by using different values. In addition, Table 8 shows the mean running

replacements

Sample size

Running time per task [s]

b

= 1

b

= 2

b

= 3

b

= 4

b

= 5

10

30

50

70

90

0

.

01

0

.

015

0

.

02

0

.

025

(a) Yearbook dataset

Number of tasks

Running time [s]

b

= 1

b

= 2

b

= 3

b

= 4

b

= 5

10

30

50

70

90

1

2

0

(b) Yearbook dataset for

n

= 10

3

Number of tasks

Running time [s]

b

= 1

b

= 2

b

= 3

b

= 4

b

= 5

10

30

50

70

90

1

2

0

(c) Yearbook dataset for

n

= 100

time per task in seconds of IMRCs for b = 1 , 2 , . . . , 5 backward steps in comparison with the stateof-the-art techniques that learn a sequence of tasks. Such table shows that the methods proposed for backward learning do not require a significant increase in complexity, and the running time of the proposed method is similar to that of other state-of-the-art methods. Figure 7 further assesses the running time of IMRCs for b = 1 , 2 , . . . , 5 backward steps varying the sample size and the number of tasks. Such figure shows that the running time of IMRCs increases moderately and linearly with the number of backward steps and the number of tasks.

In the fourth set of additional results , we evaluate the assumption of change between tasks being independent and zero-mean by assessing the partial autocorrelation of mean vectors. In particular, the partial autocorrelation at any lag would be zero if tasks are i.i.d.; while the partial autocorrelation at lag 1 is larger than zero if tasks satisfy the assumption of Section 2. Figure 8 shows the averaged partial autocorrelation of the mean vectors components +/- their standard deviations for different lags using 'Yearbook' and 'UTKFaces' datasets. Such figure shows a partial autocorrelation clearly non-zero at lag 1 that reflects dependence between consecutive mean vectors, as described by the assumption of Section 2.

Partial autocorrelations

0

-

0

.

2

0

.

2

0

.

6

1

1

5

15

20

(a) 'Yearbook' dataset

Partial autocorrelations

0

-

0

.

2

0

.

2

0

.

6

1

1

5

15

20

(b) 'UTKFaces' dataset

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an incremental minimax risk classifier (imrc) to effectively utilize forward and reverse learning and explain evolutionary tasks. Furthermore, the performance improvements provided by forward and backward learning can also be described analytically based on the expected quadratic changes of the task and the number of tasks.\n\nSTRENGTHS:\nIn the article, the effects and limitations of IMRCs are verified by rich experiments, and using IMRCs can better use forward and reverse learning to explain the evolutionary tasks and improve the effective sample size.\n\nWEAKNESSES:\nHowever, there are some deficiencies in the ablation aspect, and the experimental volume is relatively small.\n\nQUESTIONS:\n1. The setting of the task in the article mainly starts from the high similarity continuous task, so whether the high similarity continuous learning has practical application, and whether this method can have a broad development prospect in the future technology development.\n2. In the article, it is necessary to more clearly describe the task premise, between the similarity in the similarity continuous learning and the evolution in the evolutionary tasks, and then describe the clarity and rationality, why the similar continuous tasks are more consistent with the evolutionary principles.\n3. In the article, I learned the role of positive learning and reverse learning in the experiment, but I want to know whether reverse learning alone is feasible, whether positive learning has an inhibitory effect on reverse learning?\n4. In Figure 1 of the article, although the meaning of the expression can be roughly understood, the graph line is a little confused, especially the meaning of the black line is unclear.\n5. In Figure 2 of the article, the meaning of j in the first red square and k in the second red square is not clear, and the symbols introduced in the text in Article 4.2 are inconsistent with the flow chart symbols in Figure 2, which is easy to confuse the readers' memory.\n\nLIMITATIONS:\nsee questions", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIMRCs (Incremental Minimax Risk Classifiers) are a technique for incremental learning of a growing sequence of classification tasks. The key feature of IMRCs is their ability to exploit the similarity between consecutive tasks by leveraging both forward and backward learning.\nForward learning utilizes information from preceding tasks to improve the classification performance of the current task. It recursively updates the mean and mean squared error (MSE) vectors by considering the difference between the current task and the preceding task. Backward learning, on the other hand, uses information from the current task to improve the classification performance of preceding tasks.\n\nSTRENGTHS:\n(1) IMRCs leverage the evolving nature of tasks in a growing sequence. By utilizing both forward and backward learning, IMRCs effectively exploit the similarity between consecutive tasks. \n\n(2) By leveraging information from preceding and succeeding tasks, IMRCs can incorporate additional knowledge and enhance classification performance.\n\n(3) IMRCs minimize the worst-case error probabilities over uncertainty sets. This means that they provide robust classification performance, even in the face of uncertainty or variations in the underlying distributions of tasks.\n\nWEAKNESSES:\n(1) The forward and backward learning processes, along with the uncertainty set updates, can require significant computational resources and time. How about the training time compared to other methods?\n\n(2) The selection of appropriate uncertainty sets can be challenging and may require prior knowledge or assumptions about the task distributions. \n\n(3)  The authors assume that the distributional changes between tasks (p2-p1, p3-p2, ...) are independent and have a mean of 0. How can IMRCs be extended or modified to handle scenarios with non-zero-mean changes between consecutive tasks or explore novel tasks outside the existing sequence?\n\nQUESTIONS:\nSee weakness part.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper presents a novel method to tackle continual learning, called IMRCs which is able to exploit forward and backward learning to account for evolving tasks. The authors provide theoretical guarantees on the performance of IMRCs in terms of the tasks' expected quadratic change and the number of tasks. Authors compared the proposed method with prior work on multiple datasets, sample sizes, and number of tasks.

STRENGTHS:
+  The idea seems interesting and performance guarantee in the context of continual learning is a promising research direction. 
+ A thorough theoretical analysis of IMRCs is provided.

WEAKNESSES:
- My main concern about this paper is the experimental section. The diversity of the datasets and experiments with different sample sizes is a plus but at the same time it is different from the common continual learning setting in the literature and the baselines are outdated whereas the field of continual learning is rapidly growing and there has been numerous works developed between 2021 and now. 

- The paper does not discuss the computational complexity of IMRCs.

QUESTIONS:
- Authors mention they used the original hyper parameters for the baseline. But given that some methods such as MER, GEM, EWC which were introduced on image data, cannot be taken as is and applied to tabular datasets without tuning hyper parameters. How did the authors make sure of a fair comparison? Note that results for some methods such as GEM and MER are somewhat equal to those from EWC (Table 1) but in vision applications this is not expected and EWC is always outperformed by every single memory-based method.

LIMITATIONS:
No.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe papers studies a specific case of continual learning over a sequence of tasks. Authors extend the typical i.i.d. assumption of task meta-distribution to the case where only the differences between subsequent task distributions are assumed to be independent and zero-mean. Under this assumption, they devise two versions of a minimax risk classifier and prove performance guarantees that improve upon single-task bounds by relying on effective sample size instead of just the number of samples in a task. The effective sample size is no smaller than the single task size and can be larger if the quadratic change between consecutive tasks behaves in a favorable way, e.g. decreasing over time. The authors also present an experimental study of the proposed algorithms and show that it either improves the baseline performance or achieves similar one across 12 datasets.\n\nSTRENGTHS:\n### Originality\nThis is a novel approach that relies on independent differences between subsequent task distributions rather than on the typical i.i.d. assumption.\n\n### Quality\n* Paper studies the presented algorithm theoretically and experimentally.\n* Illustrative theoretical guarantees that are always no worse than a single task with a clear influence of the task distribution properties in the form of effective sample size.\n* Experiments uses 12 different datasets and cover 7 baselines.\n* Authors provide insights into impact of number of tasks and number of samples on the performance of different variants of the algorithm.\n\n### Clarity\nThe paper is well written and easy to follow in general. \n\n### Significance\nThere are numerous ways of how i.i.d. assumption can be relaxed and paper studies one of them. The significance will rely on how close the new assumption is to practical scenarios. From experimental setup it seems to be a rather specific case that relies too much on exact task ordering, thus I would expect the paper to have average impact from theoretical point of view, but quite limited impact in practice.\n\nWEAKNESSES:\nThe paper doesn’t have major weaknesses. There are things that can be improved in the presentation and discussing limitations, which I cover in questions section.\n\nQUESTIONS:\nThe following things would help with the presentation:\n* The notations used are overloaded (e.g. by using double superscripts)  and sometimes are hard to follow. The paper would benefit from a small table with all notations in one place.\n* Experiments present the result for samples size of 10 without any additional comments on exact value of 10. Appendix shows that the general conclusion is valid irrespective of the sample size, so it would be good to highlight that.\n* The main assumption on the task distribution needs to be made more precise and explicit in the theorems (introduce assumptions separately and then refer to it in theorems). A discussion on how realistic it is and providing more than one example would go a long way.\n* Theoretical analysis needs a discussion on different regimes where the ESS improvements would be realised and what they mean for the task distributions process. Authors do have a short discussion of that after Theorem 4, but it lacks the connection between different regimes of nd^2 and process properties. For example, $nd^2 \\leq \\frac{1}{j^2}$ translates into a rather fast convergence requirement of the task distribution process.\n* It would be great to discuss what happens to the algorithm when the main assumption breaks. Does it go back to single task performance? How would one figure this out in practice?\n\nLIMITATIONS:\nThe limitation of the approach are not adequantly discussed. I’ve made a few suggestion in the questions section on how to address that.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThough class-incremental learning has been studied the most as a default setting for continual learning, task-incremental study could be crucial for other areas. This paper focuses particularly on the case where tasks being introduced over time are interrelated under specific conditions. Under the supervised learning framework, the task distributions $p_t$ (of instances and labels at any time t) are independent and identically distributed, which would be a strong and less-realistic condition. In contrast, this paper covers the case where the difference of two consecutive task distributions is independent, making the difference between two distant tasks have increasing variance over time. Then the authors propose Incremental Risk Minimization Classifier (IMRC) method that effectively leverage forward and backward learning algorithms, improving the performance against existing methods on various datasets, sample sizes, and the number of tasks.\n\nSTRENGTHS:\n(1)\tThe paper formally defines a non-trivial task-incremental learning setting whose assumption is more realistic than the earlier approach.\n\n(2)\tThe paper analytically shows performance guarantees and effective sample sizes as well as forward and backward learning algorithm.\n\n(3)\tThe paper addresses empirical improvement on multiple datasets against existing state-of-the-art algorithms.\n\nWEAKNESSES:\n(1)\tCurrent draft is rarely self-contained. Readers must read the appendix frequently. Minimally, having explanations about methods and datasets in Table 1 would significantly increase the readability.\n\n(2)\tRun-time trade-off of considering the backward (possibly per different sample sizes) are missing. \n\n(3)\tCharacteristics of each experimental dataset: how much they agree and disagree the independence assumption across consecutive task distributions are missing.\n\nQUESTIONS:\n(1)\tProviding comprehensive details about different tasks and their distinctive characteristics helps readers better understanding the contributions of the paper beyond the theoretical excitement. Formalizing the time-evolving task with an independence assumption on consecutive task difference sounds exceptional. It will be great to know how much this assumption is agreed and violated on individual tasks on experiments.\n\n(2)\tIn addition to (1), accessing real examples or qualitative explanations about hard cases (that were not successful in the previous approaches, but exceptionally successful with the current IMRC) will be useful. For example, it seems IMRC is distinguished notably from other algorithms on P. Supply, Usenet1, and Spam datasets. Understanding the rationale will be insightful.\n\n(3)\tMost importantly, run-time trade-off for doing backward process must be measured for making the provable guarantees practically effective. As the trade-off would change for different number of tasks and sample sizes, practitioners require better guidance.\n\nLIMITATIONS:\nNo specific limitations are described or probed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-30"}