  In this paper, we generalize the problem of single index model to the context
of continual learning in which a learner is challenged with a sequence of tasks
one by one and the dataset of each task is revealed in an online fashion. We
propose a strategy that is able to learn a common single index for all tasks
and a specific link function for each task. The common single index allows to
transfer the informaton gained from the previous tasks to a new one. We provide
a theoretical analysis of our proposed strategy by proving some regret bounds.
Moreover, as a by-product from our work to provide an example of a within-task
algorithm, we develop a novel online algorithm for learning single index model
in an online setting and provide its regret bound.
