Nonlinear independent component analysis (ICA) provides an appealing framework for unsupervised feature learning, but the models proposed so far are not identifiable. Here, we first propose a new intuitive principle of unsupervised deep learning from time series which uses the nonstationary structure of the data. Our learning principle, time-contrastive learning (TCL), finds a representation which allows optimal discrimination of time segments (windows). Surprisingly, we show how TCL can be related to a nonlinear ICA model, when ICA is redefined to include temporal nonstationarities. In particular, we show that TCL combined with linear ICA estimates the nonlinear ICA model up to point-wise transformations of the sources, and this solution is unique -thus providing the first identifiability result for nonlinear ICA which is rigorous, constructive, as well as very general.
On REF authors leverage time-contrastive loss to learn representations leveraging the temporal structure of the data.
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
ArXiv
ArXiv
Computer Science Mathematics