Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding
Authors: Sana Tonekaboni
Danny Eytan
Anna Goldenberg
Venue: ICLR 2021 Poster (2021)

Abstract: Time series are often complex and rich in information but sparsely labeled and therefore challenging to model. In this paper, we propose a self-supervised framework for learning robust and generalizable representations for time series. Our approach, called Temporal Neighborhood Coding (TNC), takes advantage of the local smoothness of a signal's generative process to define neighborhoods in time with stationary properties. Using a debiased contrastive objective, our framework learns time series representations by ensuring that in the encoding space, the distribution of signals from within a neighborhood is distinguishable from the distribution of non-neighboring signals. Our motivation stems from the medical field, where the ability to model the dynamic nature of time series data is especially valuable for identifying, tracking, and predicting the underlying patients' latent states in settings where labeling data is practically impossible. We compare our method to recently developed unsupervised representation learning approaches and demonstrate superior performance on clustering and classification tasks for multiple datasets.

Published as a conference paper at ICLR 2021
UNSUPERVISED REPRESENTATION LEARNING FOR TIME SERIES WITH TEMPORAL NEIGHBORHOOD CODING
Sana Tonckaboni'
Danny Eytan
University of Toronto & Vector Institute The Hospital for Sick Children stonekaboni8es. toronto.edi edu
The Hospital for Sick Children bi www.ry.colle@maai.......... com
Anna Goldengerg
University of Toronto & Vector Institute The Hospital for Sick Children
anna. www.deebeergeeroroooooooooo
ABSTRACT
Time series are often complex and rich in information but sparsely labeled and therefore challenging model. In this paper, we propose self-supervised frame- work for learning generalizable representations for non-stationary time series. Our approach, called Temporal Neighborhood Coding (TNC), takes advantage of the local smoothness of a signal's generative process to define neighborhoods in time with stationary properties. Using a debiased contrastive objective, our framework learns time series representations by ensuring that in the encoding space, the distribution of signals from within a neighborhood is distinguishable from the distribution of non-neighboring signals. Our motivation stems from the medical field, where the ability to model the dynamic nature of time series data especially valuable for identifying, tracking, and predicting the underlying patients' latent states settings where labeling data is practically impossible. We compare our method to recently developed unsupervised representation learning approaches and demonstrate superior performance on clustering and classification tasks for multiple datasets.
INTRODUCTION
Real-world time-series data is high dimensional, complex, and has unique properties that bring about many challenges for data modeling (Yang & Wu, 2006). In addition, these signals are often sparsely labeled, making even more challenging for supervised learning tasks. Unsupervised representation learning can extract informative low-dimensional representations from raw time series by leveraging the data's inherent structure, without the need for explicit supervision. These representations are more generalizable and robust, as they are less specialized for solving a single supervised task. Unsupervised representation learning is well studied in domains such as vision (Donahue Simonyan, 2019; Denton et al., 2017; Radford et al., 2015) and natural language processing (Radford et al., 2017; Young et al., 2018; Mikolov et al., 2013), but has been underexplored in the literature for time series settings. Frameworks designed for time series need to be efficient and scalable because signals encountered in practice can be long, high dimensional, and high frequency. Moreover, it should account for and be able to model dynamic changes that occur within samples, i.c., non-stationarity of signals.
The ability to model the dynamic nature of time series data is especially valuable in medicine. Health care data is often organized as a time series, with multiple data types, collected from various sources different sampling frequencies, and riddled with artifacts and missing values. Throughout their stay at the hospital or within the disease progression period, patients transition gradually between distinct
'http:/ www.. toronto. wwwronto.eduu--stteee.....

---

Published as a conference paper at ICLR 2021
clinical states, with periods of relative stability, improvement, unexpected deterioration, requiring escalation of care that alters the patient's 's trajectory. A particular challenge in medical time-series data is the lack of well-defined available labels that are needed for identifying the underlying clinical state of an individual or for training models aimed at extracting low-dimensional representations of these states. For instance, in the context of critical-care, a patient's stay in the critical care unit (CCU) is captured continuously via streaming physiological signals by the bedside monitor. Obtaining labels for the patient's 's state for extended periods of these signals is practically impossible as the underlying physiological state can be unknown even to the clinicians. This further motivates the use of unsupervised representation learning in these contexts. Learning rich representations can be crucial in facilitating the tracking of disease progression, predicting the future trajectories of the patients, and tailoring treatments to these underlying states.
In this paper, we propose self-supervised framework for learning representations for complex multivariate non-stationary time series. This approach, called Temporal Neighborhood Coding (TNC), is designed for temporal settings where the latent distribution of the signals changes over time, and it aims to capture the progression of the underlying temporal dynamics. TNC is efficient, easily scalable to high dimensions, and can be used in different time series settings. We assess the quality of the learned representations on multiple datasets and show that the representations are general and transferable many downstream tasks such as classification and clustering. We further demonstrate that our method outperforms existing approaches for unsupervised representation learning, and it even performs closely supervised techniques in classification tasks. The contributions of this work are three-fold:
1. We present novel neighborhood- unsupervised learning framework for non-stationary multivariate time series data.
2. We introduce the concept of a temporal neighborhood with stationary properties as the distribution of similar windows in time. The neighborhood boundaries determined automatically using the properties of the signal and statistical testing.
3. We incorporate concepts from Positive Unlabeled Learning, specifically, sample weight adjustment, to account for potential bias introduced in sampling negative examples for the contrastive loss.
2 METHOD
We introduce a framework for learning representations that encode the underlying state of a multi- variate, non-stationary time series. Our self-supervised approach, TNC, takes advantage of the local smoothness of the generative process of signals to learn generalizable representations for windows of time series. This done by ensuring that in the representation space, the distribution of signals proximal in time is distinguishable from the distribution of signals far away, i.e., proximity in time is identifiable in the encoding space. We represent our multivariate time series signals as X RDxT where D is the number of features and T is the number of measurements over time. X $. $1 represents window of time series of length 8, centered around time 1, that includes measurements of all features taken in the interval [t 2: + 2)- Throughout the paper, we refer to this window as Wt for notational simplicity. Our goal is to learn the underlying representation of W,, and by sliding this window over time, we can obtain the trajectory of the underlying states of the signal.
We define the temporal neighborhood (N₂) of window W, as the set of all windows with centroids 1", sampled from a normal distribution L* ~ [N(,, 6). Where A is a Gaussian centered at i, ó is the size of the window, and 17 is the parameter that defines the range of the neighborhood. Relying on the local smoothness of a signal's 's generative process, the neighborhood distribution is characterized as a Gaussian to model the gradual transition in temporal data, and intuitively, it approximates the distribution of samples that are similar to W. The 1) parameter determines the neighborhood range and depends on the signal characteristics and how gradual the time series's statistical properties change over time. This can be set by domain experts based on prior knowledge of the signal behavior, or for more robust estimation, it can be determined by analyzing the stationarity properties of the signal for every W, Since the neighborhood represents similar samples, the range should identify the approximate time span within which the signal remains stationary, and the generative process does not change. For this purpose, we use the Augmented Dickey-Fulles Fuller (ADF) statistical test to
2

---

Published as a conference paper at ICLR 2021
determine this region for every window. Proper estimation of the neighborhood range is an integral part of the TNC framework. If 17 is too small, many samples from within a neighborhood will overlap, and therefore the encoder would only learn to encode the overlapping information. On the other hand, if v) is too big, the neighborhood would span over multiple underlying states, and therefore the encoder would fail to distinguish the variation among these states. Using the ADF test, we can automatically adjust the neighborhood for every window based on the signal behavior. More details on this test and how it used estimate 7) described in section 2.
Now, assuming windows within a neighborhood possess similar properties, signals outside of this neighborhood, denoted as N₁, are considered non-neighboring windows. Samples from Nr are likely to be different from W,, and can be considered as negative samples in context of contrastive learning framework. However, this assumption can suffer from the problem of sampling bias, common in most contrastive learning approaches (Chuang et al., 2020; Saunshi et al., 2019). This bias occurs because randomly drawing negative examples from the data distribution may result in negative samples that are actually similar to the reference. This can significantly impact the learning framework's 's performance, but little work has been done on addressing this issue (Chuang et al., 2020). In our context, this can happen when there are windows from N₂ that are far away from Wt, but have the same underlying state. To alleviate this bias in the TNC framework, we consider samples from N₂ as unlabeled samples, as opposed negative, and use ideas from Dositive-Ullablle (PU) learning to accurately measure the loss function. In reality, even though samples within a neighborhood are all similar, we cannot make the assumption that samples outside this region are necessarily different. For instance, in the presence of long term seasonalities, signals can exhibit similar properties at distant times. In healtheare context, this can be like stable patient that undergoes a critical condition, but returns back to a stable state afterwards.
In PU learning, a classifier is learned using labeled data drawn from the positive class (P) and unlabeled data (U) that is mixture of positive and negative samples with a positive class prior π (Du Plessis et al., 2014; Kiryo et al., 2017; Du Plessis & Sugiyama, 2014). Existing PU learning methods fall under two categories based on how they handle the unlabeled data: 1) methods that identify negative samples from the unlabeled cohort (Li & Liu, 2003); 2) methods that treat the unlabeled data as negative samples with smaller weights (Lee & Liu, 2003; Elkan & Noto, 2008). In the second category, unlabeled samples should be properly weighted in the loss term in order to train an unbiased classifier. Elkan & Noto (2008) introduces simple and efficient way of approximating the expectation of a loss function by assigning individual weights w to training examples from the unlabeled cohort. This means each sample from the neighborhood is treated as positive example with unit weight, while each sample from N is treated as a combination of positive example with weight w and negative example with complementary weight 1 w. In the original paper (Elkan & Noto, 2008), the weight is defined as the probability for sample from the unlabeled set to be a positive sample, i.e. w = p(y = 1[z) for x E U. In the TNC framework, this weight represents the probability of having samples similar to W; in N. By incorporating weight adjustment into the TNC loss (Equation I), we account for possible positive samples that occur in the non-neighboring distribution. w can be approximated using prior knowledge of the underlying state distribution or tuned as a hyperparameter. Appendix A.6 explains how the weight parameter is selected for our different experiment setups and also demonstrates the impact of weight adjustment performance for downstream tasks.
After defining the neighborhood distribution, we train objective function that encourages a distinc- tion between the representation of samples of the same neighborhood from the outside samples. An ideal encoder preserves the neighborhood properties in the encoding space. Therefore representations Zi Enc(W] of samples from a neighborhood Wi E Nr. can be distinguished from representation Z+ Ene(Wa (W&) of samples from outside the neighborhood WR E N,. TNC is composed of two main components:
1. An Encoder Bw,( that maps We E Rx6 representation Z, E RRM. in lower dimensional space (M < D 6), where D x 8 is the total number of measurements in We.
2. A Discriminator D(Z,, Z) that approximates the probability of Z being the representation of a window in N,. More specifically, it receives two samples from the encoding space and predicts the probability of those samples belonging to the same temporal neighborhood.
3

---

Published as a conference paper at ICLR 2021
1~ ((t,
Duta
Data
W1
W E A
Aio
Bind
Reighta
Representation
Representation
D
P(y)
p(y)
Neighhorhood samples
(h) Non-neighhorinn samples
Figure 1: Overview of the TNC framework components. For each sample window W; (indicated with the dashed black box), we first define the neighborhood distribution. The encoder learns the distribution of windows sampled from N₁ and N;, in the representation space. Then samples from these distributions are fed into the discriminator alongside Z₁, to predict the probability of the windows being in the same neighborhood.
TNC is general framework; therefore, it is agnostic to the nature of the time series and the architecture of the encoder. The encoder can be any parametric model that is well-suited to the signal properties (Oord et al., 2016; Bai et al., 2018; Fawaz et al., 2019). For the Discriminator D(Z Z) use a simple multi-heades binary classifier that outputs if Z and Zj are representations of neighbors in time, and 0 otherwise. In the experiment section, describe the architectural details of the models used for our experiments more depth.
Figure provides summary overview of the TNC framework. We formalize the objective function of our unsupervised learning framework in Equation 1. In essence, we would like the probability likelihood estimation of the Discriminator to be accurate, i.e., close to 1 for the representation of neighboring samples and close to for windows far apart. Samples from the non-neighboring region (N) are weight-adjusted using the w parameters account for positive samples in this distribution.
Rw,,x [EW,NN, [log D(Z,, B, ((1-w) xlog (1 - D(Z,, +xx) xlog D(Z,, 7x)] D(Enc(W.,), Enn((W)))
D(Ene Enn(WW))
We train the encoder and the discriminator hand in hand by optimizing for this objective. Note that the Discriminator only part training and will not be used during inference. Similar the encoder, it can be approximated using any parametric model. However, the more complex the Discriminator, the harder it becomes to interpret the latent space's 's decision boundaries since allows similarities to be mapped on complex nonlinear relationships.
Defining the neighborhood parameter using the ADF test: As mentioned earlier, the neighbor- hood range can be specified using the characteristies of the data. In non-stationary time series, the generative process of the signals changes over time. We define the temporal neighborhood around every window as the region where the signal is relatively stationary. Since signal may remain in an underlying state for an unknown amount of time, each window's neighborhood range may vary in size and must be adjusted signal behavior. To that end, we use the Augmented Dickey-Fuller (ADF) statistical test to derive the neighborhood range 77. The ADF test belongs to category of tests called "Unit Root Test", and a method for testing the stationarity of a time series. For every Wt, we want to find the neighborhood range around that window that indicates stationary region. To determine this, we start from = and gradually increase the neighborhood size 7, measuring the v-vulue from the test at every step. Once p-value is above a threshold (in our setting 0.01), it means that it fails to reject the null hypothesis and suggests that within this neighborhood region, the signal is no longer stationary. This way, we find the widest neighborhood within which the signal remains relatively stationary. Note that the window size constant throughout the experiment, and during ADF testing, only adjust the neighborhood's width.
4

---

Published as a conference paper at ICLR 2021
3 EXPERIMENTS
We evaluate our framework's s usability multiple time series datasets with dynamic latent states that change over time. We compare classification performance and clusterability against two state-of- the-art approaches for unsupervised representation learning for time series: 1. Contrastive Predictive Coding (CPC) (Oord et al., 2018) that uses predictive coding principles to train the encoder on probabilistic contrastive loss. 2. Triplet-Loss (T-Loss), introduced in (Franceschi et al., 2019), which employs tine-based negative sampling and a triplet loss to learn representations for time series windows. The triplet loss objective ensures similar time series have similar representations by minimizing the pairwise distance between positive samples (subseries) while maximizing it for negative ones. (See Appendix A.2 for more details on each baseline.)
For fair comparison and to ensure that the difference in performance is not due to the differences in the models' architecture, the same encoder network used across all compared baselines. Our objective is to compare the performance of the learning frameworks, agnostic to the encoder's s choice. Therefore, we selected simple architectures to evaluate how each framework can use simple encoder's 's limited capacity to learn meaningful representations. We assess the generalizability of the representations by 1) evaluating clusterability in the encoding space and 2) using the representations for downstream classification task. In addition to the baselines mentioned above, we also compare clusterability performance with unsupervised K-means and classification with a K-Nearest Neighbor classifier, using Dynamic Time Warping (DTW) to measure time series distance. All models are implemented using Pytorch 1.3.1 and trained on a machine with Quadro 400 GPU Below we describe the datasets for experiments in more detail.
3.1 SIMULATED DATA
The simulated dataset is designed to replicate very long, non-stationary, and ligh-frequency time series for which the underlying dynamics change over time. Our generated time series consists of 2000 measurements for 3 features, generated from 4 different underlying states. We use a Hidden Markov Model (HMM) to generate the random latent states over time, and in each state, the time series is generated from a different generative process, including Gaussian Processes (GPs) with different kernel functions and Nonlinear Auto-regressive Moving Average models with different sets of parameters (a and 3). Besides, for to further resemble realistic (e.g., clinical) time series, two features are always correlated. More details about this dataset provided in the Appendix A.1. For this experimental setup, we use a two-directional, single-layer recurrent neural network encoder. We have selected this simple architecture because it handles time series with variable lengths, and it easily extends to higher-dimensional inputs. The encoder model encodes multi-dimmensonnl signal windows of 50 into 10 dimensional representation vectors. The window size is selected such that it is long enough to contain information about the underlying state but not too long to span over multiple underlying states, A more detailed discussion on the window size choice is presented in Appendix A.4.
3.2 CLINICAL WAVEFORM DATA
For real-world clinical experiment, we use the MIT-BIH Atrial Fibrillation dataset (Moody, 1983). This dataset includes 25 long-tern term Electrocardiogram (ECG) recordings (10 hours in duration) of human subjects with atrial fibrillation. It consists of two ECG signals; each sampled at 250 Hz. The signals are annotated over time for the following different rhythm types: 1) Atrial fibrillation, 2) Atrial flutter, 3) AV junctional rhythm, and 4) all other rhythms. Our goal in this experiment is to identify the underlying type of arrhythmia for each sample without any information about the labels. This dataset is particularly interesting and makes this experiment challenging due to the following special properties:
The underlying heart rhythm changes over time in each sample. This is opportunity to evaluate how different representation learning frameworks can handle alternating classes in non-stationary settings;
!Code implementation can be found at htttp::///t thub. www..///attnn///ne.....
representati on_ lereriiee_eenenneeeeeeeee learning
5

---

Published as a conference paper at ICLR 2021
The dataset is highly imbalanced, with atrial flutter and AV junctional rhythm being present in fewer than 0.1% of the measurements. Data imbalance poses many challenges for down- stream classification, further motivating the use of unsupervised representation learning;
The dataset has samples from small number of individuals, but over extended period (around 5 million data points). This realistic scenario, common in healthcare data, shows that our framework still powerful in settings with limited number of samples.
The simple RNN encoder architecture used for other experiment setups cannot model the high- frequency ECG measurements. Therefore. inspired by sification problems, the encoder Enc used this experiment a 2-chamnel, 1 1-dimensional strided convolutional neural network that runs directly the ECG waveforms. We use six convolutional layers with a total down-sampling factor of 16. The window size is 2500 samples, meaning that each convolutional filter covers at least half second of ECG recording, and the representations summarized in a 64-dimensional vector.
architectures for ECG clas-
3.3 HUMAN ACTIVITY RECOGNITION (HAR) DATA
Human Activity Recognition (HAR) is the problem of predicting the type of activity using temporal data from accelerometer and gyroscope measurements. We use the HAR dataset from the UCI Machine Learning Repository that includes data collected from 30 individuals using a smartwatch. Each person performs six activities: 1) walking, 2) walking upstairs, 3) walking downstairs, 4) sitting, 5) standing, and 6) laying. The time-series measurements are pre-processed to extract 561 features. For our purpose, we concatenate the activity samples from every individual over time using the subject identifier to build the full-time series for each subject, which includes continuous activity change. Similar to the simulated data setting, we use a single-layer RNN encoder. The selected window size is 4, representing about 15 seconds of recording, and the representations encoded in a 10-dimensional vector space.
4 RESULTS
In this section present the results for clusterability of the latent representations and downstream classification performance for all datasets and across all baselines. Clusterability indicates how well each method recovers appropriate states, and classification assesses how informative our representa- tions are for downstream tasks.
4.1 EVALUATION: CLUSTERABILITY
Many real-world time series data have underlying multi-category structure, naturally leading to representations with clustering properties. Encoding such general priors is property of a good representation (Bengio et al., 2013). In this section, we assess the distribution of the representations in the encoding space. If information of the latent state is properly learned and encoded by the framework, the representations of signals from the same underlying state should cluster together. Figures 2a, 2b, and 2c show an example of this distribution for simulated data across compared approaches. Each plot is a 2-dimensional t-SNE visualization of the representations where each data point in the scatter plot is an encoding E R0 that represents a window size ó = 50 of a simulated time series. We can that without any information about the hidden states, representations learned using TNC cluster windows from the same hidden state better than the alternative approaches. The results show that CPC and Triplet Loss have difficulty separating time series that generated from non-linear auto-regressive moving average (NARMA) models with variable regression parameters.
To compare the representation clusters' consistency for each baseline, we use two very common cluster validity indices, namely, the Silhouette score and the Davies-Bouldin index. We use K-means clustering the representation space measure these clusterability scores. The Silhouette score measures the similarity of each sample to its own cluster. compared to other clusters. The values can range from - to +1, and a greater score implies a better cohesion. The Davies-Bouldin Index
Pttpps: //ter////e/../ee/e ussucii .ddu /wt.aaaaasssssaeeeeeeoes hww.sssetts//mmaaactttac wwtogiin usi usingt

---

Published as a conference paper at ICLR 2021
TNC
Triplet Loss
CPC
Mals
Mat
20
-10
-10
-20
-28
-33
(a) TNC representations
(b) T-loss representations
(c) CPC representations
Figure 2: T-SNE visualization of signal representations for the simulated dataset across all baselines. Each data point in the plot presents a 10-dimensional representation of window of time series of size 50, and the color indicates the latent state of the signal window. See Appendix A.7 for similar plots from different datasets.
measures intra-cluster similarity and inter-cluster differences. This is positive index score, where smaller values indicate low within-cluster scatter and large separation between clusters. Therefore, lower score represents better clusterability (more details on the cluster validity scores and how they are calculated can be found in Appendix A.5).
Simulation
ECG Waveform
HAR
Method
Silhouette t
DBI +
Silhouette f
DBI 4
Silhouette
DBI 4
TNC
0.71
0.36:10.01
0.44
0.74
0.61
0.52
CPC T-Loss
0.51
0.84
0.26:10.02
1.44:10.04
0.58+0.02
0.57=10.05
0.61
0.64+0.12
0.25+0001
1.30=#003
0.17=0.01
1.76=10.20
K-means
0.01:10.019
7.23
0.19
3.65
0. 0.12:10.40
2.66
Table 1: Clustering quality of representations in the encoding space for multiple datasets.
Table I summarizes the scores for all baselines and across all datasets, demonstrating that TNC is superior learning representations that can distinguish the latent dynamics of time series. CPC performs closely to Triplet loss on waveform data but performs poorly on the simulated dataset, where signals are highly non-stationary, and transitions are less predictable. However, for the HAR dataset, CPC clusters the states very well because most activities are recorded in specific order, empowering predictive coding. Triplet loss performs reasonably well in the simulated setting; however, it fails to distinguish states 0 and 2, where signals come from autoregressive models with different parameters and have a relatively similar generative process. Performing K K-means on the original time series generally does not generate coherent clusters, demonstrated by the scores. However, the performance is slightly better in time series like the ECG waveforms, where the signals formed by consistent shapelets, and therefore the DTW measures similarity accurately.
4.2 EVALUATION: CLASSIFICATION
We further evaluate the quality of the encodings using a classification task. We train a linear classifier evaluate how well the representations can used classify hidden states. The performance of all baselines is compared to supervised classifier, composed of an encoder and a classifier with identical architectures to that of the unsupervised models, and a K-nearest neighbor classifier that uses DTW metric. The performance is reported as the prediction accuracy and the area under the precision-recall curve (AUPRC) score since AUPRC is a more accurate reflection of model performance for imbalance classification settings like the waveform dataset.
Table 2 demonstrates the classification performance for all datasets. The performance the classifiers that TNC representations are closer to the end-to-end supervised model in comparison to CPC and Triplet Loss. This provides further evidence that our encodings capture informative parts of the time series and generalizable to be used for downstream tasks. In datasets like the HAR, where
7

---

Published as a conference paper at ICLR 2021
Simulation
ECG Waveform
HAR
Method
AUPRC
Accuracy
AUPRC
Accuracy
AUPRC
Accuracy
TNC
0.99+00.00
97.52++0.13
0.01
77.79:
0.94+00007
88.32
CPC
0.69+00.06
70.26++6.48
0.42++0.01
68.644+0.49
0.93+00.006
86.43+1.41
T-Loss
0.78+00.01
76.66111.40
0.47=10.00
1..5
0.71
63.6013.37
KNN
0.42:t0.00
55.53:10.65
0.38+
54.76+15.46
0.75+00.01
84.85=-0.84
Supervised
0.99:10.00
98.56:10.13
0.67
94.81:1.28
0.9810000
92.03
Table 2: Performance of all baselines in classifying the underlying hidden states of the time series, measured as the accuracy and AUPRC score.
an inherent ordering usually exists in the time series, CPC performs reasonably. However, in datasets with increased -on-stationarity, the performance drops. Triplet Loss is also a powerful framework, but since it samples positive examples from overlapping windows of time series, is vulnerable to map the overlaps into the encoding and, therefore, fail to learn other hand, samples similar windows from a wider distribution, defined by the temporal neighborhood, where many of the neighboring signals do not necessarily overlap. The lower performance of the CPC and Triplet Loss methods can also be partly because none of these methods explicitly account for the potential sampling bias that happens when randomly selected negative examples are similar to the reference W.
general representations. TNC, on the
4.3 EVALUATION: TRAJECTORY
Time series Sample Trajectory
1
Encoding Trajectory
Figure 3: Trajectory of a signal encoding from the simulated dataset. The top plot shows the original time series with shaded regions indicating the underlying state. The bottom plot shows the 10 dimensional encoding of the sliding windows W, where = 50.
This section investigates the trajectories of learned encodings over time to understand how the state transitions are captured and modeled in the representation space. This is an important property for non-stationary time series where underlying states change over time, and capturing those changes is critical in many application domains such as healthcare. Figure shows sample from the simulated dataset. The top panel shows the signal measurements over time, and the shaded regions indicate the underlying latent states. The bottom panel illustrates the 10-dimensional representation of a sliding window W, estimated over time. From the bottom panel of Figure 3, we can see that the encoding pattern changes at state transitions and settle into different pattern, corresponding to the new state. This change happens at every transition, and we can see the distinct patterns for all 4 underlying states in the representations. This analysis of the trajectory of change could be very informative for the users' post-analysiss for instance, in clinical applications, it could help clinicians visualize the evolution of the patient state over time and plan treatment based the state progression
8

---

Published as a conference paper at ICLR 2021
5 RELATED WORK
While integral for many applications. unsupervised representation learning has been far less studied for time series (Längkvist al., 2014), compared to other domains such as vision or natural language processing (Denton al., 2017; Radford et al., 2015; Gutmann & Hyvärinen, 2012; Wang & Gupta, 2015). One of the earliest approaches to unsupervised end-to-end representation learning in time series is the use of auto-encoders (Choi et al., 2016a; Amiriparian et al., 2017; Malhotra et al., 2017) and seq-to-sey models (Lyu et al., 2018), with the objective to train encoder jointly with a decoder that reconstructs the input signal from its learned representation. Using fully generative models like variational auto-encoders is also useful for imposing properties like disentanglement, which help with the interpretability of the representations (Dezfouli et al., 2019). However, in many cases, like for feeufrryy physiological signals, the reconstruction of complex time series can be challenging; therefore, more novel approaches are designed to avoid this step. Contrastive Predictive Coding (Oord et al., 2018; Löwe et al., 2019) learns representations by predicting the future in latent space, eliminating the need to reconstruct the full input. The representations are such that the mutual information between the original signal and the concept vector is maximally preserve using lower bound approximation and a contrastive loss. Very similarly, in Time Contrastive I Learning (Hyvarinen Morioka, 2016), a contrastive loss is used to predict the segment-ID of multivariate time-series way to extract representation. Franceschi et al. (2019) employs time-based negative sampling and a triplet loss to learn scalable representations for multivariate time series. Some other approaches use inherent similarities in temporal data to learn representations without supervision. For instance, in ummilarity-preserviin constrained to preserve the pairwise similarities that exist in the time domain, measured by DTW distance. Another group of approaches combines reconstruction loss with clustering objectives to cluster similar temporal patterns in the encoding space (Ma et al., 2019).
representation learning (Lei et al., 2019), learned encodings are
In healthcare, learning representation of rich temporal medical data is extremely important for understanding patients' underlying health conditions. However, most of the existing approaches for learning representations are designed for specific downstream tasks and require labeling by experts (Choi et al., 201 2016b;;; Tonekaboni et al., 2020). Examples of similar works to representation learning in the field of clinical ML include computational phenotyping for discovering subgroups of patients with similar underlying disease mechanisms from temporal clinical data (Lasko et al., 2013; Suresh et al., 2018; 8; Schulam et al., 2015), and disease progression modeling, for learning the hidden vector of comorbidities representing a disease over time (Wang et al., 2014; Alaa & van der Schaar, 2019).
CONCLUSION
This paper presents novel unsupervised representation learning framework for complex multivariate time series, called Temporal Neighborhood Coding (TNC). This framework designed to learn the underlying dynamics of non-stationary signals and to model the progression over time by defining a temporal neighborhood. The problem is motivated by the medical field, where patients transition between distinct clinical states over time, and obtaining labels to define these underlying states is challenging. We evaluate the performance of TNC on multiple datasets and show that our representations are generalizable and can easily be used for diverse tasks such as classification and clustering. We finally note that TNC is flexible to be used with arbitrary encoder architectures; therefore, the framework is applicable tasks presented in this paper, general representations can be used for several other downstream tasks, such as anomaly detection, which is challenging in supervised learning settings for time series data in sparsely labeled contexts.
many time series data domains. Moreover, in addition to
ACKNOWLEDGMENTS
Resources used in preparing this research were provided, in part, by the Government of Canada through CIFAR, and companies sponsoring the Vector Institute. This research was undertaken, in part, thanks to funding from the Canadian Institute of Health Research (CIHR) and the Natural Sciences and Engineering Research Council of Canada (NSERC).

---

Published as a conference paper at ICLR 2021
REFERENCES
Ahmed Alaa and Mihaela van der Schaar. Attentive state-space modeling of disease progression. 2019.
Shahin Amiriparian, Michael Freitag, Nicholas Cummins, and Björ Schuller. Sequence sequence autoencoders for unsupervised representation learning from audio. In Proc. of the DCASE 2017 Workshop, 2017.
Shaojie Bai, Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.0127,, 2018.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 3598):1798-1828, 2013.
Edward Choi, Mohammad Taha Bahadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James Bost, Javier Tejedor-Sojo, and Jimeng Sun. Multi-layer representation learning for medical concepts. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1495-1504, 2016a.
Edward Choi, Andy Schuetz, Walter Stewart, and Jimeng Sun. Medical concept representation learning from electronic health records and its application heart failure prediction. arXiv preprint arXiv: 1602.03666 201
Youngduck Choi, Chill Yi-I Chiu, and David Sontag. Learning low-dimensional representations of medical concepts. AMIA Summils on Translational Science Proceedings, 201 6:41, 201 6c.
Ching-Yar Chuang, Joshua Robinson, Lin Yen-Chen, Antonio Torralba, and Stefanie Jegelka. Debi- ased contrastive learning. arXiv preprint arXiv:2007.002244 2020.
Emily L. Denton al. Unsupervised learning of disentangled representations from video. In Advances in Neural Information Processing Systems, pp. 4414 2017.
Amir Dezfouli, Hassan Ashtiani, Omar Ghattas, Richard Nock, Peter Dayan, and Cheng Soon Ong. Disentangled behavioural representations. Advances Neural Information Processing Systems, pp. 1-660, 2019.
Jeff Donahue and Karen Simonyan. Large scale adversarial representation learning. In Advances in Neural Information Processing Systems, pp. 10541-10551, 2019.
Marthinus Du Plessis, Gang Niu, and Masashi Sugiyama. Analysis of learning from positive and unlabeled data. In Advances in Neural Information Processing Systems, pp. 703-711, 2014.
Marthinus Christoffel Du Plessis and Masashi Sugiyama. Class prior estimation from positive and unlabeled data. IEICE TRANSACTIONS on Information and Systems, 1386:1135 2014.
Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In
Proceedings of the 14th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 213-220, 2008
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre- Alain Muller. Deep learning for time series classification: review. Data Mining and Knowledge Discovery, (3(())311119)))) 2019.
Yean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. In Advances in Neural Information Processing Systems, pp. 4652-4663, 2019.
Michael U Gutmann and Aapo Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research, 13 (Feb):307-361, 2012.
Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ICA. In Advances in Neural Information Processing Systems, pp. 3765-3773, 2016.
10

---

Published as a conference paper at ICLR 2021
Ryuichi Kiryo, Gang Niu, Marthinus C Du Plessis, and Masashi Sugiyama. unbttie--nllbeledd learning with non-negative risk estimator. In Advances Neural Information Processing Systems, pp. 1675-1685, 2017.
Martin Längkvist, Lars Karlsson, and Amy Loutfi. A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognition Letters, 42: 10:11-22 1-2,, 2014.
Thomas A Lasko, Joshua C Denny, and Mía A Levy. Computational phenotype discovery using unsupervised feature learning over noisy, sparse, and irregular clinical data. PloS one, 8(6), 2013.
Wee Sun Lee and Bing Liu. Learning with positive and unlabeled examples using weighted logistic regression. In International Conference on Machine Learning, volume 3, pp. 448 455, 2003.
Qi Lei, Jinfeng Yi. Roman Vaculin, Lingfei Wu, and Inderjit S Dhillon. Similarity preserving representation learning for time series clustering. In IJCAI, volume 19, pp. 2845-2851, 2019.
Xiaoli Li and Bing Liu. Learning to classify texts using positive and unlabeled data. In IJCAI,
volume 3, pp. 587-592, 2003.
Sindy Löwe, Peter O'Connor, and Bastiaan Veeling. Putting an end to end-to-end: Gradient-isolated learning of representations. In Advances in Neural Information Processing Systems, pp. 3039-3051, 2019.
Xinrui Lyu, Matthias Hueser, Stephanie L. Hyland, George Zerveas, and Gunnar Rätsch. Improving clinical predictions through unsupervised time series representation learning. arXiv preprint arXiv:1812.00499, 2018.
Qianli Ma, Jiawei Zheng, Sen Li, and Gary W Cottrell. Learning representations for time series clustering. In Advances in Neural Information Processing Systems, pp. 3776-377666 2019.
Pankaj Malhotra, Vishnu TV, Lovekesh Vig, Puneet Agarwal, and Gautam Shroff. Timenet:
Pre-trained deep recurrent neural network for time series classification. arXiv preprint 2017.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representa-
tions in vector space. arXiv preprint arXiv:1301.3788 2013.
George Moody. A new method for detecting atrial fibrillation using RR intervals. Computers in Cardiology, pp. 227-230, 1983.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint 1rXiv::609. 03499, 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 187.0:180, 2018.
Slobodan Petrovic. A comparison between the silhouette index and the davies-bouldin index in labelling ids clusters. In Proceedings of the 11th Nordic Workshop of Secure IT Systems, pp. 53-64, 2006.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06433 2015.
Alec Radford, Rafal Jozefowicz, and llya Sutskever. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 2017.
Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. A theoretical analysis of contrastive unsupervised representation learning. In International Confer- ence on Machine Learning, pp. 5628-5637. PMLR, 2019.
Peter Schulam, Fredrick Wigley, and Suchi Saria. Clustering longitudinal clinical marker trajectories from electronic health data: Applications to phenotyping and endotype discovery. In Twenty-Ninth AAAI Conference Artificial Intelligence, 2015.

---

Published as a conference paper at ICLR 2021
Harini Suresh, Jen J Gong, and John V Guttag. Learning tasks for multitask learning: Heterogenous patient populations in the icu, In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 802 802--10,770 2018.
Sana Tonekaboni, Shalmali Joshi, Kieran Campbell, David K Duvenaud, and Anna Goldenberg. What went wrong and when? instance-wise feature importance for time-series black-box models. Advances in Neural Information Processing Systems, 33, 2020.
Xiang Wang, David Sontag, and Fei Wang. Unsupervised learning of disease progression models. In Proceedings of the 20th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 85-94, 2014.
Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In Proceedings of the IEEE International Conference Computer Vision, pp. 2794-28022 2015.
Qiang Yang and Xindong Wu. 10 challenging problems in data mining research. International Journal of Information Technology & Decision Making, 5(04))597-604, 2006.
Tom Young, Devamanyu Hazarika, Soujanya Poria, and Erik Cambria. Recent trends in deep learning based natural language processing. IEEE Computational Intelligence Magazine, 13(3):55-75, 2018.
12

---

Published as a conference paper at ICLR 2021
A APPENDIX
A.1 SIMULATED DATASET
Lo0
100 200
100
400
700
Figure A.1: A normalized time series sample from the simulated dataset. Each row represents single feature, and the shaded regions indicate one of the 4 underllying simulated states.
The simulated time series consists of 3 features generated from different underlying hidden states. Figure A.I shows sample from this dataset. Each panel in the figure shows one of the features, and the shaded regions indicate the underlying state of the signal in that period. We use a Hidden Markov Model (HMM) to generate these random latent states over time. The transition probability is set equal to %55 for switching
alternating state, and %85 for not changing state. In each state, the time series is generated from a different signal distribution. Table 3 describes the generative process of each signal feature in each state. Note that feature and are always correlated, mainly to mimic realistic clinical time series. As an example, physiological measurements like pulse rate and heart rate are always correlated.
State
State 2
State 3
State 4 NARMA NARMA,
Feature
GP (periodic)
NARMA,
GP (Squared Exp.)
Feature 2 GP (periodic)
NARMA
GP (Squared Exp.)
Feature 3
GP (Squared Exp.)
NARMA,
GP (periodic)
NARMA
Table 3: Signal distributions for each time series feature of the simulated dataset
In the state 1, the correlated features are generated by Gaussian Process (GP) with periodic kernel. Feature 3, which is uncorrelated with the other two features, comes from another GP with a squared exponential kernel. In addition GPs, we also have multiple Non-Linear Auto-Regressiv Moving Average (NARMA) time series models. The linear function of NARMA, and NARMA are shown in Equation 2 and 3.
NARMA, : y( 0k3 0.05 Σ ulk 1.5u(k (n -nn-n(nnnnll + 0.1 (2)
NARMA y(k = 0.1 0.25g
Σ u(k: - i) 2.5u(k - (n 1))u(k) 0.005 (3)
white Gaussian noise with a = 0.3 is added to all signals, and overall, the dataset consists 500
instances of T 2000 measurements.
13

---

Published as a conference paper at ICLR 2021
A.2 BASELINE IMPLEMENTATION DETAILS
Implementation of all baselines are included in the code base for reproducibility purposes, and hyper-parameters for all baselines are tuned using volsdvaali.
Contrastive Predictive Coding (CPC): The CPC baseline first processes the sequential signal windows using an encoder Z, = Enc(X,), with similar architecture to the encoders of other baselines. Next, an autoregressive model Par aggregates all the information in Zig: and summarizes it into context latent representation c Par (Z In our implementation, we have used a single layer, a one-directional recurrent neural network with GRU cell and hidden size equal to the encoding size as the auto-regressor. Like the original paper, the density ratio is estimated using a linear transformation, and the model is trained for 1 step ahead prediction.
Triplet-Loss (T-Loss): The triplet loss baseline is implemented using the original code made available by the authors on Github".
KNN and K-means: These two baselines for classification and clustering are implemented using the tslearn library *, that integrates distance metrics such as DTW. Note that evaluating DTW is computationally expensive, and the tslearn implementation is not optimized. Therefore, for the waveform data with windows of size 2500, we had to down-sampled the signal frequency by factor of two.
A.3 TNC IMPLEMENTATION EXTRA DETAILS
To define the neighborhood range in the TNC framework, as mentioned earlier, we use the Augmented- Dickey Fuller (ADF) statistical test to determine this range (n) the region for which the signals remain stationary. More precisely, we gradually increase the range, from a single window size up to 3 times the window size (the upper limit we set), and repeatedly perform the ADF test. We the p-value from this statistical test to determine whether the Null hypothesis can be rejected, meaning that the signal is stationary. At the point where the p-value is above our defined threshold (0.01), we can no longer assume that the signal is stationary, and this is where we set the 7) parameter. Now, once the neighborhood defined, we make sure the non-neighboring samples are taken from the distribution of windows with at least x away from W,, ensuring a low likelihood of belonging to the neighborhood. Note that for implementation of ADF, we use the stats model library 5, Unfortunately, this implementation is not optimized and does not support GPU computation, therefore evaluating the neighborhood range using ADF slows down TNC framework training. As a future direction, we working optimized implementation of the ADF score for our framework.
A.4 SELECTING THE WINDOW SIZE
The window size 8 is an important factor in the performance of representation learning framework. not only for TNC but also for similar basclines such as CPC and triplet loss. Overall, the window size should be selected such that it is long enough to contain information about the underlying state and not too long to span over multiple underlying states. In our settings, we have selected the window sizes based on our prior knowledge of the signals. For instance, in the case of an ECG signal, the selected window size is equivalent to 7 seconds of recording, which is small enough such that the ECG remains stable state and yet has enough information to determine that underlying state. Our also
understanding of the time series data can help select appropriate window size, but experiment with different to learn this parameter. Table 4 shows classification performance results for the simulation setups, under different window sizes. We can clearly the drop in performance for all baseline methods when the window size is too small or too large.
'https: hhttps://tsll httpss://tsleearn / http:////e/eeeat/e/teeeeeaeeeeee html
Phttps:/wwwl statssodetltteele https//mww...ooed///////ereeeeee adfuller.hti html
tsa. stattoo
14

---

Published as a conference paper at ICLR 2021
10
50
100
AUPRC
Accuracy
AUPRC
Accuracy
AUPRC
Accuracy
TNC CPC
0.74 + 0.01
71.60 + 0.59
0.99 + 0.00
97.52 + 0.13
0.84 + 0.11
84.25 + 9.08
0.49 + 0.02
51.85 + 1.81
0.69 + 0.06
70.26 + 6.48
0.49 + 0.05
56.65 0.81
T-Loss
0.48 = 0.06
56.70 1.07
0.78 + 0.01
76.66 1.14
0.73 + 0.008
73.29 1.58
Table 4: Downstream classification performance for different window size å on the simulated dataset
A.5 CLUSTERING METRICS
Most cluster validity measures assess certain structural properties of clustering result. our evaluation, we have used two measures, namely the Silhouette score and Davies-Bouldin index, to evaluate the representations' clustering quality. Davies-Bouldin measures intra-cluster similarity (coherence) and inter-cluster differences (separation). Let C = [CI...., Ck) be clustering of a set D of objects. The Davics-Bouldin score evaluated as follows:
mar, s(C) + s(C) 8C,C,
DB
Where s(C) measures the scatter within a cluster, and is cluster to cluster distance measure. On the other hand, the silhouette score measures how similar an object is to its cluster compared to other clusters. Both measures are commonly used for the evaluation of clustering algorithms. A comparison of metrics has shown that the Silhouette index produces slightly more accurate results in some cases. However, the Doviei-Bouldin index is generally much less complex to compute Petrovic (2006).
A.6 SETTING WEIGITT PU LEARNING
As mentioned in the Experiment section, the weight parameter in the loss is the probability of sampling positive window from the non-neighboring region. One way to set this parameter using prior knowledge of the number and the distribution of underlying states. Another way is to learn it as hyperparameter. Table shows the TNC loss for different weight parameters. The loss column reports the value measured in Equation 1, and the accuracy shows how well the discriminator identifies the neighboring samples from non-neighboring ones for settings with different weight parameters. To also assess the impact of re- weighting the loss downstream classification performance, compared these performance measures for weighted and non non-weighted settings. Table 6 demonstrates these results and confirms that weight adjusting the loss for neighbiigg samples improves the quality of learned representations.
Simulation
ECG Waveform
HAR
Weight
Loss
Accuracy
Loss
Accuracy
Loss
Accuracy
0.2
0.582+0.002
74.29-10.61
0.631
60.4442.55
0,475+0.000
85.75=0.5
0.1
0.57110.011
70.47
0.637=0.011
63.67+1.22
0.41340.003
88,21=1.29
0.05
0.576+0.002
75.23+0..24
0.622+0.023
66.04
0.083
87.33+00.17
Table Training the TNC framework using different weight parameters. The loss is the measured value determined in Equation I, and the Accuracy is the accuracy of the discriminator.
Weighting?
Simulation
ECG Waveform
HAR
True
97.52.10.13
7..79-0.84
88.32-1..12 73.2
False
97.17+00.4
1.42
Table Downstream classification accuracy on simulated data with the TNC frameworks, using different weighting strategies: 1)Trained with weight adjustment, 2)Trained with 0.
15

---

Published as a conference paper at ICLR 2021
A.7 SUPPLEMENTARY FIGURE
A.7.1 CLINICAL WAVEFORM DATA
In order understand what TNC framework encodes from the high dimensional ECG signals, we visualize the trajectory of the representations of an individual sample over time. Figure A.3 demonstrates this example, where the top rows ECG signals from two recording leads and the bottom row demonstrates the representation vectors. We see that around second 40 the pattern in the representations change result of an artifact happening in one of the signals. With the help from our clinical expert, we also tried to interpret different patterns in the encoding space. For instance, between time 80 and 130, where features 0- become more activated, has increased. Increase in HR can be indicators of arrhythmia that we believe TNC has captured. Figure A.2 shows the distribution the latent encoding of ECG signals for different basclines, with colors indicating the arrhythmia class.
heart rate (HR) increased frequency in the ECG signals and is one of the
TNC
Triplet Loss
CPC
20
-30
-11
-21
-38
Figure A.2: T-SNE visualization of waveform signal representations for unsupervised representation learning baselines. Each point in the plot is 64 dimensional representation of a window of time series, with the color indicating the latent state.
Time series Sample Trajectory
Py
Encoding Trajectory
6 02 28 0010633
Figure A.3: Trajectory of a waveform signal encoding. The top two plots show the ECG recordings from ECG leads. The bottom plot shows the dimensional encoding of the sliding windows W, where 5 2500.
A.7.2 HAR DATA
Figure A.4 and A.5 are similar plots the ones demonstrated in the previous section, but for the HAR dataset. As shown in Figure A.5, the underlying states of the signal are clearly captured by the TNC framework different patterns in the latent representations.
16

---

Published as a conference paper at ICLR 2021
TNC
Triplet Loss
Loss
CPC
Map
20
11
-20
-20
-21
-22
Figure A.4: T-SNE visualization of HAR signal representations for all baselines. Each point in the plot is a 10 dimensional representation of a window of 5 = 4, with colors indicating latent states.
series Sample Trajectory
0.
-1.0
50
100
150
200
250
Encoding Trajectory
Figure A.5: Trajectory of a HAR signal encoding. The top plot shows the original time series with shaded regions indicating the underlying state. The bottom plot shows the 10 dimensional encoding of the sliding windows W;, where 5 = 4.
A.7.3 SIMULATION DATA
In addition to the initial experiment, we also show the trajectory of the encoding for a smaller encoding size (3). In this setting, we have underlying states the signal, and only 3 dimensions for the encoding.
Time series Sample Trajectory
Encoding Trajectory
Figure A.6: Trajectory of a simulation signal encoding. The top plots shows the signals and the
bottom plot shows the dimensional encoding of the sliding windows W, where 5 50.
17


Reviews:
[{"review_id": "te77ivmOm4", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a novel unsupervised encoding scheme for time series. Utilizing a statistical test for non-stationarity, the authors derive a Temporal Neighborhood Coding (TNC) scheme and combine it with ideas from Positive-Unlabeled (PU) learning to learn informative hidden representations of time series windows. The representations are evaluated in terms of how well they can be clustered and how much they influence classification performance on three data sets. The supreme performance was demonstrated when comparing to the state of the art methods and a $k$NN (for classification) baseline. Furthermore, the authors illustrate how the learnt representations remain interpretable as long as the encoding network is reasonably small. \n\nThe presented work is very well motivated and described. The presentation of the ideas is clear and I think the authors did a great job making the manuscript accessible. I like the simplicity of the figures which do not suffer from too little information content. Starting with the analysis of a synthetic data set followed by two real-world data sets is a commendable route to choose for this kind of work. I particularly like the analysis of low sample size-high length ECG data set. I would say job well done! One aspect that is not quite clear to me is how do you combine the ADF test with the fixed-length window sizes that you mention in your experiments?\n\nSome minor comments/questions:\n\n•\tI am somewhat confused about the window size information you give in the experiments and for example in Figure 3. Here you say $\\delta=50$. I thought the window size is changed dynamically based on the ADF test which leads me to the second question\n\n•\tIs the process of determining the window size a preprocessing step or is it part of an end-to-end framework? \n\n•\tHow do you solve the classification task when you have an encoding for each window? I assume you do not have one specific label for each window. If every single measurement received a ground truth label, isn’t it possible to extract a window annotated with two different labels? Or do you take care that the windows you extract are always centred around the QRS-complex or always starts with the P-wave? \n\n•\tIt would also be interesting to evaluate how you could perform patient-wise classification (e.g. diagnosis prediction). Maybe it would be an impactful extension to find a good aggregation scheme for the embeddings to summarize single patients. \n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "qKPB-4VpXg6", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors defines the notion of temporal neighborhood to segment time series.\nIt is a location in real vector space where neighboring segments in the original time serie are projected into as well as similar but distant (in time) segments.\nThe model is trained using a discriminative loss that pushes away (in the feature space) windows distant in time while caring for the possible periodicity [Elkan & Noto 2018] \n\nOverall the paper is well written and easy to follow. \nThe idea is very interesting and I see several potential consequences and use-cases. \nNevertheless, I have several concerns.\n\n1/ I think the paper lacks classic baselines. \nI consider that HMM is part of unsupervised representation learning. \nIn its original form it does not include NN but, if needed several works have studied variants, eg.:\nFiroiu, L., & Cohen, P. R. (2002, July). Segmenting time series with a hybrid neural networks-hidden Markov model.\nThe authors rightfully raised the issue of how large (in time) a neighborhood is (\\eta). For that, Hidden semi-Markov models constitute a nice \"dynamic\" approach.\nAs well, HMMs are used to create the synthetic data-set.\n* Ok I see, interesting results. Thanks.\n\n2/ Regarding the PU methods, why choosing solution 2 over 1?\n* Thanks for the answer. My question was suggesting that it should better justified in the text.\n\n3/ \"For an ideal encoder that preserves the neighborhood properties in the encoding space, p(Z_l ∈ N_t ) should\nbe close to p(W_l ∈ N_t ), where Z_l is the representation of W_l\"\nI understood that N_t is in the encoding space. So the same letter can not be used for W_l.\nI don't see how the training steers the encoder to satisfy this property.\nOn the other hand, this constrain contradicts the PU discussion.\n* Ok I understand better. thanks for the answer.\n\n4/ Figure 1 is not very clear. The encoder outputs Z_t and P(Z_t)\n\n5/ The same network is used for all the baseline. However, Franceschi et al.'s work clearly relies on convolutional layers. Hence not using such is a bit unfair toward their method (synthetic data and HAR). \n* I understand your point. Nevertheless some models are very depend on the architecture, since the depend on some feature that not all architecture can provide. And I think your results show that. \n\n6/ Evaluation: Trajectory. Figure 3. Very interesting plot that clearly shows the benefit of this work in terms of interpretability. What happens for a 3 dimensional encoding? \n* Here it is my fault, I meant 4 dimensional encoding: as much as the number of hidden states. The point was to see if the model is would learn clearly different representation for each state.\n\n7/ The notion of  temporal neighborhood depends on \\eta and is based on a Gaussian distribution around the current timestamp. How does it behave for small and large eta? \nA model with a small eta might detect too many segments. \nIf large, I guess, the models smooths too much the variations and fails to cluster different behavior. Especially it will fail to detect abrupt changes.\nIn any case, such an analysis is missing.\n* Thank you for the explanations.\n\nI have spotted two typo in the last paragraph of page 3:\nNote that the Discrimin[a]tor...\n...the latent space, since [it] allows...\n\n\n* I update my score.", "extra_scores": {}, "extra_text": {}}, {"review_id": "cp4Zumq5KAn", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n\nThe paper proposes an unsupervised representation (embedding) learning method for time-series. While unsupervised representation learning has been extensively studied and shown good performance in fields like NLP and vision, it is relatively new to the time-series community. This paper, in contrast to recent work (CPC and Triplet-Loss), has the following differences:\n1. It estimates stationary temporal windows using statistical testing for stationarity/non-stationarity. \n2. It learns embedding using contrastive learning as in CPC and Triplet-loss, but additionally takes into account that fact that naive negative sampling may include false negatives and hurt embedding learning on time-series that have strong seasonality. Instead, it adopts the framework Positive Unlabeled learning to address this issue.\n\nQuality and clarity\n\nI believe the work itself is of good quality (or has the potential to be), but the presentation of the work is not very clear to me:\n1. The neural network architecture adopted is not well explained: if the architecture varies across problems then the authors should clearly state this and provide more explanation on the general framework using Figure 1; currently, the framework in Figure 1 is also not well discussed. \n2. The way how the statistical test (Augmented Dickey Fuller) is incorporated for window-detecting is not clear. According to the explanation in Section 2, \"For every W_t, we gradually increase the neighborhood size and measure the p-value from the test\", it seems the windows will have varying sizes according to test p-values; but in all experiments, the window sizes are fixed across the entire time-series. The authors should provide some explanation to this.\n\n\nOriginality\n\nThe paper builds on existing framework: contrastive learning based unsupervised representation learning, but proposes a new method to cater to the peculiarity of long-term time-series (non-stationarity and seasonality).\n\nSignificance\n\nGiven that exploration of unsupervised representation learning in time-series is at early stage, I believe this work will inspire future works along this direction.", "extra_scores": {}, "extra_text": {}}, {"review_id": "4UST4e4yzuG", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper Summary:\n\nThis paper proposes a self-supervised encoder-discriminator based framework for embedding the multivariate time series into a compact fixed dimensional representation. The approach dubbed Temporal Neighborhood Coding (TNC) leverages the concept of a neighborhood in time (with stationary properties), and learns time series representations by ensuring the distribution of neighboring signals is distinguishable from the distribution of non-neighboring signals, in the encoding space. Empirical evidence is provided that such embedding of time series results in clusters of higher quality, as well that use of such obtained representations for supervised tasks outperforms few competitor (unsupervised) approaches. \n\n##########################################################################\n\nStrong points:\n1. The neighborhood-based unsupervised learning framework accounts for dynamic changes that occur among samples, i.e. non-stationarity of signals.\n\n2. The introduction of the concept of a temporal neighborhood as the distribution of similar windows in time, whose range can be automatically estimated by analyzing the stationarity of time series with statistical tools such as the Augmented Dickey Fuller (ADF) test.\n\n3. Leveraging the sample weight adjustment, a concept from Positive Unlabeled (PU) learning, to account for the potential bias introduced by sampling negative examples which is a common issue most contrastive approaches suffer from.\n\n4. The proposed TNC outperforms Contrastive Predictive Coding (CPC) and Triplet-Loss (T-Loss), both being state-of-the-art unsupervised representation learning approaches.\n\n5. The paper is well structured and written in a fairly clear and comprehensive manner. \n\n##########################################################################\n\nImprovement points:\n1. Efficiency and scalability to high dimensions are mentioned as merits of TNC in the introduction, but are never discussed in further detail. Could the authors elaborate a bit on the components of TNC that allow for its claimed efficiency and scalability and how does TNC compare on these two fronts to other unsupervised representation learning approaches?\n\n2. A bidirectional, single-layer RNN encoder is used in the experiments on the Simulated and HAR datasets, whereas a 2-channel, 1-dimensional strided CNN encoder was applied on the ECG Waveform dataset. The reason behind this decision is not completely clear to me. I would encourage the authors to clearly justify the choice of different encoder architectures in the two cases.\n\n3. In the cases where an RNN architecture is considered for the encoder, the authors could have considered time-aware RNN/LSTM autoencoders as additional baselines. Some of these methods include:\n- Baytas, I. M., et al. Patient subtyping via time-aware LSTM networks. In Proceedings of the 23rd ACM SIGKDD (2017).\n- Wenjie Pei and David MJ Tax. Unsupervised Learning of Sequence Representations by Autoencoders. arXiv preprint arXiv:1804.00946 (2018).\n\n4. Although the neighborhood range $\\eta$ can be automatically determined, choosing the window size $\\delta$ still remains unclear, as the performance of TNC under different window sizes was not analysed. Moreover, the selected window sizes seem to be different for different datasets (for instance, a window size of 50 is used for the Simulated data, while a windows size of 4 is used in the case of the HAR dataset). I would like to ask the authors to provide some guidelines on how the window size is (or can be) selected?\n\n5. Some statements appear to be vague and leading away from the main point. This includes usage of phrases such as:\n- “Time series are often *complex*” - as in having multiple interacting parts? (I don’t feel that aspect was crucial for paper topic)\n- “self-supervised framework for learning *robust* and generalizable representations” - how was the robustness defined and assessed? (no empirical evidence regarding the ‘robustness’ of the learned representations is presented).\n- “the representations are general, *transferable*, etc.” - transferable in what sense? (I believe that “applicable across multiple supervised tasks” would be a more appropriate choice of words in this context).\n\n6. There are several minor textual and format typos throughout the paper that can be easily addressed. Some of them are summarized as follows:\n- In the second sentence in section 3, consider replacing “state of the art” with “state-of-the-art”.\n- In the last sentence of the introductory part of section 3, replace “:” by “.”.\n- The term “GPs” is used in section 3.1, but is not defined earlier in the text.\n- In section 3.3, “concatenate activity samples” should be replaced by “concatenate the activity samples”.\n- In the last couple of sentences in section 4.1, replace “original time series” and “coherent cluster” with “the original time series” and “coherent clusters”, respectively.\n- There are certain instances of informal language use, such as “doesn’t” and “don’t”. Please consider replacing them with “does not” and “do not”, respectively.\n\n##########################################################################\n\nQuestions during rebuttal period:\nPlease address the aforementioned remarks/questions.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an unsupervised representation (embedding) learning method for time-series. Overall, the paper is well-motivated, well-written and easy to follow. As agreed by all reviewers, the idea is interesting. To further improve the paper, the authors are encouraged to justify the choice of encoder architectures and window size, and describe more clearly how the statistical test is incorporated.
Decision: Accept (Poster)