MultiRC: Joint Learning for Time Series Anomaly Prediction and Detection with Multi-scale Reconstructive Contrast

## Abstract

Abstract Many methods have been proposed for unsupervised time series anomaly detection. Despite some progress, research on predicting future anomalies is still relatively scarce. Predicting anomalies is particularly challenging due to the diverse reaction time and the lack of labeled data. To address these challenges, we propose MultiRC to integrate reconstructive and contrastive learning for joint learning of anomaly prediction and detection, with multi-scale structure and adaptive dominant period mask to deal with the diverse reaction time. MultiRC also generates negative samples to provide essential training momentum for the anomaly prediction tasks and prevent model degradation. We evaluate seven benchmark datasets from different fields. For both anomaly prediction and detection tasks, MultiRC outperforms existing state-of-the-art methods. The code is available at https://anonymous.4open.science/status/MultiRC-CCE6 .

## 1 Introduction

With the advancement of Internet of things (IoT), an increase number of sensors are utilized in industrial facilities to collect data in the form of continuous time series, which realizes the monitoring of system status (Li et al., ). The anomaly detection technology (Li et al., ; Wen et al., ; Chen et al., ) has been widely used, which locates system malfunctions by identifying anomalies in historical data (Figure 1a). Effectively detecting anomalies helps pinpoint the sources of faults and prevent the spread of malfunctions. However, anomaly detection can only identify issues after they have occurred, which cannot meet the need for preventive maintenance timely.

As a new practical problem, anomaly prediction aims to predict whether an anomaly will occur in the future by capturing the fluctuations at the current time. In IoT, anomaly prediction can prevent full-scale failures. Previous works (Yin et al., ; You et al., ) assume and have observed that anomalies in production often do not occur suddenly but evolve gradually. Different kinds of fluctuations exist before the real anomalies, where data change from normal to abnormal. As shown in Figure 1b, the yellow period exhibits fluctuation patterns beginning distinct from the past, indicating the occurrence of possible future anomalies. The time interval of fluctuations is called reaction time, and such fluctuations are called precursor signals of future anomalies (Jhin et al., ). Following previous works (Yin et al., ), we predict future anomalies by identifying precursor signals, and emphasize that anomalies without precursor signals are unpredictable.

Due to the costs and rarity of anomalies, anomaly labels are difficult to obtain. To solve this, anomaly detection is often modeled with self-supervised (reconstructive (Zhou et al., ) or contrastive (Yang et al., )) learning. Reconstruction methods expect normal points to be accurately reconstructed, while anomalies exhibit large reconstruction errors. Contrastive approaches promote similarity in the representations of positive sample pairs. Despite the effectiveness of existing works in anomaly detection, these methods cannot be directly and effectively extended to anomaly prediction, as shown in the following challenges.

Challenge 1: Different anomalies may occur rapidly or slowly, thus resulting in reaction time with varying lengths.
Reconstruction methods focus on reconstructing individual data points which struggle to capture gradual reaction time (Li et al., ; Audibert et al., ; Zhang et al., ). Contrastive learning methods compare time series segments based on fixed time lengths which struggle to capture varying reaction time. Further, variates in the time series have different semantics, resulting in existence of distinct reaction time for different variates. Existing methods cannot identify fluctuations with such varying reaction time for different variates adaptively.

Challenge 2: Without labeled anomalies as negative samples, existing self-supervised methods will degrade into a trivial model (Ruff et al., ) that cannot learn meaningful information. For contrastive approaches, the absence of negative samples will lead to the learned features falling into a single mode where all features seem similar (Chen & He, ), thus failing to identify fluctuations. For reconstruction methods, a large model will degrade into identity transformation and a small model cannot learn complex temporal patterns (Wang et al., ), thus failing to assess the accurate magnitude of the fluctuations. Anomaly prediction requires not only identifying fluctuations but also assessing the magnitude of these fluctuations to predict the probability of future anomalies. Therefore, the existing methods fail to learn meaningful information for anomaly prediction.

In this work, we propose joint learning for time series anomaly prediction and detection with multi-scale reconstructive contrast (MultiRC), where a dual branch with joint reconstructive and contrastive learning is built upon a multi-scale structure. For Challenge 1, our novel multi-scale structure adaptively recognizes the varying reaction time for different variates with adaptive dominant period mask. Meanwhile, we exploit an asymmetric encoder-decoder to fuse cross-scale information. For Challenge 2, we incorporate controlled generative strategies to construct diverse precursors as negative samples to prevent model degradation, instead of only generating positive samples by data augmentation as in the existing contrastive methods (Woo et al., ; Zhang et al., ). Specifically, our contrastive learning judges whether there are fluctuations by learning to distinguish positive and negative samples, while reconstruction learning assesses the magnitude of these fluctuations via learning to minimize the reconstruction errors for positive samples. Moreover, we provide a novel anomaly measurement for joint learning of anomaly prediction and detection, which helps anomaly prediction by detecting the degree of fluctuations in reaction time.

Figure: (a) Anomaly Detection
Refer to caption: x1.png

Our contributions are summarized as follows:

- •
A novel multi-scale structure is proposed to both reconstructive and contrastive learning, to adaptively recognize varying reaction time for different variates with adaptive dominant period mask.
- •
We propose controlled generative strategies to prevent model degradation and propose joint learning of anomaly prediction and detection with reconstructive contrast.
- •
For both anomaly prediction and anomaly detection tasks, MultiRC achieves state-of-the-art results across seven benchmark datasets.

## 2 Related work

Multi-scale Learning.
Some multi-scale methods have been proposed for time series modeling (Chen et al., ; Shen et al., ; Challu et al., ). THOC (Shen et al., ) learn multi-scale representations through different skip connections in RNN for each time point, but it cannot capture gradual change for continuous time intervals. DGHL (Challu et al., ) maps time series windows to hierarchical latent spaces, but it cannot capture cross-scale information from windows of different lengths. Besides, none of these methods can adaptively recognize different scales or fuse multi-scale information for anomaly prediction.

Time Series Anomaly Detection. As a problem of practical significance, time series anomaly detection has remained a focal point in the fields of machine learning and data mining. Over recent years, researchers have used reconstruction paradigm (Su et al., ; Xu et al., ; Wang et al., ) or contrastive learning paradigm (Yang et al., ; Jhin et al., ) to delve into deeper data representations and patterns. In the reconstructive paradigm, OmniAnomaly (Su et al., ) captures the normal patterns of multivariate time series by learning their robust representations. Anomaly Transformer (Xu et al., ) proposes a minimax strategy that combines reconstruction loss to amplify the difference between normal and abnormal. $\text{D}^{3}\text{R}$ (Wang et al., ) combines decomposition and noise diffusion to directly reconstruct corrupted data. However, these studies either operate on a singular scale or perform indiscriminate reconstruction across the entire input range, limiting their flexibility and adaptability in extracting anomaly signals.

In the contrastive learning paradigm, DCdetector (Yang et al., ) introduces a dual-branch attention structure to learn a permutation invariant representation. However, it only focuses on relationships between positive samples, which greatly limits its performance in anomaly prediction tasks that require explicit anomaly labels. PAD (Jhin et al., ) directly uses resampling to generate artificial anomaly patterns during data preprocessing. This simple approach fails to adequately simulate the fluctuations of anomaly precursors. Although numerous contrastive paradigms (Yue et al., ; Lee et al., ; Zhang et al., ; He et al., ) have been proposed in the field of time series analysis, they often merely treat the majority of time series segments as negative samples, which constrains the potential of contrastive learning to precisely identify anomalous patterns. We extract timing information at multiple scales to adapt to changes in reaction time. Furthermore, the model identifies fluctuations and amplitude of fluctuations through contrastive learning and reconstruction. Appendix B shows the architecture comparison of three approaches.

## 3 METHODOLOGY

Problem Definition. We denote the time series ${\rm\bf X}\in\mathbb{R}^{T\times c}$ of length $T$, where each ${\rm\bf x}_{t}\in\mathbb{R}^{c}$ is the observation at time $t$ and $c$ is the dimension of multivariate, such as the number of different sensors. The reaction time is a time interval $[t-r,t]$ with length $r$, where there are fluctuations in data ${\rm\bf X}_{t-r:t}$ called the precursor, and there are real anomalies in future data ${{\rm\bf X}}_{t+1:t+f}$. Data begin to change from normal to abnormal during the reaction time. The greater the degree of fluctuations, the higher the probability of future anomalies occurring. We emphasize that the anomalies without reaction time are unpredictable (Yin et al., ; Jhin et al., ).

Figure: Figure 2: The overall architecture of MultiRC.
Refer to caption: x3.png

For time series anomaly detection tasks, it takes the input ${\rm\bf X}$ and outputs a vector ${\rm\bf y}\in\mathbb{R}^{\mathrm{T}}$ by sliding widows where ${\rm\bf y}_{t}\in\{0,1\}$ and 1 indicates an anomaly. For anomaly prediction, given a current time $t$ and the historical sub-sequence ${{\rm\bf X}}_{t-h:t}$ of length $h$, the model outputs a probability score ${\rm\bf\hat{p}}_{t}$, which indicates whether there is precursor in ${{\rm\bf X}}_{t-h:t}$ or not and whether the future sub-sequence ${{\rm\bf X}}_{t+1:t+f}$ will be anomalous, where $f\geq 1$ is the length of the look-forward window.

### 3.1 Overall framework

MultiRC consists of four modules (Figure 2). Input sequence processing normalizes the input multivariate time series through instance normalization and then uses channel-independence to split the input into univariate sequences. In Figure 2 we take one univariate sequence x as an example, and the other univariate sequences are modeled similarly. The multi-scale structure uses adaptive dominant period mask to adaptively recognizes varying reaction time for different variates, and then segments univariate sequences into patches of varying granularities. Masked time series reconstruction exploits asymmetric encoder-decoder to fuse multi-scale information and then evaluate the amplitude of fluctuations. Generative-based contrastive learning uses controlled strategies to construct negative samples to prevent model degradation. The encoder learns representations for distinguishing positive and negative samples, thus better judging the existence of fluctuation in the reaction time. Our encoder backbone consists of Transformer blocks (Vaswani et al., ) to extract temporal features from time series data.

Overall, contrastive learning is used to judge the existence of fluctuation and reconstruction is used to assess the magnitude of the fluctuation, which both help indicate the probability of future anomalies.
The anomaly score comprises the reconstruction error and the discrepancy between the representations of the positive samples.

### 3.2 Multi-scale Structure

The assumption of the existence of reaction time (Yin et al., ; Jhin et al., ), where time series begin to change from normal to abnormal, ensures anomaly prediction. However, the duration of reaction time varies across different varieties and timestamps. To solve this, at each current time $t$, we explore the most dominant periods ${\rm\bf q}_{t}$ for each univariate sequence in the frequency domain.

Specifically, the multi-scale structure contains adaptive dominant period mask and multi-scale patching. Adaptive dominant period mask takes each univariate sequence ${\rm\bf x}$ as input and produces its masked sequence ${\rm\bf x}_{m}$. Intuitively, one variate with longer periodic changes is more likely to evolve gradually while one variate with shorter periodic changes is more likely to evolve rapidly. Thus, we capture and mask with the most dominant periods for each variate to estimate the reaction time.

Multi-scale patching takes as input ${\rm\bf x}$ and ${\rm\bf x}_{m}$ in the dual branch, respectively. For masked time series reconstruction, ${\rm\bf x}_{m}$ is fed into multi-scale patching; for generative-based contrastive learning, ${\rm\bf x}$ is fed in. The multi-scale patching produces patched sequences $\{{\rm\bf x}_{m,p}\}_{p=1}^{a}$ and $\{{\rm\bf x}_{p}\}_{p=1}^{a}$ with different scales. This multi-scale patching is independent of the dominant period mask, which helps to simultaneously capture features from time intervals of different scales, thereby accommodating the varying reaction time that is inconsistent with the dominant periods.

Adaptive Dominant Period Mask. To capture the patterns of dominant periods, we first extract the periodic information in the frequency domain using Fast Fourier Transform (FFT) as follows:

$$ $\mathbf{A}=\mathrm{Amp}\left(\mathrm{FFT}({\rm\bf x})\right)$ (1) $$

where $\mathrm{FFT}(\cdot)$ and $\mathrm{Amp}(\cdot)$ denote the FFT and the calculation of all amplitude values, and $\mathbf{A}$ represents the amplitude of each frequency. It is known in FFT that the frequency with larger amplitudes represents the more dominant period (Zhou et al., ).
Thus, we calculate the frequency-based similarity between the historical univariate sub-sequences before the current time $t$ to select the most dominant periods for each variate. We use subscript $i$ and $j$ to represent different historical univariate sub-sequences before the current time $t$, e.g. $\mathbf{A}_{i}=\mathrm{Amp}\left(\mathrm{FFT}({\rm\bf x}_{t-h:t})\right)$ and $\mathbf{A}_{j}=\mathrm{Amp}\left(\mathrm{FFT}({\rm\bf x}_{t-h-1:t-h-1})\right)$, and their frequency-based similarity is:

$$ $S_{i,j}=\frac{\mathbf{A}_{i}\cdot\mathbf{A}_{j}}{\|\mathbf{A}_{i}\|_{2}\| \mathbf{A}_{j}\|_{2}}$ (2) $$

Then we select the top-$k$ frequencies from the highest similarities:

$$ $\displaystyle i_{max},j_{max}=\operatorname*{arg\,max}_{i,j}\big{(}S_{i,j}\big {)},\ \ {\rm\bf f}=\text{arg Top-}k\ \big{(}\mathbf{A}_{i_{max}},\mathbf{A}_{j _{max}}\big{)}.$ (3) $$

Specifically, ${\rm\bf f}=\{f_{1},\cdots,f_{k}\}\in\mathbb{R}^{k\times 1}$ is the top-$k$ dominant frequencies occurred most before the current time. The top-$k$ frequencies correspond to $k$ dominant period lengths ${\rm\bf r}=\{r_{1},\cdots,r_{k}\}\in\mathbb{R}^{k\times 1}$ where $r_{k}=\frac{1}{f_{k}}$. The dominant periods are used to estimate the reaction time.

The normal data can be reconstructed well and the fluctuations are hard to reconstruct (You et al., ; Campos et al., ). Thus,
we focus on the reconstruction errors mainly during the reaction time to assess the magnitude of the fluctuations for anomaly prediction. We mask each univariate sequence ${\rm\bf x}$ with adaptive mask length $r$, and $r$ is randomly sampled from ${\rm\bf r}$:

$$ ${\rm\bf x}_{m}={\rm\bf M}\odot{\rm\bf x}$ (4) $$

where $\|{\rm\bf M}\|_{1}=r$ and ${\rm\bf M}=\{0,0,\cdots,1,\cdots,1\}\in[0,1]^{T}$ denotes the mask matrix near the current time with adaptive mask length, and ${\odot}$ indicates element-wise multiplication. Thus, we obtain the masked time series sequence ${\rm\bf x}_{m}$ where each variate has varying mask lengths to estimate the reaction time.

Multi-scale Patching.
Take the masked sequence ${\rm\bf x}_{m}$ of the univariate sequence ${\rm\bf x}$ as an example. We segment ${\rm\bf x}_{m}$ into patches with multi-scale to enhance hierarchical information. Concretely, we obtain $a$ kinds of patch-based sequences from fine to coarse granularity upon ${\rm\bf x}_{m}$, where $a$ represents the number of granularities.
The multi-scale scaling process first generates a patch sequence ${\rm\bf x}_{1}\in\mathbb{R}^{N_{1}\times P_{1}}$, where ${P_{1}}$ is the finest granularity patch size and ${N_{1}}$ is the number of patches. For the ${i}$-th scale, $1<i\leq a$, we concatenate two adjacent patches from the $(i-1)$-th scale, and obtain sequence of the ${i}$-th scale containing $N_{i}$ patches with patch size ${P_{i}}$. By continuously grouping, we obtain $a$ sequences $\{{\rm\bf x}_{m,p}\}_{p=1}^{a}$ with different scales. Similarly, we obtain $\{{\rm\bf x}_{p}\}_{p=1}^{a}$ from the unmasked sequence, used for the contrastive branch.

### 3.3 Masked Time Series Reconstruction

To accurately assess the magnitude of fluctuations during the reaction time, we perform reconstruction on the dominant periods masked time series. This module is composed of asymmetric encoder-decoder architecture (as in the pink background section in Figure 2). The scale encoder is implemented based on the temporal transformer, and the output yields representations of positive sample pairs ${\rm\bf z}_{a}^{+}\in\mathbb{R}^{N_{a}\times d_{\mathrm{model}}}$, where $d_{model}$ denotes the hidden state dimensions. The decoder reconstructs inputs of different scales using lightweight MLP to fuse multi-scale information, and outputs formatted as $\mathbf{\hat{x}}\in\mathbb{R}^{T\times 1}$.

Scale Encoder.
There are $a$ encoder blocks, corresponding to patch sequences $\{{\rm\bf x}_{m,p}\}_{p=1}^{a}$ at different scales passed through. The scale encoder is implemented based on the Transformer block. First, the embedded representation ${\rm\bf x}_{m,e}$ is obtained via the embedding layer from each ${\rm\bf x}_{m,p}$. Then, ${\rm\bf x}_{m,e}$ is fed into multi-head self-attention layers, and the representations of different patches with different scales are units to learn temporal features across different time intervals. The scale encoder provides features $\{{\rm\bf z}_{p}^{+}\}_{p=1}^{a}$ for both reconstruction and contrastive learning.

Reconstruction Learning.
We concatenate the features $\{{\rm\bf z}_{p}^{+}\}_{p=1}^{a}$ and use the MLP-based decoder to learn cross-scale information for reconstruction. The reconstruction result can be obtained by:

$$ $\mathbf{\hat{x}}=\text{Decoder}(\{{\rm\bf z}_{p}^{+}\}_{p=1}^{a})$ (5) $$

MultiRC minimizes the mean squared error (MSE) loss. The losses across all channels are averaged to get the overall reconstruction loss $\mathcal{L}_{\text{Rec}}$:

$$ $\mathcal{L}_{\mathrm{Rec}}=\frac{1}{c}\sum_{i=1}^{c}\|\mathbf{{x}}-\mathbf{ \hat{x}}\|_{2}^{2}$ (6) $$

### 3.4 Generative-based contrastive learning

We introduce controlled generative strategies to construct diverse precursors as negative samples to prevent model degradation and better distinguish fluctuations. Multi-scale views from the masked time series reconstruction module are used as positive samples, while hard negative samples are generated through controlled noise pollution. The generative design of negative samples targets the need to avoid degradation and better judge fluctuations.

Negative sample generation.
For multi-scale patch sequences ${\rm\bf x}_{p}$, we apply different noise pollution strategies to generate negative samples ${\rm\bf x}_{n,p}$. Different types of fluctuations are generated and the details are shown in Appendix D. The embedded representation ${\rm\bf x}_{n,e}$ is obtained via the embedding layer from the generated negative samples ${\rm\bf x}_{n,p}$. The encoder backbone is shared with the masked time series reconstruction module. Specifically, the generated data is fed into the backbone network to derive the hard negative sample representations ${\rm\bf z}_{p}^{-}\in\mathbb{R}^{N_{p}\times d_{\mathrm{model}}}$ where $1\leq p\leq a$.

Figure: (a) Anomaly detection loss function
Refer to caption: extracted/5942937/figures/ADloss1.png

Representation discrepancy.
In anomaly prediction, we design an interval-wise contrastive loss function (Figure 3b). Three views are used as an example. Apply mean pooling to the various views generated by the scale encoder to obtain interval-level representations for each channel. Use the interval representations from the same channel $i$ as positive pairs ${\rm\bf z}^{pre}_{1[i]}$, ${\rm\bf z}^{pre}_{2[i]}$, ${\rm\bf z}^{pre}_{3[i]}$. Similarly, use three views as an example. The hard negative samples ${\rm\bf z}^{pre}_{neg1}$, ${\rm\bf z}^{pre}_{neg2}$, ${\rm\bf z}^{pre}_{neg3}$ and the representations from other channels are taken as negative pairs. The loss calculation $\mathcal{L}_{\mathrm{con}}^{inter}$ defined as:

$$ $\mathcal{L}_{\mathrm{con}}^{inter}({\rm\bf z}_{1}^{pre})=-\frac{1}{c}\sum_{i=1 }^{c}\mathit{log}\left(\frac{\exp({\rm\bf z}_{1[i]}^{pre}\cdot{\rm\bf z}_{2[i] }^{pre})+\exp({\rm\bf z}_{1[i]}^{pre}\cdot{\rm\bf z}_{3[i]}^{pre})}{ \displaystyle\sum_{i\neq j}\sum_{k=1}^{3}\exp({\rm\bf z}_{1[i]}^{pre}\cdot{\rm \bf z}_{k[j]}^{pre})+\sum_{i}\exp({\rm\bf z}_{1[i]}^{pre}\cdot{\rm\bf z}_{neg1 }^{pre})}\right)$ (7) $$

where $\exp({\rm\bf z}^{pre}_{1[i]}\cdot{\rm\bf z}^{pre}_{2[i]})$ represents the inner product of the interval representations of ${\rm\bf z}^{pre}_{1}$ and ${\rm\bf z}^{pre}_{2}$. The overall contrastive objectives defined as follows, $H$ is the total number of different views:

$$ $\mathcal{L}_{Con}=\frac{1}{H}\sum_{i=1}^{H}\mathcal{L}_{\mathrm{con}}^{inter}( {\rm\bf z}^{pre}_{i})$ (8) $$

For the anomaly detection task, we design a point-wise contrastive loss function (Figure 3a). Before calculating the loss, the outputs of encoders remain the dependence between each time point through additional upsampling:

$$ ${\rm\bf z}^{det}=\mathit{Upsampling}({\rm\bf z}_{\xi(i,l_{\mathrm{orig}},l_{ \mathrm{new}})}),\ \ \ \xi(i,l_{\mathrm{orig}},l_{\mathrm{new}})=\left\lfloor \frac{i\times l_{\mathrm{orig}}}{l_{\mathrm{new}}}\right\rfloor$ (9) $$

where $i\in\{0,1,\cdots,l_{\mathrm{new}}-1\}$ is the time index after upsampling. $\left\lfloor.\right\rfloor$ is the floor function. $z$ stands for ${\rm\bf z}_{p}^{+}$ or ${\rm\bf z}_{p}^{-}$. $l_{\mathrm{orig}}$ is the original length, that is, the number of time points in each patch. $l_{\mathrm{new}}$ is the target length, which is the size of the window size.

Representations of different views at the same time point $i$ are treated as positive pairs ${\rm\bf z}^{det}_{1[i]}$, ${\rm\bf z}^{det}_{2[i]}$, ${\rm\bf z}^{det}_{3[i]}$. Negative pairs include: representations between different time points within the same view, the representation of any time point in ${\rm\bf z}^{det}_{1}$ compared with all time points in ${\rm\bf z}^{det}_{2}$ and ${\rm\bf z}^{det}_{3}$ except that point, and the representation of any time point in the hard negative sample ${\rm\bf z}^{det}_{neg1}$, ${\rm\bf z}^{det}_{neg2}$, ${\rm\bf z}^{det}_{neg3}$. The loss calculation $\mathcal{L}_{\mathrm{con}}^{point}$ defined as:

$$ $\mathcal{L}_{\mathrm{con}}^{point}({\rm\bf z}^{det}_{1})=-\frac{1}{T}\sum_{i=1 }^{T}\mathit{log}\left(\frac{\exp({\rm\bf z}^{det}_{1[i]}\cdot{\rm\bf z}^{det} _{2[i]})+\exp({\rm\bf z}^{det}_{1[i]}\cdot{\rm\bf z}^{det}_{3[i]})}{ \displaystyle\sum_{u\neq i}\sum_{k=1}^{3}\exp({\rm\bf z}^{det}_{1[i]}\cdot{\rm \bf z}^{det}_{k[u]})+\sum_{i}\exp({\rm\bf z}^{det}_{1[i]}\cdot{\rm\bf z}^{det} _{neg1})}\right)$ (10) $$

In summary, the various designs of the contrastive loss enable the same fundamental algorithm to be flexibly applied to different task requirements, enhancing the practicality of the model.

### 3.5 Joint optimization

As previously mentioned, reconstruction and contrastive learning are interconnected. Reconstruction loss focuses on extracting key features from locally masked time series data. Contrastive loss effectively learn overall trends and patterns over time intervals. Let $\lambda$ be the weights that balance loss terms, the overall loss function is given:

$$ $\mathcal{L}=\lambda_{\mathrm{Con}}\mathcal{L}_{\mathrm{Con}}+\lambda_{\mathrm{ Re}c}\mathcal{L}_{\mathrm{Re}c}$ (11) $$

### 3.6 Model inference

During the inference phase, labeled negative sampled construction is not needed. The anomaly score is composed of the MSE between the input and the reconstructed output, as well as the representational distance between positive sample pairs, which can be determined using metrics such as Euclidean distance. The final anomaly score $f(x)$ is as follows:

$$ $f(x)=\mathit{MSE}(\mathbf{\hat{X}},\mathbf{{X}})+\mathit{Dist}({\rm\bf z}_{p}^ {+},{\rm\bf z}_{j}^{+})_{p\neq j,p,j=1,..,a}$ (12) $$

which is a point-wise anomaly score. With a threshold (Wang et al., ), we can determine whether a point is abnormal. For anomaly prediction, the probability score ${\rm\bf\hat{p}}_{t}$ is the averaged anomaly score from the look-back window. Apply a threshold (Yang et al., ), we can convert ${\rm\bf\hat{p}}_{t}$ into a binary labels, if ${\rm\bf\hat{p}}_{t}\geq\mu$, the future sub-sequence being anomaly.

## 4 Experiments

### 4.1 Experimental settings

Datasets.
We evaluated MultiRC on seven real-world datasets: (1) MSL (Mars Science Laboratory rover) (Hundman et al., ) includes operational data from multiple sensors on the Mars rover. (2) SMAP (Soil Moisture Active Passive satellite) (Hundman et al., ) provides soil moisture information collected from satellite sensors. (3) SMD (Server Machine Dataset) (Su et al., ) is a large-scale dataset collected over five weeks from a large Internet company. (4) PSM (Pooled Server Metrics) (Abdulaal et al., ) comprises data collected from eBay’s application server nodes. (5) SWaT (Secure Water Treatment) (Li et al., ) is a dataset for security research in water treatment systems. (6) NIPS-TS-SWAN (Lai et al., ) is a comprehensive multivariate time series benchmark extracted from solar photospheric vector magnetograms. (7) NIPS-TS-GECCO (Rehbach et al., ) covers data collected from multiple sensors in a drinking water supply system. The training and validation data were split in an 8:2 ratio. Additional details on the datasets can be found in Appendix A.1.

Baselines.
We compare our method with PAD (Jhin et al., ). We also modify unsupervised anomaly detection methods into anomaly prediction methods following PAD, including the reconstruction-based methods: CAE-Ensemble (Campos et al., ), GANomaly (Du et al., ), OmniAnomaly (Su et al., ), Anomaly Transformer (A.T.) (Xu et al., ), $\text{D}^{3}\text{R}$ (Wang et al., ); the contrastive learning methods: DCdetector (Yang et al., ), PAD (Jhin et al., ); the classic methods: IForest (Liu et al., ); the density-estimation method: DAGMM (Zong et al., ); the clustering-based method: Deep SVDD (Ruff et al., ). We have used all baselines with their official or open-source versions. Additional details on the baselines can be found in Appendix A.2.

Evaluation criteria.
We use the metrics Precision(P), Recall(R), and F1-score(F1) for comprehensive comparison (Su et al., ). In anomaly prediction tasks, we predict future window anomalies (Jhin et al., ), for which we use classic metrics. With respect to anomaly detection tasks, we detect point anomaly as previous works (Wang et al., ). As pointed out in (Kim et al., ), the point adjustment strategy commonly used in previous works (Song et al., ; Yang et al., ; Zhao et al., ; Xu et al., ) is unreasonable. This strategy assumes that if one anomaly is correctly detected within a continuous anomalous segment, then all points in that segment are considered correctly detected. The recently proposed affiliation-based P/R/F1 (Huet et al., ; Wang et al., ) provides a reasonable evaluation for anomaly detection in time series, so we employ this for comparison.

### 4.2 Main results

Tables 1 and Table 2 respectively present the performance comparison of the model in anomaly prediction and detection tasks. As indicated by the tables, on all five real-world datasets, MultiRC surpasses the adversary algorithms, achieving the best F1 and Aff-F1 performance, thereby confirming its effectiveness. Performance comparisons on the NIPS-TS-SWAN and NIPS-TS-GECCO datasets are presented in Appendices F and G. The visualization results of anomaly prediction are presented in Appendix E.

**Table 1: Anomaly prediction results for the five real-world datasets. Superior performance is indicated by higher metric values, with the top F1 scores emphasized in bold.**
|  | MSL | SMAP | SMD | PSM | SWaT | Average |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Method | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 | F1 |
| DAGMM | 13.94 | 17.04 | 15.33 | 11.91 | 22.00 | 15.51 | 14.74 | 15.37 | 15.05 | 39.27 | 36.14 | 37.64 | 82.14 | 61.72 | 70.48 | 30.80 |
| Iforst | 13.82 | 17.11 | 15.29 | 11.79 | 18.67 | 14.45 | 15.25 | 18.28 | 16.63 | 39.39 | 38.82 | 39.10 | 75.39 | 62.22 | 68.17 | 30.72 |
| Deep SVDD | 12.94 | 16.04 | 14.32 | 11.77 | 17.09 | 13.94 | 14.99 | 15.78 | 15.37 | 36.63 | 36.30 | 36.46 | 81.29 | 61.95 | 70.31 | 30.08 |
| A.T. | 14.62 | 12.42 | 13.43 | 11.40 | 22.80 | 15.20 | 10.52 | 19.42 | 13.65 | 30.56 | 34.64 | 32.47 | 75.00 | 60.50 | 66.97 | 28.34 |
| DCdetector | 2.35 | 3.46 | 2.80 | 8.04 | 11.30 | 9.40 | 3.71 | 9.03 | 5.26 | 28.97 | 12.05 | 17.02 | 46.45 | 36.36 | 40.79 | 15.05 |
| PAD | 14.37 | 13.22 | 13.77 | 15.19 | 34.41 | 21.08 | 11.23 | 18.76 | 14.05 | 31.23 | 33.05 | 32.11 | 74.83 | 59.09 | 55.04 | 27.21 |
| Omni | 13.66 | 21.66 | 16.75 | 11.90 | 23.39 | 15.77 | 16.22 | 18.19 | 17.15 | 39.45 | 40.88 | 40.15 | 85.73 | 58.57 | 69.59 | 31.88 |
| GANomaly | 15.36 | 17.30 | 16.27 | 12.18 | 23.41 | 16.02 | 15.06 | 21.39 | 17.68 | 37.02 | 43.06 | 39.81 | 83.99 | 59.77 | 69.84 | 31.92 |
| CAE-Ensemble | 20.35 | 18.27 | 19.26 | 15.88 | 26.68 | 19.91 | 18.41 | 19.51 | 18.94 | 40.18 | 41.99 | 41.07 | 88.61 | 59.90 | 71.48 | 34.13 |
| $\text{D}^{3}\text{R}$ | 19.99 | 20.32 | 20.15 | 15.73 | 26.89 | 19.85 | 15.78 | 19.33 | 17.38 | 40.73 | 42.21 | 41.46 | 84.13 | 61.85 | 71.29 | 34.02 |
| MultiRC | 14.74 | 69.84 | 24.34 | 16.59 | 47.11 | 24.53 | 19.99 | 23.27 | 21.51 | 36.77 | 65.30 | 47.05 | 99.38 | 60.29 | 75.05 | 38.49 |

#### Anomaly prediction.

MultiRC has achieved improvements of 4.19% (from 20.15 to 24.34), 4.68% (from 19.85 to 24.53), 2.57% (from 18.94 to 21.51), 5.59% (from 41.46 to 47.05), and 3.57% (from 71.48 to 75.05) on the MSL, SMAP, SMD, PSM and SWaT datasets, respectively.

Traditional machine learning methods, such as IForest, often underperform in generalization because they do not take into account the continuity and complexity of time series data. Despite being based on contrastive learning, the performance of DCdetector is not satisfactory. This is because it focuses only on positive samples, resulting in severely limited performance in anomaly prediction tasks that require label support from negative samples. Compared to the previous state-of-the-art model, $\text{D}^{3}\text{R}$, MultiRC significantly improves the F1 scores on benchmark datasets. In our model, multi-scale mask reconstruction and multi-noise contrastive paradigm are combined. MultiRC takes into account the periodic variations in time series and further enhances hierarchical information through a multi-scale structure, which facilitates the identification of different reaction time. The encoder outputs are simultaneously used for both reconstruction and contrastive tasks, making the learned feature representations more meaningful. These designs address the shortcomings of previous work and maintain outstanding performance across different datasets. Furthermore, the performance on the PSM and SWaT datasets is significantly better than on other datasets, likely because anomaly prediction requires the model to detect precursor signals. In some datasets, these precursors are less apparent, thereby limiting the model performance.

**Table 2: Experimental results for the anomaly detection on five time-series datasets. The best Aff-F1 scores are highlighted in bold.**
|  | MSL | SMAP | SMD | PSM | SWaT | Average |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Method | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-F1 |
| DAGMM | 4.07 | 92.11 | 68.14 | 50.75 | 96.38 | 66.49 | 63.57 | 70.83 | 67.00 | 68.22 | 70.50 | 69.34 | 59.42 | 92.36 | 72.32 | 68.65 |
| IForest | 53.87 | 94.58 | 68.65 | 41.12 | 68.91 | 51.51 | 71.94 | 94.27 | 81.61 | 69.66 | 88.79 | 78.07 | 53.03 | 99.95 | 69.30 | 69.82 |
| Deep SVDD | 49.88 | 98.87 | 65.73 | 42.67 | 68.23 | 51.94 | 65.84 | 80.43 | 72.58 | 58.32 | 60.11 | 59.95 | 55.73 | 97.34 | 70.77 | 64.19 |
| A.T. | 51.04 | 95.36 | 66.49 | 56.91 | 96.69 | 71.65 | 54.08 | 97.07 | 66.42 | 54.26 | 82.18 | 65.37 | 53.63 | 98.27 | 69.39 | 67.86 |
| DCdetector | 55.94 | 95.53 | 70.56 | 53.12 | 98.37 | 68.99 | 50.93 | 95.57 | 66.45 | 54.72 | 86.36 | 66.99 | 53.25 | 98.12 | 69.03 | 68.40 |
| PAD | 56.33 | 82.21 | 68.15 | 41.67 | 64.52 | 53.94 | 59.54 | 67.66 | 63.71 | 68.45 | 57.72 | 59.21 | 54.73 | 92.35 | 68.06 | 62.61 |
| Omni | 51.23 | 99.40 | 67.61 | 52.74 | 98.51 | 68.70 | 79.09 | 75.77 | 77.40 | 69.20 | 80.79 | 74.55 | 62.76 | 82.82 | 71.41 | 71.93 |
| GANomaly | 56.36 | 98.27 | 68.01 | 56.44 | 97.62 | 72.52 | 73.46 | 83.25 | 74.20 | 55.31 | 98.34 | 75.11 | 59.93 | 81.52 | 70.24 | 72.01 |
| CAE-Ensemble | 54.99 | 93.93 | 69.37 | 62.32 | 64.72 | 63.50 | 73.05 | 83.61 | 77.97 | 73.17 | 73.66 | 73.42 | 62.10 | 82.90 | 71.01 | 71.05 |
| $\text{D}^{3}\text{R}$ | 66.85 | 90.83 | 77.02 | 61.76 | 92.55 | 74.09 | 64.87 | 97.93 | 78.02 | 73.32 | 88.71 | 80.29 | 60.14 | 97.57 | 74.39 | 76.76 |
| MultiRC | 67.94 | 93.03 | 78.53 | 66.66 | 89.94 | 76.57 | 75.36 | 94.85 | 83.99 | 73.36 | 93.13 | 82.08 | 62.52 | 97.26 | 76.12 | 79.45 |

Anomaly detection.
As can be seen from the Table 2, MultiRC also achieves optimal performance in the exception detection tasks. This is 1.51%-5.97% higher on average than previous SOTA methods. Additionally, many previous studies were evaluated through point adjustment, leading to a false boom. Metrics based on affiliation offer a more objective and reasonable assessment for various methods, resulting in lower scores for past approaches. Overall, MultiRC not only effectively detects anomalies within complex data but also assists in predicting future anomalies.

### 4.3 Ablation studies

In order to verify the effectiveness and necessity of our designs, we conduct ablation studies focusing on key components of our model design: the multi-scale structure, the masked time series reconstruction and the generative-based contrastive learning.

**Table 3: Results of anomaly prediction ablation studies. The best scores are highlighted in bold.**
|  | MultiRC | w/o<br>multi-scale | w/o<br>adaptive mask | w/o<br>reconstruction | w/o<br>contrastive | w/o<br>generation |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 |
| MSL | 14.74 | 69.84 | 24.34 | 11.50 | 53.89 | 18.96 | 14.72 | 58.32 | 23.51 | 7.69 | 34.19 | 12.56 | 11.62 | 51.85 | 18.99 | 12.62 | 55.52 | 20.56 |
| SMAP | 16.59 | 47.11 | 24.53 | 15.48 | 31.76 | 20.82 | 14.94 | 39.95 | 21.75 | 14.94 | 26.92 | 19.22 | 16.63 | 40.41 | 23.56 | 16.37 | 40.05 | 23.24 |
| PSM | 36.77 | 65.30 | 47.05 | 42.39 | 50.27 | 46.00 | 35.84 | 62.00 | 45.42 | 28.54 | 35.60 | 31.68 | 36.33 | 65.17 | 46.65 | 37.03 | 63.79 | 46.86 |

**Table 4: Ablation results of anomaly detection. The best scores are highlighted in bold.**
|  | MultiRC | w/o<br>multi-scale | w/o<br>adaptive mask | w/o<br>reconstruction | w/o<br>contrastive | w/o<br>generation |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 | Aff-P | Aff-R | Aff-F1 |
| MSL | 67.94 | 93.03 | 78.53 | 65.87 | 93.57 | 77.31 | 67.48 | 90.32 | 77.25 | 44.65 | 44.09 | 44.37 | 67.01 | 93.08 | 77.92 | 67.37 | 93.09 | 78.17 |
| SMAP | 66.66 | 89.94 | 76.57 | 64.28 | 90.34 | 75.11 | 66.41 | 88.31 | 75.81 | 53.74 | 76.61 | 63.17 | 65.39 | 90.07 | 75.77 | 65.65 | 86.16 | 74.52 |
| PSM | 73.36 | 93.13 | 82.08 | 76.15 | 80.59 | 78.31 | 72.29 | 90.38 | 80.33 | 52.94 | 69.90 | 60.25 | 73.22 | 92.43 | 81.71 | 73.53 | 92.74 | 82.03 |

In Table 3, we introduce our ablation experimental results for the anomaly prediction. We attempted to remove the multi-scale structure (w/o multi-scale), using only a single-scale sequence. Replace the adaptively adjusted mask scale with a fully random mask (w/o adaptive mask). We also remove the masked reconstruction module (w/o reconstruction), the contrastive learning module (w/o contrastive), and the noise generation variants (w/o generated samples).

The results show that performance decreased across all datasets after removing the multi-scale framework. Notably, some datasets experienced a significant drop in performance, which is due to the different levels of dependence on the multi-scale framework caused by varying response time lengths. The random mask led to a marked decrease in performance, indicating that adjusting the mask scale based on frequency and periodicity is beneficial for identifying different response times. Removing the masked reconstruction module or contrastive learning leads to the lack of ability to effectively identify fluctuations and magnitude, thus not maintaining the best performance on all datasets. The sample generation strategy has led to significant improvements (3.78%, from 20.56 to 24.34). This emphasizes the importance of hard negative samples in enhancing the ability to avoid degradation.

In the ablation studies for anomaly detection, we define the ablation model in exactly the same manner, results are shown in Table 4. This further substantiates the importance of each component to the overall efficacy of the model.

### 4.4 Discussion and analysis

Analysis of reaction time.
For reaction time, we conducted an analysis of window size performance on three datasets, as shown in Figure 4. This study follows the methodologies mentioned in the literature (Schmidl et al., ; Wenig et al., ), adjusting the sliding window size. The results indicate that PSM achieves optimal performance within a small window size (16), suggesting a shorter reaction time and significant fluctuations within this dataset. In contrast, MSL and SMAP are evident over a longer time span (64), indicating a relatively long reaction time. This disparity underscores the necessity of adopting the Multi-scale structure to accommodate varying lengths of reaction time needs.

Figure: Figure 4: Performance of different datasets under different window sizes.
Refer to caption: x4.png

Hyperparameter analysis.
The hyperparameters that might influence the performance of MultiRC include the hidden state dimension, batch size, and patch size. To analyze their impact on the results, we conducted a hyperparameter sensitivity analysis on the MSL, SMAP and PSM datasets. The findings are presented in Figure 5. This figure primarily depicts the results for anomaly prediction tasks. For the hyperparameter results of anomaly detection, see the Appendix G. Figure 5a demonstrates the performance across different sizes of the latent space, as the performance of many deep neural networks is affected by $d_{model}$. Figure 5b displays the outcomes for MultiRC when trained with various batch sizes. Figure 5c displays the model performance at different patch sizes. The experimental results show that the performance of the model remains relatively stable in different patch size combinations. A more detailed comparison and analysis regarding multi-scale patch size can be found in Appendix C.

Figure: (a) d_model
Refer to caption: x5.png

## 5 Conclusion

We propose MultiRC, a multi-scale reconstructive contrast for time series anomaly prediction and detection tasks. Our multi-scale structure enables the model to adapt to precursor signals of varying reaction times. In addition, sample generation is used to construct hard negative samples to prevent model degradation, where our model learns more meaningful representations to better identify fluctuations with contrastive learning and evaluate the amplitude of fluctuations with reconstruction learning. Experimental results demonstrate that MultiRC surpasses existing works in both anomaly prediction and detection tasks. Our framework breaks through the limitations of traditional anomaly detection, enhancing the capability to predict future anomalies.

## Reproducibility Statement

In this work, we have made every effort to ensure reproducibility. Reproducing the results would be straightforward and require minimal additional effort, ensuring high reproducibility.
In the anonymous repository link we provide, you will find the code and evaluation datasets, which are well-documented and easily accessible. Within the code files, we provide instructions to facilitate running and reproducing our experimental results.
In Methodology 3, from input to output, we provide a detailed description of our method, including model structure, model training, and anomaly criterion.
In Appendix A.3, we summarize all the default hyper-parameters, including noise ratio, learning rate, etc.

## Appendix A Training details

### A.1 Datasets

In our study, we represent the number of samples in the training set, validation set, and test set with the labels ”Train”, ”Valid”, and ”Test”, respectively. The ’Dim’ column indicates the dimension size of the data for each dataset. Additionally, the ’AR’ (anomaly ratio) column denotes the proportion of anomalies within the entire dataset.

**Table 5: Details of original datasets.**
| Datasets | Domain | Train | Valid | Test | Dim | AR (%) |
| --- | --- | --- | --- | --- | --- | --- |
| MSL | Spacecraft | 46,653 | 11,664 | 73,729 | 55 | 10.5 |
| SMAP | Spacecraft | 108,146 | 27,037 | 427,617 | 25 | 12.8 |
| SWaT | Water treatment | 396,000 | 99,000 | 449,919 | 51 | 5.78 |
| NIPS-TS-SWAN | Space Weather | 48,000 | 12,000 | 60,000 | 38 | 23.8 |
| NIPS-TS-GECCO | Water treatment | 55,408 | 13,852 | 69,261 | 9 | 1.25 |
| SMD | Server Machine | 566,724 | 141,681 | 708,420 | 38 | 4.2 |
| PSM | Server Machine | 105,984 | 26,497 | 87,841 | 25 | 27.8 |

### A.2 Baselines

- •
CAE-Ensemble (Campos et al., ) proposed CNN-based autoencoder and diversity-driven ensemble learning method to improve the accuracy and efficiency of anomaly detection.
- •
GANomaly (Du et al., ) adopts the network structure of Encoder1-Decoder-Encoder2, and introduces the idea of adversary-training to provide unsupervised learning scheme for anomaly detection.
- •
OmniAnomaly (Su et al., ) further extends the LSTM-VAE model to capture temporal dependencies in the context of random variates, using reconstruction probabilities for detection.
- •
Anomaly Transformer(Xu et al., ) focuses on the relationship between adjacent points and designs a minimax strategy to amplify normal-anomaly resoluteness of association differences.
- •
$\text{D}^{3}\text{R}$(Wang et al., ) performs unsupervised anomaly detection on unstable data by decomposing it into stable and trend components, directly reconstructing the data after it has been corrupted by noise.
- •
DCdetector (Yang et al., ) proposes a contrastive learning-based dual-branch attention structure without considering reconstruction errors. This structure is designed to learn a permutation invariant representation that enlarges the representation differences between normal points and anomalies.
- •
IForest (Liu et al., ) directly describes the degree of distance between points and regions to find abnormal points.
- •
PAD (Jhin et al., ) presents a neural controlled differential equation-based neural network to solve both anomaly detection and Precursor-of-Anomaly (PoA) detection tasks.
- •
DAGMM (Zong et al., ) is a deep automatic coding Gaussian mixture model comprises a compression network and an estimation network. Each network measures information necessary for the anomaly detection.
- •
Deep SVDD (Ruff et al., ) finds an optimal hypersphere in a feature space trained using a neural network. Normal data points are concentrated as much as possible within the hypersphere. Points that are far from the center of the sphere are considered anomalies.

### A.3 Implementation

In our experiments, we set the blocks of the Transformer to 3. We select different sliding window size options for different datasets. For the contrastive branch, we set the noise ratio to 50% for generating samples across different scale sequences. The prediction window size is uniformly set to 4 following previous works (Jhin et al., ; Yin et al., ). Early stopping with the patience of 3 epochs is employed using the validation loss. We use the Adam optimizer with a learning rate of 1e-5. All experiments in this work are conducted using Python 3.9 and PyTorch 1.13 (Paszke et al., ), and executed on CUDA 12.0, NVIDIA Tesla-A800 GPU hardware.

## Appendix B Comparison with mainstream methods

Figure: Figure 6: Architecture comparison of three approaches. ${\rm\bf{h}_{1}^{+}}$, ${\rm\bf{h}_{2}^{+}}$, ${\rm\bf{z}_{m}^{+}}$ are positive samples, ${\rm\bf z}_{n}^{-}$ is negative samples.
Refer to caption: x8.png

In time series anomaly detection, reconstruction-based methods(Figure 6) train models to accurately reconstruct normal samples, while those that cannot be accurately reconstructed are considered anomalies. This approach is popular because of its ability to combine with multiple machine learning or deep learning models to effectively capture complex dependencies in time series and provide intuitive explanations for anomaly detection. However, they usually do not pay attention to the trend of changes in the data over a period of time, resulting in insensitive recognition of abnormal precursor signals. In addition, it is often difficult to strike a balance between the complexity of the model and the ability to reconstruct it, too simple models can not capture the time dependence, and too complex models tend to lose the ability to distinguish abnormal data. This makes it more difficult to learn a model that is flexible enough to capture time series fluctuations while maintaining good reconstruction performance.

In recent years, contrastive representative learning has emerged in the field of anomaly detection, which can detect anomalies without the need for high-quality reconstruction models. The key idea is that normal data has strong correlation with other data, while abnormal data has weak correlation with each other. However, due to the lack of negative sample labels, the features learned by such methods will fall into a single pattern, resulting in the inability to recognize temporal fluctuations. MultiRC establishes a dual-branch with joint reconstructive and contrastive learning upon a multi-scale structure, adaptively recognizing varying reaction times for different variables through the adaptive dominant period mask. Additionally, hard negative samples are constructed to prevent model degradation. The rich representation learned by the model is conducive to capturing the fluctuations of the time series and the degree of fluctuations, ensuring the effect of the prediction.

## Appendix C Scales analysis

Multi-scale approaches have a unique impact on anomaly prediction problems, particularly concerning reaction times of varying lengths. The prediction and detection performances at different scales are shown in Figure 7 and Figure 8, respectively.

Figure: Figure 7: Anomaly prediction results at different scales (F1).
Refer to caption: x9.png

Figure: Figure 8: Anomaly detection results at different scales (Aff-F1).
Refer to caption: x10.png

First, a single scale is suitable for specific ranges of reaction times but has limited performance. Second, the dependency on different scales varies across datasets. Using the PSM dataset as an example, the performance decline is greater when using the 2,8 or 4,8 scale combinations compared to the 2,4 scale combination. This indicates that shorter time scales are more suitable for anomaly prediction in this dataset. Moreover, more scales are not necessarily better. When the gap between the smallest and largest scales becomes too large, it increases the difficulty of contrastive learning.

## Appendix D Sample generation

Sample generation methods include amplitude magnification or reduction by some factor (scale), reducing resolution (compress), mirroring on the mean value (horizontal axis) (hmirror), temporal displacement by a fixed constant (shift), adding Gaussian white noise (noise), and reversal on the time axis (vmirror). To better reflect real-world conditions, the noise magnitude is randomly selected. The abnormal prediction results under different generation methods are shown in the Figure 9.

Figure: Figure 9: F1 scores for six sample generation methods.
Refer to caption: x11.png

Taking scale as an example, by a given probability, each original data point is randomly magnified or reduced, causing positive and negative samples to vary in scale at different points. The data after noise addition is:

$$ ${\rm\bf x}_{n,p}={\rm\bf x}_{p}\cdot(\sigma_{p}\cdot s_{p}+1)$ (13) $$

where $s_{p}$ represents the random factor generated for scale ${p}$, and $\sigma_{p}$ is the coefficient controlling the noise intensity. The random factor is sampled from a standard normal distribution to ensure it is suitable for all patches at scale ${p}$:

$$ $s_{p}\sim\mathcal{N}(0,1)$ (14) $$

Between different scales, $s_{p}$ is sampled independently to ensure that each scale uses a distinct random factor. This design strategy enables the model to focus on common features at the same scale.

## Appendix E Visual analysis

We visualize the anomaly prediction of MultiRC on the PSM dataset in Figure 10. The model raises warnings by identifying anomaly precursors (pink regions) before the actual anomalies (red regions) occur. This shows that MultiRC is effective in predicting whether future anomalies will occur.

Figure: Figure 10: Visualization of the anomaly prediction on the PSM dataset. MultiRC can output larger scores to identify the anomaly precursors that predict the future time series are more likely to be abnormal.
Refer to caption: x12.png

**Table 6: Anomaly prediction results for the NIPS-TS datasets. All results are in %.**
| Method | Dataset | ACC | P | R | F1 | AUC |
| --- | --- | --- | --- | --- | --- | --- |
| NIPS-TS-SWAN | A.T. | 58.19 | 52.90 | 39.77 | 45.50 | 52.98 |
| DCdetector | 74.83 | 21.04 | 10.94 | 14.40 | 50.58 |  |
| CAE-Ensemble | 58.57 | 56.33 | 44.25 | 49.56 | 64.81 |  |
| $\text{D}^{3}\text{R}$ | 58.38 | 54.96 | 44.81 | 49.37 | 63.87 |  |
| MultiRC | 61.75 | 71.28 | 55.22 | 62.23 | 66.82 |  |
| NIPS-TS-GECCO | A.T. | 89.41 | 43.62 | 27.92 | 34.05 | 50.39 |
| DCdetector | 97.21 | 1.76 | 3.46 | 2.33 | 50.79 |  |
| CAE-Ensemble | 97.71 | 51.67 | 29.92 | 37.90 | 52.94 |  |
| $\text{D}^{3}\text{R}$ | 98.01 | 51.24 | 30.91 | 38.56 | 53.09 |  |
| MultiRC | 99.10 | 82.89 | 27.39 | 41.17 | 53.35 |  |

## Appendix F Additional prediction results

**Table 7: Experimental results for the anomaly detection on NIPS-TS datasets, are presented in percentages.**
| Method | Dataset | ACC | Aff-P | Aff-R | Aff-F1 | AUC |
| --- | --- | --- | --- | --- | --- | --- |
| NIPS-TS-SWAN | A.T. | 63.89 | 48.12 | 25.89 | 33.67 | 44.74 |
| DCdetector | 66.54 | 44.47 | 8.61 | 14.42 | 43.46 |  |
| $\text{D}^{3}\text{R}$ | 68.83 | 68.73 | 31.49 | 43.19 | 37.31 |  |
| MultiRC | 67.84 | 63.28 | 41.74 | 50.30 | 59.48 |  |
| NIPS-TS-GECCO | A.T. | 95.82 | 50.39 | 88.68 | 64.27 | 51.60 |
| DCdetector | 97.50 | 52.07 | 90.79 | 66.18 | 45.38 |  |
| $\text{D}^{3}\text{R}$ | 98.90 | 77.83 | 79.56 | 78.68 | 80.72 |  |
| MultiRC | 97.67 | 71.68 | 97.71 | 82.70 | 93.11 |  |

Figure: Figure 11: F1 score results for different prediction window sizes.
Refer to caption: x13.png

Figure: (a) d_model size
Refer to caption: x14.png

We further evaluate MultiRC performance on the NIPS-TS-SWAN and NIPS-TS-GECCO datasets (Table 6), which contain a broader variety of anomaly types. Even in these more complex anomaly environments, MultiRC consistently exhibits superior performance on most metrics when compared to well-performing baseline methods.

To analyze the impact of the prediction distance on the results, we conduct experiments with different prediction window sizes. The results are shown in Figure 11. As the prediction distance increases, the difficulty of prediction gradually rises, which directly leads to a decline in model performance. This phenomenon is in line with expectations, when the prediction window is smaller, meaning the prediction distance is shorter, the fluctuations in reaction time are more closely related to future sub-sequences. However, as the prediction window expands, the model needs to consider data changes and potential dynamic patterns over a longer period, requiring the model to have stronger generalization capabilities and the ability to capture long-term dependencies.

## Appendix G Additional detection results

Table 7 presents a performance comparison between MultiRC and other well-performing baseline methods on the NIPS-TS-SWAN and NIPS-TS-GECCO datasets. Despite the two datasets have the highest (23.68% in NIPS-TS-SWAN) and lowest (1.25% in NIPS-TS-GECCO) anomaly ratio, our model consistently demonstrates good performance across these challenging conditions. On the NIPS-TS-SWAN dataset, our model surpasses the best baseline model by 7.11% in the key metric Aff-F1 (from 43.19 to 50.30). On the NIPS-TS-GECCO dataset, it shows a more significant improvement of 4.02% (from 78.68 to 82.70).

We also study the parameter sensitivity of MultiRC in anomaly detection tasks. Figure 12a shows the performance at different latent space dimensions. Figure 12b also displays the model performance under different batch sizes. The model works well with larger batch sizes. In this study, the design of multiple patch sizes is a crucial element. For our primary evaluation, the patch sizes are typically set to combinations of 2, 4, 8. The results displayed in Figure 12c demonstrate that MultiRC exhibits stable performance across various patch size combinations.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. What’s the definition of anomaly in your work? Can you give some formal definition? Is it possible to define it a temporal pattern problem?
soundness: 2
strengths: 1. Good numerical results on anomaly detection tasks. 
2. Good writing, making the proposed method understandable from a reproducibility perspective.
summary: This paper proposed a multiscale approach called MultiRC for anomaly prediction and detection. MultiRC integrates reconstructive and contrastive learning and utilizes multi-scale structure and adaptive dominant period mask to deal with the diverse reaction time. It is evaluated on 7 benchmark datasets for both anomaly prediction and detection.
weaknesses: 1. My biggest concern about this work is the vague definition of “anomaly”. If we impose a more s, then more points/subsequence including the so-called reaction time can be classified as anomalies. The setting of the reaction time and the “real” anomaly makes it hard to understand the setting of the problem to be solved. 
2. I would recommend reformulating the problem as a specific pattern prediction problem.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. F1, Precision, recall, and AUC results for all the datasets

2. The differences between the precursor and the real anomaly? And the different types of precursors?
soundness: 3
strengths: 1. The problem (anomaly prediction) studied in this work is interesting.

2. The experiments were conducted on seven datasets.

3. A joint learning framework of anomaly prediction and detection is proposed.
summary: This paper introduces MultiRC, a framework for unsupervised time series anomaly prediction and detection that leverages a multi-scale structure and sample generation techniques. Experiments were conducted on seven datasets to evaluate the effectiveness of MultiRC.
weaknesses: 1.  Some important related work are missing, especially the work on anomaly precursor detection or early anomaly detection. 

 - Unsupervised Time Series Anomaly Prediction with Importance-based Generative Contrastive Learning, Kai Zhao et al.
 - dCNN/dCAM: anomaly precursors discovery in multivariate time series with deep convolutional neural networks, Paul Boniol et al.
 - Deep Multi-Instance Contrastive Learning with Dual Attention for Anomaly Precursor Detection, Dongkuan Xu et al.
 - Early anomaly detection in time series: a hierarchical approach for predicting critical health episodes, Vitor Cerqueira et al.

2. One Url provided by the authors could potentially be used to find authors’ identity.

From the readme of the code Url the authors provided (https://anonymous.4open.science/r/MultiRC-CCE6/README.md): all benchmarks are provided in their Google Cloud (https://drive.google.com/drive/folders/1RaIJQ8esoWuhyphhmMaH-VCDh-WIluRR). But from the Google Cloud link, both the owner/author's photo and username appears there, which might violate the anonymous Url rule. 

3. Writing or presentation issues:
- Some full names are not provided: e.g., Aff-F1, Aff-P, Aff-R.
- Figure 2 is too complex to understand and it is not self-contained. 
- Table 2: Best results are not highlighted. 

4. Experimental issues:
 - The results on all metrics including F1, Precision, recall, and AUC should be shown for all datasets.
 - More showcases with different types of precursors are needed to demonstrate the effectiveness of the proposed method.

5. Formal definition of precursor is needed.
 - The differences between the precursor and the real anomaly are not clear based on current definition.

---

confidence: 5
contribution: 2
initial_rating: 6
presentation: 2
questions: 1. Supporting Evidence for Multi-scale Claims: Can the authors provide examples or data to support their claim about the relationship between periodicity and the evolution rate of variables?
2. Detailed Process of Univariate Sub-sequences: Could the authors elaborate on the process of obtaining univariate sub-sequences and the motivation behind using similarity calculations for extracting dominant periodic components?
3.Verification of Experimental Data: Can the authors explain the discrepancy in the PAD method's F1 score on the SWaT dataset? Is there a possibility of an error in the reported data, and if so, could they provide the correct values?
4.Could the authors provide more detailed information on the specific operations for constructing anomaly samples, such as whether different construction operations were performed for different datasets: what kinds of construction methods were used, and how intense the construction was? Can this information be supplemented in the hyper parameters provided in the paper?
soundness: 2
strengths: The writing is clear and easy to understand.
The pictures and tables in the paper are beautiful and clear.
summary: The paper introduces MultiRC, a novel framework designed to address the challenges in both time series anomaly detection and prediction. MultiRC innovatively integrates reconstructive and contrastive learning approaches within a multi-scale structure to adapt to varying reaction times in time series data. The framework employs an adaptive dominant period mask to recognize different scales of reaction times and uses controlled generative strategies to create negative samples, which enhances the model's ability to learn meaningful representations and prevents degradation.
weaknesses: 1. Lack of Clarity in Anomaly Prediction Module: The paper does not distinctly outline the modules dedicated to anomaly prediction within the proposed framework. This lack of specificity could hinder readers from understanding how the framework directly addresses the anomaly prediction problem.

2. Inaccurate Representation of Prior Work: The description of the Anomaly Transformer seems to overlook its handling of "prior-association" and "series-association," which are crucial aspects of its approach. This misrepresentation could lead to an incomplete comparison with the existing literature. The paper mentions the DCdetector's architecture but does not elaborate on its multi-scale sampling structure and approach. This omission might affect the comprehensiveness of the discussion on related work.

3. Potential Error in Problem Definition: In the anomaly prediction task, the definition of the antecedent range as t-r: r seems to have an error. The starting index of the future look-forward window should be t instead of t+1, as t represents a time point, not a duration.

5. Vagueness in Multi-scale Structure Explanation: The paper states that variables with longer periodic changes are more likely to evolve gradually, but it does not provide clear evidence or examples to support this claim. This lack of clarity could confuse readers and detract from the paper's credibility.

6. Insufficient Detail on Univariate Sub-sequences: The process of obtaining univariate sub-sequences and the rationale behind using similarity calculations for extracting dominant periodic components require further explanation. The current exposition is ambiguous and may impede reader comprehension.

7. Suspicious Performance Data: The PAD method's reported F1 score on the SWaT dataset is lower than both its Precision and Recall values, which is counterintuitive. This discrepancy raises doubts about the accuracy of the experimental data presented in the paper.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: - Did you consider how the complexity of the model architecture affects its interpretability in real-world implementation? 
- Given the complexity of the dual-branch, multi-scale structure, did you encounter any specific challenges during model training? How feasible is it to train this model on larger datasets or in resource-constrained environments?
- Can you provide a discussion of the computational time and resource demands associated with the added components?
soundness: 4
strengths: 1. The paper proposes a novel approach combining anomaly prediction and detection within a joint learning framework, allowing for forecasting anomalies with variable reaction times. Such a perspective is very interesting and would provide insights for the community.
2. For the reconstruction part, the use of a multi-scale structure with an adaptive dominant period mask is innovative; for the contrastive learning part, the generative strategy for creating negative samples also seems novel.
3. The authors provide a well-structured codebase that is easy to navigate and highly accessible for reproduction and further experimentation. The experimental design is very thorough, with sensitivity, scale and ablation study.
summary: This paper introduces MultiRC, a novel approach for joint time-series anomaly prediction and detection. MultiRC combines reconstructive and contrastive learning within a multi-scale framework that adapts to varying reaction times for different time-series variables using an adaptive dominant period mask. The model includes a dual-branch architecture, leveraging asymmetric encoder-decoder networks to fuse cross-scale information. To enhance training momentum and prevent model degradation, MultiRC generates controlled negative samples, unlike traditional contrastive methods that focus solely on positive samples. Evaluated on seven benchmark datasets, MultiRC achieves state-of-the-art performance in both anomaly prediction and detection tasks.
weaknesses: - The paper does not explicitly address the computational time and resource demands introduced by additional components, such as the negative sample generation strategy. Including an analysis or discussion of these overheads would provide valuable insights.
-  The dual-branch, multi-scale structure with asymmetric encoder-decoder networks and generative strategies may be overly complex, which could lead to difficulties in model training, interpretability, and implementation. It would be helpful if the author touches upon this aspect.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 1
questions: Please see Weakness.
soundness: 2
strengths: N/A
summary: This manuscript focuses on anomaly prediction, which aims to predict whether an anomaly will occur in the future by capturing the fluctuations at the current time. The proposed model is based on contrastive learning and reconstruction loss at multiple resolutions. In the proposed contrastive framework, patches' representations from different resolutions are positive to each other. Negative samples are generated by different noise pollution strategies. Those generated samples are utilized to represent fluctuations for anomaly prediction during training. Finally, anomaly scores are based on the summation of reconstruction loss and the contrastive distances.
weaknesses: My main concern is the anomaly prediction task mentioned in this paper.

In my opinion, this problem is fundamentally consistent with the anomaly detection task itself.

For example, the yellow area in Figure 1 represents the early anomaly part. Therefore, when the anomaly labels are accurate (indicating the beginning of anomalies regardless of severity), the anomaly prediction task assumed in this paper should essentially belong to the standard anomaly detection task. In other words, the anomaly labels in Figure 1 are inaccurate. It should include the yellow area. 
Therefore, I do not think the anomaly prediction problem differs from anomaly detection tasks.

I know that several multivariate anomaly detection datasets used in this paper have inaccurate anomaly labels. The widely used UCR anomaly dataset contains many time series with accurately labeled anomalies. I suggest that the authors use this dataset for further research. Moreover, the paper lacks specific examples of dataset analysis. It would be better to provide more visualized results to support the research.

The paper's methodology is limited in terms of innovation. There have been research works on contrastive representation learning for anomaly detection. The investigation into related works is insufficient. Also, the motivation for using representations from different resolutions as positive samples is unclear. 

From the source code, window sizes seem to be too short (64), much less than the actual abnormal window size (>1000). 

The writing quality of this version of the paper is unsatisfactory due to many unexplained details. In particular, mathematical formulas and algorithm descriptions lack explanation or contain errors, resulting in some details of the model being unclear.

For examples: 
- The subscript range in x_{t-h-1:t-h-1} at line 216 is incorrect.
- It's unclear how patch x_p at line 244 maps multiple scales to different timesteps due to varying patch numbers at different granularities.
-  Formulas 6 lack formal definitions for c and i, with no relevant explanations in the context or formulas.
- It's unclear how x_{n,p} at line 293 is generated, and the definitions of n and p are ambiguous.
- The settings for lambda_{Con} and lambda_{Rec} in (11) are not clearly explained.
- The variables h and f at line 417 are not utilized in the model section, raising confusion. The specific differences between the anomaly prediction process during training/testing and anomaly detection tasks lack explanation.

Since the instance normalization technique is utilized in the proposed framework when some abnormal ts are segmented and shifted from the original range, will this normalization operation produce false negative results?

Computational costs: As you use channel independence, what are the computational costs? Will dealing with multivariate time series be much slower?