Integrative Tensor-based Anomaly Detection System For Satellites
Authors: Youjin Shin
Sangyup Lee
Shahroz Tariq
Myeong Shin Lee
OkchulJung
Daewon Chung
Simon Woo
Venue:  (2020)
Keywords: Tensor decomposition
Anomaly detection

Abstract: Detecting anomalies is of growing importance for various industrial applications and mission-critical infrastructures, including satellite systems. Although there have been several studies in detecting anomalies based on rule-based or machine learning-based approaches for satellite systems, a tensor-based decomposition method has not been extensively explored for anomaly detection. In this work, we introduce an Integrative Tensor-based Anomaly Detection (ITAD) framework to detect anomalies in a satellite system. Because of the high risk and cost, detecting anomalies in a satellite system is crucial. We construct 3rd-order tensors with telemetry data collected from Korea Multi-Purpose Satellite-2 (KOMPSAT-2) and calculate the anomaly score using one of the component matrices obtained by applying CANDECOMP/PARAFAC decomposition to detect anomalies. Our result shows that our tensor-based approach can be effective in achieving higher accuracy and reducing false positives in detecting anomalies as compared to other existing approaches.

Under review as a conference paper at ICLR 2020
INTEGRATIVE TENSOR-BASED ANOMALY DETECTION SYSTEM FOR SATELLITES
Anonymous authors Paper under double-blind review
ABSTRACT
Detecting anomalies of growing importance for various industrial applications and mission-crine cisiioncciiical infrastructures, including satellite systems. Although there have been several studies in detecting anomalies based rule-based or machine learning-basse -baeed approaches for satellite systems, a tensor-based decomposition method has not been extensively explored for anomaly detection. In this work, we introduce an Integrative Tensor-based Anomaly Detection (ITAD) framework to detect anomalies tecting anomalies in a satellite system is crucial. We construct 3rd-order tensors with telemetry data collected from Korea Multi-Purpose Satellite-2 (KOMPSAT 2) and calculate the anomaly score using of the component matrices obtained by applying CANDECOMPPAARFFA decomposition to detect anomalies. Our result shows that our tensor-based approach be effective in achieving higher accuracy and reducing false positives in detecting anomalies as compared to other existing approaches.
a satellite system. Because of the high risk and cost, de-
INTRODUCTION
Due to the high maintenance cost as well as extreme risk in space, detecting anomalies in a satel- lite system is critical. However, anomaly detection in satellite system challenging for several reasons. First, anomalies occur due to complex system interactions from various factors inside and outside satellite system. For example, sensor in one subsystem in satellite system often connected to several other types of sensors or resources in other subsystem modules. Each sensor measurement encapsulated as telemetry and downlinked to the ground station. order identify anomalies, it is crucial to compare and understand not just one single telemetry but several teleme- tries as a whole. However, most of the previous studies (Fuertes et al., 2016)] (Hundman et al. [018]] OMeara et al. 2016) on detecting anomalies in satellite systems have primarily focused analyz- ing individual telemetry. This can lead to a high false positives rate, because some instantaneous glitches may not be actual anomalies, but just trivial outliers (Yairi al., (017). Additionally, false positives can be costly, requiring much manual effort from operators to investigate and determine whether they are anomalies (Hundman et al. 2018). To reduce the false positives, analyzing set of multiple telemetries as a whole can be more effective to determine true anomalies in complex system. To the best of our knowledge, this integrated approach for satellite system has not been studied extensively in the past.
In order address these challenges, we propose an Integrative Tensor-based Anomaly Detection (ITAD) framework for a satellite system, where a tensor can effectively capture set of high dimen- sional data. Specifically, we construct 3rd-order tensor for entire telemetries in one subsystem and decompose it into component matrices, which captures the characteristics of multiple telemetries as a whole to detect anomalies. We then conduct cluster analysis on one component matrix in decomposed tensor and calculate the anomaly score based on the distance between each telemetry sample and its cluster centroid. Finally, we used the dynamic thresholding method (Hundman al., 2018) to detect anomalies; the dynamic thresholding method changes the detection threshold value over time instead of using a fixed value for the entire dataset. We performed experiments on our ap- proach with subset of real telemetries from the KOMPSAT-2 satellite, and verify that approach can detect actual anomalies effectively and reduce false positives significantly, compared to other approaches.

---

Under review as a conference paper at ICLR 2020
2 RELATED WORK
The fundamental concept of a tensor and the algorithm for tensor decomposition described in Appendix A In this section, we mainly provide an overview the research that is directly relevant to our work.
Tensor-based Anomaly Detection: Tensor decomposition for anomaly detection has been devel- oped by Nomikos and MacCiregor (1994). They proposed multiway principal components analysis (MPCA) using tensor decomposition to monitor the progress of the batch process in the multivariate trajectory data. They successfully extracted information from database of batches and captured the multivariate data into a few matrices, which can represent the original data sufficiently, while reducing the space. The modern application of censor-based anomaly detection has been deployed in many different areas such as neuroscience, environmental monitoring, video surveillance, network security, and remote sensing.
The tensor- based anomaly detection method can detect anomalies in an unsupervised manner, where the score plot-based model is the most widely used tensor decomposition method in anomaly de- tection without labels. Component matrices from tensor decomposition are utilized to calculate the score plots to detect the anomalies. According to the characteristics of the tensor dataset, the score
plot can be 1-dimensional (Kosanovich et al., 1994, Gauvin et al. 2014; Papalexakis et al. 2014), 2-dimensional (Cong et al.) [ Lee et al. 2014), or 3-dimensional (Mao et al.. [014]] Fanaee-1l and Gama 2014). In our work, we adopt score- score-based method to detect anomalies using one of the component matrices, which has the comprehensive characteristics of all telemetries simultaneously along the timeline.
Anomaly Detection for Satellite Operation: Generally, the Out-Of-Limmi (OOL) is one of the most widely used methods detect anomaly, where OOL can define nominal range with the lower and upper thresholds. So far, detecting anomalies for satellite systems have primarily used the OOL along with additional methods such as dimensionality reduction algorithm (Schölkopf al., 1998 Fujimaki et al., 2005 Inui et al., 2009), nearest neighbors (Breunig et al.) [2000] Bay and Schwabacher, 2003] Lverson 2008; Kriegel et al., 2009), and clustering algorithm (Iverson 2004
Gao et al. 201 Li et al.) (017).
Recently, machine learning-based approaches have been studied in order to automate the anomaly detection process. Centre National d' d'Etudes Spatials (CNES) proposed telemetry monitoring method, NOSTRADAMUS (Fuertes et al. 2016). which transforms the original telemetry dataset into vector matrix of features and reduces the dimensions using Principal Component Analysis (PCA). After that, One-class Support Vector Machine (OCSVM) is applied with decision frontier to detect anomalies. On the other hand, Yairi et al. (2017) proposed a data-driven health monitor- ing and anomaly detection method for the Japan Aerospace Exploration Agency (JAXA), based on probabilistic dimensionality reduction and clustering. They proposed the Mixtures of Probabilistic Principal Component analyzers (MPPCA) (Tipping and Bishop 1999) and Categorical Distribution (CD). Parameters of analyzers are determined using the EM algorithm.
Furthermore, the Automated Telemetry Health Monitoring System (ATHMoS) is developed by the German Space Operation Center (GSOC) (OMeara et al. 2016 2018). They used an autoencoder to obtain compressed data representing the original dataset efficienlly. They show that the feature vec- tor extracted for the autoencoder is utilized for anomaly detection and decrease the false positives. Also, the National Aeronautics and Space Administration (NASA)'s Jet Propulsion Laboratory uti- lizes Long Short-Term Memory (LSTTM) networks with label information provided by an expert to detect anomaly for their MSI. and SMAP spacecraft (Hundman et al.) 2018). They smooth the sharp spikes in error value using an supervised model and false-positive mitigation method. However, all the prior research focused on detecting anomaly from individual telemetry, rather than from a set of telemetries.
average (EWMA). They also proposed an un-
3 DATASET
Korea Aerospace Research Institute (KARI) KARI 2019) is the aeronautics and space agency in South Korea founded for the development and research of aerospace scientific technologies. KARI has developed and is currently operating a series of multipurpose satellites for high- high-resolution opti- cal observation, and radar and IR observation, as well geostationary orbit satellites for indepen-
2

---

Under review as a conference paper at ICLR 2020
dent weather and marine observation, and communication relay. Korea Multii-Purpsse Satellite (KOMPSAT-2) [KOMPSAT-2] 2019) is one of the national monitoring satellites with high-resolution optical observation. It was launched in 2006 and had been operating for 13 years. KOMPSAT- 2 satellite generates more than 3,000 different types of telemetries from various subsystems (Lee al. 2005 eooPorta 2019).
In this work, we collected 88 types of different telemetries with more than 43,716 telemetry samples from the KOMPSAT-2 satellite for 10 months. These telemetries are categorized into different subsystems according to their characteristics. In Tablell we present the number of different teleme- try types used from each subsystem collected from May 2013 February 2014 for 10 months. Also, the collected data size for each month is shown in GB.
Table 1: Telemetry dataset description from KOMPSAT-2 satellite collected from May 2013 till Feb. 2014.
Subsystem
Sub1
Sub2
Sub3
Sub4
Sub5
Sub6
Sub7
Total
Num. of different telemetry types
7
9
14
26
15
10
7
88
Data size (GB)
1.15
1.24
1.40
1.83
1.26
1.44
1.20
9.52
INTEGRATIVE TENSOR-BASER ANOMALY DETECTION (ITAD)
In this section, we explain the overall process of our new approach, an Integrative Tensor-based Anomaly Detection (ITAD) framework for the satellite system and describe its details in Fig.
Data Interpolation and Normalization (Preprocessing): Since many telemetries are measured and sampled at different time interval (from every second to few minutes), there are many missing or unmeasured sensor values for each timestamp. To address this challenge, we apply linear inter- polation, which populates missing value with linearly increasing value between two consecutive data points. After linear interpolation, we normalize each telemetry value (feature) individually. Each value such as temperature or power is measured on a different scale; min-max normalization is used to normalize all values into the range between 0 and 1. The minimum value is mapped to 0, and the maximum value is mapped 1.
After linear interpolation, the timestamp is recorded every second in the raw dataset. However, most of the values do not change in a short period, and many telemetry values have the same value for a few minutes or much longer. Therefore, we assumed that it might be practical to compress several timestamps (rows) into single row. Also, as we add many interpolated values from the linear interpolation step. the size of the dataset increases by more than three times (9.52GB to 29.62 29.62GB) after interpolation. Therefore, the compression provides the benefit computational efficiency while maintaining the critical information of the data.
Feature Extraction: Using the above compression method, we use different statistical meth- ods extract 8 features for each telemetry time series T; mean (ø), standard deviation (s), skewness (skew), kurtosis (kurt), minimum (min), maximum (max), energy (E), and average crossing (Zerassing). Energy and average crossing are calculated from E = ạ Σn 2 and Ecrossing EN Lry>E, respectively. Each feature is calculated for every 10 minutes of the
Telemetry
Feature Vectors
Telemetry
Emur Drnamic Areschedd
Evector
Senor scae
Time
(Smpled) Time
B
Time
w MW
W
Exror
Tel
2
Tel
Tel.
Timld
Time (0
Time
(e) Construction
Preprocessing
Clastering
Calculating Error score
Anomaly Detection
Feature Extraction
Decomposition
Figure 1: The end-to-end data processing pipeline of our Integrative Tensor- based Anomaly Detection (ITAD) approach.
3

---

Under review as a conference paper at ICLR 2020
original data and we obtain the final feature vector V.I.T.) generated by concatenating the different features as shown in equation as follows:
(n.T. = {ū, 8, skew, kurl, män, mac, E, Forosiiin)
where n is the number of feature vector samples. As result, we can reduce the size of the dataset significantly from the interpolated data (0.5 Gb << 29.62 GB) and reconstitute the telemetry dataset into the matrix form consisting of feature vectors (at each column) by time samples (at each row) as shown in Fig. (b).
Tensor Construction and Decomposition: Tensor decomposition can effectively handle high di- mensional time series. Therefore, construct 3rd-order telemetry tensor consisting of time x feature vector x telemetry and decompose tensor using the CANDECOMPIPARATAC (CP) decomposition in Fig. (c), which is one of the most widely used tensor decomposition methods, as shown in equation 2 After CP decomposition, we obtain three component matrices, A, B, and C' described in Fig. (d). The component matrix A consists of time-to-factor (time X r) de- scribing the comprehensive characteristics of samples from all telemetries at the same point in time using r factors. The component matrix B shows the feature vector-to-factor matrix x feature vector) indicating how much each factor influences each feature vector. The final matrix C captures the gelemetry-toffccton (telemetry r) matrix characterize how much each factor affects each telemetry.
R
X N, [A,A,,C]]
(2)
where R is the rank, À is the weight, E RRi, E TR and e, E IR for r=l,.... R (Hackbusch) 2012).
In order to find the optimal solutions of CP decomposition, utilize the alternating least squares (ALS) updating method, which iteratively optimizes one component while leaving the others fixed. Given the 3rd-order tensor, it first fixes the component matrices B and C' to obtain the solution for the component matrix A. Then, ALS fixes A and C to find B, and lastly fixes A and B to solve for C' as follows: A arg minA||X
ACC @ B) B < arg mingl1X (2) ( C AA)'l, C- Cagrg mine (x(3) C(B A) "1, where ((1) denotes the mode-1 unfolding of tensor X into matrix, and ((2; and (a) indicates the mode-2 and mode-3 unfolding, respectively. Moreover, @ de- notes the Khatri-Rao product (Smilde et al.) 22004), which is the "matching columnwise" Kronecker product. It repeats this procedure until it reaches the specific convergence criteria or maximum iteration.
[|,
Selecting an Optimal Rank r: Since we aim to obtain the component matrices from the decom- position, it is critical to choose an optimal size of the factor (rank) r that can represent the orig- inal telemetry tensor. However, there is no general straightforward algorithm to select the opti- mal r. Instead, we measure the reconstruction error as the difference between the original ten- sor X and the approximated tensor X. From the given 3rd-order tensor X E R xlaxaa,,,, we use the Frobenius norm || X|IF to calculate the reconstruction error, where X is computed as the outer product (o) of component matrices A, B, and C, as shown in equation 2 We can compute the reconstruction error as follows: Reconstruction error = Eish (Tajk 1 (irl for i = I....,Ijj
1..... J. = 1,...,,, K,r 1...., R, where Zijk E [XXX aar E R T bjr E I and Ckr E R The smaller the reconstruction error, the closer the approximate tensor to the original tensor. We find the reconstruction error by increasing the rank from and choosing the smallest r by minimizing || X||r- until when the approximated tensor can re- construct more than 90% of the original telemetry tensor. We present an example of selecting an optimal r from the reconstruction error in Appendix B
Clustering Analysis:
The original telemetry data is highly unbalanced, where most elements normal, with only a few anomalies. This is one of the challenging issues in an anomaly detection problem. Additionally, normal telemetry data might exhibit certain repeating patterns, many satel- lite commanding sequences are not drastically different from one another during nominal operation.
Therefore, apply clustering method for the component matrix A (Time x r), in order to group major patterns of telemetry samples, as shown in Fig. (e). such that they represent normal data behavior. The primary reason we chose the matrix A among the three-component matrices
4

---

Under review as a conference paper at ICLR 2020
is that ultimately aim to identify the time, at which the anomaly occurs, and the component matrix A represents the key information and comprehensive characteristics of all telemetries at each time instance across different subsystems. Note that the clustering is applied row-wise since the component matrix A has the time information at its row.
However, clustering is challenging because not only telemetry sample is an 8-dimensional vector, but also the original tensor dataset consists of different types of telemetries. Therefore, we exten- sively experimented with several clustering algorithms such as Gaussian Mixture Model (GMM), Isolation Forest (IF), k-means, and One-class Support Vector Machine (OCSVM) to compare and determine the best approach. Clustering methods other than the k-means algorithm showed too many false positives. Hence, we only use the k-means algorithm in our work.
As it is required set the number of clusters when applying k-means clustering, we use silhouette analysis to determine optimal k. The silhouette method (Rousseeuw 1987) has coefficients ranging from to where the positive value indicates that the samples are correctly assigned and away from the neighboring clusters, and the negative value represents samples that might be assigned to the wrong cluster. The zero value indicates that the samples are on the decision boundary two neighboring clusters. We varied & from 2 to 10 and chose the value when the silhouette coefficient is the largest positive value, as shown in Appendix B
Note: Tensor decomposition can be viewed
clustering mechanism. Generally, the component matrix A (time x factor), which accounts for comprchensive characteristics of all telemetries in the same subsystem, however, we need a row-wise clustering for calculating an anomaly score by time, since our goal to identify the time instance when anomaly occurs. That is the reason we use another k- means clustering in addition to the tensor decomposition to capture the distance between normal and abnormal data.
serve
indicator for different column-wise clusters. In our research,
Calculating Anomaly Score: If time sample is normal, it might belong to one of the clusters we constructed from the previous step. If a time sample is anomalous, then would exhibit far different pattern from normal patterns, and it would not belong to any clusters. To quantify this, we calculate the Euclidean distance d(s, VE - s;}² between each time sample
s (#1, $2..... sn) and the centroid e = (c1, .2., Cn) of its nearest cluster. A short Euclidean distance means that value is similar to normal value and pattern, and a long distance indicates that the value is far different from major clusters and normal patterns. Therefore, we can define this Euclidean distance high anomaly scores.
anomaly score, shown in Fig. (f), where anomalies will typically have
Daiaenriven Dynamic Thresholding: In order to derive anomalies from the anomaly score, it is required to set a certain threshold, Although a fixed threshold is the simplest way, it cannot detect the contextual anomalies, which can be below the threshold point. In the same vein, values that are just above the fixed threshold can be normal, but they can be detected as anomalies with a fixed threshold method. Additionally, a fixed threshold approach cannot adapt to various changing conditions and can produce too many false positives. The example of the problems with a fixed threshold and high false positives is illustrated in Appendix C To address the problem with a fixed threshold, we develop the data-driven dynamic thresholding method, where a threshold value can be dynamically adjusted and changed adaptively in differing contexts. We first choose the time window w, which is defined as the number of previous anomaly score points to compute the current anomaly score. Then, we calculate the mean μ and standard deviation a of the data values in the time window w. Finally, based on the confidence interval distribution, we determine score is over m x a apart from the (4, denoted by a [(; nu2 and / X,/n, where i (n w), ..., (the number of data points in w) and m (the coefficient parameter) 2 1. This measures how far (m x σ) is apart from the mean μ of values within the window w.
anomaly, when the anomaly
5 EXPERIMENT
Tensor Size: In this experiment, we use 88 types of telemetries with more than 43,716 telemetry samples, where each telemetry sample has feature vector of 8 different statistical quantities, as discussed in Section 3 With these telemetries, we construct seven 3rd-order telemetry tensors, and the dimensions and the size of each tensor are provided as follows: time feature vector x
5

---

Under review as a conference paper at ICLR 2020
telemetry, and 43, 716 x 8 x number of telemetries. We summarize the detailed size for each subsystem in T
Convergence Criteria: Next, decompose each 3rd-order telemetry tensor into component ma- trices, A, B, and C. using CANDECOMPIPARAEAC (CP) decomposition with the alternating least squares (ALS) updating method. Updating will be stopped when the iteration reaches 100, or when the convergence tolerance is less than 10-5 in this experiment.
Optimal Rank:
The reconstruction errors are calculated from increasing the rank r from 2. Then, choose the smallest r, which minimizes ||X X until when the approx- imate tensor can reconstruct more than 90% of the original telemetry tensor. The result of the optimal the third column in Table 2 for each subsystem. As shown in Table 2 r produces differ- ent ranges of values from 11 to 29 be- cause of the different telemetry val- ues, characteristics, and structures in each subsystem.
The size of 3rd-order telemetry tensor, the optimal rank r where every r guarantees more than 90% of the reconstruction rate of estimated tensor. and the optimal number of clustering & by silhouette analysis for each subsystem.
Table 2:
Tensor size
Optimal rank
Optimal
Subsystem
(time f.ee. x tel.)
subsystemi
43, 716 x
20
r is presented
subsystem2
43, 716 x 9
28
subsystem3
43, 716 x x 14
20
subsystemd
43, 716 x x 26
29
subsystems
43, 716 x x 15
20
subsystem6
43, 416716 x TO
18
subsystem?
43, 716 x X
11
9
k-means Clustering: Among the three-componen matrices from decomposition using the optimal r. we apply k-means clustering to the component matrix A consisting of time-to-factor information. Since our goal is to detect anomaly points over different time values, we chose the component matrix A, which has the time information. Additionally, in order to determine an optimal k. we apply silhouette analysis for each subsystem. All results are presented in the last column in Table
Fine-Tume Tuning for Dynamic Thresholding: We use the dynamic thresholding method, which can dynamically adjust the threshold value based on environmental change. However, there are clear trade-offs between different dynamic thresholding parameters, window size w and coefficient pa- rameter m. To empirically evaluate the trade-off's and fine-tune the best parameters, we conducted various experiments by changing the window size from 9 to 576 to determine the optimal value of w and m for each subsystem. Since we sampled the dataset corresponding to 10 minutes into one data point, window size 9 translates to 90 minutes time period, which corresponds to the single activity cycle of the satellite operation. We can observe that the best performance is achieved when w is either 108 or 198 for all subsystems. We also empirically found that m 6 typically resulted in the best performance for any window size. An example of fine-tuning for dynamic thresholding is described in Appendix C
Comparisons with Other Methods: We compared our approach with four other well-known anomaly detection baselines developed for satellite systems. First, we compare ours with One-Class SVM (OCSVM) after feature extraction following the method from CNES's s NOS- TRADAMUS (Fuertes et al. 2016) approach. Second. Isolation Forest (IF) used instead of OCSVM. The Next approach is based on Mixtures of Probabilistic Principal Component Analy- sis and Categorical Distributions (MPPCACD) model used for JAXA (Yairi et al., [201))ss SDS-4. Lastly, single-channel LSTM (SC-LSTM) model developed by NASA-JPL (Hundman et al. 2018) is also compared.
6 RESULTS
We present the performance of each anomaly detection method in Table The domain experts label anomalies at KARL, and the number of them is shown in the second column in Table 3 If a detected point is an anomaly, is counted as a true positive (TP). Otherwise, we count it as a false positive (FP). The performance in TP and FP for each detection method is provided in Table
Note: Even with 10 months of data, we do not have many anomalies. If there are many anomalies, the satellite will not function properly. Specifically, requested by satellite operators, our objective is to detect the anomalous events accurately, while reducing false positives in a highly unbalanced dataset.

---

Under review as a conference paper at ICLR 2020
Table 3: Anomaly labels, and the performance comparison and FP with different anomaly detection
methods.
(IAXA)
SCISTM(NASA
OCSVM (CNES)
ITAD (Ours)
Subsystem
Anomaly
TP
TP
FP
TP
F.p
TP
IP
FP
Subsystem!
0
0
0
0
Subsystem2
2
Subsystem3
10
Subsystendd
10
12
12
Subsystem5
2
5
5
0
Subsystemó
Subsystem7
Total
25
18
2 18
All methods show the same detection performance in TP while detecting anomalies from differ- ent subsystems. However, our ITAD and MPPCACD detect one anomaly for subsysteml and an- other one for subsystem2. OCSVM detects all two anomalies from subsystem2, while it misses an anomaly for subsystem/. On the other hand, the ITAD framework outperforms in FP compared to others. While SCLSTM, OCSVM, and IF produces a total of 25, 18, and 18 false positives, re- spectively, our ITAD approach reduces the false positives to 1. MPPCACD shows the second-best performance among other approaches in FP, but could not reduce the FPs enough compared the ITAD approach.
To compare the overall performance, we present the precision, recall, and Fl score in Table The ITAD framework achieves the highest precision (66.67%) than other methods, because of its high performance FP. On the other hand, the recall perfor- mance was the same across all approaches. Overall, ITAD outperforms all other meth- ods in F1 score by more than two-fold (66.67% vs. 28.57%)
Table 4: Performance comparison of ITAD with other meth- ods.
Method
Precision
Recall
F1 Score
MPPCACD (IAXA)
7.41%
66.67%
13.33%
SCLSTM (NASA)
18.18%
66.67%
28.57%
OCSVM (CNES)
10.00%
66.67%
17.39%
10.00%
66.67%
17.39%
ITAD (Ours)
66.67%
66.67%
66.67%
Analysis: Most of the anomaly detection methods except our approach generate high FPs because they cannot account for multiple telemetries simultaneously. When a temporary glitch detected from only one telemetry in subsystem, it is highly likely that it is a trivial outlier, not an actual anomaly. (Note that telemetries collected from adjacent sensors the same environment are re- garded one case since they are highly correlated with each other). In the case of subsystem4, as shown in Fig. (a), the 1st telemetry. TDCSUAT, has a temporal glitch on August 17th. whereas all other different types of telemetries such as MMQTXON, TCCU2AT, and TXPOND2T do not have any glitch at the same timestamp. This glitch is confirmed as a trivial outlier, not an actual anomaly.
TOCSUAT
MMOTXON
AFSS1OMI
Data
TCCUZAT
Data Values
TXPOND2T
130027
13374
130020
120600
131200
142218
Time
Time
(a)
(b)
Figure 2: (a) Plots of different types of telemetries (TDCSUAT, MMQTXON, TCCU2A and TXPOND2T) in subsystem4. A temporal glitch not detected anywhere except TDCSUAT. According to the verification an expert, this temporal glitch a trivial outlier, not an actual anomaly. (b) Plots of different types telemetries (AFFSIOMI and ASFFIOPI) in subsystem2, Temporal glitches are discovered from different telemetries at the same timestamp. According to the verification by an expert, they are actual anomalies.

---

Under review as a conference paper at ICLR 2020
Our Integrative Tensor-based Anomaly Detection (ITAD) approach does not report this glitch as an anomaly, whereas other detection methods such as MPPCACD, SCLSTM, OCSVM, and IF record it anomaly. In the case of subsystem33, subsystem4, and subsystem5, there some temporal glitches, but no actual anomaly. The ITAD framework is the only method that does not report this type of trivial outliers, shown in FPs of subsystem4 in Table
When there are temporary glitches in multiple telemetries at the same timestamp as shown in sub- system2 in Fig. (b). ITAD reports anomalies. For example, there are temporary glitches in two different types of telemetries (AFFSJOMI and ASFFIOPI) at the same timestamp. This glitch is an actual anomaly, and the ITAD approach accurately reports an anomaly. Since the ITAD method can take and process multiple telemetries simultaneously, it significantly reduces false positives caused by the other methods based on a single-variate anomaly analysis. These results demonstrate the effectiveness of integrative analysis for multiple telemetries in subsystems using tensor-based method for satellite monitoring.
DISCUSSION AND LIMITATIONS
Determining an appropriate rank-size is an No-complete problem HHástad 1990). and there is no general algorithm find it. To choose r. we exploit the reconstruction error, which is proposed in the original CP research (Carroll and Chang 1970. Harshman 1970). However, there is possibility suffer from overfactoring and ultimately failing to obtain an optimal solution from this method. To address this possibiliiyy, we plan apply the Core Consistency Diagnostic (CORCONDIA) proposed by Bro and Kiers (2003) for determining the optimal rank r for our future work. We believe that the CORCONDIA method, which assesses the core consistency and measures the similarity between the core array and theoretical super- diagonl array, can yield more accurate results.
Even though use 10 months of real telemetry dataset, we do not have many anomalies, which is realistic scenario. Otherwise, i.c. if there are many anomalous events, most mission-criticat systems would fail very quickly. In the presence of a small number of anomalies, the main focus of our work is to reduce false positives to assist satellite operators to determine the true anomalies, as requested by KARI operators. However, we agree that because of small number of anomalies, current pre- cision, and recall metrics would be very sensitive to anomaly events. Missing one anomaly would result in 33% drop in performance. To partially address this issue, we currently in the process of collecting more datasets with anomalies within a longer and plan to evaluate our tensor-based sys- tem with datasets with more anomalies. Also, we believe we need develop better performance metric, which can capture the performance with small number of anomalies.
Lastly, we are in the process of deploying our tensor-based anomaly detection method THE KOMPSAT-2 satellite in the spring of 2020. We plan to incorporate not only 88 telemetries we experimented in this research, but also other types of telemetries and subsystems to evaluate our integrative anomaly detection method.
8 CONCLUSION
In this work, we proposed an Integrative Tensor-based Anomaly Detection framework (ITAD) to detect anomalies using the KOMPSAT-2 satellite telemetry dataset, where our approach can analyze multiple telemetries simultaneously detect anomalies. Our ITAD achieves higher performance in precision and F1 score compared to other approaches. We also demonstrate that the ITAD reduces the false positives significantly. This reduction in FPs is because it can distinguish actual anomalies from trivial outliers by incorporating information from other telemetries the same time. In the future, we plan to improve our algorithm by applying the CORCONDIA method to avoid overfac- toring and find optimal rank r and incorporate and evaluate datasets with more anomalies.
We believe our work laid the first grounds using an integrated tensor- based detection mechanism for space anomaly detection. Moreover, the result demonstrates that our proposed method can be applicable in a variety of multivariate time-series anomaly detection scenarios, which require low false positives well high accuracy.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Evrim Acar, Seyit A Çamtepe, Mukkai S Krishnamoorthy, and Bülent Yener. 2005. Modeling and multiway analysis of chatroom tensors. International Conference on Intelligence and Security Informatics. Springer, 256-268.
Evrim Acar, Seyit A Camtepe, and Bülent Yener. 2006. Collective sampling and analysis of high order tensors for chatroom communications. In International Conference on Intelligence and Se- curity Informatics. Springer, 213-224.
Evrim Acar and Bülent Yener. 2009. Unsupervised multiway data analysis: A literature survey. IEEE transactions on knowledge and data engineering 21, (2009), 6-20.
Charlotte Møller Andersen and R Bro. 2003. Practical aspects of PARAFAC modeling of fluores- cence iicitatioo-emissioon data. Journal of Chemometrics: A Journal of the Chemometrics Society 17, 4 (2003), 200-215.
Carl J Appellof and Ernest R Davidson. 1981. Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents. Analytical Chemistry 53, 13 (1981). 2053-2056.
Stephen D Bay and Mark Schwabacher. 2003. Mining distance- baaenc outliers in near linear time with randomization and a simple pruning rule. In Proceedings of the ninth ACM SIGKDD inter- national conference on Knowledge discovery and data mining. ACM, 29-38.
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. 2000. LOF: identifying density-based local outliers. In ACM sigmod record, Vol. 29, ACM, 93-104.
Rasmus Bro and Henk AL Kiers. 2003. A new efficient method for determining the number of components in PARAFAC models. Journal of Chemometries: A Journal of the Chemometrics Society 17, (2003), 274-286.
J Douglas Carroll and Jih-Jie Chang. 1970. Analysis of individual differences in multidimensional scaling via an N-way generalization "Eckart- Young" decomposition. Psychometrika 35, (1970), 283-319.
Andrzej Cichocki, Danilo Mandic, Lieven De Lathauwer, Guoxu Zhou, Qibin Zhao, Cesar Caiafa, and Huy Anh Phan. 2015. Tensor decompositions for signal processing applications: From two- way to multiway component analysis. IEEE Signal Processing Magazine 32, 2 (2015), 145-163.
Fengyu Cong, Anh-Huy Phan, Piia Astikainen, Qibin Zhao, Qiang Wu, Jari K Hietanen, Tapani Ristaniemi, and Andrzej Cichocki. 2013. Multi-domain feature extraction for small event-related potentials through nonnegative multi-way array decomposition from low dense array EEG, Inter- national journal of neural systems 23, 02 (2013), 1350006.
Lieven De Lathauwer and Joos Vandewalle. 2004. Dimensionality reduction in higher-order signal processing and rank-(R1, R2...., RN) reduction in multilinear algebra. Linear Algebra Appl. 391 (2004), 31-55.
eoPortal. 2000-2019. Earth Observation resources. hetp//pprtal/aaaellliemmssssosss 09-19.
https::/directore eoportal. org/ (2000-2019). Accessed: 2019.
Hadi Fanace-T and Joao Gama. 2014. An eigenvector-based hotspot detection. arXiv preprint arXiv:1406.319 (2014).
Sylvain Fuertes, Gilles Picart, Jean-Yves Tourneret, Lotfi Chaari, André Ferrari, and Cédric Richard. 2016. Improving Spacecraft Health Monitoring with Automatic Anomaly Detection Techniques. In 14th International Conference on Space Operations. 2430.
Ryohei Fujimaki, Takehisa Yairi, and Kazuo Machida. 2005. An approach to spacecraft anomaly detection problem using kernel feature space. In Proceedings of the eleventh ACM SIGKDD in- fernational conference on Knowledge discovery in data mining. ACM, 401-410.

---

Under review as a conference paper at ICLR 2020
Yu Gao, Tianshe Yang. Minqiang Xu, and Nan Xing. 2012. An unsupervised anomaly detection ap- proach for spacecraft based on normal behavior clustering. In Intelligent Computation Technology and Automation (ICICTA), 2012 Fifth International Conference IEEE, 478-481.
Laetitia Gauvin, André Panisson, and Ciro Cattuto. 2014. Detecting the community structure and activity patterns of temporal networks: non-negative tensor factorization approach. PloS one 9, (2014), e86028.
Wolfgang Hackbusch. 2012. Tensor spaces and numerical tensor calculus. Vol. 42. Springer Science & Business Media.
Richard A Harshman. 1970. Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multimodal factor analysis. (1970).
Johan Hästad. 1990. Tensor rank is No-comllete Journal of Algorithms 11, 4 (1990), 644-654
Frank L Hitchcock. 1927. The expression of a tensor polyadic as sum of products. Journal of Mathematics and Physics 6, 1-4 (1927), 164-189.
Frank L Hitchcock. 1928. Multiple invariants and generalized rank of p-way matrix or tensor. Journal of Mathematics and Physics 7, 1-4 (1928), 39-79.
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. 2018. Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding. arXiv preprint
(2018).
Minoru Inui, Yoshinobu Kawahara, Kohei Goto, Takehisa Yairi, and Kazuo Machida. 2009. Adap- tive limit checking for spacecraft telemetry data using kernel principal component analysis. Trans- actions of the Japan Society for Aeronautical and Space Sciences, Space Technology JAPAN 7,
ists26 (2009), PT_11-P[_16.
David Iverson. 2008. Data mining applications for space mission operations system health monitor- ing. In SpaceOps 2008 Conference. 3212.
David L Iverson. 2004. Inductive system health monitoring. (2004).
KARI. 1989-2019. Korea Aerospace Research Institute. (1989-2019). https: www.kard re.ke kr Accessed: 2019-09-19.
Henk AL Kiers. 2000. Towards standardized notation and terminology in multiway analysis.
Journal of Chemometrics: A Journal of the Chemometrics Society 14, 3 (2000), 105-122.
Tamara G Kolda and Brett W Bader. 2009. Tensor decompositions and applications. SIAM review 51, 3 (2009), 455-500.
KOMPSAT-2. 1989-2019. Korea Aerospace Research Institute. (1989-2019). https: /wwww kari. re.
02____22__000_ 01.do Accessed: 2019-09-19 19.
KA Kosanovich, MJ Piovoso, KS Dahl, JF MacGregor, and P Nomikos. 1994. Multi-way PCA applied an industrial batch process. In Proceedings of 1994 American Control Conference- ACC*94, Vol. 2. IEEF, 1294-1298.
Hans-Peter Kriegel, Peer Kröger, Erich Schubert, and Arthur Zimek. 2009. LoOP:
local outlier prob- abilities. In Proceedings of the 18th ACM conference on Information and knowledge management. ACM, 1649-1652.
J. B. Kruskal. 1977. Three-way arrays: rank and uniqueness of trilinear decompositions, with appli- cation arithmetic complexity and statisties. Linear Algebra and Its Applications 18, 2 (1977), 95-138.
J. B. Kruskal. 1989. Multiway Data Analysis. North-Holland Publishing Co., Amsterdam, The Netherlands, The Netherlands, Chapter Rank, Decomposition, and Uniqueness for 3-way and N-way Arrays, 7-18.
10

---

Under review as a conference paper at ICLR 2020
Sanguk Lee, Sungki Cho, Byoung-Sun Sunng-Sunnni Lee, and Jachoon Kim. 2005. Design, Implementation, and Validation of KOMPSAT-2 Software Simulator. ETRI journal 27, 2 (2005), 140-152.
SeungChul Lee. Hongbin Liu, MinJeong Kim, Jeong Tai Kim, and ChangKyoo Yoo. 2014. Online monitoring and interpretation of periodic diurnal and seasonal variations of indoor air pollutants in subway station using parallel factor analysis (PARAFAC). Energy and Buildings 68 (2014), 87-98.
Ke Li, Yalei Wu, Shimin Song, Yi Sun, Jun Wang, and Yang Li. 2017. A novel method for spacecraft electrical fault detection based on FCM clustering and WPSVM classification with PCA feature extraction. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering 231, 1 (2017), 98-108.
Hing-Hao Mao, Chung-Jung Wu, Evangelos E Papalexakis, Christos Faloutsos, Kuo-Chen Lee, and Tien-Cheu Kao. 2014. MalSpot: Multi 2 malicious network behavior patterns analysis. In Pacific- Asia Conference on Knowledge Discovery and Data Mining. Springer, 1-14.
Paul Nomikos and John F MacGregor. 1994. Monitoring batch processes using multiway principal component analysis. AICHE Journal 40, 8 (1994), 1361-1375.
Corey OMeara, Leonard Schlag, Luisa Faltenbacher, and Martin Wickler. 2016. ATHMoS: Auto- mated Telemetry Health Monitoring System at GSOC using Outlier Detection and Supervised Machine Learning. In 14th International Conference on Space Operations. 2347.
Corey OMeara, Leonard Schlag, and Martin Wickler. 2018. Applications of Deep Learning Neural Networks to Satellite Telemetry Monitoring. In 2018 SpaceOps Conference. 2558.
Evangelos Papalexakis, Konstantinos Pelechrinis, and Christos Faloutsos. 2014. Spotting misbe- haviors in location-based social networks using tensors. In Proceedings of the 23rd International Conference on World Wide Web. ACM, 551-552.
Evangelos E Papalexakis, Christos Faloutsos, and Nicholas D Sidiropoulos. 2017. Tensors for data mining and data fusion: Models, applications, and scalable algorithms. ACM Transactions on Intelligent Systems and Technology (TIST) 8, 2 (2017), 16.
Peter Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics 20 (1987), 53-65.
Bernhard Schölkopf, Alexander Smola, and Klaus-Roben Müller. 1998. Nonlinear component anal- ysis as a kernel eigenvalue problem. Neural computation 10, (1998), 1319.
Age K Smilde, Rasmus Bro, Paul Geladi, et al. 2004. Multi-way analysis with applications in the chemical sciences. Wiley Online Library.
Michael E Tipping and Christopher M Bishop. 1999. Mixtures of probabilistic principal component analyzers. Neural computation 11, 2 (1999), 443-482.
Ledyard R Tucker. 1963. Implications of factor analysis of whree-way matrices for measurement of change. Problems in measuring change 15 (1963), 122-137.
Ledyard R Tucker. 1964. The extension of factor analysis to three-dimensional matrices. Contribu- tions to mathematical psychology 110119 (1964).
Ledyard R Tucker. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika 31, 3 (1966), 279-311.
M Alex O Vasilescu and Demetri Terzopoulos. 2002a. Multilinear analysis of image ensembles: Tensorfaces. In European Conference on Computer Vision. Springer, 447-460.
M Alex O Vasilescu and Demetri Terzopoulos. 2002b. Multilinear image analysis for facial recogni- tion. In Pattern Recognition, 2002. Proceedings. 16th International Conference on, Vol. 2. IEEE, 511-514.
Takehisa Yairi, Naoya Takeishi, Tetsuo Oda, Yuta Nakajima, Naoki Nishimura, and Noboru Takata. 2017. A Data-Driven Health Monitoring Method for Satellite Housekeeping Data Based on Prob- abilistic Clustering and Dimensionality Reduction. IEEE Trans. Aerospace Electron, Systems 53, (2017), 1384 1401.
11

---

Under review as a conference paper at ICLR 2020
A TENSOR DECOMPOSITION
A.1 A TENSOR
A tensor is nullt-diimensionl array (Papalexakis et al. 2017), where geometrie vectors and scalars be considered the simplest tensors. A Ist-order tensor is vector, 2nd-order tensor is matrix, and a 3rd-order tensor can be represented as a cube, which has three vector spaces. In general, .Nth-order tensor X R/1 12/xxlxx"xxy is represented by the outer product of N vector spaces as follows:
x-a' o......
(3)
where IN is the Nth dimension and aN is the vector in Nth dim. A rank in tensor indicates the number components the decomposed matrices and every tensor can expressed as a sum of rank-1 tensor (Kruskal 1977 1989). Due to its ability to express multi-modaligg it is effective to handle such dataset with multi- modal aspects. Expressing a tensor as a sum of a rank-1 I tensor was
first proposed by |Hitchcock 1927 1928).
A.2 TENSOR DECOMPOSITION
In 1970, Carroll and Chang 1970) proposed canonical decomposition (CANDECOMP)
and Harshman (1970) suggested parallel factor decomposition (PARAFAC), which is an extended version of 2-way factorization for higher-order data. Since CANDECOMP and PARAFAC have similar concept, the CANDECOMPIARATACC (CP) decomposition formulated by Kiers (2000) has been widely used. It decomposes Nth order data into a linear sum of rank-1 1 tensor as described in Fig. (a) and a 3rd-order tensor can be decomposed into three component matrices A, B, and C. Ap- pellof and Davidson (1981) pioneered the use of CP model extract information from a chemical system. LnndereAn and Bro (203) contributed to developing practical description and application of tensors. And And|Acar AAcar et al. (2005) 2006) was the first to apply a tensor decomposition to data mining. They analyzed online chatroom data to understand how social groups evolved in eyberspace. They constructed a 3rd-order tensor with user x keyword x time spaces.
Ja
Ja
B
X
b1
b2
A
01
02
(b)
(c)
G
B
X
Figure 3: Examples of different tensor decomposition methods from the given 3rd-order tensor: (a) CAN- DECOMPIPARATAC (CP) decomposition, where X is an approximated sum of r rank-1 tensors. (b) Tucker decomposition, where has three component matrices with the a rank-(p, q. r) and a core tensor g. (c) A super-diagonal core tensor with ones as diagonal values. If the core tensor of Tucker decomposition is super-dingonal having ones diagonal values and all ranks are identical, it can be thought as same CAN- DECOMP/PARAFAC decomposition.
The Tucker decomposition is another commonly-used tensor decomposition, which was first intro- duced by [Tucker (1963) and refined later (Tucker) 1964 1966). As shown in Fig. (b). the Tucker decomposition has a core tensor, which can be viewed as compression the original tensor X. In the case of the 3rd-order tensor, it decomposes tensor into core tensor and three matrices with different ranks (p # g + r) as shown in Fig. (b). In fact, CP decomposition can be thought as a special case of the Tucker decomposition, where all ranks are identical (p g r) and the tensor is super-diagonal having ones as diagonal values as shown in Fig. (c). De Lathauwer and Vandewalle (2004) applied the Tucker decomposition for dimensionality reduction in higher-order
12

---

Under review as a conference paper at ICLR 2020
signal processing. Pioneers of the of the Tucker decomposition in computer vision are Vasilescu and Terzopoulos (2002a). They extended conventional Singular Values Decomposition (SVD) to N-mode SVD for tensor. They also showed that it has significantly better performance compared to the standard Principal Component Analysis (PCA) for image recognition task (Vasilescu and Terzopoulos 2002b).
The benefit of using tensor decomposition is that is one of the most effective unsupervised meth- ods for extracting characteristics of Nth-dimensional data. Traditionally, it has been required to rearrange the dimension into a 2nd-order matrix to factorize high dimensional data. However, ten- sor decomposition can offer more accurate results by keeping the Ntth-order structure of data as shown in other research (Acar and Yener 2009; Kolda and Bader 2009 Cichocki et al. 2013).
B SELECTING OPTIMAL PARAMETERS
B.1 OPTIMAL k
We use silhouette analysis to determine an optimal for k-means clustering. We varied & from to 10, and chose the value when the silhouette coefficient is the largest positive value shown in the example Subsystem2 Fig.
Select as
Silhouette Score
0.4
0.1
The number clusters
Figure 4: Selecting the optimal A for Subsystem2 using the silhouette scores by changing the number of clusters.
B.2 OPTIMAL
In order to select the optimal т, find the reconstruction error by increasing the rank r from 2 and choosing the smallest by minimizing |X X X| until when the approximated tensor can reconstruct more than 90% of the original telemetry tensor as shown in the example of Subsystem7 in Fig.
50%
40%
Reconstruction error
20%
Select us
10%
0%
11
The number of rank
Figure 5: Selecting the optimal r for Subsystem7 at the point when the reconstruction error less than 10%.
C FIXED AND DYNAMIC THRESHOLDING
C.I PROBLEM WITH FIXED THRESHOLDING
fixed threshold approach cannot adapt to changing conditions, and can produce too many false
positives. An example of the problems with fixed threshold and high false positives is illustrated
in Fig.
13

---

Under review as a conference paper at ICLR 2020
Data Value
150716
150820
Figure 6: Example of high false alarms caused by fixed threshold for telemetry AFSSIOMI of subsystem2.
C.2 FINE-TUNING FOR DYNAMIC THRESHOLDING
The X-axis indicates the time, and the Y-axis is the anomaly score value. The blue line indicates the anomaly score and the red line represents the dynamic threshold computed from the formula, + m a during w. As we can observe from Fig. 7 different number of false positives and true positives can be detected based on different w and As shown in Fig. (a) and (b). increasing the window size w from 9 to 198 tends to make the threshold line flatter. In addition, Fig. 7 (a) and (c) show that increasing the coefficient parameter m from 4 to 6 influences the overall distance between the threshold line and the anomaly score line. We can observe the best performance is achieved when is either 108 198 for all subsystems. We also empirically found that m typically resulted in the best performance for any window size.
Pobilive
Score value
( 198, T= 6
2013:11
Time
2014
False Negetive
Dynamic threshold Anomaly score
Scoree valuré
=9) T
2013-11
2013-12
Time
2014-01
2014-07
Posilivel
False Positive
Score valuee
(c) v 198, 4-a
2013-11
2013-12
2014-01
2014-02
Figure 7: Anomaly score (blue line) and dynamic threshold (red line) by the size of the time window (w) and coefficient parameter (rı): (a) when 198, T = + 6 it detects actual anomaly only (1 true positive), (b) when 9, detects many anomalies including the actual anomaly true positive and false negatives). (a) is optimal, (b) too loose, and (c) is too sensitive.
6 cannot detect any (1 false negative), and (c) when 198, T = p a,
14


Reviews:
[{"review_id": "SJlQCgb7YH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, a tensor-based anomaly detection system for satellites is proposed. The proposed method mainly consists of three parts: creating a tensor from time-series data, applying tensor decomposition, and clustering. The performance is evaluated with a real satellite data set.\n\nThough the results look interesting, I vote for rejection because the paper is not well fitted to the ICLR community. \n\nThis paper specifically focuses on the anomaly detection of satellites. I don't say the application study is less worthy but the scope of this paper is too narrow for the ML community. The methodology of this work is basically combining the existing technology and is not novel enough. The performance is validated only by a single data set. So I feel this paper is more like a case study of a specific application and is more appropriate submitting to data mining conferences such as KDD.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A"}, "extra_text": {}}, {"review_id": "BkglNHvy9H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents and evaluates an anomaly detection algorithm that is used for identifying anomalies in satellite data. The considered satellite data originates from multiple sensors that are divided into sensor groups. The anomaly detection algorithm roughly works as a 4 stage process: \nRaw data is converted into a set of features per sensor that aggregate the last 10 min of sensor data, resulting in a tensor X with dimensions (time, features, sensors)\nA tensor decomposition method is used to decompose X into matrices A,B,C. \nMatrix A with dimensions (time, latent factors) is used to cluster data using k-means. \nTime points are considered anomalous when the latent feature vector is too far away all cluster centres. \nThis algorithm is experimentally compared to baseline methods that only use one sensor at a time. \n\nOverall, I think this paper is not fully ready for publication based on a) inconsistencies in notation that make it rather hard to read at first and b) open questions regarding the baselines, and c) missing experimental evaluations that may give insights into why the algorithm works better then the baselines. Moreover, besides those shortcomings, d) I am not convinced that ICLR is the right venue for this paper. Although the data set could provide an interesting application, the paper provides no theoretical contributions and the algorithm only pipelines known algorithms in a problem-specific way. In my view, a more application-focussed conference would be a better fit. \n\n\nDetailed comments.\n\nd) in my view is the most important issue. As mentioned above, the paper contains no theoretic contributions to the field of anomaly detection. Instead, it solves data-specific challenges for anomaly detection by pipelining a variety of methods in order to predict anomalies. While this approach is fine for solving an application, I am not convinced that it warrants a publication at ICLR, regarding in particular that the experimental evaluation does not provide any insights into why the algorithm works better than the baselines. That is,\nc) I am missing experiments that show the contributions of the individual choices that were made for each stage of the detection process. E.g. how much influence does the adaptive thresholding have on the performance? (Appendix C doesn’t quantify that properly.) The same would be interesting for the other stages. \n\nb) You state that the baseline methods are single-telemetry-based. However, you do not explain how those methods detect anomalies for multi-telemetry data as presented in Tables 3,4. Do those methods detect an anomaly whenever any of the feature/telemetry-specific methods detect one?\nAlso, Table 3 is not correct and doesn’t match the analysis in the text. Labels do not seem to be correct. Does Subsystem1 have 1 anomaly and Subsystem2  2?\n\na) \n-P4: Indices for the 3 dimensions of input data X are sometimes denote I_1,I_2,I_3 and sometimes I,J,K. \n-P4: the rank or number of factors is sometimes denied by lowercase r and sometimes by uppercase R, e.g. in the definition of the reconstruction error. Here, the running variable is denoted by lowercase r.  \n-P5, ‚Calculating anomaly score‘: Shouldn’t time samples have r dimensions instead of n?\n-P5, last paragraph of Section 4: \\mu should be defined by the normaliser w instead of n.  ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper applies tensor analysis techniques to anomaly detection from satellite data. The proposed solution is simple and seems to achieve good results. However, there is limited novelty in methodology and no sufficient experiments have been conducted to explain the performance gain. The paper is not ready for publication in ICLR but could be suitable for an application oriented venue. 
Decision: Reject