# Robust Graph Factorization for Multivariate Electricity Consumption Series Clustering

Authors: Kaihong Zheng; Honghao Liang; Lukun Zeng; Xiaowei Chen; Sheng Li; Hefang Jiang; Qihang Gong; Sijian Li; Jingfeng Yang; Shangli Zhou
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

Multivariate electricity consumption series clustering can reflect trends of power consumption changes in the past time period, which can provide reliable guidance for electricity production. However, there are some abnormal series in the past multivariate electricity consumption series data, while outliers will affect the discovery of electricity consumption trends in different time periods. To address this problem, we propose a robust graph factorization model for multivariate electricity consumption clustering (RGF-MEC), which performs graph factorization and outlier discovery simultaneously. RGF-MEC first obtains a similarity graph by calculating distance among multivariate electricity consumption series data and then performs robust matrix factorization on the similarity graph. Meanwhile, the similarity graph is decomposed into a class-related embedding and a spectral embedding, where the class-related embedding directly reveals the final clustering results. Experimental results on realistic multivariate time-series datasets and multivariate electricity consumption series datasets demonstrate effectiveness of the proposed RGF-MEC model.


---

## Body

## 1. Introduction

Recently, multivariate electricity consumption series clustering has been an important issue in machine learning fields [1–3]. In multivariate electricity consumption series, each instance consists of multiple time series from different sources which often contain information related to each other [4]. For example, multivariate electricity consumption series are composed of global-active-power series, global-reactive-power series, voltage series, and global-intensity series [5, 6]. Therefore, multivariate electricity consumption series clustering needs to analyze the relationship among these series [7, 8]. In other words, the objective of multivariate electricity consumption series clustering is to discover the relationship among multiple series and divide instances into groups.There have been many works on time-series clustering, including univariate time-series clustering [9, 10] and multivariate time-series clustering [7, 11]. It is well known that k-shape [12], k-DTW Barycenter Averaging (KDBA) [13], kAVG + ED [14], and k-Spectral Centroid (KSC) [15] are effective distance-based univariate time-series clustering algorithms. k-shape [12] utilized a normalized version of cross-correlation measure in order to consider shapes of time series. KDBA [13] was a global averaging method for dynamic time warping, where a new strategy was used to reduce the length of the resulting average sequence. kAVG + ED [14] made use of an efficient indexing method to locate 1-dimensional subsequences within a collection of sequences. KSC [15] can effectively find cluster centroids with a similarity measure, which applied an adaptive wavelet-based incremental approach to clustering. Because k-shape, KDBA, kAVG + ED, and KSC have achieved great success in univariate time-series clustering, researchers considered the relationship among multiple series and extended these methods for multivariate time-series clustering (i.e., m-kShape, m-kDBA, m-kAVG + ED, and m-KSC) [16]. In addition to extending existing univariate time series methods, researchers have proposed some unsupervised feature learning works which can learn informative features of multivariate time series. For example, two-dimensional singular value decomposition [15], variable-based principal component analysis (VPCA) [17], common principal component analysis (CPCA) [18–20], and deep encoder networks [21, 22] are used to reduce dimensionality of multivariate time series and learn informative features for clustering. He et al. [17] proposed a spatial weighted matrix distance-based fuzzy clustering (SWMDFC) model for multivariate time-series clustering. SWMDFC made use of VPCA to achieve dimensionality reduction and reduce computational consumption and then utilized spatial weighted matrix distance to compute distance among multivariate time-series data. Li [20] proposed a multivariate time-series clustering method based on CPCA, which used CPCA to construct projection coordinate features from multivariate time series and reconstructed multivariate time series from coordinate features. Franceschi et al. [21] proposed an unsupervised scalable representation learning model (USRL) for multivariate time series, which utilized a deep encoder network formed by dilated convolutions to generate informative features. However, noisy data and outliers are ubiquitous in realistic multivariate electricity consumption series data. Moreover, most existing works ignore the existence of outliers and noisy data, which can significantly affect clustering performance on multivariate electricity consumption series data.In this paper, we develop a novel robust graph factorization framework (RGF-MEC) based on multivariate electricity consumption series, which is a joint learning framework for multivariate electricity consumption series clustering and discovery of outliers in multivariate electricity consumption series. After calculating the similarity matrix among multivariate electricity consumption series data, RGF-MEC performs robust orthogonal and nonnegative matrix factorization on this similarity graph structure. In addition, it can directly reveal clustering results without using k-means clustering that is sensitive to initialization. Therefore, the main contributions of this paper can be summarized as follows:(1)
This paper proposes a novel multivariate electricity consumption series framework (RGF-MEC) to simultaneously factorize the graph structure and discover outliers in multivariate electricity consumption series data(2)
RGF-MEC utilizes robust matrix factorization to simultaneously learn nonnegative class-related representations and orthogonal spectral representations(3)
We perform experiments on realistic datasets to verify the proposed RGF-MEC model, and the results demonstrate that RGF-MEC is an effective multivariate electricity consumption series clustering frameworkThe remainder of this paper is organized as follows. Section2 introduces related works, including spectral clustering, symmetric nonnegative matrix factorization (SymNMF), and orthogonal and nonnegative matrix factorization (ONMF). Section 3 details the proposed RGF-MEC framework, including specific derivation process and complexity analysis. In Section 4, experimental results show the feasibility of the proposed RGF-MEC framework. Finally, some conclusions are given in Section 5.


## 2. Related Works

This section first explains the relationship between spectral clustering and symmetric nonnegative matrix factorization and then introduces the ONMF model.SupposeX denotes original instances, Xi. denotes the ith instance, L denotes normalized Laplacian matrix of original instances, and F∈ℝN×K denotes spectral embedding. Spectral clustering decomposes eigenvalues of normalized Laplacian matrix L and constructs clusters by processing eigenvectors [23], which has the advantage of clustering on a sample space of arbitrary shape and converging to an optimal solution. Therefore, the objective function of spectral clustering can be defined as(1)minFTD−1F=ITrFTLF,where L=I−D−1/2SD−1/2, S denotes similarity graph of original instances where an element Sij is similarity between Xi. and Xj., D is the degree matrix with a diagonal element Di=∑jSij, and clustering results are obtained by performing k-means clustering on the embedding D−1/2F.Kuang et al. [24] proposed SymNMF on the basis of nonnegative matrix factorization, which can simplify the high-dimensional graph structure to low-dimensional embedding while keeping the information as unchanged as possible. The objective function of SymNMF is defined as(2)minF≥0G−FFTF2,where G denotes the symmetric high-dimensional data. Relaxing F to be orthogonal, the above problem becomes the problem of spectral clustering.Since the solution of (2) is difficult to compute, Han et al. [25] first transformed the problem of spectral clustering into the problem of symmetric nonnegative matrix factorization,(3)1⇔maxFTF=ITrFTGF⇔minFTF=I−TrGTFFT+FFTG⇔minFTF=ITrGTG−GTFFT−FFTG+FFTFFT⇔minFTF=IG−FFTF2,and then introduced an auxiliary nonnegative variate H to approximate the orthogonal variate F:(4)minH≥0,FTF=IG−HFTF2,where H is a nonnegative embedding. With orthogonal and nonnegative constraints, the reconstructed graph of ONMF naturally has clear structure about clusters. It can be seen that ONMF can simplify the calculation by orthogonalizing F while ensuring clustering accuracy.


## 3. Proposed Frameworks

In this paper, a robust graph factorization framework (RGF-MEC) is proposed for multivariate electricity consumption clustering, in which graph factorization and outlier discovery are performed at the same time.A multivariate electricity consumption series datasetMEii=1N is given, where MEi∈ℝM×D, M denotes the number of sequences in MEi, and D denotes the length of each sequence in MEi. RGF-MEC must first calculate the similarity graph structure GME of MEii=1N. If MEj belongs to k-nearest neighbors of MEi, an element GijME of GME is equal to(5)GijME=dMEi,ME′k+1−dMEi,MEj∑j′=1kdMEi,ME′k+1−dMEi,ME′j′,where dMEi,MEj=∑m=1MMEm.i−MEm.jF2 and ME′k+1 is the k+1th nearest neighbor of MEi, otherwise an element GijME is equal to zero. It is known that there are many works that can reduce dimensionality of multivariate time series, such as CPCA [20], VPCA [17], and deep encoder networks [21]. RGF-MEC can reduce dimensionality of the data to obtain more effective features before calculating the graph structure. However, in order to verify robustness of the proposed model, this paper does not perform dimensionality reduction operation.RGF-MEC takes outliers into consideration while performing matrix factorization on the similarity graphGME. The objective function of RGF-MEC can be defined as(6)minHME≥0,FTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFTF2+λ2∑i,jHi.ME−Hj.MEF2Gij,where HME is the nonnegative class-related embedding, F is the orthogonal spectral embedding, λ is a regularization parameter, RGn=Gn.ME−Hn.MEFTF2 denotes the reconstruction error of nth multivariate electricity consumption series, and ε is a parameter to filter outliers (where ε is determined according to the reconstruction error Gn.ME−Hn.MEFTF2n=1N and a hypothetical ratio r of outliers, and we need to sort Gn.ME−Hn.MEFTF2n=1N from large to small and assign reconstruction error ranked in r to the parameter ε). If RGn≤ε , 1RGn≤ε=1, otherwise 1RGn≤ε=0. It can be seen that the objective function of RGF-MEC consists of two parts, the robust graph factorization term and the graph regularization term. In other words, HME also satisfies the graph constraint of MEii=1N.In (6), RGF-MEC utilizes a squared F-norm to constrain robust graph factorization term, named RGF-MEC with squared F-norm (RGF-MECF2). At present, researchers have proved that ℓ2,1-norm nonnegative matrix factorization is effective and has a mathematical meaning [26–28]. Therefore, we also present RGF-MEC with ℓ2,1-norm (RGF-MECℓ2,1). The objective function of RGF-MECℓ2,1 can be defined as(7)minHME≥0,FTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFT2,1+λ2∑i,jHi.ME−Hj.MEF2Gij.It can be seen that the main difference between RGF-MECF2 and RGF-MECℓ2,1 is a norm of robust graph factorization term. We provide the learning procedure in the following two sections and then give a convergence analysis of RGF-MEC.


### 3.1. Learning Procedure of RGF-MECF2

RGF-MECF2 can make use of the coordinate descent method to solve problem (6), where two embeddings HMEandF are updated in turn. In addition, a detailed learning procedure of RGF-MECF2 is provided in Algorithm 1.Algorithm 1:Learning procedure of RGF-MECF2
Input: a set of a multivariate electricity consumptionseries datasetMEii=1N, the parameters λ,r.Output: the class-related embedding HME.1: Calculate the similarity graph structureGME of MEii=1N based on (5).2: InitializeF, HME, and ε in turn.3:while Not convergent do % HME is updated based on (18) in RGF-MECℓ2,1.4:  FixF and calculate HME based on (10). % F is updated based on (21) in RGF-MECℓ2,1.5:  FixHME and calculate F based on (13).6:  Calculateε based on the parameter r and reconstruction errors Gn.ME−Hn.MEFTF2n=1N.7:end while8:returnHME.


#### 3.1.1. FixF and Update HME

IfF is fixed, problem (6) will become(8)minHME≥0∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFTF2+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting the derivative of problem (8) equal to zeros, we have the following equation:(9)2HME−ΛεGMEF+4λLMEHME=0,where Λε is a diagonal matrix with nth diagonal element 1RGn≤ε and LME is the Laplacian matrix of GME. Therefore, the solution of problem (8) is(10)HME=max0,Λε+2λLME−1ΛεGMEF.


#### 3.1.2. FixHME and Update F

IfHME is fixed, problem (6) will become(11)minFTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFTF2.Due to1RGn≤ε∈0,1 and FTF=I, problem (11) can be written as(12)11⇔minFTF=IΛε1/2GME−HMEFTF2⇔minFTF=IΛεGME−HMEFTF2⇔minFTF=ITrGMETΛεHMEFT,which is the standard orthogonal procrustes problem. Defining the singular value decomposition of GMETΛεHME as USVT, the solution of problem (11) is obtained:(13)F=UVT.


### 3.2. Learning Procedure of RGF-MECℓ2,1

Similar to RGF-MECF2, RGF-MECℓ2,1 can make use of the coordinate descent method to solve problem (7), too.


#### 3.2.1. FixF and Update HME

IfF is fixed, problem (7) will become(14)minHME≥0∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFT2,1+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting the derivative of problem (14) equal to zeros, we have the following equation:(15)∑n=1N1RGn≤ε⋅an∂Gn.ME−Hn.MEFTF2∂Hn.ME+4λLMEHME=0,where an=1/2Gn.ME−Hn.MEFTF. If an is stationary, problem (14) can be written as(16)minHME≥0∑n=1Nan1RGn≤ε⋅Gn.ME−Hn.MEFTF2+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting derivatives of problem (16) equal to zeros, we have the following equation:(17)2HME−ΛεaGMEF+4λLMEHME=0,where Λaε is a diagonal matrix with nth diagonal element an1RGn≤ε.Therefore, a solution to problem (14) is(18)HME=max0,Λaε+2λLME−1ΛaεGMEF.


#### 3.2.2. FixHME and Update F

IfHME is fixed, problem (7) will become(19)minFTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFT2,1.Similarly, problem (19) can be written as(20)19⇔∑n=1Nan1RGn≤ε⋅Gn.ME−Hn.MEFTF2⇔minFTF=IΛaε1/2GME−HMEFTF2⇔minFTF=ITrGMETΛaεHMEFT,which is the standard orthogonal procrustes problem. Defining the singular value decomposition of GMETΛaεHME as U′S′V′T, a solution to problem (19) is obtained:(21)F=U′V′T.


### 3.3. Convergence Analysis

In RGF-MECF2, two embeddings HMEandF are updated in turn. When updating HMEandF, we know that (10) (i.e., the calculation of HME) is the local optimal solution of (8) and (13) (i.e., the calculation of F) is the local optimal solution of (11). Therefore, RGF-MECF2 converges to its local optimal solution.In RGF-MECℓ2,1, two embeddings HMEandF are updated in turn, too. It can be seen that (18) (i.e., the calculation of HME) is the local optimal solution of (16) instead of (14).Theorem 1.
If (18) (i.e., the calculation of HME) is a solution to (14), the objective value of (14) will not increase.Proof.
Supposeen=Gn.ME−Hn.MEFTF2 and an=1/en. In RGF-MECℓ2,1, the minimizing problem of en in (14) is transformed into the minimizing problem of anen in (16). Suppose that an is stationary and gH=λ/2∑i,jHi.ME−Hj.MEF2Gij, (18) is a local optimal solution of (16). That is,(22)∑nantent+11RGn≤ε+gHt+1≤∑nantent1RGn≤ε+gHt,⇔∑nent+11RGn≤ε+gHt+1≤∑nent1RGn≤ε+gHt⇔∑nent+11RGn≤ε+gHt+1≤∑nent1RGn≤ε+gHt.
Therefore, the objective value of (14) will not increase and Theorem 1 is proven.
It can be also seen that (21) (i.e., the calculation of F) is a local optimal solution of (19). In other words, RGF-MECℓ2,1 converges to its local optimal solution, too.


## 3.1. Learning Procedure of RGF-MECF2

RGF-MECF2 can make use of the coordinate descent method to solve problem (6), where two embeddings HMEandF are updated in turn. In addition, a detailed learning procedure of RGF-MECF2 is provided in Algorithm 1.Algorithm 1:Learning procedure of RGF-MECF2
Input: a set of a multivariate electricity consumptionseries datasetMEii=1N, the parameters λ,r.Output: the class-related embedding HME.1: Calculate the similarity graph structureGME of MEii=1N based on (5).2: InitializeF, HME, and ε in turn.3:while Not convergent do % HME is updated based on (18) in RGF-MECℓ2,1.4:  FixF and calculate HME based on (10). % F is updated based on (21) in RGF-MECℓ2,1.5:  FixHME and calculate F based on (13).6:  Calculateε based on the parameter r and reconstruction errors Gn.ME−Hn.MEFTF2n=1N.7:end while8:returnHME.


### 3.1.1. FixF and Update HME

IfF is fixed, problem (6) will become(8)minHME≥0∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFTF2+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting the derivative of problem (8) equal to zeros, we have the following equation:(9)2HME−ΛεGMEF+4λLMEHME=0,where Λε is a diagonal matrix with nth diagonal element 1RGn≤ε and LME is the Laplacian matrix of GME. Therefore, the solution of problem (8) is(10)HME=max0,Λε+2λLME−1ΛεGMEF.


### 3.1.2. FixHME and Update F

IfHME is fixed, problem (6) will become(11)minFTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFTF2.Due to1RGn≤ε∈0,1 and FTF=I, problem (11) can be written as(12)11⇔minFTF=IΛε1/2GME−HMEFTF2⇔minFTF=IΛεGME−HMEFTF2⇔minFTF=ITrGMETΛεHMEFT,which is the standard orthogonal procrustes problem. Defining the singular value decomposition of GMETΛεHME as USVT, the solution of problem (11) is obtained:(13)F=UVT.


## 3.1.1. FixF and Update HME

IfF is fixed, problem (6) will become(8)minHME≥0∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFTF2+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting the derivative of problem (8) equal to zeros, we have the following equation:(9)2HME−ΛεGMEF+4λLMEHME=0,where Λε is a diagonal matrix with nth diagonal element 1RGn≤ε and LME is the Laplacian matrix of GME. Therefore, the solution of problem (8) is(10)HME=max0,Λε+2λLME−1ΛεGMEF.


## 3.1.2. FixHME and Update F

IfHME is fixed, problem (6) will become(11)minFTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFTF2.Due to1RGn≤ε∈0,1 and FTF=I, problem (11) can be written as(12)11⇔minFTF=IΛε1/2GME−HMEFTF2⇔minFTF=IΛεGME−HMEFTF2⇔minFTF=ITrGMETΛεHMEFT,which is the standard orthogonal procrustes problem. Defining the singular value decomposition of GMETΛεHME as USVT, the solution of problem (11) is obtained:(13)F=UVT.


## 3.2. Learning Procedure of RGF-MECℓ2,1

Similar to RGF-MECF2, RGF-MECℓ2,1 can make use of the coordinate descent method to solve problem (7), too.


### 3.2.1. FixF and Update HME

IfF is fixed, problem (7) will become(14)minHME≥0∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFT2,1+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting the derivative of problem (14) equal to zeros, we have the following equation:(15)∑n=1N1RGn≤ε⋅an∂Gn.ME−Hn.MEFTF2∂Hn.ME+4λLMEHME=0,where an=1/2Gn.ME−Hn.MEFTF. If an is stationary, problem (14) can be written as(16)minHME≥0∑n=1Nan1RGn≤ε⋅Gn.ME−Hn.MEFTF2+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting derivatives of problem (16) equal to zeros, we have the following equation:(17)2HME−ΛεaGMEF+4λLMEHME=0,where Λaε is a diagonal matrix with nth diagonal element an1RGn≤ε.Therefore, a solution to problem (14) is(18)HME=max0,Λaε+2λLME−1ΛaεGMEF.


### 3.2.2. FixHME and Update F

IfHME is fixed, problem (7) will become(19)minFTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFT2,1.Similarly, problem (19) can be written as(20)19⇔∑n=1Nan1RGn≤ε⋅Gn.ME−Hn.MEFTF2⇔minFTF=IΛaε1/2GME−HMEFTF2⇔minFTF=ITrGMETΛaεHMEFT,which is the standard orthogonal procrustes problem. Defining the singular value decomposition of GMETΛaεHME as U′S′V′T, a solution to problem (19) is obtained:(21)F=U′V′T.


## 3.2.1. FixF and Update HME

IfF is fixed, problem (7) will become(14)minHME≥0∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFT2,1+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting the derivative of problem (14) equal to zeros, we have the following equation:(15)∑n=1N1RGn≤ε⋅an∂Gn.ME−Hn.MEFTF2∂Hn.ME+4λLMEHME=0,where an=1/2Gn.ME−Hn.MEFTF. If an is stationary, problem (14) can be written as(16)minHME≥0∑n=1Nan1RGn≤ε⋅Gn.ME−Hn.MEFTF2+λ2∑i,jHi.ME−Hj.MEF2Gij.Setting derivatives of problem (16) equal to zeros, we have the following equation:(17)2HME−ΛεaGMEF+4λLMEHME=0,where Λaε is a diagonal matrix with nth diagonal element an1RGn≤ε.Therefore, a solution to problem (14) is(18)HME=max0,Λaε+2λLME−1ΛaεGMEF.


## 3.2.2. FixHME and Update F

IfHME is fixed, problem (7) will become(19)minFTF=I∑n=1N1RGn≤ε⋅Gn.ME−Hn.MEFT2,1.Similarly, problem (19) can be written as(20)19⇔∑n=1Nan1RGn≤ε⋅Gn.ME−Hn.MEFTF2⇔minFTF=IΛaε1/2GME−HMEFTF2⇔minFTF=ITrGMETΛaεHMEFT,which is the standard orthogonal procrustes problem. Defining the singular value decomposition of GMETΛaεHME as U′S′V′T, a solution to problem (19) is obtained:(21)F=U′V′T.


## 3.3. Convergence Analysis

In RGF-MECF2, two embeddings HMEandF are updated in turn. When updating HMEandF, we know that (10) (i.e., the calculation of HME) is the local optimal solution of (8) and (13) (i.e., the calculation of F) is the local optimal solution of (11). Therefore, RGF-MECF2 converges to its local optimal solution.In RGF-MECℓ2,1, two embeddings HMEandF are updated in turn, too. It can be seen that (18) (i.e., the calculation of HME) is the local optimal solution of (16) instead of (14).Theorem 1.
If (18) (i.e., the calculation of HME) is a solution to (14), the objective value of (14) will not increase.Proof.
Supposeen=Gn.ME−Hn.MEFTF2 and an=1/en. In RGF-MECℓ2,1, the minimizing problem of en in (14) is transformed into the minimizing problem of anen in (16). Suppose that an is stationary and gH=λ/2∑i,jHi.ME−Hj.MEF2Gij, (18) is a local optimal solution of (16). That is,(22)∑nantent+11RGn≤ε+gHt+1≤∑nantent1RGn≤ε+gHt,⇔∑nent+11RGn≤ε+gHt+1≤∑nent1RGn≤ε+gHt⇔∑nent+11RGn≤ε+gHt+1≤∑nent1RGn≤ε+gHt.
Therefore, the objective value of (14) will not increase and Theorem 1 is proven.
It can be also seen that (21) (i.e., the calculation of F) is a local optimal solution of (19). In other words, RGF-MECℓ2,1 converges to its local optimal solution, too.


## 4. Experiments

In this section, experimental results on multiple multivariate time-series datasets and a multivariate electricity consumption series dataset are used to validate the effectiveness of the proposed RGF-MEC framework.


### 4.1. Experiments on Multiple Multivariate Time-Series Datasets

#### 4.1.1. Multivariate Time-Series Datasets

Multiple realistic multivariate time-series datasets are used to evaluate the proposed RGF-MEC framework, including ArticularyWordRecognition [29], BasicMotions [30], Epilepsy [31], ERing [32], Libras [33], NATOPS [34], StandWalkJump [35], and UWaveGestureLibrary [36]. Among these multivariate time-series datasets, the maximum number of sequences is up to 24, the maximum length of these sequences is up to 2500, the maximum size of instances is 575, and the maximum number of classes is up to 25.


#### 4.1.2. Contrast Algorithms

We compare RGF-MEC with seven representative methods, including m-kShape, m-kDBA, m-kAVG + ED, m-KSC, multiview clustering with adaptive neighbors (MLAN) [37], and multiview spectral clustering via integrating nonnegative embedding and spectral embedding (NESE) [38]. m-kShape, m-kDBA, m-kAVG + ED, and m-KSC are effective distance-based multivariate time-series clustering algorithms [16], which consider the relationship among multiple series and extend traditional univariate time-series clustering methods (i.e., k-shape, KDBA, kAVG + ED, and KSC) for multivariate time-series clustering. MLAN and NESE are multiview clustering algorithms, where each series of a multivariate time series can be seen as a view.In RGF-MEC,λ is selected from 10−5,10−4,10−3,10−2,101,100,101,102,103, and ε is determined according to the reconstruction error and a parameter r that is a hypothetical ratio of outliers in the entire data. r is selected from 0%,1%,2%,3%,5%,10%,15%,20%. In addition, parameters of these contrast algorithms are determined according to descriptions in their original papers. We utilize two widely used measurements (i.e., rand index (RI) and normalized mutual information (NMI)) for evaluation. For RI and NMI, a larger value indicates a better result.


#### 4.1.3. Performance Comparison

We first compare the performance of RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets. Performance comparisons between RGF-MEC and other contrast algorithms are reported in Tables1 and 2, where best results are highlighted in bold. It can be seen that RGF-MEC outperforms other contrast algorithms on most real-world multivariate time-series datasets, where RGF-MEC achieves five best performances on eight real-world datasets in terms of RI or NMI. We can also make the following conclusions from Tables 1 and 2. (1) RGF-MEC is clearly better than other contrast algorithms in terms of “Mean±Std,” which demonstrates that RGF-MEC is an effective multivariate time-series clustering method. (2) The performance of NESE is best on the epilepsy dataset, which indicates that each series contains complementary information to each other. If each series is sufficient to represent the similarity between multivariate time series, then the multiview model can make full use of information of each series and achieve good performance.Table 1
Performance comparisons between RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets in terms of RI.
Datasetsm-kAVG + EDm-kDBAm-kShapem-kSCMLANNESERGF-MECArticularyWordRecognition0.95220.93360.75820.95100.98990.97560.9960BasicMotions0.77180.74870.52440.77180.76280.74490.8090Epilepsy0.76840.77710.51360.60440.73300.88970.7269ERing0.80460.77470.77010.74940.74710.75400.7747Libras0.91110.91330.66050.92270.91610.90870.9255NATOPS0.85250.87550.65340.83480.79920.76370.8631StandWalkJump0.73330.69520.34850.65710.70480.64760.7429UWaveGestureLibrary0.92040.89340.80150.92590.94340.85530.9524Mean± std0.8393± 0.07640.8264± 0.08240.6288± 0.14620.8021± 0.12090.8245± 0.10190.8174± 0.10060.8488 ± 0.0946Table 2
Performance comparisons between RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets in terms of NMI.
Datasetsm-kAVG + EDm-kDBAm-kShapem-kSCMLANNESERGF-MECArticularyWordRecognition0.83360.74090.34350.79220.94350.84860.9762BasicMotions0.54320.63880.34050.55390.56480.52500.6592Epilepsy0.40890.47070.16320.38110.53180.76000.3961ERing0.40030.40580.26830.34780.31850.37780.4040Libras0.62200.62240.44740.72440.61410.54180.6724NATOPS0.64300.64270.33920.59980.82470.31390.7755StandWalkJump0.55870.46580.11630.46080.55450.39850.5555UWaveGestureLibrary0.71340.58240.41940.75820.83850.55880.8395Mean± std0.5904± 0.13710.5712± 0.10590.3047± 0.10860.5773± 0.16060.6488± 0.19160.5405± 0.17380.6598 ± 0.1910We then compare the performance of RGF-MECF2 and RGF-MECℓ2,1 on real-world multivariate time-series datasets. Table 3 gives performance comparisons between RGF-MECF2 and RGF-MECℓ2,1. It can be seen that RGF-MECℓ2,1 is clearly better than RGF-MECF2 in terms of “Mean±Std.” In other words, RGF-MECℓ2,1 achieves mean improvements of 0.74% RI and 2.31% NMI compared to RGF-MECF2. Compared with RGF-MECF2, RGF-MECℓ2,1 needs to compute weights an=1/2Gn.ME−Hn.MEFTFn=1N when updating the nonnegative class-related embedding HME, where weights ann=1N can be considered as parameters that penalize larger reconstruction errors. This may make RGF-MECℓ2,1 more robust on most multivariate time-series datasets.Table 3
Performance comparisons between RGF-MECF2 and RGF-MECℓ2,1 on real-world multivariate time-series datasets.
DatasetsRINMIRGF-MECF2RGF-MECℓ2,1RGF-MECF2RGF-MECℓ2,1ArticularyWordRecognition0.99600.99600.97620.9762BasicMotions0.79100.80900.65920.6112Epilepsy0.72690.72520.38560.3961ERing0.77470.77470.40400.4040Libras0.92550.92410.67240.6651NATOPS0.86310.86160.71550.7755StandWalkJump0.70480.74290.44890.5555UWaveGestureLibrary0.95240.95070.83950.8359Mean±Std0.8418± 0.10160.8480± 0.09450.6377± 0.19900.6524± 0.1912We also perform ablation studies on RGF-MEC. The objective function of RGF-MEC consists of two parts, the robust graph factorization term and the graph regularization term. Table4 gives the performance comparisons between the ablation model and RGF-MEC, where AblationF2 denotes RGF-MECF2 without the graph regularization term and Ablationℓ2,1 denotes RGF-MECℓ2,1 without the graph regularization term. It can be seen that RGF-MECℓ2,1 (resp. RGF-MECF2) outperforms Ablationℓ2,1 (resp. AblationF2) on most multivariate time-series datasets, which prove that the graph regularization term enforces the nonnegative embedding HME to satisfy the graph constraint and retain more class-related information.Table 4
Performance comparisons between ablation models and RGF-MEC on real-world multivariate time-series datasets.
DatasetsRINMIAblationF2RGF-MECF2Ablationℓ2,1RGF-MECℓ2,1AblationF2RGF-MECF2Ablationℓ2,1RGF-MECℓ2,1ArticularyWordRecognition0.99600.99600.99600.99600.97620.97620.97620.9762BasicMotions0.77820.79100.78850.80900.58320.65920.59610.6112Epilepsy0.72690.72690.70740.72520.38560.38560.32480.3961ERing0.77470.77470.77240.77470.40400.40400.38020.4040Libras0.92400.92550.92220.92410.67240.67240.65750.6651NATOPS0.86240.86310.85670.86160.71470.71550.70550.7755StandWalkJump0.70480.70480.74290.74290.44890.44890.55550.5555UWaveGestureLibrary0.95240.95240.94040.95070.83680.83950.80310.8359Finally, we analyze the impact of parametersrandλ on the algorithm performance. Figure 1 shows the performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as r increases. It can be seen that RGF-MECℓ2,1 takes outliers into consideration and achieves better clustering results on four multivariate time-series datasets. Figure 2 shows the performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as λ increases. It can be seen that RGF-MECℓ2,1 takes the graph constraint of the nonnegative embedding (i.e., λ/2∑i,jHi.ME−Hj.MEF2Gij) into consideration and achieves better clustering results on most multivariate time-series datasets, which is consistent with the results of the ablation study. Next, as shown in Figure 3, we use t-SNE [39] to visualize the class-related embedding HME of RGF-MECℓ2,1 on UWaveGestureLibrary. It can be seen that some classes have obvious outliers, such as class-4, class-6, and class-8. RGF-MECℓ2,1 takes outliers into consideration and performs better than other contrast algorithms.Figure 1
Performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as r increases.Figure 2
Performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as λ increases, where algorithms with NMI less than 50% cannot be displayed in this figure.Figure 3
Visualization of the class-related embeddingHME of RGF-MECℓ2,1 on UWaveGestureLibrary.


### 4.2. Experiments on Multivariate Electricity Consumption Series Dataset

#### 4.2.1. Multivariate Electricity Consumption Series Dataset

A multivariate electricity consumption series dataset of a region in China Southern Power Grid [5], denoted as the CSPG dataset, is used to evaluate the proposed RGF-MEC framework. CSPG contains four series, global-active power (household global minute-averaged active power), global-reactive power (household global minute-averaged reactive power), voltage (minute-averaged voltage), and global intensity (household global minute-averaged current intensity). In addition, it contains multivariate series for two different time periods, where the length of each series is 180 and the number of instances is 2780.


#### 4.2.2. Performance Comparison

Since the effectiveness of RGF-MEC has been verified in Section4-4.1, we first give results obtained by RGF-MEC in different series combinations on CSPG. Table 5 shows performance comparisons between the combination containing four series and combinations containing arbitrary two series. Compared with the combinations containing arbitrary two series, RGF-MEC achieves best results on the combination containing four series. Next, we analyze the impact of parameters r,λ on CSPG. Figure 4 shows the performance (i.e., RI and NMI) change process of RGF-MEC on CSPG as r increases. It can be seen that RGF-MEC takes outliers into consideration and achieves better clustering results on CSPG. Figure 5 shows the performance (i.e., RI and NMI) change process of RGF-MEC on CSPG as λ increases. It can be seen that RGF-MEC takes the graph constraint of the nonnegative embedding (i.e., λ/2∑i,jHi.ME−Hj.MEF2Gij) into consideration and achieves better clustering results on CSPG.Table 5
Performance comparisons among different series combinations on the CSPG dataset.
Different series combinationsRINMIGlobal-active powerGlobal-reactive powerVoltageGlobal intensityRGF-MECF2RGF-MECℓ2,1RGF-MECF2RGF-MECℓ2,1✓✓✗✗0.73940.73940.42200.4158✓✗✓✗0.73940.74850.39740.4218✓✗✗✓0.65540.65080.34520.3290✗✓✓✗0.64900.63510.24770.2400✓✓✗✓0.73490.73940.41250.4138✗✗✓✓0.74160.73490.40430.4153✓✓✓✓0.76240.76000.44620.4585Figure 4
Performance (i.e., RI and NMI) change process of RGF-MEC on the CSPG dataset asr increases.Figure 5
Performance (i.e., RI and NMI) change process of RGF-MEC on the CSPG dataset asλ increases, where algorithms with NMI less than 30% cannot be displayed in this figure.


## 4.1. Experiments on Multiple Multivariate Time-Series Datasets

### 4.1.1. Multivariate Time-Series Datasets

Multiple realistic multivariate time-series datasets are used to evaluate the proposed RGF-MEC framework, including ArticularyWordRecognition [29], BasicMotions [30], Epilepsy [31], ERing [32], Libras [33], NATOPS [34], StandWalkJump [35], and UWaveGestureLibrary [36]. Among these multivariate time-series datasets, the maximum number of sequences is up to 24, the maximum length of these sequences is up to 2500, the maximum size of instances is 575, and the maximum number of classes is up to 25.


### 4.1.2. Contrast Algorithms

We compare RGF-MEC with seven representative methods, including m-kShape, m-kDBA, m-kAVG + ED, m-KSC, multiview clustering with adaptive neighbors (MLAN) [37], and multiview spectral clustering via integrating nonnegative embedding and spectral embedding (NESE) [38]. m-kShape, m-kDBA, m-kAVG + ED, and m-KSC are effective distance-based multivariate time-series clustering algorithms [16], which consider the relationship among multiple series and extend traditional univariate time-series clustering methods (i.e., k-shape, KDBA, kAVG + ED, and KSC) for multivariate time-series clustering. MLAN and NESE are multiview clustering algorithms, where each series of a multivariate time series can be seen as a view.In RGF-MEC,λ is selected from 10−5,10−4,10−3,10−2,101,100,101,102,103, and ε is determined according to the reconstruction error and a parameter r that is a hypothetical ratio of outliers in the entire data. r is selected from 0%,1%,2%,3%,5%,10%,15%,20%. In addition, parameters of these contrast algorithms are determined according to descriptions in their original papers. We utilize two widely used measurements (i.e., rand index (RI) and normalized mutual information (NMI)) for evaluation. For RI and NMI, a larger value indicates a better result.


### 4.1.3. Performance Comparison

We first compare the performance of RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets. Performance comparisons between RGF-MEC and other contrast algorithms are reported in Tables1 and 2, where best results are highlighted in bold. It can be seen that RGF-MEC outperforms other contrast algorithms on most real-world multivariate time-series datasets, where RGF-MEC achieves five best performances on eight real-world datasets in terms of RI or NMI. We can also make the following conclusions from Tables 1 and 2. (1) RGF-MEC is clearly better than other contrast algorithms in terms of “Mean±Std,” which demonstrates that RGF-MEC is an effective multivariate time-series clustering method. (2) The performance of NESE is best on the epilepsy dataset, which indicates that each series contains complementary information to each other. If each series is sufficient to represent the similarity between multivariate time series, then the multiview model can make full use of information of each series and achieve good performance.Table 1
Performance comparisons between RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets in terms of RI.
Datasetsm-kAVG + EDm-kDBAm-kShapem-kSCMLANNESERGF-MECArticularyWordRecognition0.95220.93360.75820.95100.98990.97560.9960BasicMotions0.77180.74870.52440.77180.76280.74490.8090Epilepsy0.76840.77710.51360.60440.73300.88970.7269ERing0.80460.77470.77010.74940.74710.75400.7747Libras0.91110.91330.66050.92270.91610.90870.9255NATOPS0.85250.87550.65340.83480.79920.76370.8631StandWalkJump0.73330.69520.34850.65710.70480.64760.7429UWaveGestureLibrary0.92040.89340.80150.92590.94340.85530.9524Mean± std0.8393± 0.07640.8264± 0.08240.6288± 0.14620.8021± 0.12090.8245± 0.10190.8174± 0.10060.8488 ± 0.0946Table 2
Performance comparisons between RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets in terms of NMI.
Datasetsm-kAVG + EDm-kDBAm-kShapem-kSCMLANNESERGF-MECArticularyWordRecognition0.83360.74090.34350.79220.94350.84860.9762BasicMotions0.54320.63880.34050.55390.56480.52500.6592Epilepsy0.40890.47070.16320.38110.53180.76000.3961ERing0.40030.40580.26830.34780.31850.37780.4040Libras0.62200.62240.44740.72440.61410.54180.6724NATOPS0.64300.64270.33920.59980.82470.31390.7755StandWalkJump0.55870.46580.11630.46080.55450.39850.5555UWaveGestureLibrary0.71340.58240.41940.75820.83850.55880.8395Mean± std0.5904± 0.13710.5712± 0.10590.3047± 0.10860.5773± 0.16060.6488± 0.19160.5405± 0.17380.6598 ± 0.1910We then compare the performance of RGF-MECF2 and RGF-MECℓ2,1 on real-world multivariate time-series datasets. Table 3 gives performance comparisons between RGF-MECF2 and RGF-MECℓ2,1. It can be seen that RGF-MECℓ2,1 is clearly better than RGF-MECF2 in terms of “Mean±Std.” In other words, RGF-MECℓ2,1 achieves mean improvements of 0.74% RI and 2.31% NMI compared to RGF-MECF2. Compared with RGF-MECF2, RGF-MECℓ2,1 needs to compute weights an=1/2Gn.ME−Hn.MEFTFn=1N when updating the nonnegative class-related embedding HME, where weights ann=1N can be considered as parameters that penalize larger reconstruction errors. This may make RGF-MECℓ2,1 more robust on most multivariate time-series datasets.Table 3
Performance comparisons between RGF-MECF2 and RGF-MECℓ2,1 on real-world multivariate time-series datasets.
DatasetsRINMIRGF-MECF2RGF-MECℓ2,1RGF-MECF2RGF-MECℓ2,1ArticularyWordRecognition0.99600.99600.97620.9762BasicMotions0.79100.80900.65920.6112Epilepsy0.72690.72520.38560.3961ERing0.77470.77470.40400.4040Libras0.92550.92410.67240.6651NATOPS0.86310.86160.71550.7755StandWalkJump0.70480.74290.44890.5555UWaveGestureLibrary0.95240.95070.83950.8359Mean±Std0.8418± 0.10160.8480± 0.09450.6377± 0.19900.6524± 0.1912We also perform ablation studies on RGF-MEC. The objective function of RGF-MEC consists of two parts, the robust graph factorization term and the graph regularization term. Table4 gives the performance comparisons between the ablation model and RGF-MEC, where AblationF2 denotes RGF-MECF2 without the graph regularization term and Ablationℓ2,1 denotes RGF-MECℓ2,1 without the graph regularization term. It can be seen that RGF-MECℓ2,1 (resp. RGF-MECF2) outperforms Ablationℓ2,1 (resp. AblationF2) on most multivariate time-series datasets, which prove that the graph regularization term enforces the nonnegative embedding HME to satisfy the graph constraint and retain more class-related information.Table 4
Performance comparisons between ablation models and RGF-MEC on real-world multivariate time-series datasets.
DatasetsRINMIAblationF2RGF-MECF2Ablationℓ2,1RGF-MECℓ2,1AblationF2RGF-MECF2Ablationℓ2,1RGF-MECℓ2,1ArticularyWordRecognition0.99600.99600.99600.99600.97620.97620.97620.9762BasicMotions0.77820.79100.78850.80900.58320.65920.59610.6112Epilepsy0.72690.72690.70740.72520.38560.38560.32480.3961ERing0.77470.77470.77240.77470.40400.40400.38020.4040Libras0.92400.92550.92220.92410.67240.67240.65750.6651NATOPS0.86240.86310.85670.86160.71470.71550.70550.7755StandWalkJump0.70480.70480.74290.74290.44890.44890.55550.5555UWaveGestureLibrary0.95240.95240.94040.95070.83680.83950.80310.8359Finally, we analyze the impact of parametersrandλ on the algorithm performance. Figure 1 shows the performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as r increases. It can be seen that RGF-MECℓ2,1 takes outliers into consideration and achieves better clustering results on four multivariate time-series datasets. Figure 2 shows the performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as λ increases. It can be seen that RGF-MECℓ2,1 takes the graph constraint of the nonnegative embedding (i.e., λ/2∑i,jHi.ME−Hj.MEF2Gij) into consideration and achieves better clustering results on most multivariate time-series datasets, which is consistent with the results of the ablation study. Next, as shown in Figure 3, we use t-SNE [39] to visualize the class-related embedding HME of RGF-MECℓ2,1 on UWaveGestureLibrary. It can be seen that some classes have obvious outliers, such as class-4, class-6, and class-8. RGF-MECℓ2,1 takes outliers into consideration and performs better than other contrast algorithms.Figure 1
Performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as r increases.Figure 2
Performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as λ increases, where algorithms with NMI less than 50% cannot be displayed in this figure.Figure 3
Visualization of the class-related embeddingHME of RGF-MECℓ2,1 on UWaveGestureLibrary.


## 4.1.1. Multivariate Time-Series Datasets

Multiple realistic multivariate time-series datasets are used to evaluate the proposed RGF-MEC framework, including ArticularyWordRecognition [29], BasicMotions [30], Epilepsy [31], ERing [32], Libras [33], NATOPS [34], StandWalkJump [35], and UWaveGestureLibrary [36]. Among these multivariate time-series datasets, the maximum number of sequences is up to 24, the maximum length of these sequences is up to 2500, the maximum size of instances is 575, and the maximum number of classes is up to 25.


## 4.1.2. Contrast Algorithms

We compare RGF-MEC with seven representative methods, including m-kShape, m-kDBA, m-kAVG + ED, m-KSC, multiview clustering with adaptive neighbors (MLAN) [37], and multiview spectral clustering via integrating nonnegative embedding and spectral embedding (NESE) [38]. m-kShape, m-kDBA, m-kAVG + ED, and m-KSC are effective distance-based multivariate time-series clustering algorithms [16], which consider the relationship among multiple series and extend traditional univariate time-series clustering methods (i.e., k-shape, KDBA, kAVG + ED, and KSC) for multivariate time-series clustering. MLAN and NESE are multiview clustering algorithms, where each series of a multivariate time series can be seen as a view.In RGF-MEC,λ is selected from 10−5,10−4,10−3,10−2,101,100,101,102,103, and ε is determined according to the reconstruction error and a parameter r that is a hypothetical ratio of outliers in the entire data. r is selected from 0%,1%,2%,3%,5%,10%,15%,20%. In addition, parameters of these contrast algorithms are determined according to descriptions in their original papers. We utilize two widely used measurements (i.e., rand index (RI) and normalized mutual information (NMI)) for evaluation. For RI and NMI, a larger value indicates a better result.


## 4.1.3. Performance Comparison

We first compare the performance of RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets. Performance comparisons between RGF-MEC and other contrast algorithms are reported in Tables1 and 2, where best results are highlighted in bold. It can be seen that RGF-MEC outperforms other contrast algorithms on most real-world multivariate time-series datasets, where RGF-MEC achieves five best performances on eight real-world datasets in terms of RI or NMI. We can also make the following conclusions from Tables 1 and 2. (1) RGF-MEC is clearly better than other contrast algorithms in terms of “Mean±Std,” which demonstrates that RGF-MEC is an effective multivariate time-series clustering method. (2) The performance of NESE is best on the epilepsy dataset, which indicates that each series contains complementary information to each other. If each series is sufficient to represent the similarity between multivariate time series, then the multiview model can make full use of information of each series and achieve good performance.Table 1
Performance comparisons between RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets in terms of RI.
Datasetsm-kAVG + EDm-kDBAm-kShapem-kSCMLANNESERGF-MECArticularyWordRecognition0.95220.93360.75820.95100.98990.97560.9960BasicMotions0.77180.74870.52440.77180.76280.74490.8090Epilepsy0.76840.77710.51360.60440.73300.88970.7269ERing0.80460.77470.77010.74940.74710.75400.7747Libras0.91110.91330.66050.92270.91610.90870.9255NATOPS0.85250.87550.65340.83480.79920.76370.8631StandWalkJump0.73330.69520.34850.65710.70480.64760.7429UWaveGestureLibrary0.92040.89340.80150.92590.94340.85530.9524Mean± std0.8393± 0.07640.8264± 0.08240.6288± 0.14620.8021± 0.12090.8245± 0.10190.8174± 0.10060.8488 ± 0.0946Table 2
Performance comparisons between RGF-MEC and other contrast algorithms on real-world multivariate time-series datasets in terms of NMI.
Datasetsm-kAVG + EDm-kDBAm-kShapem-kSCMLANNESERGF-MECArticularyWordRecognition0.83360.74090.34350.79220.94350.84860.9762BasicMotions0.54320.63880.34050.55390.56480.52500.6592Epilepsy0.40890.47070.16320.38110.53180.76000.3961ERing0.40030.40580.26830.34780.31850.37780.4040Libras0.62200.62240.44740.72440.61410.54180.6724NATOPS0.64300.64270.33920.59980.82470.31390.7755StandWalkJump0.55870.46580.11630.46080.55450.39850.5555UWaveGestureLibrary0.71340.58240.41940.75820.83850.55880.8395Mean± std0.5904± 0.13710.5712± 0.10590.3047± 0.10860.5773± 0.16060.6488± 0.19160.5405± 0.17380.6598 ± 0.1910We then compare the performance of RGF-MECF2 and RGF-MECℓ2,1 on real-world multivariate time-series datasets. Table 3 gives performance comparisons between RGF-MECF2 and RGF-MECℓ2,1. It can be seen that RGF-MECℓ2,1 is clearly better than RGF-MECF2 in terms of “Mean±Std.” In other words, RGF-MECℓ2,1 achieves mean improvements of 0.74% RI and 2.31% NMI compared to RGF-MECF2. Compared with RGF-MECF2, RGF-MECℓ2,1 needs to compute weights an=1/2Gn.ME−Hn.MEFTFn=1N when updating the nonnegative class-related embedding HME, where weights ann=1N can be considered as parameters that penalize larger reconstruction errors. This may make RGF-MECℓ2,1 more robust on most multivariate time-series datasets.Table 3
Performance comparisons between RGF-MECF2 and RGF-MECℓ2,1 on real-world multivariate time-series datasets.
DatasetsRINMIRGF-MECF2RGF-MECℓ2,1RGF-MECF2RGF-MECℓ2,1ArticularyWordRecognition0.99600.99600.97620.9762BasicMotions0.79100.80900.65920.6112Epilepsy0.72690.72520.38560.3961ERing0.77470.77470.40400.4040Libras0.92550.92410.67240.6651NATOPS0.86310.86160.71550.7755StandWalkJump0.70480.74290.44890.5555UWaveGestureLibrary0.95240.95070.83950.8359Mean±Std0.8418± 0.10160.8480± 0.09450.6377± 0.19900.6524± 0.1912We also perform ablation studies on RGF-MEC. The objective function of RGF-MEC consists of two parts, the robust graph factorization term and the graph regularization term. Table4 gives the performance comparisons between the ablation model and RGF-MEC, where AblationF2 denotes RGF-MECF2 without the graph regularization term and Ablationℓ2,1 denotes RGF-MECℓ2,1 without the graph regularization term. It can be seen that RGF-MECℓ2,1 (resp. RGF-MECF2) outperforms Ablationℓ2,1 (resp. AblationF2) on most multivariate time-series datasets, which prove that the graph regularization term enforces the nonnegative embedding HME to satisfy the graph constraint and retain more class-related information.Table 4
Performance comparisons between ablation models and RGF-MEC on real-world multivariate time-series datasets.
DatasetsRINMIAblationF2RGF-MECF2Ablationℓ2,1RGF-MECℓ2,1AblationF2RGF-MECF2Ablationℓ2,1RGF-MECℓ2,1ArticularyWordRecognition0.99600.99600.99600.99600.97620.97620.97620.9762BasicMotions0.77820.79100.78850.80900.58320.65920.59610.6112Epilepsy0.72690.72690.70740.72520.38560.38560.32480.3961ERing0.77470.77470.77240.77470.40400.40400.38020.4040Libras0.92400.92550.92220.92410.67240.67240.65750.6651NATOPS0.86240.86310.85670.86160.71470.71550.70550.7755StandWalkJump0.70480.70480.74290.74290.44890.44890.55550.5555UWaveGestureLibrary0.95240.95240.94040.95070.83680.83950.80310.8359Finally, we analyze the impact of parametersrandλ on the algorithm performance. Figure 1 shows the performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as r increases. It can be seen that RGF-MECℓ2,1 takes outliers into consideration and achieves better clustering results on four multivariate time-series datasets. Figure 2 shows the performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as λ increases. It can be seen that RGF-MECℓ2,1 takes the graph constraint of the nonnegative embedding (i.e., λ/2∑i,jHi.ME−Hj.MEF2Gij) into consideration and achieves better clustering results on most multivariate time-series datasets, which is consistent with the results of the ablation study. Next, as shown in Figure 3, we use t-SNE [39] to visualize the class-related embedding HME of RGF-MECℓ2,1 on UWaveGestureLibrary. It can be seen that some classes have obvious outliers, such as class-4, class-6, and class-8. RGF-MECℓ2,1 takes outliers into consideration and performs better than other contrast algorithms.Figure 1
Performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as r increases.Figure 2
Performance (i.e., NMI) change process of RGF-MECℓ2,1 on four multivariate time-series datasets as λ increases, where algorithms with NMI less than 50% cannot be displayed in this figure.Figure 3
Visualization of the class-related embeddingHME of RGF-MECℓ2,1 on UWaveGestureLibrary.


## 4.2. Experiments on Multivariate Electricity Consumption Series Dataset

### 4.2.1. Multivariate Electricity Consumption Series Dataset

A multivariate electricity consumption series dataset of a region in China Southern Power Grid [5], denoted as the CSPG dataset, is used to evaluate the proposed RGF-MEC framework. CSPG contains four series, global-active power (household global minute-averaged active power), global-reactive power (household global minute-averaged reactive power), voltage (minute-averaged voltage), and global intensity (household global minute-averaged current intensity). In addition, it contains multivariate series for two different time periods, where the length of each series is 180 and the number of instances is 2780.


### 4.2.2. Performance Comparison

Since the effectiveness of RGF-MEC has been verified in Section4-4.1, we first give results obtained by RGF-MEC in different series combinations on CSPG. Table 5 shows performance comparisons between the combination containing four series and combinations containing arbitrary two series. Compared with the combinations containing arbitrary two series, RGF-MEC achieves best results on the combination containing four series. Next, we analyze the impact of parameters r,λ on CSPG. Figure 4 shows the performance (i.e., RI and NMI) change process of RGF-MEC on CSPG as r increases. It can be seen that RGF-MEC takes outliers into consideration and achieves better clustering results on CSPG. Figure 5 shows the performance (i.e., RI and NMI) change process of RGF-MEC on CSPG as λ increases. It can be seen that RGF-MEC takes the graph constraint of the nonnegative embedding (i.e., λ/2∑i,jHi.ME−Hj.MEF2Gij) into consideration and achieves better clustering results on CSPG.Table 5
Performance comparisons among different series combinations on the CSPG dataset.
Different series combinationsRINMIGlobal-active powerGlobal-reactive powerVoltageGlobal intensityRGF-MECF2RGF-MECℓ2,1RGF-MECF2RGF-MECℓ2,1✓✓✗✗0.73940.73940.42200.4158✓✗✓✗0.73940.74850.39740.4218✓✗✗✓0.65540.65080.34520.3290✗✓✓✗0.64900.63510.24770.2400✓✓✗✓0.73490.73940.41250.4138✗✗✓✓0.74160.73490.40430.4153✓✓✓✓0.76240.76000.44620.4585Figure 4
Performance (i.e., RI and NMI) change process of RGF-MEC on the CSPG dataset asr increases.Figure 5
Performance (i.e., RI and NMI) change process of RGF-MEC on the CSPG dataset asλ increases, where algorithms with NMI less than 30% cannot be displayed in this figure.


## 4.2.1. Multivariate Electricity Consumption Series Dataset

A multivariate electricity consumption series dataset of a region in China Southern Power Grid [5], denoted as the CSPG dataset, is used to evaluate the proposed RGF-MEC framework. CSPG contains four series, global-active power (household global minute-averaged active power), global-reactive power (household global minute-averaged reactive power), voltage (minute-averaged voltage), and global intensity (household global minute-averaged current intensity). In addition, it contains multivariate series for two different time periods, where the length of each series is 180 and the number of instances is 2780.


## 4.2.2. Performance Comparison

Since the effectiveness of RGF-MEC has been verified in Section4-4.1, we first give results obtained by RGF-MEC in different series combinations on CSPG. Table 5 shows performance comparisons between the combination containing four series and combinations containing arbitrary two series. Compared with the combinations containing arbitrary two series, RGF-MEC achieves best results on the combination containing four series. Next, we analyze the impact of parameters r,λ on CSPG. Figure 4 shows the performance (i.e., RI and NMI) change process of RGF-MEC on CSPG as r increases. It can be seen that RGF-MEC takes outliers into consideration and achieves better clustering results on CSPG. Figure 5 shows the performance (i.e., RI and NMI) change process of RGF-MEC on CSPG as λ increases. It can be seen that RGF-MEC takes the graph constraint of the nonnegative embedding (i.e., λ/2∑i,jHi.ME−Hj.MEF2Gij) into consideration and achieves better clustering results on CSPG.Table 5
Performance comparisons among different series combinations on the CSPG dataset.
Different series combinationsRINMIGlobal-active powerGlobal-reactive powerVoltageGlobal intensityRGF-MECF2RGF-MECℓ2,1RGF-MECF2RGF-MECℓ2,1✓✓✗✗0.73940.73940.42200.4158✓✗✓✗0.73940.74850.39740.4218✓✗✗✓0.65540.65080.34520.3290✗✓✓✗0.64900.63510.24770.2400✓✓✗✓0.73490.73940.41250.4138✗✗✓✓0.74160.73490.40430.4153✓✓✓✓0.76240.76000.44620.4585Figure 4
Performance (i.e., RI and NMI) change process of RGF-MEC on the CSPG dataset asr increases.Figure 5
Performance (i.e., RI and NMI) change process of RGF-MEC on the CSPG dataset asλ increases, where algorithms with NMI less than 30% cannot be displayed in this figure.


## 5. Conclusion

This paper builts an effective robust graph factorization framework for multivariate electricity consumption clustering, which performs robust matrix factorization on the similarity of multivariate electricity consumption series and obtains nonnegative class-related representations. The proposed RGF-MEC framework also enforces the nonnegative embedding to satisfy a graph constraint and retain more class-related information. In addition, RGF-MEC utilizes the squared F-norm and theℓ2,1-norm to constrain the robust graph factorization term, named RGF-MECF2 and RGF-MECℓ2,1. Experimental results demonstrate that RGF-MECF2 and RGF-MECℓ2,1 achieve competitive results on multiple multivariate time series datasets and a multivariate electricity consumption series dataset.



---

*Source: 4310417-2021-08-26.xml*