## 3 Proposed Framework: PE-DOCC
### Periodicformer Encoder for Unsupervised Representation Learning
#### 3.1.3 Unsupervised Representation Learning  
Electricity consumption data are treated as a univariate time series of length \(T\), expressed as \(X=\{x_{1},...,x_{T}\}\). To determine whether the data point \(x_{t}\) at time \(t\) is an anomaly, we analyze a contextual window of length \(L\), defined as \(w_{t}=\{x_{t-L+1},...,x_{t}\}\), instead of focusing solely on a single point. The training set can be represented as \(X^{\prime}=\{w_{L},...,w_{T}\}\). For simplicity, windows \(w_{t}\) are omitted when \(t<L\)[30].  
Unsupervised representation learning involves two steps: creating a Boolean mask \(M\) and calculating the mean squared error (MSE) loss. The mask \(M\) is generated with alternating segments of \(0\) s and \(1\) s, where the lengths of the \(0\) s and \(1\) s follow geometric distributions with means \(l_{m}\) and \(l_{u}\), respectively. The proportion \(r\) determines the ratio of \(0\) s in the mask, and the length \(l_{u}\) is computed as \(l_{u}=\frac{1-r}{r}l_{m}\). The masked detection window \(\hat{w}_{t}\) is then passed through the encoder, producing feature representation \(Z\). The output \(Z\) is flattened and passed through a linear layer to forecast the masked values, yielding \(\widehat{w}_{t}\). The MSE loss is computed by comparing the predicted values \(\widehat{w}_{t}(i)\) to the actual masked values \(w_{t}(i)\) at the indices in \(M_{0}\), which are the positions where \(M(i)=0\), as Equations (7)-(9).  
\[\widehat{w}_{t}=M\odot w_{t} \tag{7}\]  
\[\widehat{w}_{t}=flatten(Z)W+b \tag{8}\]  
\[\mathcal{L}_{MSE}=\frac{1}{|B|}\sum_{t\in B}\sum_{i\in M_{0}}\left[\widehat{ w}_{t}(i)-w_{t}(i)\right]^{2} \tag{9}\]  
where \(\odot\) is the Hadamard product, \(B\) is a batch of training samples, and \(|B|\) represents the batch size. The overall process of unsupervised representation learning is described in detail in Algorithm 2.  
```
Input: \(L\): Length of mask and sequence to be masked, \(l_{\text{m}}\): Average length of masking subsequences (streaks of 0 s), \(r\): Proportion of \(L\) to be masked, \(X^{\prime}\): Training set composed of detection windows \(w_{t}\), \(\phi\): Mapping function of Periodicformer encoder, \(M\): A mask of length \(L\) with all values initialized to one, \(E\): Maximum number of epochs. Output: \(Z\): Latent representation of \(X^{\prime}\). Initialize: \(p_{m}\leftarrow\frac{1}{l_{\text{m}}}\) ; // Probability of each masking sequence stopping. Parameter of geometric distribution \(p_{u}\gets p_{m}\cdot\frac{r}{1-r}\) ; // Probability of each unmasked sequence stopping. Parameter of geometric distribution. \(p\leftarrow[p_{m},p_{u}]\); \(state\leftarrow 1(\text{rand}(>r)\) ; // State 0 means masking, 1 means not masking for epoch\(\gets 1\)to\(E\)do foreachbatch of windows\(w_{t}\)in\(X^{\prime}\)do Randomly create a boolean mask\(M\): for\(i\gets 1\)to\(L\)do \(M[i]\leftarrow state\) ifrand() \(<p[state]\)then \(state\gets 1-state\)  end if  end if Calculate MSE loss: \(\widehat{w}_{t}\gets M\odot w_{t}\) ; // Mask the input window \(Z\leftarrow\phi(\widehat{w}_{t})\); \(\widehat{w}_{t}\leftarrow\text{flatten}(Z)W+b\); \(\mathcal{L}_{MSE}\leftarrow\frac{1}{|B|}\sum_{t\in B}\sum_{i\in M_{0}}\left[ \widehat{w}_{t}(i)-w_{t}(i)\right]^{2}\) ; // Compute MSE loss over masked elements Make a gradient step to minimize \(\mathcal{L}_{MSE}\).  end if end foreturn\(Z\leftarrow\phi(W)\)
```  
**Algorithm 2**Unsupervised representation learning  
The algorithm is based on the following two steps:  
* \(\widehat{w}_{  
Note that the objective of the unsupervised representation learning is to learn latent representations \(\mathrm{Z}\) for subsequent one-class classification-based anomaly detection. In the subsequent anomaly detection process, the linear layer originally used for outputting the masked time series is discarded.