# Anomaly Detection in High-Dimensional Time Series Data with Scaled Bregman Divergence
## 2 Materials and Methods
### Scaled Bregman Divergence-Based Dissimilarity Measure
Given the measure \(m_{\alpha}(x)=\alpha p_{tr}(x)+(1-\alpha)p_{te}(x)\), where \(\alpha\in(0,1)\), the Pearson-like scaled Bregman divergence between \(p_{tr}\) and \(p_{te}\) is defined as  
\[PL(p_{tr}\|p_{te};m_{\alpha}(x))=\frac{1}{2}\int m_{\alpha}(x)\bigg{(}\frac{p_{ tr}(x)}{m_{\alpha}(x)}-\frac{p_{te}(x)}{m_{\alpha}(x)}\bigg{)}^{2}dx,\]  
is bounded and can be expressed as [35]:  
\[PL(p_{tr}\|p_{te};m_{\alpha}(x))=\frac{1}{2}E_{p_{tr}}[w_{\alpha}(x)]-\frac{2- \alpha}{2(1-\alpha)}E_{p_{tr}}[w_{\alpha}(x)]+\frac{1}{2(1-\alpha)},\]  
where \(w_{\alpha}(x)=\frac{p_{tr}(x)}{m_{\alpha}(x)}\), and \(E_{p}[\cdot]\) denotes the expectation with respect to the density \(p(x)\). The detailed proof of Theorem 3 can be found in our recent work [35]. The PLsBD divergence \(PL(p_{tr}\|p_{te};m_{\alpha}(x))\) can be approximately computed by the empirical average:  
\[\widehat{PL}(p_{tr}\|p_{te};m_{\alpha}(x))\approx\frac{1}{2n_{tr}}\sum_{i=1}^{ n_{tr}}\psi(x_{i}^{tr})-\frac{2-\alpha}{2(1-\alpha)}\sum_{i=1}^{n_{tr}}\psi(x_{i}^ {te})+\frac{1}{2(1-\alpha)}. \tag{4}\]  
It is worth noting that when \(n=3\), it is straightforward to derive another, more complex divergence:  
\[PL(p_{tr}\|p_{te};m)=\frac{1}{3}\int_{\mathcal{X}}m(x)\bigg{(}\frac{p_{tr}(x)} {m(x)}-\frac{p_{te}(x)}{m(x)}\bigg{)}^{2}\bigg{(}\frac{p_{tr}(x)}{m(x)}+\frac{ 2p_{te}(x)}{m(x)}-3\bigg{)}dx.\]  
However, this divergence is not utilized in our work, not only due to our preference for the simpler form obtained when \(n=2\), but also because this divergence is not symmetric under all conditions. The \(\alpha\)-relative Pearson (\(\alpha\)-PE) divergence is another divergence commonly used to quantify the discrepancy between two distributions. By replacing \(p_{te}\) in the standard Pearson divergence with \(p_{te,A}(x)=\alpha p_{tr}(x)+(1-\alpha)p_{te}(x)\), which represents a mixture of the training and test densities, we obtain the \(\alpha\)-PE divergence, which can be viewed as a specific instance of the scaled Bregman divergence. This modification addresses the unboundedness problem associated with the standard Pearson divergence. It is defined as  
\[PE_{\alpha}(p_{tr}\|p_{te,A}) =\frac{1}{2}\int p_{te,A}(x)\bigg{(}\frac{p_{tr}(x)}{p_{te,A}(x)} -1\bigg{)}^{2}dx,\] \[=\frac{1}{2}E_{p_{tr}}(w_{\alpha}(x))-\frac{1}{2}.\]  
The \(\alpha\)-PE divergence \(PE_{\alpha}(p_{tr}\|p_{te,\alpha})\) can be approximately computed by the empirical average:  
\[\widehat{PE}(p_{tr}\|p_{te};p_{te,A}(x))\approx\frac{1}{2n_{tr}}\sum_{i=1}^{n _{tr}}\hat{w}(x_{i}^{tr})-\frac{1}{2}. \tag{5}\]  
RuLSIF (relative unconstrained least-squares importance fitting) applied the \(\alpha\)-PE divergence in previous studies for change-point detection to overcome the unboundedness issue, but has not been applied for the anomaly detection. However, \(\alpha\)-PE measures the dissimilarity between \(p_{tr}\) and the mixture distribution \(p_{te,A}\), rather than between two distinct distributions. Our work [35] has shown that the RuLSIF method is highly sensitive to the choice of \(\alpha\), with large \(\alpha\) values potentially introducing numerical instability. The PLsBD directly measures the disparity between two individual distributions, rather than a mixture, making it more robust and reliable in applications such as change-point detection. As shown by the empirical averages in Equations (4) and (5), PLsBD demonstrates greater effectiveness in anomaly detection by incorporating data from both training and test distributions for divergence estimation. This approach enhances its ability to detect subtle anomalies arising from shifts in the test data, albeit at the cost of increased computational complexity. In contrast, the \(\alpha\)-PE divergence relies predominantly on the training data, reducing its sensitivity to anomalies that emerge exclusively in the test distribution. In this work, we will apply both PLsBD and RuLSIF methods for anomaly detection and compare their performance with the uLSIF-based anomaly detection method.