## 3 Methods
### Overview
 \tag{8}\]

where \(W_{Q},W_{K}\), and \(W_{V}\in R^{d_{model}\times d_{model}}\) represent parameter matrices for the scientific department, \(X_{in}\in R^{L\times d_{model}}\), \(Q\in R^{L\times d_{model}}\) represents the query matrix, \(K\in R^{L\times d_{model}}\) represents the key matrix, and \(V\in R^{L\times d_{model}}\) represents the value matrix. To compute the attention output, the following steps should be followed: Given the Q, the K, the V, and the scaling factor \(d_{model}\), the computation process is as follows. First, compute the dot product between the Q and the transpose of the K to generate similarity scores between each Q and K. To prevent issues with large numerical values leading to gradient vanishing, scale the dot product results by dividing by the \(d_{model}\). Next, apply the Softmax function to the scaled results for normalization, converting the similarity scores into weights that reflect the importance of each K to the corresponding Q. Finally, use these weights to perform a weighted sum of the V to obtain the final attention output. This process can be represented by Formula (9):

\[Attention(Q,K,V)=Softmax(\frac{QK^{T}}{\sqrt{d_{model}}})V \tag{9}\]

This study employs a multi-head self-attention mechanism, as shown in Figure 2c. It demonstrates excellent performance in feature extraction while maintaining the same number of parameters. Here, \"self-attention\" refers to linearly projecting queries (Q), keys (K), and values (V) from the same input matrix \(X_{in}\in R^{L\times d_{model}}\). The concept of \"multi-headed\" implies splitting the output into \(h\)-independent blocks (referred to as heads) along the channel dimension, with each block employing different projection weights. Subsequently, the diverse outputs from individual single-headed self-attention mechanisms are concatenated along the channel dimension. Introducing this multi-headed self-attention mechanism is of significant importance for enhancing feature extraction capabilities and capturing complex relationships within the input data. This process can be described by Equation (10). \[ \begin{split} head_{i}=Attention(Q_{i},\ \ K_{i},\ \ V_{i})\\ MHSA(X_{in},X_{in},X_{in})=Concat(head_{1},head_{2},\dots,head_{h })w^{0}\end{split}  \tag{10}\]

where \(Q_{i},K_{i}\), and \(V_{i}\in\mathbb{R}^{L\times\frac{d_{model}}{h}}\), respectively, represent the query matrix, key matrix, and value matrix for each attention head. \(\text{head}_{i}\in\mathbb{R}^{L\times\frac{d_{model}}{h}}\) represents the output of the \(i-\)th attention head. To combine the results of all attention heads, we introduce an additional weighted matrix \(w^{0}\in\mathbb{R}^{d_{model}\times d_{model}}\). ### Time Block

To model the complex dynamic characteristics of time-dependent relationships in distributed photovoltaic grids, we propose the Time-Block component. Its purpose is to capture both long- and short-term dependencies in time series through a multi-headed self-attention mechanism, while dynamically optimizing weight allocation based on input. The architecture is depicted in Figure 2a. Specifically, the Time Block comprises a multi-headed self-attention mechanism (MHSA), layer normalization with residual design, and a feedforward network. Assuming we have S Time Blocks, where the input of the \(s\)-th Time Block is \(X_{s-1}^{en}=\text{Time}-\text{Block}(X_{s-2}^{en})\), the expression for the Time Block is given in Equation (11):

\[ \begin{split} z_{T}^{s}&=Layer-Norm(MHSA(x^{s-1})+ x^{s-1})\\ x^{s}&=Layer-Norm(Feedward(z_{T}^{s})+z_{T}^{s}) \end{split}  \tag{11}\]

where \(x^{s-1}\in R^{L\times d_{model}}\) represents the input of the Time Block, and \(x^{s}\in R^{L\times d_{model}}\) represents the output of the Time Block. Layer normalization (Layer-Norm) can alleviate issues like gradient vanishing or exploding, while also enhancing the efficiency of neural network training. FeedForward denotes a multi-layer feedforward network.