# TCNformer Model for Photovoltaic Power Prediction
## 3 Methodology
### Long- and Short-Time Series Feature Extraction (LSTFE) Module  
In this study, we designed an LSTFE module, and we used it to extract time series features from each time scale. The structure of the LSTFE module is shown in Figure 4. The LSTFE mainly includes the LSTM unit, the SkipGRU unit, and the CycleEmbed unit.  
We transferred the week-level time-series-related data and the month-level time-series-related data to the LSTM network and the SkipGRU network in the LSTFE module for prediction. The prediction results of the LSTM network made full use of the short-term time series features, while the SkipGRU network made full use of the long-term time series features:  
\[f_{l}=LSTM(d_{l}) \tag{10}\]  
\[f_{s}=SkipGRU(d_{s}) \tag{11}\]  
\[X=Integration(d_{t},f_{l},f_{s}) \tag{12}\]  
\[X^{0}_{en}=CycleEmbed(X) \tag{13}\]  
In Formulas (10) and (11), \(f_{l}\in\mathbb{R}^{l}\) and \(f_{S}\in\mathbb{R}^{l}\) represent the month-level time series feature extraction results and the week-level time series feature extraction results in the LSTFE module, respectively. Using the excellent feature extraction capabilities of the LSTM and the SkipGRU, the extracted feature results were transformed into the input length \(l\) of the Encoder module.  
Using the LSTM and the SkipGRU, the time series features at weekly and monthly levels were extracted, but we were left wondering how to extract the time series features at an annual level? To solve this problem, we designed the CycleEmbed module.  
Figure 4: The structure of LSTFE module.  
The structure of the CycleEmbed unit is shown in Figure 5, including data projection, position coding, cycle coding, and timing coding.  
Data projection is based on the results of correlation and periodic analysis, mapping the output data to the vector of dimension, and aligning the dimensions. The alignment tool is a one-dimensional convolution filter.  
The position coding is calculated in the same way as in Transformer:  
\[P(pos,2j)=\sin\left(\frac{pos}{(2L_{x})^{\frac{2j}{d_{model}}}}\right) \tag{14}\]  
\[P(pos,2j+1)=\cos\left(\frac{pos}{(2L_{x})^{\frac{2j}{d_{model}}}}\right). \tag{15}\]  
In Formulas (14) and (15), \(j\in\left\{1,\ldots,\left|\frac{d_{model}}{2}\right|\right\}\), \(L_{x}\) is the input sequence length, and \(d_{model}\) is the Encoder input dimension.  
Cycle coding is divided according to the results of periodic analysis and calculation. \(\tau\) is the number of cycle data steps, which is determined by the results of periodic analysis \(T\) and the granularity of the data sampling time \(g\); that is, \(\tau=T/g\). Then, the cycle information of the input data is coded according to the results of \(\tau\); that is, there are \(\tau\) results in cycle coding, \(C_{i}=C_{i\%\tau}\).  
Timing coding is used to add the month and year to the coding to extract the longer time series features. In this way, the annual time series features of the data are introduced into the codec along with the embedding operation.  
Combining the results of the four parts, the output result of the final period embedding module is the input of the Encoder:  
\[CycleEmbed_{t}[i]=U_{i}+P(L_{x}(t-1)+i)+C_{i}+M_{i}+Y_{i} \tag{16}\]  
Figure 5: The structure of the CycleEmbed module.