# Enhancing Photovoltaic Power Predictions with Deep Physical Chain Model
## 2 Methodology and Resources
### Features Engineering
#### 2.4.4 Model Architecture  
Photovoltaic power prediction represents an intrinsic challenge that requires the conversion of various sequential variables into power forecasts. Due to the superior performance of the Transformer architecture [22], and LSTM [20], elements of this architecture have been incorporated as the basis for our proposal.  
Based on the Transformer encoder from [37], our model simplifies the conventional structure by removing the positional coding and adding a linear layer with the same number of neurons as the number of input features. The model was chosen for its simplicity and the promising preliminary results obtained at the beginning of the research. This adaptation seeks to maintain computational efficiency without sacrificing performance in photovoltaic forecasts. In our design, the Transformer block acts as an encoder or feature generator, see Figure 3.  
The yellow block represents the Linear Layer that processes input features. The purple block corresponds to the Attention mechanism, which focuses on the most relevant parts of the input sequence. The red blocks denote Layer Normalization (Layer Norm), stabilising the learning process by normalizing inputs. Finally, the green section represents  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**No.** & **Name** & **Reference** & **Predictors** & **Comment** \\ \hline
1 & [PERSON] & [25] & \(k_{T}\) & \\
2 & Skartveit & [26] & \(k_{T},\Theta_{Z}\) & \\
3 & DISC & [27] & \(k_{T},\,AM\) & \(AM\) is calculated using [28] formula \\
4 & DIRINT & [29] & \(k_{T},\Theta_{Z},\,W,\Delta k_{T}\) & The model is used with the universal bin for \(W\) since the exact value is not available \\
5 & DIRINDEX & [30] & \(k_{T}^{\prime},\Theta_{Z},\,W,\,\Delta k_{T}\), \(GH_{cs},DNI_{S}\) & Paired with the clear sky model Ineichen, and \(W\) is treated similarly as for DIRINT \\
6 & BRL & [31] & \(k_{T},\Theta_{Z},\,t_{s},\,K_{T},\,\psi\) & Paired with the clear sky model Ineichen and used with the 15 min parameterization presented in [33] \\
7 & Engerer & [32] & \(k_{T},\,\Theta_{Z},\,t_{s},\,\Delta k_{T,\mathcal{S}},\,K_{de}\) & Used with the parameterization for the temperate climate zone \\ \hline \hline \end{tabular}
\end{table}
Table 5: Summary of tested physical decomposition models.  
the Feed Forward Network (FFN) with a ReLU activation function, providing non-linear transformations to the data.  
The output is fed into a bidirectional LSTM connected to an attention layer and a two-layer feed-forward network, followed by a linear layer of four neurons; the architecture is represented in Figure 4. The components of the model are colour-coded as follows: The blue block represents the Bi-LSTM layer, which processes the sequence in both temporal directions. The purple block represents the attention mechanism, highlighting the most relevant time steps. The green blocks indicate the feed-forward network with a ReLU activation, responsible for further transformation of the input data. Finally, the yellow blocks correspond to the output linear layer with ReLU activation, producing the forecasted power values.  
This configuration is especially suitable for forecasting power production in the next hour and is fully illustrated in Figure 5.  
Figure 4: Diagram of the proposed Bi-LSTM decoder, where each (\(y_{t}\)) represents the power measurement at (\(t\)), and the indices from -3 to 4 indicate measurements taken every 15 min, either before or after (\(t\)). For instance, (\(y_{t-3}\)) is the measurement taken 45 min before (\(t\)) and (\(y_{t+4}\)) is the measurement taken 60 min after (\(t\)).  
Figure 5: Simplified architecture of the proposed Transformer Bi-LSTM model.  
Figure 3: Diagram of the proposed Transformer encoder.  
A crucial element in our proposal is the attention mechanism, which allows us to discern which parts of the input have the greatest relevance for the forecast. This mechanism is illustrated in Figure 6.