# Solar Irradiance Forecasting with Transformer Model
## 5 Conclusions  
The results show that it is advantageous to use space-time bonds between features as well as point and regional data in solar irradiance prediction. The paper also resulted in a model capable of predicting a specified number of timesteps ahead based on historical data as well as its own prediction of the future. The Transformer model proved to be a convenient solution not only for tasks dealing with images or natural language but also for tasks related to predicting the future values of a physical quantity. The basis for this method is a rich knowledge of historical data, which the model can draw on during learning and thus adapt not only to the average weather. Given a sufficiently long time, the dataset could include several dramatic natural events, such as volcanic activity, large-scale fires, or global warming. Another advantage of the Transformer model proved to be the coupling of regional data to predict point (target) solar irradiance. This feature of the proposed model is based on the idea of translation from one natural language to another natural language. Moreover, the property of matching words with the same or similar meaning from two different dictionaries is reflected by linking the local regions across India according to their influence in predicting future solar irradiance. The Transformer model has proved to be a suitable solution for finding the correlations, with the advantage that its computation is massively parallelized. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Name** & **Description** & **Value** \\ \hline Attention dropout & Dropout applied to the attention matrix & 0.25 \\ Batch size & Number of examples per one training batch & 32 \\ Dropout & Dropout applied between hidden layers & 0.15 \\ Embed layer size & Size of the Key, Value, Query matrices & 64 \\ Epochs & Number of training epochs & 1000 \\ FC layer size & Size of the fully connected feed-forward network block & 256 \\ Global clipnorm & Global clipnorm for the optimizer & 2 \\ Learning rate & Learning rate for the optimizer & 0.005 \\ No. decoder layers & Number of decoder blocks & 6 \\ No. encoder layers & Number of encoder blocks & 3 \\ No. heads & Number of heads in every Multi-head Attention block in the model & 6 \\ Optimizer & Optimizer used during training & AdamW [35] \\ Warmup steps & Warmup steps of the learning rate cosine scheduler & 70 \\ Weight decay & Weight decay for the optimizer & 0.00001 \\ Window size & Windows size defines the size of known history & 7 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The optimized hyperparameters. **Supplementary Materials:** The Solar Transformer code and further description can be downloaded at: [[https://github.com/markub3327/Solar-Transformer](https://github.com/markub3327/Solar-Transformer)]([https://github.com/markub3327/Solar-Transformer](https://github.com/markub3327/Solar-Transformer)) (accessed on 1 July 2022). The NASA Power bot code and further description can be downloaded at: [[https://github.com/markub3327/NASA-POWER-BOT](https://github.com/markub3327/NASA-POWER-BOT)]([https://github.com/markub3327/NASA-POWER-BOT](https://github.com/markub3327/NASA-POWER-BOT)) (accessed on 1 July 2022). The online interactive charts at: [[https://wandb.ai/markub/solar-transformer?workspace=user-markub](https://wandb.ai/markub/solar-transformer?workspace=user-markub)]([https://wandb.ai/markub/solar-transformer?workspace=user-markub](https://wandb.ai/markub/solar-transformer?workspace=user-markub)) (accessed on 1 July 2022). **Author Contributions:** Conceptualization, I.D.L. and J.P.; methodology, M.K. ; software, M.K. ; validation, M.K. ; formal analysis, J.P.; investigation, M.K. ; resources I.D.L. ; data curation, M.K. ; writing--original draft preparation, M.K. ; writing--review and editing, J.P.; visualization, M.K. ; supervision, I.D.L. and J.P.; project administration, I.D.L. ; funding acquisition, I.D.L. All authors have read and agreed to the published version of the manuscript.