# Stride-TCN for Energy Consumption Forecasting and Its Optimization
## 3 Methodology
### TCN Architecture  
A reasonable volume of training data was available in this work; thus, we decided to use deep learning models to predict hour-ahead energy consumption. The TCN is a variation of the CNN for sequence modeling tasks by combining aspects of the RNN and CNN architectures. The TCN achieves better performance than RNNs in many tasks while avoiding the common drawbacks of recurrent models, such as the exploding/vanishing gradient problem or lack of memory retention [7]. The original architecture of the TCN (Figure 1) introduced includes two components: dilated and causal 1D convolutional layers [13], which smooth the input time series. Thus, we do not need to add the rolling mean or rolling standard deviation values in the input features.  
Dilated convolution can be applied in the long information dependency problem of the sequence to determine the output \(\mathcal{Y}\) at position \(t\) for a sequence input \(x\ \in\ \mathbb{R}^{n}\), expressed as:  
\[\mathcal{Y}(t)=(x*w)(t)=\sum_{i=0}^{K-1}w(i)\cdot x_{t-di} \tag{3}\]  
where \(d\) represents the dilation factor, \(K\) indicates the 1D convolutional window size, and \(t-di\) denotes the direction of the past with kernel \(w:\{0,\dots,K-1\}\rightarrow\mathbb{R}\). However, to construct a deep model with the increased depth of the architecture to capture a more extended history based on the TCN, using skip connections is recommended [7]. Accordingly, the shortcut connections across layers were added to TCNs against the degradation problem, and accuracy saturates as the network converges.  
Stacking multiple dilated convolutions enables networks to have extensive receptive fields and capture long-range temporal dependencies with a smaller number of layers [14]. Beside \(d_{l}\) is increased consecutive layers within a block, calculated as \(d_{l}=2^{l}\) for layer \(l\) in the network. Therefore, each TCN block contains \(\gamma\) elements identified based on \(R_{fieldmax}\), the maximum supported \(R_{field}\):  
\[\gamma\ =\ \Big{[}\log_{2}\Big{(}R_{fieldmax}-1\Big{)}\Big{]}+1 \tag{4}\]  
As defined in Equation (4), we can consider \(\gamma\) a parameter to determine the number of dilations of a TCN block. However, setting the TCN hyperparameters by hand requires an empirical and time-consuming trial-and-error process and is not optimal [7]. In this paper, we reduce this work by automatically searching the hyperparameters. Table 1 lists the hyperparameters automatically searched for in the TCN using Bayesian optimization.  
\begin{table}
\begin{tabular}{c c c} \hline \hline \multicolumn{1}{c}{**Hyperparameters**} & **Symbol** & **Choices** \\ \hline
1D convolutional window size & \(K\) & \(1,3,5,7,9\) \\ Number of filters in each & \(N_{i}\) & \(8,16,32,64,128,256,512\) \\ convolution layer & \(N_{i}\) & \(\geq 2\) \\ Number of TCN layers & \(N_{t}\) & \(\geq 2\) \\ Dilation factor & \(\gamma\) & \(\geq 1\) \\ Skip connection & & Yes, No \\ Batch Normalization & & Yes, No \\ \hline \hline \end{tabular}
\end{table}
Table 1: Search space of the TCN using Bayesian optimization.  
Figure 1: TCN architecture.