# Soft Periodic Convolutional Recurrent Network for Spatiotemporal Climate Forecast
## 3 Materials and Methods
### Periodic Representation
#### 3.4.2 Load Mechanism  
To use periodicity in the model, the representations from the PRD are loaded with three different proposals. Note that the load mechanism proceeds before the update mechanism. The representation at the bottom of \(P\) (where \(z=d\)) is the representation of the previous periodic cycle when the time index is \(I_{P}\). Thus, \(p_{i,z}\) (where \(i<I_{p}\)) are the representations of a current cycle. In contrast, \(p_{i,z}\) (where \(i\geq I_{p}\)) are the representations of the previous cycle as shown in Figure 3a. **1. Nearby-Time Representation:** In this work, to capture a phase shift that may change every year, we extended the periodic attention mechanism component idea from the original Periodic-CRN to include nearby (months) representations for the previous year as shown in Equation (5). In this proposal, \(d\) is \(2\), and \(z\in\left\{1,2\right\}\) indicate which depth of \(P\) will be used for a load mechanism. Since the update mechanism is proceeded at every time step, representations  
Figure 3: Examples of the three proposals for the load mechanism to predict the climate variable of July 1999 (index 1 starts from the top left of \(P\)). (**a**) Nearby-Time representation with the nearby representation of 2, (**b**) Periodic-Depth representation with a depth of 5, and (**c**) Periodic-Depth-Differencing representation with a depth of 5.  
in the same depth (\(z\)) may not be in the same periodic cycle. \(I_{Pn}\) is the nearby time index; \(n\) is the incremental number indicating nearby time index representation; and \(n_{pre}\) is the pre-defined length. The example of the nearby-time representation proposal is shown in Figure 3a. To find a yearly periodic representation of July 1999, \(m\) was defined as 12 (yearly), \(t\) was 247 (when the time index started from January 1979); and \(\left\lfloor\frac{t}{m}\right\rfloor\) was calculated as 20 (years); and \(I_{P}\) is 7 (July). If \(n_{pre}\) is defined as 2, then the representations of \(p_{1,5}\), \(p_{1,6}\), \(p_{2,7}\), \(p_{2,8}\), and \(p_{2,9}\) which are May, June, July, August and September 1998 are loaded. \[h_{p_{n}}^{(t+N)}\Leftarrow P[I_{Pn}][z]\quad\text{(for $n=-n_{pre},...,0,...,n_{pre}$)} \tag{5}\]  
where  
\[I_{Pn} = \begin{cases}(I_{P}+n)-m,&\text{if $I_{P}+n>m$}\\ m-(I_{P}+n),&\text{else if $I_{P}+n\leq\ 0$}\\ I_{P}+n,&\text{otherwise}\end{cases} \] \[z = \begin{cases}2,&\text{if $n\geq\ 0$}\\ 1,&\text{otherwise}\end{cases} \]  
**2. Periodic-Depth Representation:** The periodic representation in the first proposal, the nearby time representation, considers only one year's prior representation. We extended the representation to be more than one period prior, and use only the same time index (phase) as shown in Equation (6) where \(d\) is the depth of \(P\) representing the number of consecutive periodic cycles. The load mechanism loads all the representations that are in the same phase of the previous cycles. An example of periodic-depth representation proposal is shown in Figure 3b. To find a periodic representation of July 1999, \(d\) is 5, \(t\) is 247, \(m\) is 12, \(\left\lfloor\frac{t}{m}\right\rfloor\) is 20, then \(I_{P}\) is 7 (July). The representation of July 1998, 1997, 1996, 1995, and 1994 are loaded according to \(d=5\). \[h_{p_{z}}^{(t+N)}\Leftarrow P[I_{P}][z]\quad\text{(for $z=1,...,d$)} \tag{6}\]  
**3. Periodic-Depth-Differencing Representation:** Similar to the second proposal, we considered the first-order derivative of the representations to capture the changes over cycles as shown in Equation (7). In this proposal, we used a differencing of two years. The load mechanism loaded the differencing of representation \(p_{ij}\). The example of periodic-depth-differencing representation proposal is shown in Figure 3c. According to the parameters set in the second proposal, to find the periodic representation of July 1999, the representation of July 1998 minus 1997, 1997 minus 1996, 1996 minus 1995, and 1995 minus 1994 were loaded.