## 4 Implementation  
In order to implement our proposed network, we used the scientific deep learning framework PyTorch [38] to accomplish our task. The architecture could be implemented in the Constructor and Forward functions, which are the interface provided by PyTorch. As mentioned in Section 3.1 and the single-layer CARU [8], a CARU unit can be composed of several Linear layers as presented in Algorithm 1. The hidden state and current input feature were first processed by the Linear layer that focuses on the calculation of the states and weight, and their results were passed on to the content-adaptive gate for the transition of the hidden state. The gate was used to combine all the obtained weights and produced the next hidden state directly. Note that there were two Sigmoid functions which we added (line 11 in Algorithm 1) to ensure that the produced state could be clamped in the range of \((-1,+1)\), providing a more stable feature for the subsequent processing.  
```
1 defConstructor\((in\gets input\_size,out\gets hidden\_size)\):
2 Lw\(\leftarrow\)Linear\((out,out)\)
3 LL\(\leftarrow\)Linear\((in,out)\)
4 Weight\(\leftarrow\)Linear\((out,out)\)
5 Linear\(\leftarrow\)Linear\((in,out)\)
6 enddef
7 defForword\((word,hidden)\):
8\(feature\leftarrow\)Linear\((word)\)
9\(n\leftarrow\)Tanh\((\texttt{Weight}(hidden)+feature)\)
10\(l\leftarrow\)Sigmoid\((feature)\odot\texttt{Sigmoid}(\texttt{Lw}(hidden)+\texttt{LL}(word))\)return\(((1-l)\odot hidden)\oplus(l\odot n)\)
11 enddef
```  
**Algorithm 1**Pseudo code of CARU unit architecture, with regard to Figure 2.  
Once we prepared the CARU unit, it could be directly embedded in our flow. We only focused on the design of stacking layers, and the multilayer CARU could also be implemented by Algorithm 2, such as in Figure 3.  
```
1 defConstructor\((input\_size,hidden\_size,num\_layers)\):
2 CARU\({}_{1}\leftarrow\)CARU\((input\_size,hidden\_size)\)
3 for\(l\gets 2\)to\(num\_layers\)do
4 CARU\({}_{l}\leftarrow\)CARU\((hidden\_size,hidden\_size)\)
5 endfor
6 enddef
7 defForword\((hidden\gets input)\):
8 foreachCARU\({}_{l}\)\((l\)increments from \(1)\)do
9 foreach\(h^{(t)}\in hidden\)do
10\(h^{(t)}_{l}\leftarrow\)CARU\({}_{l}\left(h^{(t)-1}_{l-1},h^{(t-1)}_{l}\right)\)
11 end for
12 hidden\(\leftarrow\left\{h^{(0)}_{l},h^{(1)}_{l},\cdots,h^{(t)}_{l}\right\}\)
13 end for
14 returnhidden
15
16 endfor
```  
**Algorithm 2**Pseudo code for complete multilayer CARU.  
On the other hand, the Chebyshev pooling could be achieved by using average pooling and maximum pooling as standard modules. The complete implementation was presented in Algorithm 3. In addition, there was an exceptional case we had to handle. It could cause the denominator in Equation (3) to be zero when the entire tensor contained the same value, that would make the result become NaN (not a number) as shown below,  
\[\begin{cases}\mu=t\\ \sigma^{2}=0\end{cases}\implies P_{max}(t)=\frac{0}{0+(t-t)^{2}}=\frac{0}{0}= \texttt{NaN}.\]  
This issue would always occur in the early stage of training, and we had to replace the NaN to 0 through a function nan_to_num provided by PyTorch to ensure smooth learning.  
```
1 defConstructor(\(k\gets kernel\_size\)):
2 AvgPool2d(\(k\times k\))
3 MaxPool2d(\(k\times k\))
4 enddef
5
6 defForward(\(feature\leftarrow input\)):
7\(\mu\leftarrow\texttt{AvgPool2d}(feature)\)
8\(\sigma^{2}\leftarrow\texttt{AvgPool2d}\big{(}feature^{2}\big{)}-\mu^{2}\)
9\(t\leftarrow\texttt{Softplus}(\texttt{MaxPool2d}(feature))\)
10 return\(\dfrac{\sigma^{2}}{\sigma^{2}+(t-\mu)^{2}}\)\(\triangleright\) Equation (3)
11 enddef
```  
**Algorithm 3**Pseudo code for the whole processing of Chebyshev pooling.