## 4 Methodology
### Model Backbone: Hyena Hierarchy  
Processing ultra-long documents like central bank policy statements is one of the key challenges of this task. Traditional Transformer-based self-attention mechanisms have \(O(L^{2})\) computational and memory complexity, making direct processing of sequences with tens of thousands of tokens impractical. To overcome this limitation, we employ Hyena Hierarchy [19] as the model's backbone feature extractor. Hyena is a novel deep learning operator that, by cleverly alternating stacks of implicitly parameterized long convolutions and data-controlled gating, can effectively capture long-range dependencies within sub-quadratic complexity (specifically, \(\mathcal{O}(NL\log L)\), where \(N\) is the depth of the Hyena operator and \(L\) is the sequence length). It also demonstrates faster inference speeds on long sequences compared to highly optimized attention mechanisms, like FlashAttention. Figure 5 provides an illustration of the Hyena architecture and its core mechanism. The core of the Hyena operator is a recursive process. For an input sequence \(u\) (after initial embedding and positional encoding), linear projections yield a set of sequences \((v,x_{1},\ldots,x_{N})\). Then, an \(N\)-step recursion is performed:  
\[z_{1}=v \tag{3}\] \[z_{n+1}=x_{n}\odot(h_{n}*z_{n})\quad\text{for }n=1,\ldots,N \tag{4}\]  
The final output is \(y=z_{N+1}\). In this recursion, \(h_{n}\) is the \(n\)-th layer's long convolution kernel. Its length can be as long as the input sequence \(L\), providing a global receptive field. Crucially, \(h_{n}\) is implicitly parameterized rather than having its weights learned directly. Figure 5: Illustration of the Hyena backbone architecture and the gated convolution recurrence mechanism. The \(*\) symbol denotes convolution, implemented efficiently using Fast Fourier Transform (FFT)-based algorithms with \(\mathcal{O}(L\log L)\) complexity. \(x_{n}\) is the \(n\)-th linear projection of the input sequence \(u\), serving as a data-controlled gate. The \(\odot\) symbol represents element-wise multiplication (Hadamard product), allowing the convolution results to be modulated by the input data itself, introducing data dependency similar to attention mechanisms. Hyena's long convolution kernel \(h_{n}\) is not learned directly, but dynamically generated by a small feed-forward network (FFN). Specifically, for time step \(t\) (or sequence position \(t\)), the kernel value \(h_{n}(t)\) is determined by:  
\[h_{n}(t)=\text{Window}(t)\cdot(\text{FFN}\circ\text{PositionalEncoding})(t) \tag{5}\]  
Here, \(\text{PositionalEncoding}(t)\) maps position \(t\) to an embedding vector. The FFN (e.g., a shallow MLP) maps the positional embedding to a basis for the filter response, often using periodic activation functions (like sin) to capture high-frequency information. \(\text{Window}(t)\) is a window function (e.g., a parameterized exponential decay \(\exp(-\alpha t)\) plus a bias term) that shapes the filter, enabling it to adaptively focus on information at different time scales (like macroeconomic cycles). This parameterization decouples the number of parameters from the sequence length \(L\) (sub-linear parameter scaling) and allows the effective filter length to be learned during training. To make the model aware of sequence position information, we inject Rotary Positional Embeddings (RoPE) [40] after the input embedding layer for both the text sequence \(x\) and the interest rate sequence \(r\). RoPEs have demonstrated excellent performance in various Transformer architectures by effectively encoding relative position information. The Hyena architecture is designed to be compatible with and benefit from modern positional encoding methods like RoPEs. In this study, we adopt a variant of Hyena-Small (125M parameters). The specific configuration involves \(N=6\) layers of Hyena recursion with a model hidden dimension \(D=768\).