Numerion: A Multi-Hypercomplex Model for Time Series Forecasting
Authors: Hanzhong Cao
WenBo Yan
Ying Tan
Venue: ICLR 2026 Poster (2026)
Keywords: Time Series Forecasting
Hypercomplex Numbers
Hypercomplex Time Series Models
Multi-Hypercomplex Space
Primary area: learning on time series and dynamical systems
TL;DR: We propose Numerion, a hypercomplex space-based model, decomposes and forecasts time series using multi-dimensional RHR-MLPs, achieving state-of-the-art results.

Abstract: Many methods aim to enhance time series forecasting by decomposing the series through intricate model structures and prior knowledge, yet they are inevitably limited by computational complexity and the robustness of the assumptions. Our research uncovers that in the complex domain and higher-order hypercomplex spaces, the characteristic frequencies of time series naturally decrease. Leveraging this insight, we propose Numerion, a time series forecasting model based on multiple hypercomplex spaces. Specifically, grounded in theoretical support, we generalize linear layers and activation functions to hypercomplex spaces of arbitrary power-of-two dimensions and introduce a novel Real-Hypercomplex-Real Domain Multi-Layer Perceptron (RHR-MLP) architecture. Numerion utilizes multiple RHR-MLPs to map time series into hypercomplex spaces of varying dimensions, naturally decomposing and independently modeling the series, and adaptively fuses the latent patterns exhibited in different spaces through a dynamic fusion mechanism. Experiments validate the model’s performance, achieving state-of-the-art results on multiple public datasets. Visualizations and quantitative analyses comprehensively demonstrate the ability of multi-dimensional RHR-MLPs to naturally decompose time series and reveal the tendency of higher-dimensional hypercomplex spaces to capture lower-frequency features.

NUMERION: A MULTI-HPERCCMPPEE MODEL FOR TIME SERIES FORECASTING
Hanzhong Caol.", Wenbo Yan & Ying Tan
"School of Electronics Engineering and Computer Science, Peking University, Beijing "Scooo of Intelligence Science and Technology, Peking University, Beijing "Computational Intelligence Laboratory, Peking University, Beijing "Inttitute for Artificial Intelligence, Peking University, Beijing 'State Key Laboratory of General Artificial Intelligence, Peking University, Beijing {22 (220001898) 1929, wenboyan }@stu. /estu.pku. edu. en, ytanêpku. edu. en
ABSTRACT
Many methods aim to enhance time series forecasting by decomposing the series through intricate model structures and prior knowledge, yet they are inevitably limited by computational complexity and the robustness of the assumptions. Our research uncovers that in the complex domain and higher-order hypercomplex spaces, the characteristic frequenciess of time series naturally decrease. Leveraging this insight, we propose Numerion, a time series forecasting model based on multiple hypercomplex spaces. Specifically, grounded in theoretical support, we generalize linear layers and activation functions to hypercomplex spaces of arbitrary power-of-two dimensions and introduce a novel HeaalHHyprroomlee--Re Multi-Layer Perceptron (RHR-MLP) architecture. Numerion utilizes multiple RHR-MLPs map time series into hypercomplex spaces of varying dimensions, naturally decomposing and independently modeling the series, and adaptively fuses the latent patterns exhibited in different spaces through dynamic fusion mechanism. Experiments validate the model's performance, achieving state-of- the-art results on multiple public datasets. Visualizations and quantitative analyses comprehensively demonstrate the ability of multi-diment multi-diimnnsional RHR-MLPs naturally decompose time series and reveal the tendency of higher-dimensional hypercomplex spaces to capture lowerffrequency features.
Domain
INTRODUCTION
Time series prediction is crucial in fields like power, finance, transportation, and industry, with focus uncovering temporal patterns for accurate forecasting. Recent advancements in deep learning, including CNN-based (e.g., TCNBai et al. (2018), SCINe(Liu et al. (2022)), RNN-based (e.g., LSTNe(Lai et al. (018)), SPh- et (2019)), Transformer-basedd (e.g., Informe ct al. 2021), Autoforme Wu et al. (202I) PatchTS et al. (2022). iTransforme et al. (2023b)). and MLP-based methods (c.g.. N-BeatsOreshkin al. (2019). DLineafZeng et al. (2023 )) have
significantly enhanced this domain.
In cutting-edge solutions, techniques like sequence decomposition and multi-period pattern recog- nition are integrated into models, often through complex structures or manual rules, increasing computational demands and model complexity. Alternatively, some models achieve these tasks in the frequency domain with simpler architectures. For instance, SCINe(Liu et al. (2022) and FiLM FiLMZZhu (2022a) use frequency domain decomposition, FEDforme (2023) capture mulli-pperie patterns, and FilterNe{Yi et al. (2024) learns amplitude relationships via filters. This stems from the natural divergence of time series properties in complex spaces compared to real domains, prompting the question: Can time series exhibit additional properties in higher-dimensional hypercomplex spaces, enabling modeling through simple transformations rather than intricate structures?
al. (2022b) and Fre et al.
Inspired by hypercomplex number theory (Appendix D. our experiments reveal that mapping time series from the real domain to higher-dimensional hypercomplex spaces (e.g., complex numbers,

---

Weather
Time Series Rea Domain
w
25
120
125
275
202
ETT
Aalar-Energy
Thme Iote
0.359
A.29
Time Series
Complex Domain
Time Series in Qualernion Domain
R-424
0.300
4.325
Eiins
6,325
0.353
Exchange
8.325
Electricity
75
20
20 20.0% 27.50
25
Rensency
-00
0.275
Thandkenc
Traffic (h)
Figure 1: (a) Time Series in Real, Complex, and Quaternion Domains. (b) Performance of Numerion.
quaternions) naturally alters their characteristics, favoring lower-frequency features. As illustrated in Figure I mapping to the complex domain eliminates high-frequency fluctuations, unveiling hierarchical spatial structure. In quaternion space, high-frequency features further diminish, resulting clustering phenomenon where nearly all data points align with the same wwe.uuequncc feature.
Based on our experimental findings, we propose that mapping time series into hypercomplex spaces of varying dimensions inherently reveals distinet latent features, eliminating the need for complex model structures. To this end, we introduce Numerion, multi-Hypercomples time series forecasting model. Specifically, we generalize linear transformations and tanh activation functions from the real domain to hypercomplex spaces of arbitrary power-of-two dimensions, establishing unified framework. We then design a novel Real- HolalH-yerropplexx Real domain Multi-Layer Perceptron (RHR- MLP) to model linear and nonlinear relationships in hypercomplex spaces. By mapping time series into multiple hypercomplex spaces, we independently model temporal features using the simple RHR-MLP structure. Finally, better integrate information from different hypercomplex spaces, designed the Multi-Hypercomplex Adaptive Fusion mechanism to adaptively fuse them. Experiments show our method achieves state-of-the performance across multiple datasets. Ablation studies, visualizations, and quantitative analyses confirm that hypercomplex spaces naturally enable multi- frequency decomposition of time series, uncovering diverse latent temporal patterns, with higher- dimensional spaces tending to model low-frequency features. Our contributions are summarized as follows:
We generalize the linear layer and Tanh activation function to hypercomplex spaces, sup- ported by rigorous theoretical foundations, and introduce novel Real- Hypercopeee Real domain Multi-Layer Perceptron (RHR-MLP) architecture.
We propose Numerion, multi- Miltiihypercomplex space time series prediction model that maps time series into different hypercomplex spaces to naturally achieve multi-frequency decomposition, model distinct temporal patterns, and adaptively fuse them.
Experiments validate effectiveness of our method. Visualizations and quantitative analy- ses demonstrate that hypercomplex spaces naturally enable multi-frequency decomposition and reveal their underlying decomposition patterns.
2 PRELIMINARY
2. DEFINITION OF LONG-TERE TIME SERIES PREDICTION PROBLEM
The long-term time series prediction problem involves forecasting the values of target variables (IT+1." ... 2 E "R'") over a future time period, utilizing historical time series data (21. FrLE, E R Here, T represents the length of the historical sequence, P denotes the
'The first author and second author contribute equally this work.
2

---

prediction horizon, and F indicates the number of features in the time series. The input is represented as X E RTx and the label is denoted as Y E RPx
2.2 DEFINITION OF HYPERCOMPLEX NUMBERS
Hypercomplex numbers extension of complex numbers, generalizing their properties to higher- dimensional spaces through the introduction of additional dimensions or more intricate algebraic structures.
Lemma 2.1 Cayley-Dickson Algebra System: Let be an algebraic system equipped with a conju- gation operation (denoted as *), where elements are the form (a.3), 3). Through iteration, algebraic system can be constructed, adhering to the following rules:
(a1,8) B1) + (a2,82) (2) =(11- 02 31 B2); (01, B1) (02, (a102 -353, B201 31a5); (01, (ai. -11
By Lemma 2.1 can begin with the real numbers and iteratively construct algebras of any power- of-two dimension, such as the complex numbers (2D), quaternions (4D), octonions (8D), sedenions (16D), and beyond. These algebras comprise real part and multiple imaginary units, adhering to specific multiplication rules. The properties of real numbers manifest distinct behaviors within these higher-dimensional spaces.
We define the n-dimensional hypercomplex numbers uniformly represented as:
(x) == ao + ail +... + Gn 6n-1in-1
where ag is the real part coefficient, {aI..... an- } are the imaginary part coefficients, and 1i,-...,nn-1 } are the imaginary units. Since hypercomplex numbers are generated iteratively, the dimension n must be a power of two (e.g.. 1, 4..... When n = 0, c0)) represents a real number.
To optimize these parameters within neural architectures, Numerion is rigorously grounded in the Generalized Hamilton-Real (GHR) Calculus framework. As dictated by Liouville's Theorem in hypercomplex analysis, strict differentiabilityy is mathematically incompatible with the bounded non- linear activation functions necessary for deep learning. GHR Calculus bridges this gap, establishing a formal mathematical basis for optimizing real-valued loss functions with respect hypercomplex parameters while ensuring the gradient flow remains strictly constrained by hypercomplex algebraic rules. Comprehensive theoretical details and derivations regarding these gradient differentiation mechanisms are provided in Appendix H.2
Similarly, in this paper, all vectors. matrices, and modules in hypercomplex space are denoted with superscript (n) to indicate their dimensionality. The absence of this superscript implies n 1, which corresponds to the real number domain.
3 METHOD
In this section, we first generalize the linear layer and Tanh activation function hypercomplex spaces, proposing sall-hypperomplex-eeee domain multi-layer perceptron (RHR-MLP) architecture. This architecture maps inputs to arbitrary hypercomplex spaces, uncovering unique features in high- dimensional spaces. Building on this, we introduce Multi-Hypercomple time-series prediction model, named Numerion, which achieves natural multi- fretuiifrrquncyy decomposition, modeling, and fusion of time series through RHR-MLP across multiple hypercomplex spaces.
3.1 REAL-HYPERCOMPLESS REAL DOMAIN MULTI-LAYER PERCEPTRON
Firstly, we define the mapping principles and generalize the linear layers and Tanh activation functions to the hypercomplex space. Subsequently, we propose Haperco-eecc Real domain Multi- Layer Perceptron (RHR-MLP) architecture, designed to model both línear and nonlinear relationships within the hypercomplex space.
3

---

Mattil Livll Patch Embodding
RHIR-MEP
Lincar
00O
KHR-MLP: Reak(n-t)
000
Linrrr
concal
Linear
RHIR-MPP: Complex(a-l)
0
Denpeut
RHR.MLP: Quatcruionin
Hyptr-llpeermmple Adapthe Faddon
INTTas
000
0
RHI-MMP.: Octoslon(orst)
Ju
MLP: Sedenlonio-11)
IMAP
I0I
(1)
Figure 2: Overall Structure of Numerion. Primarily includes Multi-Level Patch Embedding, Multi- Dimensional RHR-MLP, and Multi-Hypercomppie Adaptive Fusion.
3.1.1 HIGH-DIMENSIONAL HYPERCOMPLEX MAPPING
We first define the mapping method from lower-dimensional hypercomplex numbers to higher- dimensional ones. Since higher- ligee--dimensional hypercomplex numbers are derived from lower- dimensional ones, for any ny-dimensional hypercomplex number, when mapping higher dimension Vıa. (where n& > ni), we adopt the principle of preserving the lower-dimensional coefficients and padding the higher-dimensional ones with This process denoted as ( HMAP( (x()((()))) (x))...
(m) == ao + +aii + ... an--i -i [IMAp(n-*HA)
(ma) ani + an On + Oi,, + + Oina-s
3.1.2 HYPERCOMPLEN LINEAR LAYER
We generalize the linear layer hypercomplex spaces of arbitrary dimension n. The linear transfor- mation from dj to d2 space is expressed as:
O w()) X (n) + (()
where the input vector ((),, the mapping matrix W(n), the bias matrix b(0), and the output (n) are all composed of hypercomplex numbers of the same dimension
c
t,000 C00,1, Cwd,, C. c 10
C.aa cha
n
co
Cod
Cwrdz.0 1wddg.1
Curdyy
Czids
Coyd
evident that the output of the hypercomplex linear layer can still be represented in a manner analogous to real-valued matrix multiplication:
[m)
(n) Cusi
(n) Cardy + Cb:
(m)
Copk
C250
Cwiids
(4)
where the product of two hypercomplex numbers must conform the multiplication rules of the hypercomplex space. In Appendix G multiplication rules hypercomplex spaces of any power-of-twe dimension, along with an efficient computational approach for hypercomplex linear mappings.
we present comprehensive techniques for generating
For simplicity, we consistently represent the linear layer in an n-dimensional hypercomplex space as
(xx
(5)
3.1.3 HYPERCOMPLEX NORM TANH ACTIVATION FUNCTION
To capture non- non-linear mapping relationships in hypercomplex spaces, we introduce the Hypercomplex Norm Tanh (HN'Tanh) activation function. This function applies non-linear transformations both
4

---

the real and multiple imaginary components of the hypercomplex number. The computation process of (NTanh(") is defined as:
03 tanh (1((())))))
d; |c
(6)
where G2 denotes the coefficients the hypercomplex number, and || lpp represents the p-norm. HNTanh enables non-linear transformation of the modulus of the hypercomplex number while maintaining its phase. When the hypercomplex number simplifies to the real number space, this activation function becomes equivalent the tanh activation function. Additionally, this function is both continuous and real-differentiable. Relevant theoretical derivations and advantage analyses are provided in the Appendix and H.1
3.1.4 LOW-DIMENSIONAL HYPERCOMPLES MAPPING
Similar the process of mapping higher dimensions, when mapping to low-dimensional hy- percomplex spaces, we follow the principle of retaining the lower-dimensional coefficients while discarding the higher-dimensional ones. This operation is denoted LMAP!
(ma) : -u aiin + ... + Onj Oni-1in + anzine + ... + ann ann-11ma- 1mma-1
LMAP!Ng (tp)) ((n)) := do + arin + + an--thn-t
where and represent the dimensions of the hypercomplex numbers, and
3.1.5 REAL-HYYPERCOMPLXX-RA.. DOMAIN MULTI-LAYER PERCEPTRON
The Real-Hyyerromplex-RRe domain Multi-Layer Perceptron (RHR-MLP) architecture utilize comprises High-Dimensional Hypercomplex Mapping, Lx Hypercomplex Linear Layers with Hy- percomplex Norm Tanh Activation Function and Dropout, prediction layer, and www-iiensiional Hypercomplex Mapping, as depicted in Figure 2 We denote the RHR-M MLLP in the re-dimensional hypercomplex space as RHR-MLP
(X), and its computational process is as follows:
(n) (0)
HMAP (1NAp(1-m)) (X)
X (n)
Dropout (HNTanh (n) (HLinear"")
(x (-1)))) i= L..,
(8)
0 (n) HLinear (n) (( [[x (m)
x ))
(1)
0
(0
where [[I.] represents matrix concatenation.
3.2 NUMERION ARCHITECTURE
We introduce multi-hhperrcompeex space time series prediction model, named Numerion, which transcends the constraints of the real number domain by mapping sequences to complex, quaternion, and higher-dimensionall hypercomplex spaces. This approach seeks to uncover distinct latent temporal patterns inherent in various hypercomplex spaces and adaptively integrate muutt--dimensionall features. The model architecture, as depicted in Figure primarily comprises three components: Multi-Level Patch Embedding, Multi-dimensional RHR-MLP, and Multi-Hypercomp Adaptive Fusion. Prior to inputting the sequence and after generating the prediction, we apply mean normalization and denormalization, respectively.
3.2.1 MULTI-LEVEL PATCH EMBEDDING
Time series often exhibit both long-term and short-term characteristics, which are challenging for simple MLP structure to model directly. However, Patch et al. (2022) enables segmentation of the time series into varying lengths. allowing the MLP to capture temporal features across different scales. Thus, before mapping the time series to multi- multt-hhyerccomplex spaces, we enhance long-term and short-term features through Multi-Level Patch Embedding al. (0022).

---

We denote the number of patch levels lp and define the segmentation length for each level as 1 ii 0, o..,, 1, 1. The time series is segmented according to the length of each level, forming multiple subsequences, where sequences within the same level have identical lengths. We linearly map the time series at each level to unified encoding dimension de, average the encodings of multiple sequences at the same level, and concatenate the encodings from different levels to form a time series Xp E (x) that incorporates features of varying periodicities.
3.2.2 MULTI-DIMENSIONAL RHR-MLP
Next, X, fed into the nulti-diimnsionnl RHR-MLP to uncover latent patterns in hypercomplex spaces of varying dimensions. Given the stepwise derivation of hypercomplex spaces, we select con- secutive power-of-two spaces starting from real numbers: Real Numbers (Real), Complex Numbers (Comp), Quaternions (Quat), Octonions (Octo), and Sedenions (Sede).
O Real RHR-MLP(2) (Xp); O Comp RHR-MLP(2) (Xp), QQuar RHR-MLP (Xp)
O = RHR-MLP (Xp): O CHR-MMLP(1) (Xp)
where O Reat, O Comp: Quat, O0et. and O denote the prediction outputs in each hypercomplex space. All RHR-MLPs operate in parallel, independently uncovering latent temporal patterns in their respective spaces. We avoid higher-dimen dimensional spaces for two reasons: first, computational time and memory demands escalate rapidly; second, experiments reveal that tend to model lower-frequency features, as shown in Section 4.2 and the benefits of excessively low-frequency features for time series prediction are limited.
spaces
3.2.3 MULTI-HYPEECOMPPEX ADAPTIVE FUSION AND Loss
We design an adaptive fusion mechanism to integrate features from different spaces and generate the final prediction. The outputs from all spaces are stacked, and an MLP learns adaptive weights for each space. The softmax function ensures these weights sum to 1, and the weighted sum of the outputs yields the fused prediction result.
O, R = ( (WT, o (WT, O, + by.1) + by,2), Ý sum(R O.)
O Real, OComp Qquat, O O Sede >
(10)
where < <>> denotes matrix stacking, sum(-) represents the summation operation, ju(-) is the softmax activation function, ((.) is the GeLU activation function, and Y is the model's prediction result. The model trained using the MAE Loss as the loss function.
Regarding computational complexity, Numerion operates with an asymptotic time complexity of OI. d), where L, represents the time series length and denotes the encoding dimension. While hypercomplex multiplication inherently scales quadratically with the chosen algebra dimension n, this dimension acts as a fixed structural constant rather than a variable dependent on input length. Consequently, the O(n?) cost introduces only constant multiplier that does not alter the model's fundamental linear scaling (L) with respect to the sequence length. This preserves a crucial asymptotic advantage over the (L?)) scaling typical of standard Transformer architectures. Furthermore, because these hypercomplex operations are formulated as highly parallelizable matrix multiplications, modern GPU infrastructure effectively absorbs this constant computational overhead during training.
4 EXPERIMENT
we present the comparisons with other methods. Section 4.2 reveals the multi- frequency natural decomposition patterns of time series through visualization and quantitative analysis. 4.3 and Section 4.4iiouuu. on parameter experiments and ablation studies, respectively. Section 4.5 a.aaneayzes efficiency. In Appendix G we introduce a High-Efficient Hypercomplex Linear Layers approach to accelerate hypercomplex computations. Appendix H Hprooiiees theoretical supplements. Appendix I includes detailed analysis of HNTanh. Appendix uncover the natural decomposition patterns of time series. JoaffrrJJfffrs offers additional visualizations to quantitative statistics across various hypercomplex spaces. Appendix reports the error bars of our model results.
In Section 4.1

---

Time Domain Contriution Busoo (Channel 04
Time Domain: Real Uinor us Ground Trush
Timee Domoin Comp & Linar Ground Truch
Invay
Cent Liner
out tieer
Time (T)
Domain Quat Ground ruth
Doman: Octo Liver va Ground Truth
Time Domain Sade vs Ground Tuth
Ground
Grann Tuan ass Linear
Coatlhtorr
Sade Linwar
10
20
36
Figure 3: Visualization of MLP outputs on ETTh] (channel 0). The first plot shows the weighted sum of five RHR-MLPS, with color indicating the module and thickness representing its contribution
4.1 MAIN RESULT
Table 1: Results of the Long-Term Time Series Forecasting task. We averaged the results across four prediction lengths: (96, 192, 336, 7203. The best result is indicated in bold, and the second-best result is underlined.
Numerion
PatchMLP
TimeMiser [Transforuerr
TexHilter
Paitillter
PatebTTT
[Crosformer
ThmesNet |FEiformer
DLincar
MAE
MSF
MAF MSE
MAF
MAR
|AA
MAF.
MSE
MAF
MSE
[MAR
MAR MSF
MSR
MAE
ETThi
0.414
01456
0.447
[0449
0:440
0.469
0.450
0.484
0.408
0.388
0.364
0:307
0.375
0.409
0.388
0.383
0.407
0.378
0.407 0.387
0.684
0.427
0.414
0.599
ETTml
0.378
0.370
0.391
0.39
0.398
0.381
0.410
0:410
0.401
0.368
0.384
0.400
0.387
0.495
0.513
0.606 0.600
0.422
0.448
0.407
0.403
ETTu2
0.315
0.270
0.326
0.282
0.325
1.275
0.332
0.258
0.328
0.285
0,322
0.276
0.326
0.281
0.611
0.757
0.333
0.291
0.349
0.305
0.401
0.550
Wather
0.271
0.26
0.222
0.240
0.277
0.346
0.278
0.258
0.22
0.245
0.274
0.248
0.781
0.359
0.320
0.264
0.284
0.250
0.360
0.309 0.317
0.265
Solar Energy
0.262
0.24
0.202
0. 809
0.278
0.342
0.284
0.295
0226
0.298
0.200
0.277
0.307
0.270
0.443
0.006
0.374
0.401
0.383
0.328 0.401
0.330
Electricity
0,267
0.181
0:275
0.293
0.273
0.182
0.270
0.178
0.285
0.198
0.278
0.197
0.290
0.205
0.334
0.244
0,301
0.193
0.327
0.214 0,300
0.212
Traffie
0.281
0.468
11258
0.485
01298
0.485
0.263
0.428
0.300
0.470
I.340
0.521
0.304
0.481
0.426
0.667
0.336
0.620
0.376
0.620
0.383 0.625
Exchango
0.399
0.555
0.432
0.416
0.43 0.391
0,470
0,519
464
0.515
0.414
0.379
0.404 0.367
0.707
0.940
0.443
0.416
0.299
0.519
0.414 0.354
Table I showcases the comparison results of our method against other approaches, with all out- comes representing the average across four prediction lengths (96, 192. 336, 720}. The complete experimental results are detailed in Appendix B and error bar are provided in Appendix F It is evident that our method achieves the best MAE across all datasets, particularly on datasets such as ETThl and ETTh2, where the performance improvement is substantial, with an overall average MAE enhancement exceeding 4%. Our method consistently ranks within the top two in terms of MSE, demonstrating improvement of over 3% the ETT datasets. On datasets with more variables. such as Traffic, our method, being variable-ing pitiaabl-indeeeddent, occasionally exhibits slightly lower MSE performance compared to variable-fusion models like timemixer and itransformer. Nonetheless, our method demonstrates significant advancements over other models.
4.2 VISUALIZATION ANALYSIS OF RHR-MLP
To better understand the functional roles of different RHR-MLP in our model, we visualized the outputs of each RHR-MLP under the best best-performing configuration. As shown in Figure 3 and detailed further in Appendix J.T] a clear trequency-based specialization emerges: high- high-dimensional hypercomplex layers are primarily responsible for capturing lowfrrequency, fow-dimensional trend components, whereas real and complex-vai -vauudd layers focus on high-frequency, high-dimg high-dimensional fluctuations, such as local seasonal signals and noise. This emergent decomposition enables the model to preserve long-term trend accuracy without compromising the on short-term patterns.
Quantitatively, the visualization shows that hypercomplex layers contribute about 60% of the final output, with the hexadecimal layer alone accounting for nearly 40%, underscoring its dominant role in modeling temporal trends. In contrast, the real and complex layers- sprecsalize for mid- to fegg-frequency content-contribute the remaining 40%. This separation is not due to any handcrafted constraints predefined modules. Instead, it emerges naturally from training, allowing the model to
7

---

learn flexible statistical decomposition directly from data. We offer theoretical insights into this phenomenon in Appendix L.I] conjecturing that hypercomplex representations naturally introduce an implicit stable rank reduction and low-pass bias via symmetry-inducee spectral degeneracies. This structural perspective helps explain the method's built-in regularization, robustness to measurement noise, and strong capacity for modeling long-horizon trends, standing in contrast to classical methods that rely on rigid assumptions. To further support these visualization results, we also present a quantitative statistical analysis Appendix K which independently reaches similar conclusion, reinforcing our interpretation.
These insights suggest that low- fwweeueuency components may inherently pose greater modeling chal- lenges. and that addressing limitations in low-frequency prediction could further enhance the model's overall
its ability to represent high-frequency structures. Future work may refine the allocation of representational resources to better balance the frequency spectrum.
4.3
HLYPER-PARAMETO STUDY
Analysis on Patch Level (Electricity)
Analysis en Patch Embedding Dim (Electricity)
Analysis on Number of Layer (Electricitt)
46
0.22
0.22
0.24
192
195
0.20
720
H 0.20
720
720
0.20
0.38
0.16
0.16
0.36
3
1632
64
128
256
2
Patch Level
Patch Embedding Dim
Number Layer
Analysis FeedForward Dim (Electricity)
Analysis Dropout Rate (Electricity)
Analysis Learning Rate (Electricity)
96
336
192
0.24
192
336
0.22
162
336
0.22
720
0.20
720
0.20
0.18
0.36
0.16
3264 128
512
0.3
0.9
0.005
0.01
FeedForward Dim
Dropout Bale
Learning Bate
Figure 4: Hyper- paremmerreettr Study on Electricity with P (96, 192, 336, 720)
We performed a sensitivity analysis on the Electricity dataset for four prediction lengths, focusing on key parameters: patch level l, in Multi-Level Patch Embedding, patch embedding dimension de. RHR-MLP layer number, feedforward dimension, dropout rate, and learning rate. The results, shown in Figure 4 indicate that the Numerion model is less sensitive to patch-related parameters, favoring larger dimensions to capture more information, though performance gains are marginal. Conversely, it is more sensitive to RHR-MLJ parameters. Additionally, Numerion benefits from larger dropout rates and lower learning rates, improving its ability to fit and generalize in high-dimensional spaces. Additional parameter studys are provided in Appendix C
4.4 ABLATION STUDY
We conducted ablation experiments on ETThl (P=720) and Electricity (P=96) to evaluate each component of Numerion (Table (). Removing the Multi-Level Patch Embedding caused sharp degradation, confirming role in capturing multi-scale temporal cues. Excluding any power-of- two RHR-MLP also weakened performance, showing that distinct hypercomplex spaces specialize in different temporal patterns, with higher dimensions particularly effective for low-frequency periodicity. Adaptive Fusion proved essential, as its removal greatly harmed results on Electricity. A detailed breakdown of 17 ablation cases is in Appendix D
We also ablated the hypercomplex settings (Table 6 Appendix D). Removing the HNTanh activation or either of its parts, tanh on the modulus or p-norm normalization consistently degraded performance, despite minor dataset-speciffc gains (e.g., ETTml). Full HNTTnnh provided the most
8

---

Table 2: Ablation Study
ETThi with P=720
Electricity with P=96
MSE
MAE
MSE
MAE
Numerion
0.449
0.449
0.152
0.239
W/o Multi-Level Patch
0.455
-1.34%
0.456
-1.56%
0.164
-7.89%
0.250
-4.60%
w/o Real
0.451
-0.45%
0.451
-0.45%
0.160
-5.26%
0,246
-2.93%
w/o Comp
0.244
-2.09%
w/o Qual
0.453
-0.89%
0.451
-0.45%
0.160
-5.26%
0.246
-2.3%%
w/o Octo
0.453
-0.89%
0.455
-1.34%
0.161
-5.92%
0,247
-3.35%
0.454
-1.11%
-1.56%
0.166
-9.21%
0.252
-5.44%
w/o Adaptive Fusion
0,450
-0.22%
0.451
-0.45%
0.177
-16.45%
0.263
-10.04%
stable improvements, underscoring the synergy both components. To isolate dimensional effects, expanded real inputs for standard MLP with comparable parameters. Its inferior results confirmed that hypercomplex gains stem not from dimensionality but from structured multiplication and phase coupling, which enforce stronger inductive bias for frequency decomposition and multichannel interactions.
4.5 EFFICIENCY ANALYSIS
We assessed the efficiency of our model in comparison with several baseline models using a single NVIDIA A5000 GPU and Intel Xeon Gold 6326 CPU. As illustrated in Figure our model exhibits competitive performance while maintaining reasonable resource consumption. Although incurs higher computational overhead and parameter count due to the simulation of hypercomplex operations using real numbers, overall efficiency remains practical and acceptable for real-world applications. It is important to contextualize this overhead: the inflation in parameters and memory is primarily software-leve artifact stemming from the current lack of native framework support and hardware acceleration for multi-dimensional hypercomplex algebra. Despite the necessity of simulating these spaces relative to standard real-valued linear layers, Numerion's 's MLPP-ased architecture inherently preserves highly favorable computational complexity compared heavier Transformer- or Mamba- based models. Furthermore, these resource constraints are not fundamental architectural limitations. As the field increasingly recognizes the representational power of hypercomplex spaces, future dedi- cated low-level implementations will significantly reduce memory and time requirements, mirroring the historical optimization trajectory of complex-vall -vauued neural networks. Detailed reasoning behind these design choices and efficiency trade-off's is provided in the Appendix F
Efficiency Experiments ETThl
Efficiency Experiments on Solar Energy
1.22
0.34
1.20
0.30
0.40
0.38
9:24
0.36
0:23
0.34
100
700
300
400
1001
6.00
7101
100
000
9930
10000
5000
10300
15000
40200
41000
5000)
19000
(9100
650E0
70000
mas opoch
mB epoch
Figure Efficiency Analysis on ETTh! and Solar Energy dataset
RELATED WORK
5.1
TIME SERIES FORECASTING
In this section, classify most time series forecasting tasks based whether they adopt the standard benchmark datasets widely recognized by the academic community, as used in our work. We focus particularly those studies that inspired the design of our model. First, number of classical models

---

based on MLP architectures are closely related to ours. These include N- N-BEAT SreeSkSresskinn et al. (2019)
N-HITSChallu et al. (002)) MLinearLi et al. (2023). DLinearZeng et al. (003)) Pat Tang &
Zhang 2025), FilterNet al. (2024) Net et al. (202)). TimeMixer Wang et al. (024c))
and its extension l'imeMixe++ Wang et al. (024b). Among them, TimeMixer combines linear and sampling layers implement bottom-up and top-down mixing, achieving unique trend-seasonality
decomposition that guided our exploration. MLinear adopts Mixture of Experts (MoE) framework that combines predictions from multiple linear layers. Its design and aggregation mechanism provided key insights into the architectural design of our own model.
Secondly, Transformer- based models form another major line of research in time series forecasting. Prominent models in this category include InformefZhou et al. (2021). AutoformerWi Wu et al.
(222 ).
FEDformetZhou et al. (2022b). CrossFormerZZang & Yan (2023) Na al. 2022).
STAFformerLLiu et al. (2023a). L'raansformerLuu et al. (2023b). TimeXerWang et al. (202dd) and
Timer-XI al. 2024). Notable examples such as Autoformer et al. (2021). FEDiormetZhou al. (022b)) and PatchTSTNNie et al. (2022) explored trend-season decomposition and hierarchical embeddings tailored for time series, laying groundwork that we build upon. Other models like iTrans- formerLiu et al. (023b) and Timer-XILin et al. (2024) address temporal ordering in multivariate settings, providing insights into sequence processing strategies that inform our design. Many of the latest methods, such FBMMang al. 20244), have mined undiscovered temporal patterns from the frequency domain.
Several recent works based the novel Mamba architecture, such as Time-MachnnAAhamae & Cheng (202)), BIGSTMYYan et al. (2025b) and Bi-Mamb: Tang et al. (2024), as well as those using spatio-temporal graph convolutional networks like AGCRN Bai et al. (2020) and ASTGCNGuo et al. (2019), have contributed meaningfully to the field. Other advances include multi-channel aggregation methods (e.g., CCMChen et al. (2024)). contrastive learning techniques (RCL Yan al. (2025a)), and label refinement approaches (FreDF Wang et al. (2024a)). These works have all made outstanding contributions time series forecasting.
5.2 HYPERCOMPLEX NEURAL NETWORKS
Although hypercomplex neural networks are not yet widely adopted in machine learning partly due to the challenges discussed in Appendix[[Ittee they offer valuable insights relevant to our work. One recent study /Zhang et al. (2021) proposes parameter-efficien design with learnable hypercomplex number multiplication, inspiring improvements in our model structure. Another work Lopez et al.) (2024) introduces cosine similarity transformation aligned with feature weights, enabling direct visualization of attention in quaternion convolutional layers and guiding our interpretability approach. Additional research on hypercomplex networks Comminiello et al. (2024), including quaternion convolutions Ciamirano-Gomee & Gershenson (2023), quatermion transformers Singh et al. (2024). and octonion and sedenion networks [Popal (2016); : Pavlov et al.) (2023), also contributes ideas, particularly for our activation function design. A more detailed theoretical connection between hypercomplex representations and time series modeling is provided Appendix
CONCLUSION
In this paper, we extend linear layers and the Tanh activation hypercomplex spaces of arbitrary power-of-two dimensions, providing a unified framework with the real domain and an efficient computation method. Building on this, we propose the Hypercollleo tron (RHR-MLP), which transforms inputs into hypercomplex spaces for enhanced modeling. This leads to Numerion, forecasting model that integrates multiple hypercomplex spaces to decompose, model, and adaptively fuse multi-frequene components. Numerion is simple and interpretable, with experiments confirming its effectiveness and showing that digeee-dimesiional spaces capture low- fww-ureyencyy features. Looking forward. as noted in Appendix expect progress from exploring the characteristics of hypercomplex spaces, advancing theoretical analysis of their representational power, and developing optimizers and hardware tailored to hypercomplex arithmetic.
Multi-Layer Percep-
10

---

ACKNOWLEDGMENTS
This work is supported by the National Natural Science Foundation China (Grant No. 62276008, 62250037, and 62076010), and partially supported by the National Key R&D of China (Grant #2022YFF08006011,
REFERENCES
Md Atik Ahamed and Qiang Cheng. Timemachine: A time series is worth 4 mambas for long-term forecasting. In ECAI 2024: 27th European Conference Artificial Intelligence, 19-24 October 2024, Santiago de Compostela, Spain-Including 13th Conference on Prestigious Applications of Intelligent Systems. European Conference on Artificial Intelli, volume 392. pp. 1688, 2024.
Silvia Maria Alessio. Spectral Analysis of Deterministic Discrete-Timme Signals, pp. 153-182. Springer International Publishing, Cham, 2016. ISBN 978-3-319-25566-5. 25468-2688-
doi: 10. 10.1007/
978-3-319-25468-5555 URL sttps://don wtg.
Gerardo Coamirano-GGomer and Carlos Gershenson. Quaternion convolutional neural networks: Cur- rent advances and future directions, 2023,
Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in neural information processing systems, 33:17804 17815, 2020.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arxiv. arXiv preprint arXiv:/803.01271, 10, 2018.
ukasz Baszczyk and Kajetana M. Snopek. Octonion fourier transform of real-valued functions of three variables selected properties and examples. Signal Process., 136(C):29-37, July 2017. ISSN 0165 1684. doi: 10. si6p:i.ipproo221..1111
sigpro. 2016 6, 11.1 021
Younis Ahmad Bhat and Neyaz A. Sheikh. Windowed octonionic fourier transform. Circuits, Systems, and Signal Processing, 42(5): May 2023. ISSN 1531-5878. doi: 10.1007/ 5000034-02--0224-- URLhttps: /o1i http:///ooooro//o/o//oo//o -220000000000000000000000002---0
Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: Neural hierarchical interpolation for time series forecasting. In Proceedings of the AAAI conference artificial intelligence, volume 37, pp. 6989-6997, 2023.
Jialin Chen, Jan Eric Lenssen, Aosong Feng, Weihua Hu, Matthias Fey, Leandros Tassiulas, Jure Leskovec, and Rex Ying. From similarity to superiority: Channel clustering for time series forecasting. Advances in Neural Information Processing Systems, 37:153665-130666 2024.
Dong Cheng and Kit Ian Kou. Plancherel theorem and quaternion fourier transform for square integrable functions. Complex Variables and Elliptic Equations, 64(2):223-242, Jamuary 2018 ISSN 1747-6941. doi: 10. 100110007777693..00 14127080. URL http:://txx doi. www.orrggll.... 1080/17476933 211116633..11...... 1427080
Danilo Comminiello, Elconora Grassucci, Danilo P. Mandic, and Aurelio Uncini. Demystifying the hypercomplex: Inductive biases in hypercomplex deep learning. IEEE Signal Processing Magazine, 41(3):59-71, 2024, doi: 10. 10.1099022222244400000
Todd Anthony Ell. Hypercomplex spectral transformations. PhD thesis, USA, 1992. UMI Order No. GAX92-31031.
Peter J. Forrester. Octonions in random matrix theory. Proceedings of the Royal Society A: Mathe- matical, Physical and Engineering Sciences, 473(2200):0016880000 April 2017. ISSN 1471-2948 doi: 10. (098///pa.206..08800
rspa 201 6 0800
D. A. S. Fraser. On Information in Statistics. The Anmals of Mathematical Statissssss 36(3):890 896, 1965. doi: 1 177700011. URL httL:/tttt///.o..o..t /ttp::/:://011..0011....222.. aoms/ 1177700061
11

---

Sara Fridovich-Keil. Raphael Gontijo-Lopes and Rebecca Roelofs. Spectral bias in practice: The role of function frequency in generalization, 2022. URL https: http::://axxvvvvvo/o org/ abss/////// /2110
02424
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 922-929, 2019.
S. Hahn and Kajetana Snopek. The unified theory of n-dimensional complex and hypercomplex analytic signals. Bulletin of the Polish Academy of Sciences. Technical Sciences, 59, 06 201 1. doi: 10.2478/v 10175-011-00220
Matz A. Haugen, Bala Rajaratnam, and Paul Switzer. Extracting common time trends from concurrent time series: Maximum autocorrelation factors with application tree ring time series data. arXiv: Methodology, 2015. URL https: /api. www.eeeiiiie..eeeeee cscho ar. www.oorr///ooo...
88513313
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information reirieval, pp. 95-104, 2018.
Wei Li, Xiangxu Meng, Chuhao Chen, and Jianing Chen. Mlinear: Rethink the linear model for time-series forecasting. arXiv preprint arXiv:2305. 04800, 2023.
Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Staeformer: spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting, arXiv preprint arXiv:2308) 10425, 2023a.
Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 55:5816-5825 2022.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.0660 06625, 2023b.
Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer-xl: Long-context transformers for unified time series forecasting. arXiv preprint arXiv:2410.04880 2024.
Eleonora Lopez, Eleonora Grassucci, Debora Capriotti, and Danilo Comminiello. Towards explaining 0ttp//2433.1999 17929
hypercomplex neural networks, 2024.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64
words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14733 2022.
Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905. 10437, 2019.
Stanislav Pavlov, Dmitry Kozlov, Mikhail Bakulin, Aleksandr Zuev, Andrey Latyshev, and Alexander Beliaev. Generalization of neural networks on second-order hypercomplex numbers. Mathematics, 11(18), 2023. ISSN 2227-7390. doi: 10.3390/math 183973. URL sittps://wwe mdpi
com/ 200m/2227-1399/1/////999
Calin-Adrian Popa. Octonion-valued neural networks. In Alessandro E.P. Villa, Paolo Masull,, and Antonio Javier Pons Rivero (eds.), Artificial Neural Networks and Machine Learning-ICA. ICANN 2016, pp. 435 443, Cham, 2016. Springer International Publishing. ISBN 978 978-3-319-44778-0.
Helmy Rahadian, Steven Bandong, Augie Widyotriatmo, and Endra Joelianto. Image encoding selection based on pearson correlation coefficient for time series anomaly detection. Alexandria Engineering Journal, 82:304-322 2023. ISSN 1110-0168. doi: http:://to.oo//100001/// 2023.09 09.300 URL https: www.:/ssiiinnndeee.......m. com http:///wwe...ecooococc////aaaaa
S 002
12

---

Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks, 2019. URL https://arxiv wwwpg///w.//......///. 08734
Sayed Salehi, Rasoul Amirfattahi, and Keshab Parhi. Pipelined architectures for real-valued fit and hermitian-symmmrricc ifft with real datapaths. Circuits and Systems II: Express Briefs, IEEE Transactions on, 60:507-511, 08 2013. doi: 10.1 119/TTCS11.0013.22666
S.J. Sangwine. Fourier transforms of colour images using quatemninn or hypercomplex, num- bers. Electronics Letters, 32:1979-1980, 1996. doi: 10.11439/ URL https
Shun- Shih, Kan-Keng Sun, and Hung-yi Lee. Temporal pattern attention for multivariate time series forecasting. Machine Learning, 108:1421-1444 2019.
Sukhendra Singh, Sushil Kumar, and B. K. Tripathi. A comprehensive analysis of quaternion deep neural networks: Architectures, applications, challenges. and future scope. Archives of Computa- tional Methods in Engineering, Dec 2024. ISSN 1886- 1784. doi: 10. 1831-024-- 10211661-.1
URL httttttss:///oo.o. /rtp:::////ooo//////////// 0. 1007 00000//1111 024---0--2---0 10216 6-1
A. Sudbery. Quaternionic analysis. Mathematical Proceedings of the Cambridge Philosophical Sociity, 85(2):199-223 1979. doi: 10.1017/80305004 1515500000005566 100055638.
Peiwang Tang and Weitai Zhang. Unlocking the power of patch: Patch-based mlp for long-term time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 12640-12648, 2025.
Shengkun Tang, Liqun Ma, Haonan Li, Mingjie Sun, and Zhiqiang Shen. Bi-mamba: Towards accurate 1-bit state space models. arXiv preprint arXiv:2411.11844 2024.
Hao Wang, Licheng Pan, Zhichao Chen, Degui Yang. Sen Zhang. Yifei Yang. Xinggao Liu, Haox- uan Li, and Dacheng Tao. Fredf: Learning to forecast in frequency domain. arXiv preprint 240.0299900 2024a.
Shiyu Wang. Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Shengtong Ju, Zhixuan Chu, and Ming Jin. Timemixer++: general time series pattern machine for universal predictive analysis. arXiv preprint 1rXiv:2410.16033 2024b.
Shiyu Wang. Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint arXiv:2405. 14616, 2024c.
Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jianmin Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. arXiv preprint arXiv:2402. 1t..22222.99977 /9072, 2024d.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances neural information processing systems, 34:22419-9222 22430,, 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022.
Dongpo Xu, Cyrus Jahanchahi, Clive Took, and Danilo P Mandic, Quaternion derivatives: The ghr calculus. Royal Society Open Science, 158)55 2015.
Wenbo Yan, Hanzhong Cao, and Ying Tan. Repetitive contrastive learning enhances mamba's selectivity in time series prediction. Neural Networks. pp. 108290, 2025a.
Wenbo Yan, Shurui Wang, and Ying Tan. Hierarchical information guidedd spatio-tempont mamba for stock time series forecasting. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 22-40. Springer, 2025b.
13

---

Runze Yang, Longbing Cao, JIE YANG, et al. Rethinking fourier transform from basis functions perspective for long-term time series forecasting. Advances in Neural Information Processing Systems, 37:8515-8544 2024.
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mips are more effective learners in time series forecasting. Advances in Neural Information Processing Systems, 36:76656-7669, 2023.
Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, and Wei Fan. Filterntt Harnsssing frequency filters for time series forecasting, Advances in Neural Information Processing Systems, 35:551 2024.
Joseph P. Zbilut and Norbert Marwan. The wiener-khinehin theorem and recurrence quantification. Physics Letters A, 372(44):6622 6626, 2008. ISSN 0375-9601. doi: http:///bb.orrg/10.0010j physleta.2008.09.. 09.009902222. 027. URL https:// httpp:://wwwww.o
e11/8037596010801110
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp.
11121-11128, 2023.
Aston Zhang, Yi Tay, Shuai Zhang, Alvin Chan, Anh Tuan Luu, Siu Cheung Hui, and Jie Fu, Beyond fully-connected layers with quaternions: Parameterization of hypercomplex multiplications with 1/n parameters, 2021. URL http://psss//rsv.ooo/////// htwp:///g..///.......... 02820
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting, In The eleventh international conference on learning representations, 2023.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-serics forecasting. In Proceedings of the AAAI conference artificial intelligence, volume 35, pp. 11106-11115, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, al. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in neural information processing systems, 35:12677-12690, 2022a.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pp. 37268-27286 PMLR, 2022b.
14

---

APPENDIX
A EXPERIMENTS SETTING SUPPLEMENTS
A.1 DATASETS
We evaluated the performance the model on nine publicly available long-term time series fore- casting datasets: Weather, Solar Energy, Electricity, Traffic, Exchange, ETTh1, ETTh2, ETTml, and ETTm2. Detailed information about the datasets is provided in Table
Table 3: Basic Information Dataset.
Dataset
Data Partition
Frequency
Feature Dim
Type
Weather
06692,5271.10544
10min
21
Weather
Solar Energy
(36601,5161.004)
10min
137
Electricity
Electricity
(18317,2633,526)
Hourly
321
Electricity
Traffic
(12185,1757.3509)
Hourly
862
Traffic
Exchange
(5120,665,
Daily
8
Weather
ETTh1
(8545,2881
15min
Temperature
ETTh2
(8545,2881,288)
15min
Temperature
ETTml
(151,1111))(111521
15min
7
Temperature
ETTm2
(34465,11521,11522
15min
7
Temperature
A.2 BASELINES
We selected variety of state-of-the-art baselines to compare with our method order to evaluate the performance of Numerion. Specifically, utilize Transformer-basee models: FEDformer, Crossformer, iTransformer, and PatchTST; convolutional neural network (CNN)-based models: TimesNet; MLP-based models: TimeMixer, PatchMLP, and DLinear; well filter-based models: TexFilter and PaiFilter.
A.3 METRICS
We evaluated all models using Mean Squared Error (MSE) and Mean Absolute Error (MAE) as the evaluation metrics.
MAE
N Σ Y, Y,
MSE Σ (Y; Y,)
A.4 EXPERIMENT DETAILS
We repeat each experiment three times server equipped with an NVIDIA GeForce RTX 3090 GPU and an AMD EPYC 7282 16-Core Processor. We employ Adam as the optimizer, with the learning rate between le-4 and 1e-2 depending the dataset. For Numerion, the patch level is configured between and 4, the patch embedding dimension is set between 64 and 256, the number of RHR-MLP layers is set with subsequent HLinear dimensions decreasing, and the dropout rate is set between 0.5 and 0.7. For datasets with feature number greater than 100, the batch size is set 100, while for other datasets, the batch size is set to 512. The HLinear layers are initialized using random sampling from standard normal distribution, followed by normalization. and the biases are initialized as zero vectors.
3, the initial dimension of RHR-MLP is between 64 and 256
B FULL RESULTS
Table 4 provides the detailed comparison results the Long- Long-Term Time Series Forecasting task. We set the input length to 96 and the prediction lengths to {96, 192, 336, 720). Avg is averaged from all four prediction lengths.
15

---

Table 4: Full results for the Long-Term Time Series Forecasting task. We compare competitive models under different prediction lengths 19,,1922336,7200, Avg is averaged from all four prediction lengths.
Numerioa
PatchMLP
Timehliser
FTrandormerr
TexFilter
Paiiiittr
PatchTST
Crosdoemer
TimesNet
PEDGocmer
DLinear
Model
MAE
MSE
MAE MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE MSE
MAE MSE
MAE
MSE
MAE MSE
MAR MSE
MAE MSG
96
0,380
0.359
0,350
0,385
0.400
0.325
0,405
0.386
0,402
0,382
0.384
0.375
0,419 0.414
0,448
0.423
0.402 0.384
0.424
0.395
0,400 0.386
192
0.409
0.407
0.431
0.458
0.421
0.429
0,512
0.441
00.429
0.430
0-422
0.466
0.445
0.460
0.474
0.471
(0.429
0:436
11:770
0.469
D.AR2 0.437
336
0.429
0.444
0,500
0:458
0.487
00:551
0.472
0.443
0.476
(.466
0,501
0.546
0.570
(0,469
1.499
0,530
0:489
0.481
720
0,449
0.447
0.468
0,482
0.462
0,498
0,491
0,503
0:475
0,681
0:469
0.674
0.468
0,500
0.621
0.683
(0,500
0,521
0.564
0.996
0,516 0,519
0.417
0.414
0:437
0:467
0.454
0:439
0.441
0.422
0.448
0.454
0.469
0.522
(0.450
0:458
0:484
0.408
0:452 0:456
96
0.326
0.279
0.344
0.296
0.345
0.296
0,349
0.297
0.343
0.293
0.343
0.292
0,348
0,302
0.584
0.745
01.374
0.540
0.399
0.358
0.387 0.335
0.379
0.389
0.
0.506
0.374
0:395
0.369
(,400
0,358
0.686
0,400
0:499
0.479
0:476 0:477
ETTh2
336
0.416
0.406
0,324
0,417
0.428
0.417
0,432
0.428
0,430
0.417
0.432
0.420
0.433
0.426
0,731
1,043
00,452 0,452
0.487
0.496
(1544 0,594
720
0.433
0.414
0.432
0.415
0:445
0,649
0.405
0,430
0.446
0,431
0.462
0.474
0.463
0.657 0,831
0,388
0.364
0.392
0.375
0.409
0.988
0.407
0.583
0-407
0,883
0-404
0.378
0.407
0.857
0.684
0.942
00.427 0.434
0.449
0:487
0.515 0,559
0.337
0.305
0.361
0,521
0.358
0.967
0.339
0.375
0.598
0:419
0:599
192
0,367
0.386
0.391
0,363
0
0.361
0.361
0,999
0,990
0,387
0,967
0.383
0,364
0.365
0.967
0.451
0.450
0..887
0.594
0.441
0:426
0.389 0,380
ETTml
336
0.380
0.297
0.350
0.420
0.426
0.409
0.401
0.405
0.596
0.410
0.399
0,515
(0,413
0.430
0.459
0.445
0,413 0.413
720
0.423
0.439
0.448
0,499
0.441 0.454
0,459
0.491
0.448
0.477
0.444
0,456
(1450 0.454
0.589
0.666
00.450
0.478
0.490
0,543
0.453 0.474
0.377
0.370
0.394
(0:410
0.410
00:401
0,392
0.395
0.384 0.400
0.367
0.495
0.513
(0,406
0.400
0:452
0.448
0.407 0:403
96
0,249
0.170
0.259
0,178
0.258
0,175
0,264
0.180
0.258
0,175
0.287
0.174
0,259
0,175
0,366
0.267
[0.267
0.187
0.287
0,205
0.292 0.195
192
0.292
0.231
0.299
0.237
0,300
0,250
003001
0.240
0.300 0.240
0.300
0.241
0,492
0.414
(0,309
0.249
0.325
0,269
0.362 0.284
KTTm2
336
0.330
0.289
0.341
0,302
0.340
0.298
0,348
0.511
0.347
0,311 0.339
0.207
0.343
0,305
0,542
0.597
(1,35]
0.321
1.366
0,325
(1,477 0,369
0.387
0.390
0.389
0,405
0 399
0.391 0.407
0.412
0.405
0.414
0.393 0.992
0,400
0,402
1.042
1,730
00.40%
0,408
0:415
0:421
0.522 0.554
0.315
0.270
0.326
0.282
0.323
0.275
0,332
0.288
0.328
0,285
0.322
0.276
(.326
0.261
0.611
0.757
003333
0.291
0.349
0,305
01.401 0,350
96
0.203
0.189
0.304
0.164
0.166
0.214
0.174
0.207
0.162
0:310
0.164
0.218
0.177
0.221
0.195
00.111
0.170
0.206
0.217
0.255 0.196
192
0.250
00.111
0.250
0,212
0.253 0.209
0,255
0.221
0.250
0.210
0.252
0,214
(0259
0.235
0.277 0.209
00,290
0.223
0.336
0.276
n.296 0.237
Weather
336
0.289
0.270
0.293
0,269
0 0.293
0.264
0.296
0.278
0.290
0,265
0.293
0,268
0.297
0,218
0.332
0.273
00,006
0,280
0.380
0,339
0.338 0,283
720
0.345
0,350
0.347
0.340
0.342
0.342
0.344
0.348
0.354
0.401
00.399
0.965
11:266
0.345
pvg
0.271
0.230
0.272
0,249
0.277 0.246
0.278
0,258
0.222
6.345
0.274
0.248
0.281
0,219
0.320
0.264
0..284
0,259
0.360
0,509
0.317 0,265
96
0.244
0.209
0,200
0,769
0.254
0,236
0.258
0.250
0.246
0.278
0.245
0.254
0.296
0.234
0.302
0.333
0.355
0.573
0.341
0.286
0.376 0.290
192
0.261
0.250
0.293
0.314
0.284
0.235
0,286
0.298
0.283
0.297
0.281
0.272
0,310
0,267
0.410
0.371
0.3376
0.397
0.337
0.291
0.398 0,320
Solar- Energy
336
0.270
0.273
0,304
0,325
0.267
0.244
0.296
0.334
0.305
0.307
0.315
0.290
0,315
0,290
0,515
0.485
0.380
0.420
0.416
0.354
0.415 0.355
0.272
0.224
0.903
0.327
0.286
0.253
0.204
0,319
0.311
0,509
0.318
0.202
0.317
0.269
0.542
0.526
(0.38)
0.420
n.487
0.580
0.413 0.356
axg
0.262
0.252
0.292
0.242
0.284
0.295
0.286
0.298
0.290
0.277
0.307
0,270 0,442
0.337
0.403
0.383
0.328
0.403 0,330
96
10.399
0.152
0.255
0.065
0.247
0.153
0,240
0.148
0.259
0.169
0.256
0.175
0.290
0.181
0.314
0.219
10.272
0.168
0.308
0.195
0.282 0.197
192
0.251
0.167
0.255
0.368
0:253
0.162
0.275
0.183
0.264
0.182
0.274 0.188 0.322
0.231
00.322
0.184
0.315
0.201
0.285 0.196
Electricity
996
0.220
0.182
0.276
0.189
0.277
0.185
6.269
0.17%
0:287
0.200
0.279 1.19 0.299 0.204
0.337
0.246
(0.000 0. 198
0.329
0:214
0.301 0.009
720
0.205
0.322
0.313
0.237
0.934 0.246
0.155
0.2267
01.811
0.275
0,190
0.273
0.182
0.270
0.17M
0.2885
0.198
0.378
0.197 0,200 0,205
0.334
0.244
[0.304 0,193
1.327 0,214
0.3000 0,212
0.264
0.441
0.285
0.268
0.395
00.244
0.439
0:336
0.506
0.205
0.462
0.429
(0.32)
0.903
0.366
0.587
0.396 0.650
192
0.276
0.457
0.287
0,064
0.296
0.473
0,226
0.417
0.293
0,555
0.333
0.508
0.296
0.666
0,431 0.665
00,336 0.617
0.373 0.004
0,370 0.598
Traftic
0.465
0.285
0.673
0.335
0,518
0.300 0.652
720
0,301
0.507
0.327
0.547
0.313 0.506
0,302
0.467
01.324
0,512
0.354
0,553 0.322
0.514
0.424
0.683
11330
0.640
[13322
0,626
11.944 0.645
pvg
0.281
0.46
0,282
0.428
0.300
0.470 0.340
0.521
10.304
0.081 0,426
10.336
0.620
0.376
0,520
0.383 0.625
96
6.2000
0.083
1..211
0.092
0.335 0.000
0:213
0.092
0:000
0.DRK
0.207
0:087
0.200
0,088
0.367
0.256
[0.034 0.100
0:976
0.1448
0.215 0.088
192
0.172
0,302
0.181
0.316
0.196 0.305
0.186
0.299
0.176 0.509
00.344
0.226
0.315
0.271
0.315 0.176
Exchonge
336
0.410
0.322
0.448
0.386
0.473
0.353
0.435
0.360
0.490
0.458
0.437
0.367
0.397
0.301 0.983 1.268
00.448
0.367
0.427
0.460
0.427 0.313
720
0.855
0.767
1.004
0.984
069288
0.541
1,318 0.706 0.874
0.714 0.901
1.767
(0.746
0.964
0.605
1.195
01605 0.839
avg
0.399
0.358
0.432
0.416
0.453
0
00.470
0,519
0..644
0,515
0.414 0.379 0.404 0,367
0,707
0.940
00,443 0.436
11,429
0.519
0.414 0.354
16

---

C ADDITIONAL HYPER-PARAMETER STUDY
Analysis on Patch Level (ETTm2)
Analysis on Patch Embedding Dim (ETTm2)
Analysis on Number of Layer (ETTm2)
0.40
0.40
192
192
192
336
0.35
336
336
720
720
720
0.30
0.25
0.25
0.20
0.20
0.0
1632
64
128
256
Patch Level
Patch Embedding Dim
Number of Layer
Analysii FeedForward Dim (ETTm2)
Analysis Dropout Rate (ETTm2)
Analysis on Learring Rate (ETTm2)
96
0.40
96
0.0
96
192
192
192
336
316
0.35
336
0.30
0.25
0.25
0.5
88
0.20
0.20
0.20
32.64 64
128
256
512
0.1
0.3
0.5
0.7
0.9
0.01
FeedFFoward Bim
Dropout. Bate
Leaming Rate
Figure 6: Additional Hyper- Study on ETTm2 with P (96, 192, 336, 720}
Analysis on Patch Level
Analysis on Patch Embedding Dim
Analysis on Number of Layer
0.46
0.46
0.44
0.44
0.44
96
0.42
0.42
192
0.42
96
0.40
0.40
96
0.38
336
720
720
0.36
16332
64
256
Patch
Patch Embodding Dim
Number of Layer
Analysis on FendForward Dim
Analysis Dropout Rate
Analysis on Learning Rate
0.46
0.46
0.40
192
0.40
0.40
96
336
0.38
336
536
720
720
720
0.36
0.36
1264
1.28
256
512
0.1
0.3
0.5
0.7
0.9
oggiles
a00s
0.01
FoodFooward Dim
Dropout Rate
Loaming
Figure 7: Additional Slyper-parameter Study ETTh1 with P {96, 192, 336, 720}
We further conducted hyperparameter experiments on the ETTh] and ETTm2 dataset, and the results are presented in Figure 6 and Figure 7 Overall, our method demonstrates insensitivity to patch parameters, moderate sensitivity to RHR-MI.P parameters, and high sensitivity to the learning rate, consistent with findings from prior parameter analyses. For shorter prediction lengths, the model tends to utilize fewer layers and smaller widths, while for longer prediction lengths, it requires more layers, larger widths, and higher patch levels to enhance information capture. The model achieves superior prediction performance at lower learning rates, albeit at the cost of increased training time.
D ADDITIONAL. ABLATION STUDY
In Table we provide a comprehensive ablation study. In Cases 1-5, we individually remove each hypercomplex space (RHR-MLP), and the results indicate that eliminating any layer results in performance degradation. Higher-dimensional hypercomplex spaces exhibit a more substantial impact on performance. Moreover, since adjacent dimensions partially compensate for the missing
17

---

Table The full results of the ablation study. The check mark (V) and the wrong mark (x) indicate with and without components.
Hyper Hyper-Complex
ETThl with P=720
Electricity with P=96
ETThi with P-96
Patch
Real
Comp
Quat
Octo
Sede
Adaptive Fusion
MSR
MAR
MSR
MAE
MSE
MAE
0.451
0.451
0.160
0.246
0.367
0.384
0.451
0.452
0.156
0.244
0.369
0.387
0,453
0.451
0.160
0.246
0.360
0.384
0.161
0.387
0.454
0.456
0.166
0,252
0.369
0. 0,386
0.45
0.451
0.177
0.263
0.375
0.391
0.488
0.483
0.235
0.314
0.392
0.469
0.469
0.214
0.292
0.369
0.391
0.461
0.461
0.202
0.281
0.369
0.393
0.457
0.182
0.266
0.394
0.457
0.454
0.180
0.263
0,369
0,394
0.455
0.456
0.164
0.250
0.376
0.396
0.452
0.451
0.178
0.263
0.374
0.395
0.451
0.452
0.178
0,262
0.372
0.393
0.45
0.451
0.176
0.259
0.366
0.386
0.451
0.452
0.170
0.255
0.368
0,390
17
0.45
0.453
0.175
0.259
0.365
0.391
0,449
0.449
0.152
0.2.9
0.359
0.380
spatial information, the performance does decline significantly when single hypercomplex layer is removed. In contrast. Cases 7-11 utilize only single hypercomplex space linear layer, and the notable performance decline emphasizes the importance of integrating multiple hypercomplex linear layers. Each hypercomplex space inherently learns distinct temporal patterns. In Case 12, the removal of Multi-Level Patch Embedding leads a performance drop, highlighting the necessity of incorporating temporal information of varying lengths, which enables RHR-MLP to model multi- period temporal features. In Case 6, the elimination of the Adaptive Fusion mechanism, where predictions from all spaces are fused with equal weights, results in significant performance decline, underscoring the critical role Adaptive Fusion. Case 13-17, We further compare the ablation of each RHR-MLP in the absence of Adaptive Fusion, and the conclusions remain consistent: adjacent hypercomplex spaces can partially compensate for the missing space, causing a performance drop but not drastic decline.
Table 6: Ablation and baseline comparison ETT benchmarks. Lower is better. 'HNTanh' is our full method. Best per (dataset, P. metric) is bold; ties are bolded for all winners.
ETThl
ETTh2
ETTml
ETTm2
Method
P
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MLP
96
0.385
0.370
0.331
0.284
0.343
0.307
0.252
0.173
w/o Activation
96
0.383
0.372
0.331
0.285
0.341
0.313
0.253
0.175
w/o Tanh
96
0.388
0.366
0.334
0.286
0.343
0.306
0.252
0.174
w/o Dividenorm
96
0.385
0.372
0.334
0.289
0.337
0.310
0.251
0.173
HNTanh (Ours)
96
0.380
0.359
0.326
0.279
0.337
0.305
0.249
0.170
ML.P
192
0.413
0.416
0.381
0.364
0.369
0.362
0.292
0.236
w/o Activation
192
0.412
0.420
0.387
0.374
0.369
0.365
0.297
0.238
w/o Tanh
192
0.418
0.410
0.390
0.372
0.367
0.352
0.295
0.239
w/o Dividenorm
192
0.414
0.418
0.384
0.369
0.365
0.362
0.294
0.237
HNTanh (Ours)
192
0.409
0.407
0.379
0.359
0.367
0.356
0.292
0.231
We ablate the proposed hypercomplex activation by disentangling its two components tanh on the modulus and p-norm normalization-and compare against removing activations entirely. On ETT benchmarks, "w/o activation" (only hypercomplex linear layers), "w/o tanh" (y/ = |
and "w/o dividenorm" (y: = x, tanh([[=[((p)) each show occasional wins on single datasets (e.g.. ETTml), but the full HNTanh consistently yields the best aggregate performance. Theoretically, for the unnormalized variant Yi x, (a=h([[==)p)) the Jacobian entries shrink as ||æ||p + +0 (diago- tanh" n) n induction see
nal tanh(n) + [c tanh? n) n--P, off-diagonal ne|æ,|1-
18

---

producing vanishing gradients; adding normalization bounds the input norm and stabilizes gradient flow, matching the empirical gains. We further test a "dimensional expansion" control that pads real inputs (zcro-padding/MLP) and trains standard MLP: despite similar pa- rameter counts, it underperforms, indicating that hypercomplex algebras contribute more than mere width- structured multiplication/phase coupling provides superior inductive bias for frequency decomposition and multichannel interactions.
Appendix H.2
E ADDITIONAL EFFICIENCY EXPERIMENTS
conducted an efficiency evaluation our model alongside a series of related models. The tests were performed single NVIDIA A5000 GPU, with the CPU being an Intel(R) Xeon(R) Gold 6326 2.90GHz. The experimental results on the ETThl dataset are presented Table those the solar-energy dataset shown in Table g
while
Table 7: Comparison of model performance and resource usage in ETThl dataset
Model Name
s/epoch
Parameters
GPU Memory (MB)
MSE
MAE
Numerion
0.48
904,309
700.4
0.360
0.380
TimeMixer
0.71
75,497
1053.76
0.377
0.388
iTransformer
0.20
224,224
156.68
0.378
0.393
DLinear
0.10
18,624
29.18
0.379
0.386
PaiFilter
0.10
49,614
27.17
0.380
0.390
PStchTST
0.17
21,040
173.95
0.381
0.386
TimeXer
0.60
1,390,944
687.13
0.384
0.394
TimesNet
9.94
605,479
10103.31
0.399
0,404
PatchMLP
0.26
2,470,830
506.18
0.399
0.401
Informer
0.79
422,407
1106.81
1.204
0.800
Table 8: Comparison of model performance and resource usage in solar-energy dataset
Model Name s/epoch
Parameters
GPU Memory (MB)
MSE
MAE
Numerion
53.53
1779749
2111.33
0.217
0.250
TimeMixer
50.26
76537
2665.77
0.236
0.254
PaiFilter
1.36
49874
179.23
0.245
0.254
iTransformer
4.17
224224
374.72
0.246
0.255
PatchTST
6.26
21040
353.62
0.246
0.258
TimeXer
40.66
1424224
1746.3
0.248
0.256
TimesNet
59.76
613929
1638.73
0.277
0.296
PatchMLP
13.3
2508530
866.04
0.282
0.280
Informer
9.68
539017
303.38
0.298
0.298
DLinear
0.95
18624
151.84
0.303
0.323
The results indicate that, although our model achieves competitive performance, it occupies a middle ground terms of GPU memory consumption, parameter count, and computational time. Compared to several recent mainstream models, our computational overhead is relatively higher; however, it remains within an acceptable and practical range.
In fact, the core architecture of our model was originally designed with only five linear layers. The increase in parameter count primarily stems from limitations the underlying Py PyTorch framework, which lacks native support for high-dimensional hypercomplex operations. As result, each hyper- complex linear layer is implemented as a pair real-valued linear layers, effectively simulating hypercomplex computations using real operations. This leads to substantial increase in the reported parameter count from five layers to the equivalent of 63 vallev layers.
The relatively large memory footprint is attributed to the design of multiplication matrices used to simulate hypercomplex multiplications. To accelerate hypercomplex multiplication, we precompute and store multiplication matrices, effectively trading additional memory (up to 16-fold increase in
19

---

quadratic space for 16-dimensional hypercomplex numbers) for improved computational speed. This design choice constitutes the primary source of memory overhead.
In terms of computation time, while reduced the complexity of hypercomplex multiplication from quadratic matrix multiplications to a single matrix multiplication step, the absence of native data type support the low-level computational layer still results in slower operations. Additionally, the use of a norm tanh activation function, while essential for ensuring stable transformations in the hypercomplex space, inherently incurs higher computational overhead compared to standard activation functions. This inefficiency is further amplified in the hypercomplex setting. Together, these factors contribute the observed computational overhead.
Nevertheless, despite these inherent challenges posed by the framework and mathematical formulation, the overall efficiency of the model remains tolerable for practical use. Moreover, the architecture leaves room for further optimization, especially with potential future support for hypercomplex data types at the framework level.
F
ERROR BARS
Table 9: Standard deviation across all datasets
ETTh1
ETTh2
ETTml
MSE
MAE
MSE
MAE
MSE
MAE
96
0.380#0.001
0.359=0.001
96
0.326#0.000
0.279=0.001
96
0.337=0.002
0.305=0.001
192
0,409=0.0001
0.407=0.0000
192
0.379#0.001
0.359=0.000
192
0.367=0.003
0.356#0.002
336
0.429#0.001
0.44420.001
336
0.416#0.002
0.40620.002
336
0.386
0.38=0.002
720
0.449+0.001
0.447=0.002
720
0.433+0.001
0.41420.001
720
0.423+0.003
0.439=0.002
ETTm2
Weather
Solar-Energy
MSE
MAE
MSE
MAE
MSE
MAE
96
0.249
0.170=0.000
96
0.20340.004
0.159:0.002
96
0.244+0.003
0.209
192
0.29240.000
0.231=0.001
192
0.25#0.005
0.21120.002
192
0.261
0.25=0.005
336
0.330+0.001
0.289=0.001
336
0.289+0.005
0,270=0.003
336
0.270#0.008
0,273+0.004
720
0.387+0.001
0.390#0.001
0.340+0.005
0.345=0.003
720
0.27240.004
0.274+0.007
Electricity
Traffic
Exchange
MSE
MAE
MSE
MAE
MSE
MAE
96
0.239+0.002
0.152=0.000
96
0.26440.000
0.441=0.002
96
0.200=0.002
0.083+0.001
192
0.201
0.167=0..01
192
0,276+0.007
0.457=0.003
192
0.2993+0.003
0.172+0.001
336
0,270 0.182+0.002
0.28240.006
0.465=0.003
336
0.270=0.003
0.32240.001
720
0.308+0.003
0.224+0.004
720
0.301
0.507=0.004
720
0.693+0.004
0.855+0.002
In Table 9. we present the standard deviations calculated from three repeated experiments on Nume- rion, highlighting its exceptional stability as model. The results consistently exhibit low standard deviations across all datasets, with particularly notable performance on the ETT dataset. This underscores Numerion's 's robustness and reliability in time series forecasting tasks.
G HIGH EFFICIENT HYPERCOMPLEX LINEAR LAYERS
In this paper, The defination of hypercomplex algebra based the Cayley-Dickson algebra system. The Cayley-Dicksee construction provides general recursive method for generating higher-dimensional algebras by repeatedly doubling lower-dimensional ones.
20

---

We begin with the complex numbers, which can be viewed as two-dimensional algebra IR x R over the real field IR, where each element represented as a pair of real numbers (a1, 51), (a2, 32). The addition of these elements follows the natural coordinate-witg rule:
(a1: B1) (a2, (2)) (a1 02 82)
Under the guidance of the Cayley-Dickson construction, the multiplication is defined by:
(on1, B1) x (a'2, (2) = (ar102 - -3 32a1 + 3102)
where d' denotes the conjugate of d. Since d is real in this base case, the conjugate has effect. It straightforward to verify that this multiplication satisfies the bilinearity property, thus constituting valid algebraic structure. By iterating this construction, can systematically define multiplication rules for higher-dimensional algebras such as quaternions, octonions, and beyond.
Theoretically, the time complexity of layer O(T E), which is linear with respect to the time series length T. However, the recursive nature of Cayley-Dickss Dickson multiplication means the computational cost scales quadratically with the algebra dimension n. In actual running, specifically for high-order algebras like Sedenions (n 16). this results in very large constant multiplier (approx. 63x FL.Pss vs. scalar), making naive sequential implementation extremely slow. Therefore, to optimize this "large constant" and ensure practical efficiency, we propose the following vectorized implementation:
In practice, real-number computations enjoy significant efficiency advantages in frameworks like PyTorch. Therefore, in our implementation, we represent hypercomplex numbers as real-valued vectors, leveraging extra dimensions encode hypercomplex coefficients. Specifically, the Cayley-Dickse construction to generate two matrices: a coefficient selection matrix and sign matrix. The former specifies which coefficients of the operands should be multiplied at each step, while the latter encodes the corresponding signs to apply to the resulting products. During each multiplication operation, these matrices are instantiated using PyTorch tensor operations, enabling efficient vectorized computation.
employ
As an example,suppose p, q is the quaternion numbers, have quaternion multiplication
p x q = (poto - Piςi - p2(2 - Paga)
+ (pogi + P1 + p243 P342]ii
(11)
+ (po92 - P1ę3 + P290 + P3411
+ (pons + p: p291 P3g
We can explicitly extract both the coefficient selection matrix and the sign matrix corresponding to the Cayley-Dickson construction each algebraic level. The coefficient selection matrix determines (left below) which elements from the input vectors should participate in each multiplication term, while the sign matrix (right below) encodes the appropriate sign (positive or negative) for each product.
0
1
2
37
1
0
3
2
1
3
0
-1
1
1
3
2
1
0
1
By precomputing these matrices, we effectively transform the recursive multiplication process into fixed matrix operation. Specifically, quaternion (or more generally, hypercomplex) multiplication can be reformulated as a linear transformation:
y = S. ((M w)
Here, M the coefficient selection matrix, the sign matrix, O denotes element-wise multiplica- tion (interpreted as gather-and weight process), x is the input vector, and represents the learnable weights or operands. The weight tensor w is shaped as output
input x n, where n is
the number of hypercomplex coefficients. Let w((") denote the i-th coefficient of the hypercomplex
21

---

vector The coefficient selection matrix M N is used to construct the multiplication matrix shaped as output.dim x input.dim x n x n according to:
A..
(n) M
Mi.j]] is not a shallow copy but a newly constructed tensor derived from the original w(n) via an index-based gather operation. Despite creating a new tensor, the autograd mechanism retains the connection to w(n) the computational graph. As a result, multiplying the input with matrix A remains algebraically equivalent to hypercomplex multiplication with w, and gradients are correctly and efficiently propagated back to the original j-th coefficient of w(() during backpropagation, without introducing redundancy.
In this expression, w (n)
This approach enables to express hypercomplex multiplication as unified matrix multiplication step. Although it necessitates temporarily building the multiplication matrix for each computa- tion- iion-introducing some additional CPU/GPU overhead-it substantially accelerates the overall computation by reducing the number separate operations and fully utilizing hardware-optimizee matrix kernels. Consequently, this parallelized strategy effectively neutralizes the "large constant" cost identified in the complexity analysis above, allowing the model to maintain linear scaling without the wall-clock latency penalty.
The multiplication rules for octonions is provided below:
0
2
5
6
77
1
0
3
5
4
7
6
2
3
7
4
5
3
2
1
6
5
4
-1
4
2
5
4
7
1
0
3
2
1
-1
1
-1
6
7
3
0
-1
I
7
6
5
3
2
1
0
-1
-1
H THEORETICAL SUPPLEMENT
H.1 GRADIENT OF ACTIVATION FUNCTION
Although HNTanh activation function in this paper operates on hypercomplex numbers, it can activation function for multivariate real vectors. Given input vector equivalently be viewed x e Rm, the output after applying the activation function is = f(x) ) E R.
Our goal to derive the Jacobian matrix J = 8y Suppose we use z; to represent the i-th coefficient of the vector æ. Due the inherent symmetry of the function, it suffices to compute the diagonal entries Oy. and the off off-diagonal entries dv. for + ōz,
dz.
The activation function is defined coefficient-wise as:
F;
V/1
|
(anh([|=11)))
where | denotes the p-norm:
m 1/D
||0||p
Σ
Io IP
Let := =||>|| for brevity.
We differentiate y, with respect to a;;
0y: 0 ( tanh(n))
Ox, 8x n
22

---

Applying the product derivative rule:
dy:
tanh(n)
(tan((n)
+ Is Or:
n
Next, by the chain rule:
tanh(n)
On (tanh(n)
Da, л
Dz, On
Compute the derivative with respect to n:
a
(tanh(n) n(1 tanh tanh?(n)) tanh(n)
Ön n
n²
Compute the derivative of n with respect to zy:
On
xx P-1 (i:n(x:))
Therefore, the final expression for
Ou is:
Oy
tanh(n)
[n(1 tanh?(n)) - tanh(n) n1-p
|r,11
n²
Similarly, differentiating ys with respect to Ty:
Oy. θ (tanh(n) dz,
x; dz,
7
Again applying the chain rule:
On 0 (tanh(n) dx, Dx, On
ay: 24
We already computed on
@ taab((n) ) For On 0E,""
On θx,-
1r,11-1 sign( n
Thus:
Oy: z. 21|1, 10 sag)( n1( - (anh2(n)) tanh(n) n 1-p
dx,
n
The Jacobian matrix entries are thus fully determined, Diagonal entries by the formula above for Ous Off-diagonal entries by the formula above for dus with i + j. dzz
Bing
This derivation completes the gradient calculation of the HNTanh activation function.
H.2 GRADIENT OF HYPERCOMPLEX NUERAL NETWORK
In the context of training hypercomplex neural networks, it is essential to demonstrate that the results obtained through rell-valued gradient descent are valid within the hypercomplex domain. Our justification is twofold: first, establish the mechanical validity of the gradient via the Jacobian matrix; second, we provide the theoretical grounding for optimization using Liouville's Theorem and GHR Calculus.
23

---

Mechanical Validity: The Real-Valued Jacobian First, the network is demonstrably trainable when treated as a real-differentiable function. By viewing hypercomplex multiplication as linear transformation real-valued vectors, the entire network inherits differentiability in the real sense.
To verify this, we compare the gradient computed in our network against the true Jacobian matrix derived from real differentiation. Taking a quaternion-based linear layer example:
F R -R, F(c(4) = W x
(12)
Here, the input c(4) weights W. and bias are quaternions treated as vectors of four real coefficients. Multiplication follows standard quaternion rules (Equation TT. Computing the Jacobian of the output F(c(9) with respect to c(4) yields:
Wo
W1
-W2
-W3
W
Wo
W3
-W2
Jw
(13)
W2
W
Wo
W1
W3
W2
WW1
Wo
This structure reveals that each coefficient of W contributes to the output via a linear transformation consistent with the coefficient matrices defined in our implementation (Appendix G Furthermore, is not difficult to extend this analysis to gigher-iimensionll hypercomplex numbers. Consequently, the gradients derived from standard backpropagation are exactly equivalent to this Jacobian, confirming that real-valued optimization methods are mechanically applicable.
Theoretical Validity: Liouville's Theorem and GHR Calculus While the network is mechanically trainable, a theoretical distinction must be made regarding hypercomplex differentiability, Our implementation does not strictly satisfy the generalized Still taking quaternion function as an example, strict differentiability requires:
(CRF) equations.
- (1=00x OX.
θ 0x, f=0 =0
Df =
0x, + Ox2 + +1s
(14)
However, enforcing this condition is detrimental to neural network design. According to Liouville's Theorem for hypercomplex analysis (Sudbery 1979), any bounded function that strictly satisfies the CRF conditions is necessarily constant. Even in relaxed definitions, the class of functions that are strictly cerrcooppex-diffeeenttae is restricted to linear or affine transformations. This restriction would render the construction of deep, non-linear neural networks impossible.
Therefore, we do not seek strict hypercomplex differentiabilityy Instead, our optimization framework relies GHR (Generalized Hamilton- RRell) Calculus. (Xu al. 2015) GHR calculus generalizes the complex Wirtinger calculus to quaternions and other algebras, providing a formal framework for minimizing real-valued loss functions (which non-holomorphic by definition) with respect hypercomplex parameters.
By employing GHR calculus, treat the hypercomplex weights and their conjugates as independent variables during differentiation. This theoretically validates our approach: the "real-valued gradient descent" we employ is, in rigorous terms, an implementation of GHR calculus, allowing for effective training without being constrained by the rigidity of Liouville's Theorem.
ACTIVATION MECHANISMS IN HYPERCOMPLEX NEURAL NETWORKS
Although hypercomplex networks remain niche area, researchers have increasingly explored their potential in various deep learning tasks. One major limitation hindering their broader success is the lack of effective activation functions tailored to hypercomplex spaces. Prior approaches often apply real-valued activation functions independently each component, which breaks the inherent symmetry of hypercomplex representations and restricts expressive capacity during training. Notably, the modulus shared across all hypercomplex restems-possedd desirable mathematical properties.
24

---

This motivates to develop activation functions that operate the modulus directly, aiming to preserve structural integrity and improve learning efficiency.
In this paper, propose HNTanh, activation function that leverages the modulus property of hypercomplex numbers and treats the hypercomplex input holistically rather than coefficient
Specifically, given hypercomplex input x, the activation function is defined
cln)
HNTanh(c!m))
(anh(]le(")
|c
where
6" =
tanh(a:)
x-xx
1/8
||
1lp
||c( denotes the p-norm of n-dimensional hypercomplex number clm). In this work, we set p We visualize the state space of the activation function on two-dimensional plane. Specifically, Figure shows the activation function itself; Figure 9 depicts the gradient of the activation function with respect to the modulus; Figures TO and TT illustrate the gradients with respect the first and second coefficients, respectively. These visualizations demonstrate that our activation function exhibits stronger overall symmetry and maintains a smoother gradient distribution across most regions of the space.
I.1 WHY DO WE CHOOSE SIX-NORM?
To determine the optimal choice of norm, we conducted both visualization and empirical evaluations. Visualization on the complex plane revealed that using six-norm, shown in Figure O, produces pronounced fluctuations and sharper transitions around extreme values. In contrast, lower-order norms, such the two-norm depicted in Figure exhibit smoother behavior in non-extreme regions. Higher-order norms, including the infinity norm,depicted in Figure [3] introduced excessive extreme points, leading to poorer overall continuity in the function's pnd from the formula in H.l we can see that the Jacobian matrix is almost 0 except for the diagonal and the rows and columns where the maximum value elements are located, and the optimization is unstable.
For the experimental evaluation, we trained models on multiple variants of the ETT dataset (ETTh1, ETTh2, and ETTm1) using p-norm values ranging from 8 and inf, with all other settings kept identical. The results, summarized in Table TO reveal that due to differences in the data distributions across datasets, is not feasible to depict a single unified performance curve showing the relationship between the norm value and overall model performance. Nevertheless, it is evident that activation functions with medium to high norm values consistently outperform those with lower norms. Notably, the six-norm, part of this medium-tor to-ggh norm range, achieves relatively strong performance across datasets. Based on these empirical findings and the smoother, more balanced properties observed in the visualization, we adopt the six-norm as our activation function.
1.2 WHY DO WE CHOOSE TANH?
The norm-based activation function for hypercomplex numbers can effectively induce a symmetric and relatively smooth functional space. However, determining the optimal choice of activation function remains open question worthy of deeper investigation. Indeed, selecting an appropriate activation function a highly complex problem. The study of activation functions itself challenging, as it lacks universally rigorous theoretical frameworks and definitive solutions that generalize well across applications. On the one hand, the performance of an activation function is closely tied to the specific characteristics of the downstream tasks; on the other hand, research activation functions in hypercomplex spaces remains scarce.
In this work, we therefore build upon well-established activation functions with widespread empirical success. We conducted an analysis of the currently mainstream ReLU-like activation functions and the sigmoid function, and visualized their gradient behavior with respect to the norm (modulus) of
25

---

Hintnh Gradient Norm Surface
HNTanh Output Norm Surface
06
41
Figure 9: Visualization of the gradient of the norm induced by the HNTanh activation func- tion.
Figure 8: Visualization of the HNTanh activation function over complex numbers.
HNTanh Gradient dy1/dk Surface
HNNlanh Gradient dy2/dx Surface
14
E4
Figure 10: Visualization of the gradient of the real coefficient (Re) in the HNTanh activation function.
Figure 11: Visualization of the gradient of the imaginary coefficient (Im) in the HNTanh activa- tion function.
26

---

HNTanhZ Dutput
HNTanhinf Output Surface
X2
Figure 12: Visualization the 2- 2-norm tanh ac- tivation function space. The function exhibits noticeably over-smoothed field characteristics.
Figure 13: Visualization of the oc-norm tanh activation function space. The function displays clearly sawtooth-like field patterns.
Table 10: Performance of different activation function norms on the ETT dataset, evaluated under identical experimental conditions.
ETThl
ETTh2
ETTml
Order of Norm
MAE
MSE
MAE
MSE
MAE
MSE
0,3838
0.3629
0.3319
0.2877
0.3415
0.3046
2
0.3805
0.3604
0.3329
0.2911
0.3425
0.3049
3
0.3800
0.3604
0.3313
0.2870
0.3424
0.3063
0.3801
0.3609
0.3313
0.2869
0.3419
0.3072
5
0.3795
0.3599
0.3309
0.2870
0.3412
0.3056
0.3798
0.3602
0.3306
0.2854
0.3401
0.3057
7
0.3798
0.3600
0.3301
0.2853
0.3405
0.3057
8
0.3797
0.3597
0.3310
0.2867
0.3398
0.3049
inf
0.3795
0.3593
0.3306
0.2846
0.3415
0.3069
hypercomplex numbers. As illustrated in Figures 14 and 13 span both positive and negative domains, the modulus inherently non-negative. Consequently. here by the SiLU function exhibit a nearly
unlike the real-valued case where inputs
activation functions in the ReLU comily-exempee uniform gradient across the complex plane, effectively activating only the positive half of the input domain. Sigmoid- activations also suffer from similar limitations under this setting.
Gradient Surface
Sigmold Gradient
200
33
32
Figure 14: Visualization of activation function gradient under SiLU replacement
Figure 15: Visualization activation function gradient under Sigmoid replacement
27

---

In validation experiments, selected several widely used and influential activation functions. Within the Norm-Activation framework, we fixed the overall structure while varying only the activation function, ensuring that all other experimental parameters remained identical. The results are presented the table below.
Table 11:
Performance of different activation function norms on the ETT dataset, evaluated under identical experimental conditions.
ETTh1
ETTh2
ETTml
Type of Function
MAE
MSE
MAE
MSE
MAE
MSE
Tanh
0,3800
0.3602
0.3337
0.2901
0.3416
0.3068
Sigmoid
0.3855
0.3671
0.3366
0.2944
0.3409
0.3041
Relu 0.3815
0.3633
0.3316
0.2861
0.3483
0.3234
Mish 0,3824
0.3667
0.3324
0.2899
0.3507
0.3337
SiLU
0.3842
0.3695
0.3324
0.2904
0.3438
0.3193
LogSigmoid
0.3993
0.3945
0.3398
0.3068
0.3562
0.3314
RRelu
0.3814
0.3632
0.3319
0.2868
0.3515
0.3362
Notably, Tanh achieved the best performance on the ETThl dataset and performed comparably well on both ETTh2 and ETTml. Although it did not attain the top score across every dataset, Tanh consistently delivered the most competitive overall results among all tested activation functions. These findings provide empirical support for the effectiveness of selecting Tanh as the activation function, validating choice from practical experimental standpoint.
J VISUALIZATION
J.1 VISUALIZATION HYPERCOMPLEX LINEAR PREDICTION RESULTS
We visualized the outputs of each linear block in the trained model under its best-performing parameters. For this analysis, we selected the prediction results of the first channel from the first sample in the ETTh1 test set. F8gure presents the predictions from five different linear layers in the time domain, while Figure 17 shows their corresponding representations in the frequency domain. Figure illustrates the weighted average weights assigned to each linear block, which are used to average the outputs the RHR-MLP.
The results reveal that, under optimal settings, the model implicitly assigns the high-dimensional hypercomplex layer to predict the now-dimensional, low-frequency components of the time series, while the real and complex linear layers focus capturing the high-dimensional, high-frequency components. This separation enables the model to effectively fit localized, noisy seasonal signals while maintaining accurate predictions of the overall trend. Additionally, the average weight distri- bution indicates that the real and complex linear layers collectively contribute approximately 40% of the prediction sequence, primarily capturing medium and high-frequency seasonal signals. In contrast, the hypercomplex layers account for about 60% of the results, with the hexadecimal linear layer alone supporting nearly 40% of the predictions, while the remaining hypercomplex layers play an auxiliary role in refining the overall trend.
Interestingly, while this decomposition behavior resembles the effects classical time series decom- position, it emerges naturally through model training. Unlike artificially designed decomposition models, it is not constrained by the limitations of fixed-form decomposition. This statistical, machine leavning-driven decomposition ability enables the model to adapt flexibly to different datasets, ulti- mately contributing to its strong predictive performance across multiple benchmarks. It also provides glimpse into the nature of the model's 's low- and high- gigh-frequency features in learning time series data. We will explore this further the next paragraph, and the section quantitative analysis these features.
28

---

J.2 VISUALIZATION RESULTS WITH OTHER TRAINING SETTING
In order to better explore the inspiration brought by this method, We conducted visualization ex- periments under alternative experimental settings further validate our findings. Figures[20) and illustraae the visualization effects of the ETTh dataset under non-optimal parameters, while Figure 19 presents the corresponding average weight distribution. The results indicate that even under non-optimal configurations, the model consistently employs high- high-dimensional hypercomplex layers as the primary mechanism for capturing low-frequency information, while relying on real and complex layers to capture high-frequency signals. However, under these suboptimal conditions, the low-dimensional layers exhibit a relatively poor learning effect on the high-frequency components, with their corresponding weight proportion significantly lower than the 40% observed under optimal configurations. Consequently, their contribution to the overall decomposition is minimal, leading to subpar model prediction results.
We also performed visualization experiments on other Figures
23
24 and
22
present the
visualization of specific channels the ETTh2 dataset. while Figures 29 show the corresponding visualizations for the ETTm2, ETTml dataset. Although these datasets exhibit more complex high-frequency signals, the dominance of hypercomplex numbers in the learned weights, along with the observation that different hypercomplex layers capture distinct low-frequency features, suggests that the model primarily focuses learning low-frequency trend characteristics. Based on these experiments, we propose the following hypothesis: the model's inability to accurately predict low-frequency signals limits its capacity to further analyze high-frequency components, thereby constraining overall performance improvement. While this conclusion may not be entirely definitive- given the inherent limitations of the datasets and model architecture-iit nonetheless offers valuable insights for further enhancing our time-series prediction model.
30
28
26
27 and 25
29

---

Average Weight over Time (Feature
1.2
Resl Uneer
Comp Linear
Welgght
Sede Linear
02-
0.0
Figure 16: Visualization of weights of MLP on ETThl
Frequency Spectrum: Real Linear Truth (Channel
Linear Ground
Ground Thuth 1gt) Real Linear
60
40
20
Ground Truth (gt)
Real Linear
Frequency Spectrum: Comp Linear vs Ground Truth (Channel D)
Time Domain: Comp Linear vs Ground Truth
80
Ground Truth 18t)
Hoph
value
Ground Truth (gt)
10
30
40
20
40
60
Frequency Spectrum: Quat vs
Qual Ground Truth
Ground Thuth 1gt)
Quat Lineer
Hognitute
80
40
20
Ground (gt)
Quet Linwar
Frequency Spectrum: Octo Linear vs Ground Truth (Channel 01
Time Domain: Octo Linear vs Ground Truth
80
Ground Truth
Hopntare
50
value
40
Ground (gt)
4-
10
30
60
Frequency Spectrum: Sede vs (Channel
Domain: Sede Linear Ground Truth
Ground Truth 1g
Sede L hear
80
80
20
Ground Truth (gt)
Sede Linear
Frequency Domann Contribution (Channel 0)
Time Domain Contribution (Channel 05
40
Quat
-30
Octo Uner
Value
Teal Lineer
Sedc Uinea
Comp Uncar
20
Quast Liwer
10
1.0
30
80
Frequency (Hld
Time
Figure/ 17: Visualization of MLP prediction in Figure 18: Visualization of MLP prediction in frequency domain on ETTh1
time domain on ETTh1
30

---

Average Weight over Time (Feature
1.2
Resl Uneer
Comp Linear
Weiight
Sede Linear
02-
0.0
Figure 19: Visualization of weights of MLP on ETThl on non-optimal parameters
Frequency Spectrum: Linear
(Channel
Time Domain: Linear Ground
Ground Truth 1gt) Real Linear
60-
20
Ground Truth (g)) Real Linear
Froquency Spectrum: Comp Linear vs Ground Truth (Channel O)
Time Domain: Comp Linear vs Ground Truth
80
Ground Truth 18t)
volve
Ground Truth (gt)
10
20
30
20
40
60
Frequency Spectrum: Quat Ground Truth
Qual Linear Ground Truth
Ground Truth 1gt)
Quet Lineer
6.
40-
20-
Ground (gt)
Quet Linwer
Frequency Spectrum: Octo Linear vs Ground Truth (Chaninll 0)
Time Domain: Octo Linear vs Ground Truth
80
Ground Truth 1gt)
60
value
60
(gt)
4-
10
20
60
Frequency Spectrum: Linear Ground Truth (Channel
Time Domain: Sede Linear Ground Truth
125
Ground Ruth 1g
1.00
Sede L hear
Hoghaude
50
Ground Truth (gt)
Sede Linear
Trequency-Demainnccm Demain Contribution (Channel 0)
Time Domain Contribution (Channel 03
60
Quat
Octo Unerr
Value
Real Unewr
Sedc Unnea
Comp Lincor
Qual Linear
Linear
20
60
Fruquency
Time
Figure 20: Visualization of MLP prediction in frequency domain on ETTh (non-optimal)
Figure 21: Visualization of MLP prediction in time domain on ETTh (pon-opiia))))
31

---

Average Weight over Time (Feature
1.2
Real Liner
Comp Linwar
Sede Linear
Welgght
02-
0.0
Figure 22: Visualization of weights of MLP on ETTh2
Frequency Spectrum: Real Linear Truth (Channel
Domain: Real Linear Truth
Ground Thuth 1gt)
Real Linear
10
Valie
Ground Turth (22)
Roaal Linear
175
Frequency Spectrum: Comp Linear vs Ground Truth (Channel D))
Time Domain: Comp Linear vs Ground Truthh
20
Ground Truth 18t)
12
Viduo
11
Ground Truth
20
40
60
80
50
75
100
125
150
175
200
Frequency Spectrum:
Domain:
vs
Ground Thuth 1g
An
Quel Lineer
0.0
10
-0.5
-1.0
1pt)
-2.
Qual Linwar
Frequency Spectrum: Octo Linear vs Ground Truth (Channel 01
Time Domain: Octo Linear vs Ground Truth
Ground Truth
18
Hognt,oe
15
value
10
60
80
100
125
150
175
200
Frequency Spectrum: Sede
(Channel
Domain:: Sede
Ground Ruth 1g
25
Sede L hear
Hognaude
10
Ground Turth 1977 Sede Liner
175
Frequency Domann Contribution (Channel 0)
Time- Domain Contribution (Chanmel 0)
15
Rigg
Quat
Octo Uner
Value
0.4
Revll iower
Sedi Linea
Comp Uncar
Gust Liveer
75
100
150
175
200
Frequency (Hil
Tim (1)
Figure 23: Visualization of MLP prediction in Figure 24: Visualization of MLP prediction in frequency domain on ETTh2
time domain on ETTh2
32

---

Average Weight over Time (Feature
1.2
Resl Uneer
Comp Linear
Welgght
Sede Linear
02-
0.0
Figure 25: Visualization of weights of MLP on ETTml
Frequency Spectrum: Real Linear Truth (Channel
Domain:
20
Ground Thuth 1gt) Real Linear
Ground Trthh (gt) Real Linear
15
0.0
Frequency Spectrum: Comp Linear vs Ground Truth (Channel 4)
Time Domain: Comp Linear vs Ground Truth
Ground Truth 18t)
1.0
Gmunnd Tith (gh)
Hogniuse
15
0.5
value
0.0
-0.5
-1.
10
30
20
60
Frequency Spectrum:
vs
Linear vs
15
Ground Thuth 1gt)
Ground Tuth (gt)
70
Quat Lineer
Quat meer
Hoghiaate
0.5
-0.5
Frequency Spectrum: Octo Linear vs Ground Truth (Channel 4)
Time Domain: Octo Linear vs Ground Truth
Ground Truth
Ground Truth
Hognt,aee
1.0
15
volue
0.0
10
30
Frequency Spectrum: Sede
(Channel
Domain: Linear Ground
Ground Ruth 1g Sede L hear
1.5
Ground Truth Igt)
30
Sede hear
Hognaude
20
0.0
-1.0
Frequency
Frequency-Demain Domain Contribution (Channel 4)
Time-Domain Contribution (Channel 4)
30
Migg
20
Quat
Linee
Octo Unerr
Value
Octo Liner
Sedc Uinea
Side Linear
0.9
-0.5
1.0
Freqanccy (Hld
Time
Figure 26: Visualization of MLP prediction in Figure 27: Visualization of MLP prediction in frequency domain on ETTml
time domain on ETTml
33

---

Average Weight over Time (Feature
1.2
Resl Uneer
Comp Linwar
Weigght
Sede Linear
02-
0.0
Figure 28: Visualization of weights of MLP on ETTm2
Frequency Spectrum: Linear Ground
Real Linear Ground Truth
Ground Puth 1gt Real Linear
Megihtuoe
-0.5
-1.0
-1.5
Ground Truth (g))
-2.0
Real Linear
Frequency Spectrum: Comp Linear vs Ground Truth (Channel 0)
Time Domain: Comp Linear vs Ground Truth
Ground Trath 1gt)
Value
Ground Tith igt)
Comp Linsar
20
30
40
20
Frequency Spectrum: Quat Linear Ground Truth
Qual
Ground Truth 1gt
Quel Linee
Negntan
-0.5
-1.5
(97)
-20
Quei Linwar
Frequency Spectrum: Octo Linear vs Ground Truth (Channel 0)
Time Domain: Octo Linear vs Ground Truth
Truth 1gt)
-0.5
Megitaan
-1.0
-1.5
Ground
20
20
Frequency Spectrum: Sede Linear Ground Truth (Channel
Sede Linear
Ground Truth 1gt Swde teer
0.5
Magnhulte
-1.5
1.5
Ground Truth (g))
-2.0
Sede Liner
20
Frequency- Domann Contribution (Channel 0)
Time Domain Contribution (Channne 0)
(Channel
0.3
Linear
Megnkute
Qual
Deho Uner
Real L near
Sode Unear
0.0
Comp Uncar
Quat Limeer
-0.
20
40
20
Frequency (Mdd
Time (1)
Figure 29: Visualization of MLP prediction inFigure 30: Visualization of MLP prediction in frequency domain on ETTm2
time domain on ETTm2
34

---

K QUANTITATIVE STATISTICAL ANALYSIS TO THE DECOMPOSITION METHODS
To further validate the characteristics of the decomposition approach described the previous section and to derive concrete insights from the unique decomposition learned by the model, conducted extensive quantitative analyses on the ETT dataset. Specifically, we computed three key metrics for the predictions made by real, complex, and hypercomplex linear layers in the frequency domain: the Proportion of Variance Explained [PE]rrrsser (1965), the Mean Absolute Frequency (MAF Haugen al. (2015), and the Pearson correlation coefficien (Rahadian predictions and ground truth in the time domain.
Raahaian et al. (2023) between
PVE, a classical metric in variance analysis, measures the proportion of variance in the dependent variable that explained by the independent variable. Letting y denote the ground truth and Y denote the predicted value at frequency f, the PVE is mathematically defined as:
PVE, = 1 Var(Y, - Y Var(Yr)
A PVE value closer to indicates a higher explanatory power. In this study, we partitioned the frequency domain into intervals to analyze the PVE across different frequency bands, thereby revealing how well each RHR-MLP fits different frequency regions.
The MAF metric reflects the frequency specificity of the predictions by the linear layers. A lower MAF indicates that the predictions are dominated by lower-frequency components, whereas a higher MAF suggests greater presence of high- fighffreueney components in the predictions. The MAF in this study is computed as:
E,111 P(F) P()(F
MAF
where P(f) denotes the power at frequency f.
Table 12: Mean PVE per RHR-MLP per frequency bin on channel of ETThl, with corresponding Mean Absolute Frequency (MAF) and Pearson correlation coefficient.
[0.00-0.01]
[0.01-0.04]
[0.04-0.09]
[0.09-0.14]
[0.14-0.20]
[0.20-0.27]
[0.27-0.50]
MAF
Pearson
Real 0.0004
0.0156
0.0377
0.0091
0.3040
0.1878
0.4311
0.241223
0.097605
Comp 0.0432
0.0075
0.9196
0.0065
0.0083
0.082320
0,382733
Quat 0,2199
0.0604
0.6982
0.0138
0.0035
0.0017
0.0023
0.037663
0.521406
Octo 0,1036
0.0230
0.7892
0.0740
0.0037
0.004]
0.0024
0.051302
0.631457
Sode
0.1000
0.0116
0.8445
0.0362
0.0026
0.0028
0,0022
0.046143
0.688479
Table 13: Mean PVE per RHR-MLP per frequency bin on channel of ETTh1, with corresponding Mean Absolute Frequency (MAF) and Pearson correlation coefficient.
10.00-0.01]
[0.01-0.04]
[0.04-0.09]
[0.09-0.14]
[0.14-0.20]
[0.20-0.27]
[0.27-0.50]
MAP
Pearson
Real 0.0004
0.0087
0.0208
0.0066
0.277
0.1997
0.4685
0.257347
0.114217
Comp
0.0612
0.0107
0.8371
0.0173
0.0153
0.0428
0.0155
0.089623
0,304849
Quat 0.326
0.0784
0.5738
0.0146
0.0034
0.0016
0.0022
0.032386
0.45547
Octo 0,1592
0.0192
0.7165
0.0797
0.007
0.0121
0.0062
0.05182
0.479334
Sede 0.168
0.7587
0.0057
0.0078
0.045574
0.565875
The third metric, the Pearson correlation coefficient, measures the overall similarity between the predicted and ground truth signals in the time domain. On typical time series datasets, a Pearson coefficient closer to I indicates that the predicted sequence better aligns with the overall trend of the true sequence.
Tables 12 13
present the results of our quantitative analyses channels 0 and of the ETTh1 dataset. The corresponding MAF and Pearson correlation coefficients are reported alongside the interval-based PVE values. From these results, observe similar patterns to those discussed in
35

---

Table 14: Mean PVE per RHR-MLP per frequency bin on channel 0 of ETTml, with corresponding Mean Absolute Frequency (MAF) and Pearson correlation coefficient.
[0.00-0.01]
[0.01-0.04]
[0.04-0.09]
[0.09-0.14]
[0.14-0.20]
[0.20-0.27]
[0.27-0.50]
MAF
Pearson
Real 0.2485
0.7226
0.0237
0,0033
0.0007
0.0004
0.0008
1412
0.446613
Comp 0,4764
0,4804
0.0315
0,0073
0.0019
0.0007
0.0017
0.010222
0.037417
Quat 0.3179
0.4730
0.1607
0.0172
0.0135
0.0044
0,0133
0.026225
0.408534
Octo 0.0173
0.9675
0.0139
0.0006
0.0002
0.0001
0.0003
0.013652
0.788544
Sede
0.1989
0.7693
0.0268
0.0010
0.0005
0,012790
0.678040
Table 15: Mean PVE per RHR-MLP per frequency bin on channel 3 of ETTml, with corresponding Mean Absolute Frequency (MAF) and Pearson correlation coefficient.
[0.00-0.01]
[0.01 -0.04)
0.09-
[0.09-0.14]
[0.14-0.20]
[0.20-0.22
[0.27-0.50]
MAP
Pearson
Real 0.1378
0.8323
0.0222
0.005
0.001
0.0006
0.0011
0.01 3584
0,208974
Comp 0.6085
0.3462
0.0315
0,0089
0.0022
0.0007
0,0019
0,009141
0.028703
Quat 0,4405
0,3668
0,1465
0.0181
0.0117
0.0043
0.0122
0.023216
0,305707
Octo 0.0069
0.9701
0.0205
0.0011
0.0004
0,0002
0.0006
0.014868
0.660863
Sede
0,5131
0,4609
0.0211
0.0023
0.0009
0.0005
0.0011
0.00871
0.602638
the previous section: the lower- freeufrry regions (below 0.09) are predominantly predicted by the higher-dimmn ligher-dimensional hypercomplex layers, effectively capturing global trend patterns; whereas the higher-frequency regions are predicted by the lower-dimensional complex and real-valued layers, capturing localized seasonal signals. And because the model is channel-independent, this prediction rule does not change with the channel.
However, notable divergence from conventional decomposition design emerged (cite). Traditional modeling intuition often assumes that low-frequency components are more difficult to predict than high-frequency components, leading to designs that allocate greater model capacity to handle low- frequency signals. Contrarily, our model, after training, reveals an opposite pattern: the complex and hypercomplex MIP, which constitute the majority of the model's parameters, primarily focus their predictive capacity on a specific low-frequency band-particularly the interval [0.04 0.09) -whie selectively predicting auxiliary low-frequency components in [0 0.01], [0.01 0.04), and (0.09 0.14]. Surprisingly, the Real MLP, despite having the fewest parameters, is responsible for predicting the complex high-frequency components.
Tables 14
15 and 16 summarize the quantitative results of our analyses on the ETTml and ETTm2 datasets, both of which exhibit lower frequency characteristics compared the ETThl dataset.
When comparing across datasets. we observe that the low-frequency nature of ETTml and ETTm2, coupled with their training challenges, results in the Real MLP struggling to capture prominent high-dimensional features. Consequently, the predicted signals from the MLP tend to concentrate in lower frequency ranges overall.
Nonetheless, a closer examination of the frequency distribution across low- wwwffrequecyy intervals, along with metrics such as the Mean Amplitude Frequency (MAF) and Pearson correlation coefficient, reveals clear stratification pattern in the predictive frequencies of the different MLP. Despite operating predominantly at low frequencies, the predictions from each RHR-MLP exhibit distinct characteristics that reflect the integration of diverse feature sources.
These findings suggest that the decomposition mechanism learned by the model is indeed functioning as intended, effectively disentangling components with different frequency properties. However, the
Table 16: Mean PVE per RHR-MLP per frequency bin on channel 6 of ETTm2, with corresponding Mean Absolute Frequency (MAF) and Pearson correlation coefficient.
[0.00-0.01]
[0.01-0.04]
[0.04-0.09]
[0.09-0.14]
[0.14-0.20]
[0.20-0.27]
[0.27-0.50]
MAF
Pearson
Real 0.1545
0.7596
0.0506
0.0326
0.002
0.0004
0.0003
0.017574
0.016364
Comp 0.9773
0.0108
0.0043
0.0027
0.0015
0.001
0.0023
0.002114
0.055557
Quat 0.3066
0.4821
0.1502
0.0454
0.005
0.0031
0.0075
0.026367
0.096686
Octo
0.0671
0.872
0.0454
0.0046
0.0033
0.0023
0.0053
0.017973
0.315934
Sede
0.9916
0.0068
0.0008
0,0004
0.0001
0.0001
0,0002
0.000295
0.169416
36

---

results also indicate that there is potential for further improvement and optimization to expand the decomposition frequency range and improve the prediction performance.
L WHY THE METHOD WORKS: RMT AND HYPERCOMPLEX-SPECTRAA PERSPECTIVES
L.1 J.1 RANDOM-MATRIX PERSPECTIVE: EFFECTIVE-RANE COLLAPSE AND SPECTRAL BIAS IN HYPERCOMPLEX LAYERS
Let Rd be embedded into hypercomplex algebra, quaternions H octonions O, by identifying R with the zero-imaginary subfield and applying a random hypercomplex weight W E HH or W E gmxx Via standard real block representations, this induces real linear map
Although R(W) is generically full rank and has more real parameters than a comparable real/complex layer, its effective number of modes is smaller.
Key claim (stable rank reduction). Define the stable rank
sr (R(W)) =; (R(W)
(15)
[2
Relative to real/complex random matrices, hypercomplex ensembles exhibit symmetry-induced eigenvalue/singular--iile degeneracies, yielding repeated eigenvalues and concentrated spectra This speculated from symmetry-induee degeneracies simple cases: when N=2, quaternions show 2- to 4-fold eigenvalue repetitions (aligned with Dyson ensemble 3 = 4), while octonions display up to 8-fold degeneracy (3 Forrester 2017). The spectral entropy
Let o; be the singular values and p; a ( F2.
(oo)= Σ ip, log
(16)
reduced by these degeneracies, and the ratio in equation 15 correspondingly decreases [R(W)]2 inflates relative to [R(W)
Lower stable rank means optimization gravitates toward solutions with few dominant singular direc- tions. Functionally, the layer acts as a low-pass operator: it favors smooth, low-frequency structure while attenuating high-Ereyuency oscillationsnnoise, consistent with implicitsspectrl low-rank biases observed in over- para-aaettteeeeed linear networks (Fridvicc--Kell et al. 2022 Rahaman et al. 2019).
In noisy time series, truncating high frequencies reduces variance. The algebra induced rank collapse thus serves as a built-in regularizer: despite large parameter counts, admissible mode diversity is curtailed by symmetry, improving stability against outliers and measurement noise.
L.2
J.2 HYPERCOMPLEX-SPEE -SERLEEX--PPETRLLLLLE PERSPECTIVE: QFT/OFT PROPERTIES AS INDUCTIVE BIAS FOR MULTIVARIATE TIME SERIES
The quatermion Fourier transform (QFT) Ell 1992 Cheng & Kou 2018 Sangwine
1996) and octonion Fourier transform (OFT) (Hahn & Snopek 2011 Baszezyk & Snopek 2017) extend the complex Fourier transform to vector-valued signals while preserving core theorems:
Hermitian symmetry. For real inputs, spectra are conjugate-symmetry climinating redundant frequencies without loss (Salehi al. 2013). hypercomplex embeddings, this symmetry extends across channels, reducing spectral redundancy in multichannel data.
Plansherel-Parsevall. Energy is preserved between time and frequency domains, supporting power preservin linear operators and interpretable orthogonal decompositions (Alessio) 2016).
Khinter-Kinnehine, Autocorrelation equals the inverse transform of the power spectral density, now in the hypercomplex domain, exposing both intra-channel and cross channel dependencies (Zbilut & Marwan 2008).
Windowed variants (e.g., windowed OFT) provide time- fren-frryenccy localization while retaining these guarantees, improving detection of transient, multiscale structure (Bhat & Sheikh 2023).
37

---

Why this helps modeling.
1. Phase-awarr aware coupling. Hypercomplex units encode amplitudes joint phases/orientations via non con--mmmttaiive products; the induced real block operator performs coupled rota- tions/scalings across channels, enabling orientation-awar filtering that scalar models cannot match at the same parameter budget.
2. Energy- and correlation porrelation-preservinng structure. Because QFT/OFT preserve energy and connect spectra to autocorrelation, hypercomplex hyperoomplex- constrrieddd linear maps behave as phase-coherent, power-preserving filters, stabilizing optimization and aiding interpretabil- ity.
3. Built-nn multichannel priors. Algebraic constraints impose cross-channel couplings (akin Cauchy-Riemann-yype relations), pruning spurious degrees of freedom and sharpening generalization for vector-valued time series.
The spectral theorems specify what structure is preserved (energy, symmetry, correlations); the RMT view explains how algebra enforces compact dominant-modk set (implicit low rank). Together, they imply a thaae-cchhrees low-variance estimator that captures trends and stable cross channel relations while suppressing noise noise-matching empirical behavior.
Summary. Hypercomplex representations provide (i) an implicit low-rank, low-pass bias via spectral degeneracies (RMT), and (ii) the right spectral invariants (Hermitian symmetry, Manccherll-Pasevain Wiener (hmner--hitchheee) for multichannel time series (QFT/OFT), This combination explains why our method is data-efficient, noise-robust, and strong on long-horizon trends even when nominal parameter counts are large.
M SHOW CASE
In this section, we present the complete results of the case study. For each dataset, we randomly selected 2-3 channels from the first seven channels for visualization. The detailed outcomes are summarized as follows. While the predictive performance and visualizations appear broadly similar across models-reflecting the strong overall baseline performance- model consistently demon- strates more accurate predictions of low- now-frequency components, particularly evident in the weather dataset and other representative cases.
N LIMITATIONS
Our method introduces a novel time series forecasting model leveraging multiple hypercomplex spaces, but also faces several limitations. First, PyTorch natively supports only real and complex numbers, so all hypercomplex data structures, linear layers, and activation functions had to be implemented from scratch. Computations are simulated in the real domain, leading slower training and higher memory demands-ea, a sedenion layer consumes 16x more memory than its real counterpart. Second, optimizers such as Adam are inherently designed for hypercomplex o
arithmetic, resulting in slower convergence and increased training fluctuations. Third, despite our efficient implementation. memory usage and runtime scale unfavorably with dimensionality due to the lack of specialized hardware or data structures. Beyond hardware constraints, our approach is also limited representation design. Current practice relies mainly on zero zero-padding to extend real inputs and Cayley-Dickson constructions to define hypercomplex spaces, but alternative transformations into different hypercomplex spaces remain underexplored. is unclear whether other structured spaces mappings might yield stronger inductive biases for time series, and these directions require substantial theoretical investigation. Finally, advanced transformations, such as generalized hypercomplex Fourier methods, not yet implemented due both theoretical and computational barriers. We remain optimistic that future wovk-developing new transformation schemes, theoretical foundations, and hardware/optimizer support-will greatly enhance the practicality and performance of hypercomplex models.
38

---

16
Numerion
TixeMixer
PatchMLP
TimeXer
FilterNet
iTransformer
PStchTST
TimesNet
Figure 31: ETThl Channel
Numerion
DLinear
PatchMLP
TimeXer
FilterNett
iTransformer
PSt
TimesNet
Figure 32: ETTh1 Channel
2-
Numerion
DLinear
PatchMLP
TimeXer
[._
240
FilterNet
iTransformer
PST
TimesNet
Figure 33: ETTh2 Channel O
39

---

10
Numerion
TimeMixer
PatchMLP
TimeXer
FilterNet
iTransformer
PStchTST
TimesNet
Figure 34: ETTh2 Channel 6
WA
P/
Numerion
TimeMixer
PatchMLP
TimeXer
FilterNet
iTransformer
PatchTST
TimesNet
Figure 35: ETTml Channel 6
M
Numerion
DLinear
PatchMLP
TimeXer
FilterNet
iTransformer
PST
TimesNet
Figure 36: ETTml Channel 0
40

---

10
Numerion
DLinear
PatchMLP
TimeXer
FilterNet
iTransformer
PStchTST
TimesNet
Figure 37: ETTm2 Channel 6
Numerion
DLinear
PatchMLP
TimeXer
FilterNet
iTransformer
PStchTST
TimesNet
Figure 38: ETTm2 Channel
Numerion
DLinear
PatchMLP
TimeXer
20 30
25
20
20
20 150
FilterNet
iTransformer
PSt
TimesNet
Figure 39: Solar Energy Channel 0
41

---

10
Numerion
Tixer
PatchMLP
DLinear
FilterNet
iTransformer
PStchTST
TimesNet
Figure 40: Solar Energy Channel 4
Numerion
TimeMixer
PatchMLP
DLinear
FilterNet
iTransformer
PatchTST
TimesNet
Figure 41: Weather Channel 2
Numerion
TimeMixer
PatchMLP
DLinear
20 34
FilterNet
iTransformer
PStchTST
TimesNet
Figure 42: Weather Channel 3
42

---

Numerion
TimeMixer
PatchMLP
DLinear
TimeXer
iTransformer
PatchTST
TimesNet
Figure 43: Electricity Channel
Numerion
TimeMixer
PatchMLP
DLinear
TimeXer
iTransformer
PatchTST
TimesNet
Figure 44: Electricity Channel
43


Reviews:
[{"review_id": "9k9Ras7QCD", "reviewer": "Reviewer_cJjH", "summary": "This paper proposes $\\text{Numerion}$, an $\\text{MLP}$-based model for time series forecasting. Its core idea is that mapping time series into higher-order hypercomplex spaces ($\\mathbb{C}$, $\\mathbb{H}$, $\\mathbb{O}$, $\\mathbb{S}$) naturally decomposes the signal by frequency; higher dimensions capture lower frequencies. The method generalizes linear layers and $\\tanh$ to create a $\\text{Real-Hypercomplex-Real MLP}$ ($\\text{RHR-MLP}$). The $\\text{Numerion}$ model runs parallel $\\text{RHR-MLPs}$ from $\\mathbb{R}^{1}$ to $\\mathbb{R}^{16}$ and fuses their outputs. The concept is highly novel but undermined by contradictory ablations and severe efficiency limitations. I recommend a Score 4 (Marginally Below Threshold).", "questions": "Q1. Please explain Table 2. Why does removing components like \"w/o Real\" or \"w/o Adaptive Fusion\" lead to equal or better performance on $\\text{ETTh1}$? This suggests these parts are not contributing.\n\nQ2. Given the $16\\text{x}$ memory cost, parameter inflation, and slow speed, how is $\\text{Numerion}$ practically useful compared to efficient $\\text{SOTA}$ models like $\\text{iTransformer}$ or $\\text{DLinear}$?\n\nQ3. The choice of $p=6$ for $\\text{HNTanh}$ seems arbitrary. Table 10 shows $p=\\text{inf}$ is best for $\\text{ETTh1}$ and $p=7$ for $\\text{ETTh2}$. How was $p=6$ chosen, and how sensitive is the model to this value?\n\nQ4. Figure 4 shows $\\text{MSE}$ on $\\text{ETTm2}$ improves as layers increase from 1 to 5, suggesting underfitting. How does this align with the text claiming the model \"prefers fewer layers... [to avoid] overfitting\"?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "S1. The core idea is using the algebraic hierarchy ($\\mathbb{R} \\to \\mathbb{C} \\to \\mathbb{H} \\to \\mathbb{O} \\to \\mathbb{S}$) as an implicit frequency decomposition mechanism. This is a novel and elegant approach, distinct from traditional methods.\n\nS2. The paper formally generalizes linear layers and activations to $n$-dimensional hypercomplex spaces. The $\\text{Hypercomplex Linear Layer}$, $O^{(n)} = {W^{(n)}}^{T}X^{(n)} + b^{(n)}$, and the $\\text{Hypercomplex Norm Tanh}$ ($\\text{HNTanh}$) activation create a unified framework based on the $\\text{Cayley-Dickson construction}$.\n\nS3. The central hypothesis—higher dimensions capture lower frequencies—is well-supported by visualizations and quantitative analysis. Appendix L shows higher-dimensional $\\text{RHR-MLPs}$ have lower $\\text{Mean Absolute Frequency}$ ($\\text{MAF}$) and higher $\\text{Pearson correlation}$, indicating better trend-fitting, while the $\\text{Real-MLP}$ handles high frequencies.", "weaknesses": "W1. Contradictory Ablation Study (Major Flaw)The ablation in Table 2 contradicts the paper's claims. On $\\text{ETTh1}$, the full $\\text{Numerion}$ model ($\\text{MSE}$ $0.449$, $\\text{MAE}$ $0.449$) performs worse than or equal to variants where components are removed (e.g., \"w/o Real\", \"w/o Adaptive Fusion\"). This contradicts the text stating these components are essential and suggests they are not synergistic, at least on this dataset. This is a major red flag.\n\nW2. Severe Practical and Efficiency LimitationsThe paper admits to severe inefficiency. Lacking native $\\text{PyTorch}$ support, operations are simulated. This causes extreme memory overhead (a sedenion layer is $16\\text{x}$ larger), parameter inflation (5 layers become 63), and slow training/convergence. Efficiency tables confirm $\\text{Numerion}$ is much slower and larger than $\\text{iTransformer}$, $\\text{PatchTST}$, or $\\text{DLinear}$, making it impractical.\n\nW3. Vague Justification for DecompositionThe paper shows decomposition happens but not why. The theory in Appendix M is a hypothesis linking $\\text{Random Matrix Theory}$ ($\\text{RMT}$) and stable rank reduction to a \"low-pass bias\". This is an insightful post-hoc explanation, not a formal proof. The causal link between $\\text{Cayley-Dickson}$ rules and frequency filtering is not established.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BzmIZmwccV", "reviewer": "Reviewer_auNm", "summary": "This paper proposes Numerion, which achieves multi-frequency decomposition and prediction by mapping time series to multiple hypercomplex spaces. The core innovation lies in generalizing linear layers and Tanh activation functions to hypercomplex spaces of arbitrary power-of-two dimensions and designing the RHR-MLP architecture. The paper also provides extensive ablation experiments, visualizations, and frequency analyses to validate the effectiveness of the Numerion model.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors' idea of achieving decomposition by mapping time series data to hypercomplex spaces of different dimensions is very inspiring.\n2. The authors provide a mathematical formulation of the Numerion framework based on the Cayley-Dickson algebra system. They use the Cayley-Dickson construction and random matrix theory principles of spectral bias as the theoretical foundation of the model. This is very novel and interesting.\n3. The paper provides extensive comparative and ablation experiments, visualizations, and frequency-domain analyses to support their claims. The experimental design is relatively complete, and the results are clear.", "weaknesses": "1. There are multiple instances of missing spaces throughout the manuscript, such as after model names.\n2. The mathematical formula formatting needs revision, as punctuation marks (commas or periods) are missing at the end of equations.\n3. While the authors claim that experimental results are averaged over multiple runs, standard deviations are not reported.\n4. The authors only analyze the training time of the model but lack analysis of time complexity.\n5. It is suggested that the authors move critical theoretical analyses to the main text to improve readability.\n6. The authors acknowledge that the hypercomplex linear layers do not satisfy the hypercomplex differentiability conditions, and the gradients are not strictly hypercomplex. Does this imply that the backpropagation process is actually performed in the real domain? What are the implications of this approach for model optimization and convergence?\n7. The authors acknowledge that hypercomplex networks consume significantly more memory than real-valued layers. Have the authors considered approximation methods (e.g., low-rank decomposition, sparsification) to reduce computational costs?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Mb7mPDOheq", "reviewer": "Reviewer_Dzt7", "summary": "This paper introduces Numerion, a novel framework for long-term time series forecasting that leverages multi-hypercomplex algebraic representations to model complex inter-channel dependencies and cross-frequency interactions. The core idea is to embed different temporal and channel-wise features into multiple hypercomplex spaces, and use hypercomplex convolutions and mixing operations to enhance expressive power while maintaining compactness.", "questions": "As in Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "It is the first work to systematically apply multi-hypercomplex algebra to time series forecasting and provides a unified representation that captures both amplitude and phase interactions across channels.\nIt outperforms DLinear, TimesNet, PatchTST, and other baselines on multiple benchmarks.\nThis paper includes comparisons across different algebraic spaces, and studies of layer depth and hypercomplex dimensionality. The ablation results clearly isolate the contribution of each module.", "weaknesses": "How are the results of PatchMLP and TimeMixer in Table 1 obtained? According to the original papers of PatchMLP [1] and TimeMixer [2], there is an excessively large gap between their results and those presented in your paper. Additionally, Numerion's performance on some datasets is far from achieving the state-of-the-art performance when compared to PatchMLP. For example, on the Weather dataset, Numerion achieves an MSE of 0.246 and an MAE of 0.271, while PatchMLP yields an MSE of 0.231 and an MAE of 0.256. On the Solar-Energy dataset, your model’s results are an MSE of 0.252 and an MAE of 0.262, whereas PatchMLP achieves an MSE of 0.211 and an MAE of 0.261. Besides, Numerion also underperforms PatchMLP on the two datasets of ETTh2 and ETTm2.\nAlthough the paper mentions the properties of partial hypercomplex multiplication in the appendix, it lacks a systematic theoretical analysis to explain why the multi-hypercomplex space can improve prediction performance.\nWhile the model is parameter-efficient, hypercomplex multiplication scales quadratically with algebra dimension. Can the authors provide explicit complexity formulas?\n\n\n[1] Tang P, Zhang W. Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(12): 12640-12648.\n[2] Wang S, Wu H, Shi X, et al. Timemixer: Decomposable multiscale mixing for time series forecasting[J]. arXiv preprint arXiv:2405.14616, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Og6Alpg0lM", "reviewer": "Reviewer_ugew", "summary": "The paper proposes Numerion, a time-series forecaster that maps inputs into multiple hypercomplex spaces (complex, quaternion, octonion, sedenion) and learns with a unified Real–Hypercomplex–Real MLP (RHR-MLP). The key idea is that higher-dimensional hypercomplex spaces naturally bias toward lower-frequency components, enabling a simple, parallel decomposition and an adaptive fusion head to combine per-space predictions. Across standard long-term forecasting benchmarks, Numerion reports SOTA or competitive results, and provides visual/quantitative analyses indicating specialization of higher dimensions for trends and lower dimensions for fluctuations. The method generalizes linear layers and tanh to arbitrary power-of-two dimensions and supplies ablations, sensitivity, and efficiency studies.", "questions": "How sensitive is Numerion to patch levels/embedding dims under noisy domains (e.g., Exchange, where MAE improvement is only 0.005), beyond the Electricity study?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper extends linear/activation layers to arbitrary hypercomplex dimensions and composes them into a parallel multi-space forecaster with adaptive fusion—distinct from typical decomposition or spectral methods.\n- Offers a simple, backbone-agnostic path to multi-frequency modeling without heavy architectural complexity; could inspire broader use of hypercomplex representations for TS.\n- Comprehensive experiments across 10 datasets with consistent improvements and honest discussion where variable-fusion models can outperform on high-dimensional data; includes ablations, visualization, and sensitivity.", "weaknesses": "- This paper lacks comparisons against strong, explicit frequency or time–frequency baselines, such as FBM[1], to support the claimed advantage brought by operations in higher-dimensional hypercomplex spaces. \n\n- It remains unclear whether improvements stem from genuinely hypercomplex algebra or simply from vector factorization at dyadic sizes ($n \\in$ \\{$1,2,4,8,16$\\}). The Cayley–Dickson construction does not mathematically guarantee a low-pass (frequency-selective) behavior; thus the observed “low-frequency preference” might arise from architecture/parameterization effects rather than algebraic properties. The authors may add controls that replace hypercomplex layers with *grouped real MLPs* (same splits $n $); also visualize learned frequency responses to disambiguate where the bias truly comes from.\n\n\n[1] Yang, R., Cao, L., & YANG, J. (2024). Rethinking Fourier transform from a basis functions perspective for long-term time series forecasting. Advances in Neural Information Processing Systems, 37, 8515-8540.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)