TimeKAN: KAN-based Frequency Decomposition Learning Architecture for Long-term Time Series Forecasting

Authors: Songtao Huang, Zhen Zhao, Can Li, LEI BAI
Primary area: learning on time series and dynamical systems
Keywords: Kolmogorov-Arnold Network; Time Series Forecasting

Abstract:
Real-world time series often have multiple frequency components that are intertwined with each other, making accurate time series forecasting challenging. Decomposing the mixed frequency components into multiple single frequency components is a natural choice. However, the information density of patterns varies across different frequencies, and employing a uniform modeling approach for different frequency components can lead to inaccurate characterization. To address this challenges, inspired by the flexibility of the recent Kolmogorov-Arnold Network (KAN), we propose a KAN-based Frequency Decomposition Learning architecture (TimeKAN) to address the complex forecasting challenges caused by multiple frequency mixtures. Specifically, TimeKAN mainly consists of three components: Cascaded Frequency Decomposition (CFD) blocks, Multi-order KAN Representation Learning (M-KAN) blocks and Frequency Mixing blocks. CFD blocks adopt a bottom-up cascading approach to obtain series representations for each frequency band. Benefiting from the high flexibility of KAN, we design a novel M-KAN block to learn and represent specific temporal patterns within each frequency band. Finally, Frequency Mixing blocks is used to recombine the frequency bands into the original format. Extensive experimental results across multiple real-world time series datasets demonstrate that TimeKAN achieves state-of-the-art performance as an extremely lightweight architecture. Code is available at https://github.com/huangst21/TimeKAN.

Decision: accept (poster)
Reviewer scores: 3, 5, 8, 8

Reviews and discussion:
<thread>
creation date:
1730066448618

thread_type:
official_review

<thread_object>
creator: reviewer_bgvs

<content>
summary:
This paper explores a method for decomposing mixed frequency components into distinct single-frequency components to improve time series forecasting accuracy. The proposed approach, called TimeKAN, is based on the Kolmogorov-Arnold Network (KAN). TimeKAN's process consists of three key components: (1) Cascaded Frequency Decomposition (CFD) blocks, which use a bottom-up cascading approach to obtain series representations for each frequency band; (2) Multi-order KAN Representation Learning (M-KAN) blocks, which capture and represent specific temporal patterns within each frequency band; and (3) Frequency Mixing blocks, which recombine the separated frequency bands back into the original series format.

The study demonstrates that TimeKAN outperforms several state-of-the-art forecasting methods, including Autoformer, FEDformer, and iTransformer, by achieving lower MSE and MAE across multiple time series datasets such as Weather, ETTh2, and ETTm2.

strengths:
1. Figure 1 is beautifully designed and provides an intuitive overview of each component in the new TimeKAN method, as well as how they connect.

2. The study makes effective use of large-scale datasets and performs comparisons with a variety of other methods (including CNN-based and Transformer-based models), demonstrating the advantages of TimeKAN. The model is also tested across different prediction lengths, and for datasets where performance is less optimal, the paper offers thorough explanations and detailed insights.

3. The analysis delves into several key components of TimeKAN, such as Upsampling, Depthwise Convolution, and Multi-order KANs. I especially appreciated this section, as it not only establishes that TimeKAN outperforms other deep learning methods but also shows that each individual component of TimeKAN is optimally designed.

weaknesses:
1. Section 3.2 appears somewhat disorganized. While the overall logic is clear, the expression could be refined for clarity. Additionally, more mathematical details and background should be provided, which can be included in the appendix.

2. If possible, please add more data to Table 5 in Section 4.3. Supplement it with the performance of other methods in Table 1 on parameters (params) and MAC across these six datasets.

questions:
N/A
</content>
</thread_object>

</thread>
<thread>
creation date:
1730350949404

thread_type:
official_review

<thread_object>
creator: reviewer_8zzz

<content>
summary:
TimeKAN is a time series forecasting model that combines frequency decomposition, representation learning, and mixing. It first uses a moving average to separate high and low frequencies, creating multi-level sequences that are embedded into a high-dimensional space. Cascaded Frequency Decomposition (CFD) blocks progressively isolate each frequency band. The Multi-order KAN Representation Learning (M-KAN) blocks use Kolmogorov-Arnold Networks to capture temporal patterns within each frequency band independently. Finally, the Frequency Mixing blocks recombine these decomposed bands to restore the original sequence, which is then used for forecasting through a linear layer

strengths:
1	Clarity and Structure: The paper follows a logical flow from problem statement to conclusions, making complex ideas accessible.
	2.	Thorough Background: A strong review of related work provides valuable context, situating the contribution within the field.
	3.	Detailed Experiments: Comprehensive experiments across multiple datasets support the model’s performance claims, with ablation studies highlighting component effectiveness.
	4.	Focused Writing: The paper stays on topic, avoiding unnecessary details and maintaining focus on the core contribution

weaknesses:
Depth in Explanation: The methodology section could offer more detail on complex components like Kolmogorov-Arnold Networks for greater accessibility.

questions:
What if we split the frequency band to more layers (more than 3 for example ). Will it increase performance ?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730698948816

thread_type:
official_review

<thread_object>
creator: reviewer_gwx6

<content>
summary:
In this paper, the authors propose a time series forecasting method based on KAN. TimeKAN uses frequency decomposition and KAN to effectively capture temporal correlations of the data. Experiments show the effectiveness of TimeKAN based on real-world datasets.

strengths:
1. A vivid introduction and related work section to explain the background of time series forecasting and KAN.
2. A clear figure to illustrate the overall framework of TimeKAN. In the methodology section, all components are detailed. 
3. Good ablation study to test the effectiveness of the different components of the model.

weaknesses:
1. The author does not explain why TimeKAN does not perform well in the Electricity dataset. It would be helpful if the authors could provide potential reasons or hypotheses for why TimeKAN underperforms on the Electricity dataset specifically. 
2. From Table 4, I do not see a huge increase with KAN compared to MLP models. Generally, if these results are similar, mostly, KAN is much slower than MLP. It would be good to see runtime comparisons between KAN and NLP implementations. Additionally, if KAN is slower than MLP in practice, it would be beneficial for authors to discuss more reasons why we prefer KAN over MLP. 
3. For the look-back window, the authors do not compare TimeKAN with other models. For most models, when the prediction length is fixed, the prediction accuracy will increase as the look-back window increases. It is beneficial to provide a comparative analysis of TimeKAN's performance with varying look-back windows compared to other baseline models. This would provide a more comprehensive evaluation of TimeKAN's capabilities relative to existing methods.
4. For baseline methods, it is better to choose more frequency-based (such as FreTS) methods since frequency decomposition is a key contribution.

questions:
1. See weaknesses.
2. Could you provide a short theoretical analysis about why in some cases in time series forecasting, KAN is better than MLP?
3. For Table 2, why not include the electricity dataset?
4. For KAN, you mentioned that the Kolmogorov-Arnold representation theorem states that any multivariate continuous function can be expressed as a combination of univariate functions and addition operations. Could you explain more about how it can capture multivariate correlations?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730996526270

thread_type:
official_review

<thread_object>
creator: reviewer_xpef

<content>
summary:
The paper presents TimeKAN, a Kolmogorov-Arnold Network (KAN)-based model for long-term time series forecasting, designed to handle complex multi-frequency patterns in real-world data. Traditional models struggle with mixed frequencies, but TimeKAN addresses this with a three-part architecture: Cascaded Frequency Decomposition to separate frequency bands, Multi-order KAN Representation Learning to model each band’s specific patterns using adaptable polynomial orders, and Frequency Mixing to recombine frequencies effectively. Experiments show that TimeKAN achieves superior accuracy and efficiency compared to existing models, making it a robust, lightweight solution for complex TSF tasks.

strengths:
**Exploratory Application of KAN to Time Series Forecasting**: The paper attempts to introduce Kolmogorov-Arnold Networks (KAN) into time series forecasting, using multi-order polynomial representations to handle the complexities of different frequency components. While KAN has not been widely applied in this area, this effort demonstrates its potential flexibility in data fitting and offers an alternative approach to traditional MLPs.

 **Comprehensive Experimental Design**: The paper includes experiments across various time series datasets, such as weather, electricity, and energy data, covering diverse scenarios. Additionally, it conducts ablation studies to examine the effects of each module. These experiments help to assess TimeKAN’s performance and may provide a reference point for further research.

weaknesses:
**Lack of Innovation**: The primary contribution of this paper is the integration of the Kolmogorov-Arnold Network (KAN) into time series forecasting, yet the work does not introduce novel methods or substantial breakthroughs in methodology. While the inclusion of KAN is somewhat new, other components, such as frequency decomposition and mixing, are mature techniques, and the paper does not propose innovative applications or enhancements to these. Overall, this work appears more like a combination of existing technologies rather than a genuinely innovative study.

**Absence of Comparison with Cutting-Edge Models**: The experiments lack direct comparisons with state-of-the-art models, especially those in high demand for time series forecasting, such as large language models (LLMs) and foundation models. Given current research trends, these models have become widely adopted benchmarks. Without such comparisons, the effectiveness of the proposed method remains unclear, especially as the improvements presented are relatively limited compared to advancements in mainstream approaches.

 **Reliance on KAN, a Model with Limited Validation**: The foundation of TimeKAN is the KAN model, which has not yet been widely validated or accepted. Its theoretical correctness and practical effectiveness remain uncertain, which casts doubt on the reliability and generalizability of TimeKAN as a whole. If there are inherent issues with KAN, the predictive performance and stability of TimeKAN could be compromised, making the paper's conclusions less convincing.

**Insufficient Analysis of Computational Efficiency**: While the paper claims that TimeKAN is more lightweight than existing methods, it lacks an in-depth analysis of its actual computational efficiency, especially compared to more mainstream and optimized time series models. Additionally, there is no quantification of the computational cost associated with KAN’s multi-order polynomial calculations when handling long-sequence data. Given that many time series tasks require efficient real-time computations, focusing solely on parameter reduction does not adequately demonstrate TimeKAN’s advantage in computational efficiency; the absence of data on inference speed and computational cost undermines its practical applicability.


**Focus on Single-Task Performance Rather Than Generalized Representation**: A dominant trend in time series modeling now follows the approach of large language models (LLMs) to develop foundation models and leverage self-supervised representation learning. This approach enables generalization across various tasks and domains, ultimately aiming for a “one-fit-all” solution. However, this paper remains focused on improving single-task performance in time series forecasting (TSF), which may be of limited value in light of the broader goals of the field. Furthermore, the improvements reported in the experimental results are relatively modest, and without statistical significance testing, it remains unclear if these gains are truly meaningful or could simply be attributed to random variation.

questions:
**To my knowledge, KAN itself has not been formally accepted, meaning it has not undergone rigorous peer-reviewed validation. If KAN’s theoretical foundation is later found to be flawed, would this impact the validity of this paper?** – If the underlying theory or structure of KAN is later shown to have limitations or inaccuracies, would the overall reliability of TimeKAN be compromised? Has the author considered this risk, and are there alternative solutions in place?

**In the experimental section, this paper does not compare TimeKAN with current state-of-the-art models (such as large language models or foundation models), making it difficult to assess its actual performance** – Without direct comparisons with these advanced models, can TimeKAN demonstrate a significant advantage? If the authors believe TimeKAN holds particular value in computational efficiency or predictive accuracy, could more data be provided to quantify this advantage?

**A major trend in time series research is developing foundation models, inspired by large language models, to generalize across domains and tasks. However, it is unclear if TimeKAN’s current design can support such robust representation learning** – Can TimeKAN truly compete with established frameworks like Transformers in terms of generalization and adaptability? If not, have the authors considered alternative approaches to enhance TimeKAN’s structural robustness and flexibility for broader applications?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732544802561

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We sincerely appreciate all the reviewers for their thoughtful feedback and constructive suggestions, which have been immensely helpful in guiding us to enhance our paper. We would like to clarify three points in order to address the remaining concerns of some reviewers to the best of our ability:

(1)**The core foundation of TimeKAN lies in the Decomposition-Learning-Mixing architecture rather than KAN**, which decomposes time series into multiple frequency bands represented in the time domain and leverages time-domain processing techniques to learn representations for specific frequencies.  Multi-order KAN are carefully designed to efficiently adapt this framework and improve forecasting performance across multiple datasets. TimeKAN has been demonstrated state-of-the-art performance across multiple long-term time series forecasting tasks and showcasing its lightweight and parameter-efficient architecture.

(2)We understand the the concerns of some reviewers regarding the effectiveness of KAN. However, **we emphasize that KAN is not a seamless replacement for MLP and may not work in all cases. It requires careful design to  fully realize its potential.** As shown in Table 4 of the paper, directly replacing MLP with KAN results in a performance that is actually worse than MLP. Therefore, in TimeKAN, we have designed the Multi-order KAN to better match the modeling of different frequency components, thereby maximizing the advantages of KAN’s variable complexity. And the results demonstrate that Multi-order KAN outperforms both the direct use of KAN and the direct use of MLP. 

(3) To the best of our knowledge, **the vanilla KAN has received positive feedback on OpenReview, with five reviewers consistently agreeing on its contribution to the community, which to some extent demonstrates its effectiveness.** Therefore, we hope the reviewers will approach KAN with a more open perspective.

We sincerely hope the reviewers will reconsider their score and look forward to further feedback.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a novel forecasting method that exploits the frequency structure of time series. The method, TimeKAN, has a decomposition component, a representations learning component based on the Kolmogorov-Arnold Network, and a mixing component. 

Overall, this paper has a solid technical contribution and convincing experiments. The reviewers also praised the clarity of the paper, the thorough presentation of background work, and the ablation studies. 

There were some reservations expressed because the KAN algorithm has not yet been accepted at a conference - however, it has been public on arXiv since April 2024 and the proposed method is sufficiently different from that algorithm, so I do not see it as a problem, particularly since the authors have conducted additional experiments replacing the KAN structure with an MLP, showing the merits of the other components in their architecture (i.e. the method is not exclusively reliant on the success of KAN). 

The concerns raised by reviewer XpEF about comparisons to state-of-the-art models have been adequately resolved in the rebuttal through the addition of comparisons to moment and TimeFFM.

Thus, I recommend acceptance of this submission.