Title: Kolmogorov-Arnold Fourier Networks
Primary area: foundation or frontier models, including LLMs
Keywords: Deep Learning, KAN, MLP, Fourier

Abstract: Although Kolmogorov-Arnold based interpretable networks (KAN) have strong theoretical expressiveness, they face significant parameter explosion and high-frequency feature capture challenges in high-dimensional tasks. To address this issue, we propose the Kolmogorov-Arnold-Fourier Network (KAF), which effectively integrates trainable Random Fourier Features (RFF) and a novel hybrid GELU-Fourier activation mechanism to balance parameter efficiency and spectral representation capabilities. Our key technical contributions include: (1) using Fourier transform to avoid the parameter explosion problem caused by KAN's B-spline function and significantly reduce the number of parameters; (2) introducing learnable RFF initialization strategies to eliminate spectral distortion in high-dimensional approximation tasks; (3) implementing an adaptive hybrid activation function that progressively enhances frequency representation during the training process. Comprehensive experiments demonstrate the superiority of our KAF across various domains, including vision, NLP, audio processing, and differential equation-solving tasks. We will release the source code of our method in accordance with the review policy.

TL;DR: We propose the Kolmogorov-Arnold-Fourier Network (KAF), which integrates trainable Fourier features to solve the parameter explosion of KANs and improve high-frequency representation in high-dimensional tasks.