Title: Accelerating Eigenvalue Dataset Generation via Chebyshev Subspace Filter
Venue: NeurIPS 2025
Accepted: no
Keywords: Data Generation

Eigenvalue Problem

AI4PDE
Abstract: Eigenvalue problems are among the most important topics in many scientific disciplines. With the recent surge and development of machine learning, neural eigenvalue methods have attracted significant attention as a forward pass of inference requires only a tiny fraction of the computation time compared to traditional solvers. 
However, a key limitation is the requirement for large amounts of labeled data in training, including operators and their eigenvalues.
To tackle this limitation, we propose a novel method, named **S**orting **C**hebyshev **S**ubspace **F**ilter (**SCSF**), which significantly accelerates eigenvalue data generation by leveraging similarities between operators---a factor overlooked by existing methods. 
Specifically, SCSF employs truncated fast Fourier transform (FFT) sorting to group operators with similar eigenvalue distributions and constructs a Chebyshev subspace filter that leverages eigenpairs from previously solved problems to assist in solving subsequent ones, reducing redundant computations.
To the best of our knowledge, SCSF is the first method to accelerate eigenvalue data generation. 
Experimental results show that SCSF achieves up to a $3.5\times$ speedup compared to various numerical solvers.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel method called Sorting Chebyshev Subspace Filter (SCSF) to accelerate the generation of eigenvalue datasets for training neural eigenvalue methods. The key insight is that operators in datasets often share similarities since they describe related physical phenomena, but existing approaches solve each eigenvalue problem independently. SCSF addresses this by first using truncated FFT sorting to group operators with similar eigenvalue distributions, then employs Chebyshev filtered subspace iteration to leverage eigenpairs from previously solved problems when tackling subsequent ones. They claim this is the first method specifically designed to accelerate eigenvalue dataset generation and demonstrate up to 3.5× speedup compared to state-of-the-art solvers across four different operator types.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: I think the authors have tackled an important and practical problem. The computational cost of generating eigenvalue datasets is indeed a significant bottleneck for training neural eigenvalue methods, and the authors correctly identify that existing approaches ignore the inherent similarities between operators in these datasets. The technical approach seems sound and well-motivated. The authors cleverly combine two existing techniques - truncated FFT sorting and Chebyshev filtered subspace iteration - in a novel way that's specifically tailored for dataset generation. I appreciate that they provide detailed algorithmic descriptions and complexity analysis. The experimental evaluation is quite comprehensive, covering four different operator types (Poisson, elliptic, Helmholtz, vibration) and comparing against multiple baseline solvers.\n\nWeaknesses: However, I have several concerns about the work. First, the theoretical analysis is somewhat limited. While the authors provide complexity analysis, they don't give much insight into when we can expect the method to work well versus when it might fail. The success of SCSF heavily depends on the similarity structure in the dataset, but there's limited discussion of how to characterize or predict this similarity. Second, I think the authors could better analyze the sorting overhead. While they show that truncated FFT sorting is faster than naive greedy sorting, the overall cost-benefit analysis could be more thorough. For small datasets or when operators are not very similar, the sorting overhead might outweigh the benefits. Third, the experimental section, while comprehensive in breadth, could use more depth in some areas. For instance, the authors don't thoroughly explore failure cases or discuss when their method performs poorly. The hyperparameter analysis is relegated to the appendix, but some of this seems important for understanding the method's sensitivity.\n\nOn the clarity front, the paper is generally well-written, though some sections feel dense. The method description is clear, but I think the authors could better motivate why Chebyshev filtering specifically is the right choice here versus other subspace iteration methods.\nIn terms of significance, this addresses a real practical problem, but the impact might be somewhat limited since it's specifically focused on eigenvalue dataset generation rather than eigenvalue solving more broadly. The originality is good - while the individual components exist, combining them for this specific application appears novel.\n\nQUESTIONS:\nI have several questions that I think could help improve the paper:\n\n1. Can the authors provide better theoretical insight into when SCSF is expected to work well? Specifically, what properties of the operator distribution or dataset characteristics make SCSF most beneficial? This would help practitioners understand when to apply the method.\n2. How sensitive is the method to the quality of the initial sorting? The authors show that sorting helps, but what happens when the similarity metric doesn't capture the true relationships between operators well? Have they considered adaptive or iterative sorting strategies?\n3. The paper focuses on Hermitian matrices, but many practical eigenvalue problems involve non-Hermitian or generalized eigenvalue problems. How would SCSF extend to these cases, and what modifications would be needed?\n4. Could the authors provide more analysis of the computational trade-offs? While they show speedups, a more detailed breakdown of when the sorting overhead is justified would be valuable. For instance, what's the break-even point in terms of dataset size?\n\nLIMITATIONS:\nI think the authors have done a reasonable job addressing limitations, particularly in Section 5. They acknowledge that the method is currently focused on general eigenvalue problems and that extending to nonlinear eigenvalue problems would require additional work. They also mention the potential for better distance metrics in the sorting algorithm. However, I think they could expand the limitations discussion to include scenarios where the method might not work well, such as when operators in the dataset are highly dissimilar or when the dataset is small.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides an algorithm that speedup the eigenvalue computation for a sequence of matrices that all originated from the same partial differential equations. It has two contributions 1)  To benefit the successive solving sequence of the eigenvalue problem, one way is to sort the matrices that pull matrices with similar spectral properties close in the solving sequence. The solution of one matrix can then be used to provide a warm start to the solution of the next matrix in the sequence. Typically, this can be done with the F-norm. However, since the eigenvalues are dependent on low-frequency info, the paper first uses truncated FFT to preprocess the matrices to retain only low-frequency information, then applies the sorting using the F-norm. 2) It proposed a method that can do a warm start well, based on Chebyshev filtered subspace iteration.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n\n- The paper is well written and easy to understand\n- The proposed approach makes intuitive sense\n- The results do show that there is a speedup of 3.5x overall compared with other methods.\n\nWeakness:\n- I wonder if NeurIPS is the best venue for this paper? I can understand that the motivation for developing this approach is to speed up the generation of benchmark matrices and the computation of the labels (eigenvalues). But a conference in the numerical analysis field could provide a better gauge as to the significance of the proposed methods\n\n- A factor of 3.5x is significant in general. But if benchmark generation is a one-off, I wonder if it is that important, or if we can afford to wait for the results. Perhaps solving the eigenvalues of a sequence of matrices has other applications that make this contribution more significant? Or perhaps the methods, esp Chebyshev filtered subspace iteration, can motivate improvement to other fundamental numerical analysis methods (see my question as to whether a numerical analysis conference is a better fit)\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the modern \"parametric operator learning framework\" such as DeepONet, which tries to directly learn a mapping from an operator to its eigenvalue pairs. This is a supervised learning framework that requires such a \"labeled\" dataset {(operator, eigenvalue pairs)}, and this work aims to reduce the time to generate the labeled dataset. The parametric operator framework works upon the assumption that operators in a dataset have enough commonalities to be exploited, so, for example, they assume a class of PDEs with varying parameters. In this setting, they want to expedite to get the eigenvalue pairs for different parameters. Their methodology is essentially as follows: \n1) Sort the parameters by applying \"FFT\" to the parameters. \n2) Given the sorted parameters, they sequentially solve each eigenvalue problem, but utilizing the previous iteration via the \"Chebyshev filtered subspace iteration.\"  \nThe resulting algorithm is evaluated on four different eigenvalue problems, and the results compared to several existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- the problem is one of growing interest in a range of \"AI for science\" problems\n\n- it highlights the role that the established Chebyshev subspace filtering methods can play in efficiently solving eigenvalue problems at scale\n\n- the empirical results demonstrate the computational complexity can be significantly smaller than several existing methods.\n\nWeaknesses:\n\n- the novelty of the paper is rather limited.   In particular, the addition to the methodology beyond the standard Chebyshev filtering appears to be introduction of the \"parameter FFT\", which while useful is a somewhat incremental contribution.\n\n- generally the writing is good, but the transition to Section 2.2 is rather abrupt...the motivation and context are absent, and is disconnected from what precedes it.  Moreover, the writing is too technical to appreciate at the point it appears.\n\nQUESTIONS:\n1.  To the extent that it differs from what is written above, could the authors summarize what they would characterize as most novel in the work's methodology?\n\n2.  Does the proposed method have any concrete optimality properties?  To what extent does it bound achievable speeds?\n\n3.  How were the datasets chosen for the experiments?  The choices seem a bit arbitrary.   Are they representative in some way?  Are there examples of operator eigenvalue problems that create particular challenges for the methodology?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduced a new approach to generating eigenpairs for supporting data-driven eigenvalue methods. The main method is based on a Chebyshev subspace filter and makes use of the structures of operators. The authors presented numerical experiments to demonstrate the effectiveness of the proposed algorithm.\n\nSTRENGTHS AND WEAKNESSES:\n### Strength\nGenerally, the technical writing is easy to follow; the overall analyses are technically sound. Experiment setups are reasonable and give positive results.\n\n### Weakness\n1. [Significance] It is unclear to me whether the dataset generation is the bottleneck for designing data-driven eigensolvers. Particularly, there have been recent works on solving the eigen/SVD problems directly from the operator itself, without requiring eigenpairs in its training. Examples include:\n- Pfau, David, et al. \"Spectral inference networks: Unifying deep and spectral learning.\" arXiv preprint arXiv:1806.02215 (2018).\n- Deng, Zhijie, Jiaxin Shi, and Jun Zhu. \"Neuralef: Deconstructing kernels by deep neural networks.\" International Conference on Machine Learning. PMLR, 2022.\n- Ryu, J. Jon, et al. \"Operator SVD with neural networks via nested low-rank approximation.\" arXiv preprint arXiv:2402.03655 (2024).\n\nIn comparison, this paper primarily focuses on classical numerical methods as baselines, which may be less relevant given current deep learning practices.\n\n2. [Methodology] Since the proposed method relies on some existing numerical solvers, it is not guaranteed that the generated dataset can be used as a ground truth. In particular, matrix-based approaches are sensitive to the discretization process and can hardly be scaled to high-dimensional problems. \n3. [Performance] Without context, it is not obvious to see whether a 3.5x speedup is a significant performance gain. Specifically, parallization could provide similar gain for the particular dataset generation problem.\n\nQUESTIONS:\n1. How is the boundary condition dealt with in discretizing the operator problem to the matrix problem?\n2. Could the authors comment on how the parameters in FDM affect the error of computed eigenpairs?\n### Comments:\n+ Eq. (2) is not a good example for a 2D problem; at least $3 \\times 3$ grids are needed to see how the discretization works. \n+ L86, missing a space after the second comma;\n+ Sec 2.2 is mostly facts; the authors might consider moving some parts to the supplementary.\n\nLIMITATIONS:\nThe intrinsic error and scalability issues introduced by FEM or similar discretization processes shall be included as a limitation of applicability.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-10"}