FAME: Adaptive Functional Attention with Expert Routing for Function-on-Function Regression
Authors: Yifei Gao
Yong Chen
Chen Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Functional Attention; Function-on-Function Regression;Bidirectional NCDE;Mixture-of-Experts
Primary area: deep_learning
TL;DR: Introduce a functional attention mechanism that leverages bidirectional NCDEs and MoE to model function-on-function regression.

Abstract: Functional data play a pivotal role across science and engineering, yet their infinite-dimensional nature makes representation learning challenging. Conventional statistical models depend on pre-chosen basis expansions or kernels, limiting the flexibility of data-driven discovery, while many deep-learning pipelines treat functions as fixed-grid vectors, ignoring inherent continuity. In this paper, we introduce Functional Attention with a Mixture-of-Experts (FAME), an end-to-end, fully data-driven framework for function-on-function regression. FAME forms continuous attention by coupling a bidirectional neural controlled differential equation with MoE-driven vector fields to capture intra-functional continuity, and further fuses change to inter-functional dependencies via multi-head cross attention. Extensive experiments on synthetic and real-world functional regression benchmarks show that FAME achieves state-of-the-art accuracy and strong robustness to arbitrarily sampled discrete observations of functions.

FAME: Adaptive Functional Attention with Expert Routing for Function-on-Function Regression
Yifei Gao
Yong Chen
Department of Industrial Engineering Tsinghua University
Department of Industrial and Systems Engineering
University of Iowa
Beijing 100084, China gao-yf @mail. tsinghua. edu.
Iowa City, IA 52242, USA
yong-chen&uiowa edu
Chen Zhang*
Department of Industrial Engineering
Tsinghua University Beijing 100084, China
zhangchen01@tsingguue edu. on
Abstract
Functional data play a pivotal role across science and engineering, yet their infinite- dimensional nature makes representation learning challenging. Conventional sta- tistical models depend on pre-chosen basis expansions kernels, limiting the flexibility of data-driven discovery, while many deep-learning pipelines treat func- tions as fixed-grid vectors, ignoring inherent continuity. In this paper, we introduce Functional Attention with a Mixture-of-E Expertss (FAME), an end-to-end, fully data- regression. FAME forms continuous
driven framework for function-on-functeo attention by coupling bidirectional neural controlled differential equation with MoE-driven vector fields to capture intra- futatffuntiioaal continuity, and further fuses change to inter-functionll dependencies via multi-head cross attention. Extensive experiments synthetic and real-world functional regression benchmarks show that FAME achieves state-of-the-an accuracy and strong robustness to arbitrarily sampled discrete observations of functions.
Introduction
Functional data samples whose individual elements random continuous functions, providing rich temporal-spatial dynamics domains such as biology [3] marketing [11,, transportation [17],
and meteorology [42] With the growing demand for high-resolution measurements, functional data analysis (FDA) is attracting increasing attention both scientific research and practical applications
48 Among canonical problems in functional data analysis (FDA), dunction-on-function regression (FoFR) is widely regarded as the most challenging and representative task, as it requires the model to accommodate both infinite-dimensionaal inputs and outputs. (35 25 D. While functional
41 27
20
data offer new opportunities, they also bring intertwined challenges: (1) fntra-functioaal continuity: each function lies in infinite-dmennee
continuous functional space with features such as local dynamics and global trends [29]; (2) inter-functions interactions: different dimensions of the input function can have nonlinear couplings with each other (]: (3) Feature heterogeneity: different functions may exhibit vastly different properties such as scale, smoothness, or noise, and may even reside in different functional spaces
Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To address these challenges, existing approaches broadly fall into two methodological categories. The first category comprises classical statistical methods, which manage functional complexity by projecting each function onto fmite-dimensional representations. Common techniques include basis expansions, such as B-splines [5), wavelets [49), and functional principal component analysis (FPCA) [8. as well as kernel- based methods that embed functions into functional reproducing kernel Hilbert spaces via operator-valued kernels [4]. Although these methods facilitate linear or additive modeling in lower-dimensional spaces, their performance heavily depends on predefined bases or kernels and strong smoothness assumptions, limiting their ability to capture intra-functional and inter-functions features prevalent in real-world functional data. The second category involves recent deep learning approaches, which offer greater flexibility inspired by advances in natural language processing and computer vision. These methods typically feed discretized curves into deep neural networks. For instance, Shi et al.
[33 proposes a smooth-kernel neural network tailored to nonlinear functional [45] introduces adaptive basis learners that adjust expansions according to
regression. Yao al. specific tasks. Although such models enhance nonlinearity and support end-to-end learning, they still assume all functions lie in discretized regular sampling grids with the same dimension, and treat functional data as finite-dimensional vectors, Consequently, these methods have limited power to model intra-functional continuous dynamics, and cannot be applied to irregularly sampled cases, let alone handle feature heterogeneity.
To overcome these limitations, introduce Functional Attention with Mfxture-of offterrr- (FAME), an end-to-end framework designed expressly for FOFR. Specifically, our contributions are: (I) FAME is the first FoFR model that directly operates on irregularly sampled functional space without relying on predefined basis functions or discretisation grids. Its effectiveness is rigorously established through both theoretical guarantees and extensive experimental validation. (2) FAME includes a novel functional attention mechanism composed of continuous attention via bidirectional Neural controlled differential equations (NCDEs), which can efficiently capture intra-functional continuity, and multi- head cross attention, which can efficiently model inter-functional interactions. (3) FAME enhances a mixture-of-experr (MoE) architecture, enabling adaptive modeling of feature heterogeneity, and incorporates an NCDE decoder capable of generating continuous functional outputs at arbitrary query locations, naturally accommodating misaligned target indices.
2 Related work
Function-on--function regression FoFR has witnessed substantial advances recent years and is drawing increasing attention from the research community. Existing approaches can be concretely divided into three categories: linear decomposition, reproducing kernel Hilbert space (RKHS). and deep learning methods. Linear approaches employ finite bases such as FPCA, wavelets, or splines to project input and response curves into oow-dimesiional coordinates, enabling conventional 28 Despite computational efficiency, these approaches rely on fixed bases, restricting their ability to capture non-stationary dynamics and complex interactions. RKHS-based methods use operator-valued kernels, providing nonlinear function regression within linear frameworks [19 14. Extensions have incorporated robust losses [T6] and optimal estimation strategies (35), yet kernel selection and scalability remain persistent challenges. Deep learning methods, including functional neural networks (FNNs) [30. FPCA-based neural networks [39 40). and adaptive basis expansions [45). offer data-driven flexibility. However, these techniques typically discretize functions onto fixed grids, ignoring the inherent continuity and heterogeneity of functional data. In summary, current methods rely either predefined prematurely discretized feature representation for functional data, limiting their ability to model complex FoFR problems comprehensively.
regression techniques 44 24
23
22
Neural differential equations Neural differential equations provide a powerful framework that combines continuous-time dynamies with the high-capacity function approximation of neural net- works, making them particularly effective for handling irregular time series [31 12]. Among these methods, Neural ODEs map discrete data to continuous trajectories and leverage adjoint methods for memory-efficient gradient computation, enabling scalable training for large models [.. Building on this concept, Neural CDEs (15 extend the approach by capturing the continuous time dynamics of RNNs through learned control function for hidden states. This strategy has shown superior performance in modeling irregular time series compared to Neural ODEs or RNNs, particularly in offline prediction tasks [3]. Recent work also emphasizes their usefulness in enhancing stability
2

---

and generalization by injecting noise directly into the training process [26. However, Neural CDEs operate in a strictly causal fashion, building hidden-state dynamics solely from past observations. whereas FoFR can exploit the full input function to capture global dependencies and non-causal relationships.
Attention mechanism for feature representation The attention mechanism was initially proposed to mitigate the limitations of encoding sequences into fixed-length vectors in RNN-based models [2]. Xu et al. [43] introduces soft alignment during decoding, enabling better handling of long sequences. This concept evolves through variants like global and local attention, and reaches a milestone with the Transformer architecture [37, which entirely replaces recurrence with self-attention, improving both modeling power and computational efficiency. Building on this foundation, numerous attention- based models have emerged for time series modeling. For example, iTransformer(2] reorganises the input so that time steps serve as channels and variables act as tokens, allowing self-attention to capture long- range temporal dependencies more effectively, whereas CrossFormer(47] employs frequency-aware, cross-scale attention mechanism to model long- and short-term patterns simultaneously. However, these attention mechanisms designed for discrete sequences defined on fixed grids. They do not provide a native framework for continuous-space mapping of functions.
3 Function-on-fuuction regression
We consider an unknown operator T X' + y that maps a d-tuple of input functions X (X(1), ................................ (()) to an m-tuple of output functions Y = (Y ..... (Y" ((m)). For every input channel j E (1......d d} the function ( [0, + IR is assumed to have finite 1-variation; we denote the space by
V1 = =
10,
< <o0,, X 11 x ... x Va-
For each output index Ç E {1....,m,) m). the output function (() [0, Sc] + R is continuous and equipped with the supremum norm || supse (s)).)). We set
W:- = ((0,, y= Wh ... Wm.
|||| max ||
In practice, for each sample i we observe X; = (x x x EX, Y= (v 2, but only irregular time points ((e,,, for each X and ((s,,,,i,r)) for each y.(5) Given a dataset {(X,, M X x y, our goal is to learn a umite-paramette approximation Ta that recovers T from these discretely and non-uniformly observed functions.
v.(m) E
4 Functional attention with a mixture-of-experts (FAME)
In this section, we describe how FAME addresses the core challenges of FoFR by designing a custom functional attention mechanism. Figure D FAME To account for the truly infiniteddimensional nature of each function XO), we introduce in Section 4. continuous attention block built bidirectional NCDE that integrates both forward and backward over l. This construction produces smoothly varying Query, Key, and Value trajectories, ensuring that every query instant attends both past and future contexts and remains insensitive to the discrete sampling along the input function. To capture heterogeneity across functions, Section 4.2 enhances this continuous attention block by replacing its single vector field with a mixture of K expert fields (MoE), each specialised for distinct scales, amplitudes, or noise regimes. A learnable router allocates a weight vector over the K experts, thereby boosting the model's s ability to capture feature heterogeneity, removing the need to instantiate separate encoder for each function, and enabling smooth generalisation to unseen features. Section
presents high-level overview of the overall architecture of
introduces a cross attention fusion block that integrates these per-function representations into unified global context, propagating information across functions and modelling their continuously evolving, nonlinear couplings, before decoding via NCDE head to produce the final continuous output.
3

---

Y(s) (s), (2)) ym (s)
201(
Neural CDE Decoder
(0gg
(9
Cross Attention
(0
Wr
Contimuous Attention
d
200
MoE- GoE--enhancedwee vector field
Badlward NCDE
Eqpest.1
Enppert
Export
4
A
Forward NCDE
Leamable Router
X1g
Xx
C2
T;
(t) (t), .... (.)
Figure I: Architecture of the FAME.
4.1 Capturing intra-functional continuity through continuous attention
Controlled differential equations. The core motivation of our method is to construct attention mechanisms that operate directly on continuous functional data. We adopt the framework of neural controlled differential equations (NCDEs), whose integrals are interpreted in the Young sense because every input function has finite l-variation. In a controlled differential equation (C5) an external driver continuously influences the internal dynamics, providing principled route for learning from irregularly sampled paths. Formally, the latent trajectory is defined as the solution of:
=(1) = =11)) + I' [d(((u)) dx),,,
L E (lo, Tj},
(1)
where the initial state 2(tu)= & (to). is learnable, 2(t) encodes evolving summary of the input path, and fo governs how this summary changes in response to the signal X.
Bidirectional NCDE. A one-sided CDE only observes the driving signal on [to, t). so its latent state lacks access to the future segment (t, Tz]. To capture the essential global structure of functional data, we propose bidirectional NCDE that augments the standard forward integration with complementary backward pass. For each function X(1), we solve pair of controlled differential equations:
((i((1) (1)) + I" Sogul Sojul (T) (x)
EE Tyl.
S. (old) dx (T).
2 zú) (T;)
where fare and fara are direction-specife vector fields, and (i) denotes the reversed input function (() = (T) - t).
Assumption 1 (Directional Lipschitz regularity). Each vector field is globally Lipschitz: that is, Sopn is 1tndd Lipschitz and Soya is Lindd -Lpschiic
Theorem 1 (Existence and uniqueness of the Bi-NCDE). Under Assumption I both the forward and backward CDES (2) admit unique solutions on [to. Ty}. Moreover, letting L,, = max I1food I. bwd the latent paths satisfy
[z - 10 S <e's(T)--to to) l 8101

---

Then, the final latent representation is defined as ()) (Z 2 (4) E IR combining local dynamics with global context.
Continuous attention. To extend attention from discrete sequences to continuous functions, we apply three bounded matrices We, WK, Wr E IR x2h to ( thereby obtaining continuous Query, Key, and Value functions:
(t) = Wq (t), (t) = Wx (t), (t) = Wv (t),,, t [to. Tj]. (3)
Because bounded linear maps preserve Young-integrability and Lipschitz regularity, the ensuing attention operations well defined over [T] and reserve the continuous structure of the data. For any (f, T), we compute the attention score and normalized weight
(t, (t) (t)
(t()
(t,T+ = { (nb)( du
(4)
vdz
and form the attended representation
2 20-0) (0000 (L)
t EE Til.
(5)
Assumption 2 (Normalization regularity). There exists y > 0 such that fod (al))),)) (1,0) ds 2 y for all t and admissible inputs. In practice we ensure this by a temperature parameter temp 2 temp > and log-sum-exp stabilisation.
Proposition 1 (Lipschitz stability). Under Assumptions) Tand?
|| - 1 S |Wvll (4,111,10) [T, - to) ( -1:0-x16100 x(0 |1.arr
Y
Denote the constant on the right by forr later reference.
Proposition 2 (Universal approximation). Let k C(too Týl3) and E O. If 2 h and the width of so is sufficiently large, then there exist parameters (Sa, Wo, WK, Wv) with ||a(0) k Hence x + 2(5) can approximate any bounded continuous operator on functions, independent of the sampling grid;
Proposition confirms the Lipschitz stability of our continuous attention mechanism: a small perturbation of the input function produces only proportionally small change in its attended representation. Moreover, Proposition (orrsstttinnssoowss shows that, with sufficient width, the continuous attention can approximate any bounded continuous kernel and thus emulate arbitrary integral operators in FoFR. Consequently, by embedding continuous attention within a bidirectional NCDE, our model not only captures both local dynamics and global context but also can realize any mapping in the function space. More details are provided in Appendix A
4.2 Encoding feature-wise heterogeneity with MoE-Driven Bi-NCDE
Real-world functional data exhibit evident feature heterogeneity, as each function may vary on its own scale, frequency, and noise level. single vector field can scarcely accommodate such diversity, To address this, we assign each function its own MoE vector field. Specifically, shared router computes, once per function, a fixed convex combination of K specialist fields, which then governs the bidirectional NCDE dynamics for that function.
Function-wise router and expert fields. Let (fo, 1 - Jk=1 be K expert vector fields. For each input function ( we compress its functional information with a lightweight I-D convolution followed by global average pooling, obtaining a summary vector s(i) ConvPool ( e R
The router 9& t - R assigns softmax weights g() = softmax (s((())) E A K-1 to the K specialist fields, yielding the MoE vector field
K
[6 (2) = Σ
( T
(2)
θ { 0x.0). Φ).
(6)
5

---

Directional experts with shared router. To preserve the bidirectional representation we instanti- ate disjoint expert sets {fere } and (frm }. yet reuse the same gates π (5)) in both directions:
fo (2) E (1) Th
(z)
S (2) - TT fepu((2)
Assumption 3 (MoE regularity). Each expert fo and Jor is globally Ie and -ipschitn
respectively, and bounded by B and B The router 96 is bounded on the summary space: ()) S B all E Rho.
Lemma (Mixed-ficld Lipschitz bound). Under Assumption globally Lipschitz with
the mixed fields feg and f are
Lemix max
Σ
( mh
Σ
(3) "
s max mmax(T. I.ud).
Consequently, the bidirectional NCDE driven by satisfies the existence, uniqueness, and stability properties of Theorem
MoE-Driven Bi-NCDE. Replacing fara and for in with fgge and f yields the MoE-
Driven Bi-NCDE, which enables each input function to employ a custom vector field mixture and thereby model input functions with markedly different characteristics.
Theorem (Well-posed MoE-Driven Bi-NCDE). The MoE augmentation admits unique solutions, and the stability bound ( holds with L max fiod bbvd'
Thus the MoB design endows the encoder with function-wise expressiveness while retaining all continuity, sampling invariance, and Lipschitz stability guarantees of the base architecture.
4.3 Integrating inter-functional interactions via cross attention
Having obtained a function -wise representation 20) for each input function, we now capture the inter-functional dependencies that single-channel encoders miss. We realise this with multi-head cross attention that, at every time point f,, allows each function to attend to the concurrent states of all other functions.
For every head E {l,..., M) and function j E d). shared linear maps w-f)) w(P) w(l Edy project the latent paths into query, key, and value functions:
Cross attention.
QlAP)) (t) w (t),,,
(()))(t) = w (t). v(PP) = w (t)
(7)
Cross attention weights are defined by
glaee) exp((Quin) (t), K (tt)
e (E d}.
(8)
exp((QU<)()) (1), (00,))(r))))))))) (())) Vdc}
and each head outputs
H (1) = Σ 30
(9)
Concatenating the M heads and applying Wo 2hx Mde gives
Hb) Wo (TO[HH)))))))ll (H) LE Ty). Here Wo projects the concatenated representation back the original 27-dimensional latent space, keeping the hidden width uniform throughout the network.
(10)
Assumption (Cross attention Lipschitz). For every head p (I....,M) M} the projection matrices satisfy [ [lop. || |lpp [ |lop |Wollop & Mmat for some constant M > O. Under this condition, the cross attention map 2 + is globally Vecosr-liscchi with Temss S (Mmat)? 2 oit with respect to the L-variation norm function space; see Appendix A.l for the derivation.

---

CDE decoder for functional targets. To produce continuous functional outputs, we feed the stacked path (t) = [ [(H (t)l (t)ll... ....1 (t)] E R2h into a shallow neural CDE:
(s) (0) ( f (o((( (H()),
Sc)
(11)
where Jo
is a learnable vector field. Because the solution of (1I can be evaluated at any index 8, the decoder naturally accommodates misaligned or irregular target grids.
Assumption 5 (Decoder Lipschitz regularity). The decoder field fo is globally Lg LipLiischitz and bounded: [ S B
Theorem 3 (Well-Posedness and Lipschitz stability of the decoder). Under Assumptions
the
encoder output path H has finite 1 1-variation, and (TT) admits unique solution g E E[10, S R) with
|| 1l S S
-H
Hence, for each output index Ç. the decoder Dv H + (() is a bounded, continuous operator from Sc), R2hd), ), || /l1.var) to the Banach space (C([0, Sc], B), llso).
End-to-end Lipschitz bound and generalization. Composing the encoder Eg : X + H (continuous and cross attention) with the decoder D& : H + y yields the operator Te = "D Eg. Denote Smax max; Sç. Under the preceding assumptions, Te is globally Lipschitz with
L Lene C Lou Smax SupSmmax
Lenc = Leress Lsingle-
This single constant summarizes the theory behind FAME: model provably stable, invariant to sampling grids, and enough to represent any continuous operator functions. Moreover, the hypothesis class has Rademacher complexity guarantee for FoFR. Further details and auxiliary lemmas are provided Appendix A
supplying quantitative generalization
Experiments
To evaluate the proposed model, we benchmark FAME against wide range of state-of-the art methods for FoFR (see Section 2). For basis-expansion pipelines (38), we pair two orthonormal systems-B-splinee and Fourier functions-wiis four classical regressors: Ordinary Linear, Ridge, Lasso, and Elastic Net. Beyond these eight variants, we include functional principal component analysis (FPCA) (8), operator-vall valued kernel regression [14, Gaussian processes [32], and the functional neural network (FNN) (36] as stronger baselines. We evaluate FAME and the baselines on both synthetic datasets and several real-world datasets, using mean- mean-squared error (MSE) as the primary evaluation metric.
5.1 Datasets
Synthetic data. We generate functional data using Gaussian ppoccess trajectories because GP paths have finite 1-variation almost surely, hence X; E X. Specifically, we draw x(5)(1) (O, (t) for j I,...,d and t E [0, and define the response by Yi(() =
sin E (() are sampled on (possibly distinct) irregular grids (4,,)),,, and { We set = 3. m A r; = 20 and N. = 200 unless otherwise specified. particular, we evaluate FAME under seven controlled settings. Case 1 fixes the input and output grids at A, I' = 20, perfectly aligned across all samples. Case increases the shared resolution A, = I; 50, while preserving grid alignment. Case 3 keeps the output grid at T; = 20 but samples the input resolution independently as (10,20.35
(t) At,(t), with ex(t) iid. NN(0,0²) and noise level X 2 0. Input and output Ir
A (10, 20, 50) for each sample, thereby introducing variable input granularity. Throughout Cases 1-3 we vary the dataset size N, E {100, 200, 500} to disentangle the effects of resolution and sample size. Moving to stress tests, Case introduces feature heterogeneity by assigning each input function distinct RBF kernel width o, E (0.2, 0.3, 0.5). rather than the fixed value of 0.3 used in the other cases. Case probes noise robustness by adding independent Gaussian perturbations of noise level
A {0.1, 0.2, 0.3) to every output. Case 6 tests multivariate prediction by extending the target
7

---

Table I: Average test MSE for different methods in regression. Detailed results (mean + standard The best MSE for each case highlighted in bold.
deviation) are provided Appendix B
Case
Case 2
Case
Model
100
200
500
100
200
500
100
200
500
Linear
0.4720
0.3947
0.3123
0.4135
0.3412
0.2822
0.4958
0,4002
0.3260
Ridge
0.4264
0.3893
0.3117
0.3960
0.3393
0.2817
0.3869
0.3740
0.3222
B-spline
Lasso
0.4098
0.3830
0.3052
0.3856
0.3351
0.2766
0.4132
0.3584
0.3188
Elastic Net
0,4510
0.3650
0.2874
0.3725
0.3152
0.2583
0.3986
0.3618
0,3190
Linear
0,5002
0,4092
0.3224
0,4923
0.3636
0.2896
0,4762
0.3921
0.3547
Ridge
0.4255
0.3780
0.3149
0.3616
0.3343
0.2841
0.3755
0.3568
0.3328
Fourier
Lasso
0.3550
0.3493
0.3135
0.3361
0.3280
0.3001
0,3808
0.3560
0.3247
Elastic Net
0.3540
0.3325
0.2914
0.3167
0,3070
0.2720
0.3737
0.3496
0,3366
FPCA
0.3717
0.3554
0.3200
0.2890
0.2733
0.2624
0.3812
0.3563
0.3295
Kernel Method
0,2441
0.1728
0.1058
0.1741
0.0923
0.0700
0.2654
0.2445
0.1485
Gaussian Process
0.3405
0.2941
0.2036
0.3905
0,3917
0.2588
0.3031
0.2926
0,3945
FNN
0.3123
0,2083
0.1013
0.1941
0.1142
0.0811
0.3678
0.3571
0.1366
FAME w/o Bi-dir
0.1832
0.0812
0.0654
0.1530
0.0528
0.0355
0.1919
0.0813
0.0368
FAME w/o MoE
0.1870
0.0828
0.0663
0.1578
0.0538
0.0362
0.1972
0.0856
0.0374
FAME wfo Cross-atth
0.1902
0.0815
0.0668
0.1602
0.0544
0.0375
0.1997
0.0879
0.0381
FAME
0.1806
0.0783
0.0635
0.1532
0.0511
0.0342
0.1954
0.0796
0.0352
Table 2: Average test set MSE regression under simulation. Basis Expansion (best) shows the best result among the basis expansion methods presented in Table 1.
Model
case
5
case
case
case
0.1
0.2
0.3
5
10
Basis Expansion(best)
0.3610
0.3665
0.3944
0.4419
0.3669
0.4501
0.4705
0.5204
FPCA
0.3802
0.3879
0.3956
0.4570
0.3844
0.5284
0.5573
0.5737
Kernel Method
0.1928
0.1440
0.1919
0.2435
0.2022
0.4659
0.5236
0.5895
Gaussian Process
0.2302
0.3079
0.3356
0.3830
0.3434
0.4120
0.4498
0,4762
FNN
0.2102
0.1879
0.2250
0.3438
0.1744
0.4801
0.5164
0.5384
FAME
0.0798
0.0846
0.1076
0.1420
0.0824
0.2285
0.3330
0.3530
to the two-dimensional map [sin(E, (t). cos(E, xos(E,x(( (t)) Case 7 stresses scalability by increasing the number of input functions to d E (5, 10} while keeping all other settings unchanged. Finally, to isolate the impact of the hyperparameter K under high dimensionality and heterogeneity, Case 8 fixes the number of input functions at d 10, generates inputs with RBF kernels of widths 0 E (0.2, 0.3, 0.5}, and varies K (,8
Real-world data. We evaluate the same set of models on three public datasets: 1) Hawaii Ocean, which contains five hydrographic depth profiles-temppee temperature, salinity, oxygen, chloropigment, and density- among which different variables treated regression targets in turn, with the remaining serving as input functions: 2) Human3.6M a human motion capture dataset consisting of 3-D joint trajectories, where define three action-speciff regression tasks (Walking, Eating, and Sitting); and 3) E7T-small, a monthly deccricity-transooor dataset used to forecast oil temperature from transformer load curves. Full preprocessing details and task definitions provided in Appendix
5.2 Experimental Results
Table summarizes the performance of FAME and various baselines the synthetic datasets, clearly demonstrating that FAME consistently achieves the lowest test-set MSE. Other methods based on fixed basis expansions, FPCA. Gaussian processes, neural networks achieve inferior performance,

---

Table 3: Average test set MSE in regression different real-world datasets.
Hawaii ocean
Human3.6M
ETDataset
Model
Salinity
Temp
Walking
Eating
Sitting down
Oil Temp
Basis (xpansion(best)
0,0780
0.0014
0,0359
0.04841
0,0122
0.0365
FPCA
0.0865
0.0025
0.0373
0.0099
0.0121
0.0355
Kernel Method
0.0754
0.0025
0.0373
0.0099
0.0121
0.0355
Gaussian Process
0.0931
0.0022
0.0360
0.0075
0,0107
0.0380
FNN
0,0766
0.0020
0.0373
0.0099
0,0121
0.0355
FAME w/o Bi-dir
0.0751
0.0012
0.0327
0.0035
0.0071
0.0264
FAME w/o MoE
0.0759
0.0013
0.0332
0.0038
0.0075
0.0271
FAME w/o Cross-attn
0.0773
0.0014
0.0344
0.0044
0.0083
0,0286
FAME
0.0748
0.0012
0.0325
0.0034
0.0070
0.0262
Tompeeature
Salinity
2.0
1.00
6.8
0.8
Volue
0.6
0.5
0.25
0.00
0.4-
0.4
- 0.25
0.2
02
0.75
-1.00
0.30
0.25
0.50
0.75
0.00
0.2
0.50
0.00
0.25
0.50
1.30
0.00
0.75
1.00
Depth
Depth
Time
(a) Real data functional features
(b) Synthetic data functional feabures
Figure 2: Visualization of Functional Features.
largely due their limited capacity adapt nonlinear functional mappings, sensitivity irregular sampling points, or challenges in preserving continuous functional structures. In contrast, FAME's continuous attention mechanism, supported by adaptive MoE-driven dynamics and inter-dimensional cross attention, provides stable and expressive mapping that directly operates in continuous function spaces, which the ablation study in Table clearlly demonstrates. Table further evaluates model robustness across a variety of stress-test scenarios incluuiingiiin feature heterogeneity, noise levels, output structures, and input superior predictive performance under challenging conditions. Table reports results on real-worle datasets, confirming the practical applicability of our model. The performance advantage over strong baselines is reduced compared to the synthetic setting. Figure ([a) shows that the HAWAII OCEAN dataset occupies a smooth, wwrriiiabillity subspace dictated by ocean stratification, a structure that fixed-basis and kernel methods can already approximate competently. Figure contrasts this with synthetic trajectories drawn from independent Gaussian processes, whose broad, irregular function space contains pronounced local fluctuations. Under this more demanding regime, the continuous- attention encoder in FAME secures a substantially larger accuracy gain, underscoring its flexibility across disparate function-space complexities and corroborating the universal aiprosalllpprooxmmationn and sampling-invariance invariance guarantees established in Section 4.3
that FAME consistently maintains
As shown in Figure 3] the model's performance varies systematically with key parameters, illustrating clear relationships between predictive behavior and data characteristics. Specifically, in Case the observed test еттоr progressively decreases with increasing sample size (Fig. B()). in agreement with our theoretical generalisation result. Case illustrates clear improvement in model performance as the number of sampling points grows, with FAME achieving superior accuracy even at sparse resolution of sampling points (Fig. (b).. Case 3 reveals that while baseline methods experience substantial performance degradation under mixed input resolutions, FAME maintains stable and
(Fig.Baa))

---

3.0-
631
6443
100-
604
530
Scan Cod
(4)
by sample
sampling desalty
Tost MSE strung1
(4) MSE by
Figure 3: Parameter-sensitivity curves for FAME.
0.375
0.08
0.59
0.5
MSE
0.365
0.36
0.4 H
0.360
0.3589
0.355
0.3530
0.350
0.00
0.0
K
Figure 4: Sensitivity to K-tesst MSE (left axis) and H (right axis).
superior performance. This result highlights model's sampling invariance capability, endowed by the Young integral formulation. As the input becomes noisier or more high-dimensional (Cases and 7), all models exhibit increased error (Fig. (d)) but FAME remains within a practically useful error range. Taken together, these parameter sweeps confirm that FAME's empirical behaviour matches its theoretical guarantees, maintaining robust accuracy across diverse data regimes. To assess how the number of experts K affects performance, we plot the test MSE together with the normalized routing entropy H H/ log K in Figure 4 where H denotes the Shannon entropy of the gating distribution and the normalization by log K yields a [0, 1]-scaled quantity comparable across different K. The curves show that accuracy improves as K increases from to 3 and plateaus around K = 3~ 5, while H remains moderate moddeaat--indicatingg healthy, non-collapsed expert utilization. Beyond that, accuracy does not meaningfully improve and computation grows with K. Consequently, K = 3 and K 5 show stable accuracy and healthy expert utilization; since computation scales with K, we adopt = 3 as the default. More extensive visualizations and discussions are provided in Appendix [ which also details the experimental setup hyperparameters, and training schedules) and comprehensive analysis of computational efficiency (time and memory complexity).
Conclusion
We have presented FAME, fully end-to-end framework that learns FoFR mappings by coupling bidirectional neural CDEs with continuous attention mechanism, enriching the resulting latent dynamics through a mixture-of-expertt router, and fusing inter-functional information via multi-head cross attention before decoding with NCDE. This design yields resolution-agnostic. Lipschitz- stable, and universally expressive operator that consistently outperforms classical bases, RKHS models, and recent deep networks both synthetic and real benchmarks. A natural limitation of FAME is that when the underlying operator varies little across the function domain, simpler linear or low-rank models may offer a more attractive trade-off between statistical and computational complexity. In future work, we plan to extend the same functional attention principle beyond FoFR to prediction, where the continuity-aware
tasks such as functional classification and function-on- architecture of FAME is expected to provide similar advantages.
10

---

Acknowledgement
Yifei Gao and Chen Zhang would like to acknowledge the support of NSFC Grant 72271138 and Tsinghua-Nur NUS Joint Funding 20243080039.
References
[1] Ardizzone, L.. Kruse J.. Wirkert, S., Rahner D., Pellegrini, E. W., Klessen R. S., Maier-Hein L... Rother, C.. & Köthe, U. (2018) Analyzing inverse problems with invertible neural networks. arXiv preprint arXiv:1808.0444 04730
[2] Bahdanau, D., Cho, K., & Bengio. Y. (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.047.
[3] Bellot, A. & Van Der Schaar, M. (2021) Policy analysis using synthetic controls in continuous-time In International Conference on Machine Learning pages 759-768. PMLR.
[4] Bouche D.. Clausel, M., Roueff F,, & Buc F. (2021) Nonlinear functional output regression: A dictionary approach. In International Conference Artificial Intelligence and Statistics pages 235-243. PMLR.
Crambes, C., Kneip A., Sarda, P. (2009) Smoothing splines estimators for functional linear regression
[6] Fan, Y. James, G. M., & Radchenko P. (2015) Functional additive regression
[7] Febrero-Bande, M., Gonzälez-Manteiga, W., & Oviedo De La Fuente, M. (2019) Variable selection in functional additive regression models. Computational Statistics 34:469-487.
(2007) Methodology and convergence for functional linear regression Technometrics 63(3):396-409.
[8] Hall, P. & Horowitz
[9] Hullait, H., Leslie.. D. S., Pavlidis NN. G.. King, S. (2021) Robust function-on- -onfnnctionnnnnnnnno regression.
[10] Jonescu C., Papava. D., Olaru, & Sminchisescu, (2013) Human3, 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence 36(7):1325 -3399
[11] Jank, W. & Zhang, S. (2011) An automated and data-driven bidding strategy for online auctions. INFORMS Journal computing 23(2):238-25)
[12] Jia J. & Benson A. R. (2019) Neural jump stochastic differential equations. Advances in Neural Information Processing Systems 32.
[13] Jiang F., Cheng Q. Yin, G., & Shen H. (2020) Functional censored quantile regression. Journal of the American Statistical Association 1355599)93119444
[14] Kadri, H., Duflos E., Preux, Canu, S., Rakotomamonjy. A., Audiffren. (2016) Operator-valued kernels for learning from functional response data. Journal of Machine Learning Research 17(20): 1-54.
[15] Kidger P., Morrill, J., Foster J., & Lyons, T. (2020) Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems 33:6696-6707.
[16] Lambert A., Bouche D., Szabo Z., & Buc F. (2022) Functional output regression with infimal convolution: Exploring the huber and e-insensitive losses, In International Conference on Machine Learning pages 11844 11867. PMLR.
[17] Lan, T., Li, Z., Li, Z., Bai L., Li M., Tsung F., Ketter, W. Zhao R., & Zhang C. (2023) Mm-dag: Multi-task dag learning for multi-modal data-with application for traffic congestion analysis. In Proceedings the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining pages 1188-1199.
[18] Ledoux Probability in Banach Spaces: Isoperimetry and Processes: Springer.
Talagrand, M. (1991) Probability Banack Spaces: Isoperimetry and Processes.
[19] Lian H. (2007) Nonlinear functional models for functional responses in reproducing kernel hilbert spaces. Canadian Journal of Statistics 35(4):597-6066
[20] Liu H., You J., & Cao J. (2023. A dynamic interaction semiparametric conction-on-scalee model. Journal of the American Statistical Association 1186541):30-3377

---

[21] Liu Y. Hu, T.. Zhang H.. Wu, H., Wang S., Ma. L.., & Long M. (2023. itransformer: Inverted transformers effective for time series forecasting. arXiv preprint arXiv:2310.06625
[22] Luo, R. & Qi. X. (2017) Function-on -onettio-on-uuctionn linear regression by signal compression. Journal of the American Statistical Association 112(518):690-700
[23] Luo, R., Qi X., & Wang Y. (2016) Functional wavelet regression for lincar cunction-on-funcce models
[24] Müller, H.-G. & Yao F. (2008) Functional additive models. Journal of the American Statistical Association 103(484):5544-544.
[25] Mutis, M., Beyaztas, U., Karaman, , F., Lin Shang, H. (2025) On function-on-function linear quantile regression. Journal of Applied Statistics 52(4):814 840.
[26] Oh, Y., Lim, D., & Kim, S. (2024) Stable neural stochastic differential equations in analyzing irregular time series data, arXiv preprint arXiv:2402. 14989
[27] Park, J., Ahn, J., Jeon, Y. (2022) Sparse functional linear discriminant analysis. Biometrika 109(1): 209-226.
[28] Qi X. & Luo, R. (2019) Nonlinear function-on-ffunctin additive model with multiple predictor curves. Statistica Sinica 29(2)719-7739
[29] Ramsay, J. & Silverman, B. (2007) Applied Functional Data Analysis: Methods and Case Studies. Applied Functional Data Analysis: Methods and Case Studies: Springer.
[30] Rossi, F. & Conan-Gued B. (2006) Theoretical properties of projection based multilayer perceptrons with functional inputs. Neural Processing Letters 23( 10.55-00
[31] Rubanova, Y., Chen R. T., & Duvenaud, D. K. (2019) Latent ordinary differential equations for irreuulary--smplled time series. Advances in neural information processing systems 32.
[32] Shi, J. Q. & Choi, T. (2011) Gaussian process regression analysis for functional data. Gaussian process regression analysis for functional data: CRC press.
[33] Shi. Z., Fan, J.. Song L., Zhou, D.-X., & Suykens J. A. (2024) Nonlinear functional regression by functional deep neural network with kernel embedding. arXiv preprini 4rX1v:240..08890
[34] Stone, M. H. (1948) The generalized weierstrass approximation theorem. Mathematics Magazine 21(5): 237-254.
[35] Sun X., Du P., Wang X., & Ma P. (2018) Optimal penalized function-on funnctioonn regression under reproducing kernel hilbert space framework. Journal of the American Statistical Association 113(524): 1601-1611.
[36] Thind, B., Multani, K., & Cao. (2023) Deep learning with functional inputs. Journal of Computational and Graphical Statistics 32111171--
[37] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit J., Jones L., Gomez, A. N., Kaiser L., & Polosukhin T. (2017) Attention is all you need. Advances in neural information processing systems 30.
[38] Wang J.LL, Chiou J.-M., & Müller, H.-G. (2016) Functional data analysis. Annual Review of Statistics and its application 3(1):257-295.
[39] Wang Q.. Zheng. S., Farahat, Serita, S., Saeki, T., & Gupta. C. (2019) Multilayer perceptron for sparse functional data. In 2019 International joint conference on nearal networks (IJCN) pages 1-10. IEEE.
[40] Wang Q. Wang H., Gupta, C., Rao A. R., & Khorasgani, H. (2020) A non-linear function-on-funae model for regression with time series data. In 2020 IEEE International Conference on Big Data (Big Data) pages 232-239. IEEF.
[41] Wang S., Cao G., Shang Z., & Initiative A. D. N. (2023) Deep neural network classifier for multidimensional functional data. Scandinavian Journal of Statistics 1064)1166
[42] Wen, C., Wang X., & Zhang, A. (2023) E0 trend filtering. INFORMS Journal on Computing 35(6): 1491-1510.
[43] Xu K., Ba, J.. Kiros, R., Cho, K., Courville A., Salakhudinov R., Zemel, R., & Bengio, Y. (2015) Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning pages 2048 2057. PMLR.
12

---

[44] Yao, F, Müller H.-G., & Wang J.-L. (2005) Functional linear regression analysis for longitudinal data
[45] Yao, J., Mueller J., & Wang J.-L.. (2021) Deep learning for functional data analysis with adaptive basis layers. In International Conference on Machine Learning pages 11898-1 1998-11908. PMLR.
[46] Young, L. C. (1936) An inequality of the hölder type, connected with stieltjes integration
[47] Zhang Y. & Yan J. (2023) Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations
[48] Zhang. Z. Wang X., Kong L., & Zhu H. (2022) High-dimensional spatial quantile function-on -oniona- regression. Journal of the American Statisticul Association 117(539)11563-1158
[49] Zhao, Y., Ogden, T., & Reiss, T. (2012) Wavelet-based lasso in functional lincar regression. Journal of computational and graphical statistics 21(3):600-617.
[50] Zhou , H., Zhang. S., Peng. J., Zhang S., Li J., Xiong, H., & Zhang, W. (2021) Informer: Beyond efficient transformer for long sequence time-series forecasting. In The Fhirty-Fitth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference 35, pp. 11 11106-11115 15. AAAT Press.
13

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer:
[TODO]Yes
Justification: [TODO]The abstract and introduction present the proposed model and key innovations, which are fully supported by the methodological and experimental sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: Yes
14

---

Justification: [TODO]We discuss model limitations in both the experimental results and the conclusion (Section 5.2 and Section 6). highlighting that simpler tasks, lightweight baselines may offer better Sfficiency-perfomme tradeoffs.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [TODO]Yes
Justification: [TODOJAII theoretical assumptions and guarantees are explicitly stated in Section 4, with complete proofs provided in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: YeSDO
15

---

Justification:
[ToDo]Synthetic data generation is detailed in Section 5.1; real-world datasets are described in Appendix B.1; model configurations and hyperparameters are listed in Appendix B.2.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [TODO]No
Justification: [TODOJAML experiments use publicly available datasets, and every model configuration and training detail is listed in Section 5 and Appendix B.2. Any researcher familiar with Py PyTorch can reproduce our results from this information alone. If appropriate, we would be happy to share a GitHub link our implementation alongside the camera-ready version of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https //nips. http://www../..eeeeeoe//etttaaaa
for more details.
16

---

The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
reproducible, they
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [TODO]Ye
Justification: [TODO]Section (Experiments) and Appendix B.2 list the optimiser (Adam), learning-rate schedule, dropout, batch-size, epoch budget, and train/validationntest splits for both synthetic and real-world tasks.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [TODO]No
Justification: [TODO]Each experiment is deterministic once the random seed is fixed; running five independent seeds confirmed the ranking of all models, and we report the mean across seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
17

---

For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [T
Justification: Appenddpp B.2 specifies that all runs were executed on a laptop with AMD R9-7940HS CPU, 16 GB RAM, and a single NVIDIA RTX 3090 GPU;since both our model and datasets are relatively lightweight, runtime was not a limiting factor and thus not primary concern in evaluation.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [TODO]Yes
Justification: [TODO]The work uses only publicly available, properly licensed datasets and poses no foreseeable misuse risks.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [TODO]Yes
[TODO]Section 1 (Introduction) and Section 5 (Experiments) describe the positive societal impacts of the proposed method, such as its applicability to scientific and real-world domains. As a theoretical framework for function modeling, the approach itself poses no foreseeable negative societal impacts.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
18

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [TODOJNA
Justification:
[TODO]The released code and datasets pose no known high-risk misuse
scenarios.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer:
[TODO]Ye Yes
Justification: [TODOJAII external datasets and libraries are cited with their licences.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwpppswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- e-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [TODOJNA
19

---

[TODO]The work does not create new datasets; synthetic data are generated on-the-fly by scripts released with the code.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [TODOJNA
Justification:
[TODO]No human-subject or crowdsourcing data were collected.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [TODOJNA
Justification: [TODO]The study does not involve human participants and thus requires no IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [TODOJNA
20

---

Justification: [TODO]LLMs were not used in the core methodology or experiments, only for minor editing support.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non stanssand components.
Please refer to our LLM policy hattps://neurips.o 220p://p.....ooooeeeeeee///22/// for what should or should be described.
21

---

A Theoretical Guarantees of FAME
This appendix presents complete proofs for all theoretical claims made in the main text. We begin by deriving Lipschitz constants for each architectural block and compose them to obtain an end-to-end stability bound. We then establish sampling invariance and universal approximation, and conclude with quantitative generalisation result based Rademacher complexity.
Notation. Each input function X() defined on its own horizon [0, Tý]: we write
X' II C((O, R)),,
| m
mmax|xx(>1|1
Each output function ((s) lives [0, Sc]: correspondingly.
m y II S(, C=1
| max ||
For any (vector-valued) path X, || denotes the usual total variation and is taken component- wise as above. All integrals are understood in the Young sense [46], so they depend only on the underlying continuous functions, not on the choice of discretisation.
A.1 Module-wise Lipschitz constants
(i) Continuous attention. Under Assumption the bidirectional NCDE (forward and backward flows) is well posed for every input function. Theorem in the main text shows that the mapping
X + Z satisfies
WVv cL,(T)-(o)) LG(T, tt).
2 21100 S singte ||X
X I1lavars
Lsingle
(ii) Cross attention. Let soltmax( d......
Lemma (Soft-max contraction). The map Rd - Ad is li.p/ from (loo, || (1o0) to (41, 111).
Proof. The Jacobian of a is diag(o) oo its operator norm Loo +6 equals 1/Vdo. Integrating this bound along the line segment joining any two inputs yields the claim.
If the projection matrices satisfy [| || llop: [IW(P))opo 1lop. |Wolloo I Mmat for every head Lemma 2 implies, path-wise in 1-variation.
M M
| -H-H H| I Leross 1 2 2112-vrr
mar
Leross
dd
(iii) CDE decoder. If fo is globally Læ-Lipschitz and bounded, the Young- framework together with Grönwall's inequality yields, for every output function Ç.
110 I Ldee || H [|
Lass = Sc L
Writing Smax max; Sc. the end-to-end Lipschitz radius
L Lage Lecoss Lsingles
Like e Smox S Smax..
A.2 End-to-end stability and sampling invariance
Theorem (Global Lipschitz bound). For the composite operator To = Deo o Es.
[TT(X))
To(X)]l I L. |X -111-var.
22

---

Proof. Chain the inequalities of Section
A.1 along X x
Proposition (Sampling invariance). If two observation grids encode the same underlying functions, the operator Te returns identical outputs.
Proof. Young integrals depend only on the driving functions, not on the chosen partitions 46
A.3 Universal approximation
Lemma (Density separable kernels). Define
s- T = Ei- ui(l) vz(T) EEN, ua al1, {(1o,T)))) Ril)
(12)
The uniform closure of S equals the full kernel space:
S C(too. T713).
Proof. (i) Sub-algebra. S is closed under point-wise addition, scalar multiplication, and multiplica- tion, is a sub-algebra of ([to, T,
(ii) Constants. Choosing = 1 and = gives the constant kernel k(t, = E
(iii) Point separation. For two distinct points (t1, T1) & (22,
If 1 12, pick C( T;I) with u(t1) A u(t2) (by Urysohn's lemma) and set v = 1.
If t1 = 22 (hence 7I # T2). pick C[(o. Ty]) with v(ri) * v((2) and set = 1.
In both cases the resulting kernel belongs to S and takes different values the two points, S separates points of the compact Hausdorff space [to. T,I².
(iv) Stone-Weierstrass: Because S is sub-algebra that contains the constants and separates points, the real Kkone-Weierstrand theorem [34] yields = (((oo,,,1))
Theorem 5 (Density of the FAME hypothesis class). For any bounded continuous operator F X' + y and any & > 0 there exists a parameter set θ such that
T Fl <.
Proof. Neural CDEs are universal approximators on path space 15 Lemma 3 shows that cross- attention can approximate any continuous kernel, and the decoder NCDE universal curve generator. Universality is preserved under composition, the class {Te} is dense in C).
A.4
Generalisation bound
Theorem 6 (Rademacher complexity). For N i..dd. samples, the hypothesis class with radius L, satisfies Ry LL./VN for universal constant c.
Proof. Apply the Ledoux -aalagrad contraction principle 18 to Theorem
B Experimental details
B.1 Real-world Dataset
Hawaii ocean dataset. The Hawaii ocean dataset, part of the Hawaii Ocean Time-Series program, includes measurements of hydrographic, chemical, and biological characteristics at station north of Oahu, Hawaii, from January 1, 1999, December 31, 2018. The dataset consists of five functional variables: Temperature, Oxygen concentration, Potential density, Salinity, and Chloropigment, measured every 2 meters between and 200 meters below the sea surface. After data preprocessing. we retained 265 samples, with irregular sampling (A, = r, = 20) as the number of observation points for each function sample. For our analysis, Temperature and Salinity alternately serve as the target variable, with the remaining variables used as input predictors.
23

---

Human3.6M dataset. The Human3.6M, 6.6 dataset [10] consists of 11 subjects performing 15 different actions, with 3D joint coordinates provided for 32 body parts. Each action captured with high precision across different subjects, making the dataset suitable for studying human motion. In our approach, we enhance the dataset by performing data augmentation through random sampling of time series data from different subjects for the same action. Specifically, for each action, we sample 30 points, and each subject contributes 30 samples, ensuring a diverse set of data for training.
We have designed three specific regression tasks:
Walking:
For the Walking action, the input consists of the X and Y coordinates of the right knee, and the output is the corresponding Z coordinate. This task includes 180 samples from multiple subjects.
For the Eating / action, the input includes the XYZ coordinates of the right forearm and the XY coordinates of the right wrist, with the output being the 7. coordinate of the right wrist. This task also contains 180 samples.
Eating I:
Sitting down: For the Sitting down and Sitring down/ actions, the input consists of the XYZ coordinates of Spinel and Spine2, and the output is the XYZ coordinates of Spine3. This task includes 390 samples.
ETT-small dataset. The ETT-small dataset [50] contains data from two electricity transformer stations, including variables such as oil temperature and power load. Each sample is constructed from monthly data, comprising 30 data points, with each point representing a day's worth of measurements. For the experiments, we use a total of 48 samples, aiming to assess the performance of regular time sampling in this context.
B.2 Experimental Setup
All experiments were performed on workstation equipped with an AMD Ryzen 5950X CPU and an NVIDIA RTX 3090 GPU. Unless otherwise stated, every model is trained for 100 epochs with the Adam optimiser, an initial learning rate of 10-3, and a dropout rate of 0.2. Each dataset is randomly split into 80% training and 20% test instances; for the synthetic benchmark we repeat this split five times and report the average performance. For FAME, the neural controlled differential equation that constructs the continuous attention and the decoder CDE share an identical architecture. The input lifting network &e and the neural vector field fo are both implemented as two-layer MLPs with hidden widths 32 and 64, respectively, followed by Tanh activations. The same MLP configuration is used for the MoE expert fields and for the decoder vector field fa, ensuring architectural consistency throughout the model.
B.3 Results and discussion
While the main text already provides extensive quantitative evidence, additional results -sanninn Tables 4-6 and Figures 5-7-together with analyses of stress tests and computational efficiency, merit inclusion for completeness.
Supplementary Stress-Test Analysis Across all supplementary simulations, FAME preserves decisive lead over competing approaches. The heterogeneity configuration in Case 4 assigns distinct length-scales {0.2, 0.3, 0.5) the three input coordinates; ablating the mixture-of-expprrs router in this setting raises the test MSE from 0.0798 to 0.0842, demonstrating the value of specialist vector fields for channel-specific dynamics. As the input dimensionality rises (case 7), the test errors of all baselines increase markedly, highlighting how swiftly FoFR becomes more demanding in higher dimensions. Kernel methods, which performed strongly in the low-dimensional settings of Cases 1 1-3, deteriorate the most. Basis-expansion pipelines and FPCA regressors also degrade, since fixed bases struggle to model the richer cross- coordinate interactions that emerge with additional functional inputs. Although FAME is likewise affected by the added complexity, its relative performance drop considerably smaller, and its absolute accuracy remains within practically useful range. These findings indicate that the continuous attention encoder and mixture-of-expert vector fields endow FAME with a robustness that scales more gracefully than existing alternatives when moving to high-dimensional functional spaces.
24

---

Table 4: Runtime comparison (training inference).
Case I (Train Infer s)
Method
100
200
500
Basis Expansion
0.01 0.011
0.01 0.01
0.01
FPCA
0.01 / 0.01
0.01 0.01
0.01 / 0.01
Kemel Method
0.01 / 0.01
0.01 / 0.01
0.01 / 0.01
Gaussian Process
0.03 / 0.03
0.03 / 0.03
0.03 / 0.03
FNN
0.40 10 0.36
0.62 10/0.37
0.97 10 10/0.33 0.36
FAME
48.79 20 (epoch) / 1.02
57.46 20 1.00
79.53 20 1.03
Case 2 (Train Infer s)
Method
100
200
500
Basis Expansion
0.01 / 0.01
0.01 / 0,01
0.01 / 0.01
FPCA
0.01 / 0.01
0.01 0.001
0.01 0.01
Kernel Method
0.01 0.01
0.01 0.01
/ 0,01
Gaussian Pracess
0.03 / 0.03
0.03 0.03
0.03 / 0.03
FNN
1.02 10/0.81
1.98 x 00/0.82
3.32 x 10 10/0.84 0.84
FAME
134.42 x 2.60
151.47 x 2.66
207.20 x 2.92
Table 5: Peak memory usage (MB).
Method
Basis Expansion
FPCA
Kernel Method
Gaussian Process
FNN
FAME
Memory (MB)
0.30
0.25
2.91
12.70
3.82
15.36
Loss Dynamics Figure traces the optimisation trajectory of FAME The loss decreases mono- tonically and stabilises after roughly 20 epochs, demonstrating both rapid convergence and training stability for the continuous attention architecture.
Runtime and memory. Cases 1-2 jointly span the two principal axes of computational load-the number of imput functions and the number of sampling points per function-ss they are well suited for reporting cost. All timings in Table 4 were obtained on the same ordinary desktop. In brief, basis/FPCA/kerrel methods train fastest when grids are fixed and models are small, but their reliance on pre-specified bases kernels costlier but remains lightweight in absolute terms, and inference latencies are comparable to classical baselines. Even in the most demanding configuration samples), FAME trains in about 207.20 s per epoch; over 20 epochs this totals RJ 4144 (As 69 min) on this CPU, while post-training inference remains the seconds level (Table Memory usage varies little across tasks and is modest overall (Table5); the peak footprint for FAME is only 155 MB, and datasets are purely numeric and small disk, making deployment straightforward.
limit accuracy. By contrast, FAME is fully data-driven: training
considered (Case 2 with 500
Regression Visualisation Figure reports analogous comparison for the synthetic Case setting. Across the entire domain the predicted and true curves are almost indistinguishable, illustrating FAME's ability to recover fine-gran fine-grained function-to-functto
mappings even under irregular sampling. Figure presents model outputs on Sitting down sequence from the Human3.61 M.6 benchmark-2 representative real-world task. Basis-function baselines capture global trends, whereas conventional neural networks reproduce local variations; FAME accurately follows both scales and achieves the closest alignment with ground truth.
25

---

FAME Te Loss FAME Voi dation Loss
0.0
030
020
030
010
Epach
Figure 5: Training and validation loss epochs. The monotonic convergence illustrates stable optimisation behaviour.
Suoplo
Saria 6-
Samepio
16
02
18
Figure 6: Prediction curves for Case /.
26

---

115
100
Bust Bosi Expasoon
PNN
3.5
Tree
Kernel Merine
1.-
Expersion
INN
Couasian
PAME
Kamne Vernice
Dest Basia Expesion
MNN
True
Kemmni Merine
68
Figure 7: Prediction curves on the Sitting down task (Human3.6MM 6.6M.
27

---

Table 6: Average test MSE for different methods in regression, reported as mean d. standard deviation. The best MSE for each case is highlighted in bold.
0.3260 + 0.0261
0 + 0.0258
0.0255
0.3190 + 0.0055
0.3547 0.0284
0.5128 + 0.0266
0 1 0.0260
0.0269
+ 0.0064
0.1488 I 0.0119
0.3945 1: 0.0016
0.1366 0,0109
0.0299
0 # 0,0030
0,0000
=1 #.0022 MM22
500
0.3666
0.0668
00381
(00355
0,4002 4: 0.0320
0.3740 1 0.0299
0.3544 0,0287
0,36 - 0.0289
0.3921 + 0.0314
0.3568 + DA12K5
0.3560 1 0.285
0.499 0.0200
03563 + 00288
02445 1 0.1966
02926 1 09234
03571 0.286
0.065
1 0.0088
0.0000
+ 0.0064
Case
2,00
00813
00856
0,0879
(M079
+: 0.0397
0.3869 4 0.0310
0.4132 0.031
0.3986 :: 0:0819
0.4762 + +.0.88
0.3755 + 0.0300
0.3808 :: 0.030
0.3812 4 00000
0.2684 E 000012
0.3051 1. 00024
0.3678 1.00
0.1919 1.054
0. 1999 1 00158
0.0100
+ 9.0156
100
0.3737
0.1997
0.1954
0.2822 0.0226
0.2817 = 0.0225
0,2583 = 0.0207
0.0232
0.0227
= 0.0200
0.0218
0,00000
0.9866
0,0811 1.006
0.0006
= 00017
1IMIT7
500
0.2896
0.2641
0. 0.3001
0.27200
0-2624
0.0100
0.2588
0.0355
0.0362
0,0375
0,1144
0.0273
0.3393 = 0.0271
0.0268
0.3152 = 0.0052
0.3636 = 0.0291
0.5343 + 0.0267
0.3280 = 0.002
0,0246
0.2233 = 09219
0,9923 E 020044
0 29177-1 2 00013
0.1142 0.000
00025
= 00033
0 010200
0M23
Clsc 2
200
0.0628
0.0338
0,044
116611
0.1335 1 0.0831
0.3960 + 0.0017
05856 0.0008
03725 - 0.0098
0,923 = 0.094
033116 1 0.0289
0336 = 0.009
03167 0.005
02890 + 00011
0.744 11741-1 1 10030
1 00012
0.194 1991 1.055
0.025
01578 1578 1 0.003
0.112 = 01045
= 0:01
100
033 13995
A. 1532
0.3123 + 0.0250
0,3117 + 0,0249
1. 0.0244
0,2874 = 0,0230
0.522 0.255
1.3149 1 0.052
#
0.022
0/026
E 0.0085
09060
1,00
0.055
1 0,002
01004
+ 9.0056
500
0.3135
02914
0.200
0.10%
1058
02200
01011
00654
0006
00098
0. 0.307 0.0316
0.3893 + 0.311
0.0306
0. 0,3650 + 0.0292
0:4092 0.9327
0.3780 + 0.1302
0.279
0.0266
00288
100188
00233
1000
0,0000
1 0002
15155-50000 0000
04066
Case
200
0.3493
0.3325
0.3554
0.1728
0.2941
0.208
0.0812
0.0828
0.0815
0,0783
0.4720 0.0378
0-4264 = 0.0341
0.0328
0.4510 15 = 0.036
0.5002 0.0-00
014255 4 0.0300
0.3550 = 0.0284
0.3540 0.0283
0.3717 0.0297
24441 = 0.0195
0.3405 0.0272
0.3123 11 0.0250
0.1832 0.0150
0. 1870 = 0.0161
O. 0.01588
0,0152
100
0. 1.16
Elostic Not
Elastic Not
FAME weo Cross-ORE ontn
Lincar
Radge
Lawon
Linear
Ridge
Lasso
Kernel Method
Gausiian Process
ES
FAME MNdO MoE
FAME
PPCA
FNN
B splime
Medel
Fourier
28

---

0,04166
0,0459
0.0422
0,030
0,0431
1: 00180
case 8
Table 7: Average test set MSE in simulation (mean + standard deviation). The best MSE for cach case is highlighted in bold.
0.5204
0.5737
0.5895
0.4762
0.5384
0.3350
0.4703 1. 0.0376
0,5573 = 0.0044
0.5236 3: 0.0419
0.4498 # 000300
0.5164 = 0,0413
= 0.0266
10
0.3330
case
04501 1. 0,0000
0.2844 0,0023
0.4659 + 0.0573
0.4120 0.0300
0.4801 + 0.0384
: 00183
0,2285
0. 0.3669 1. 0.0284
0.3844 + 0.008
0.2022 + 0.0162
0.3434 + 0.0275
0.1744 + 0.0140
0.0824 0.0966
case 6
0.4419 1 0.0354
0.477 = 0.0366
0,2435 = 0.0195
0.3830 = 0.0306
0.3488 0.0275
= 0.0114
0.3
0.1420
0.3944 1: 00016
0.0166
0.1919 1 0015
0.2088
00180
= 09086
case
0.3956
0.3356
0,2250
0.1076
0.3665 1. 0.0293
03387 = 0.0310
0.1440 - 0.0115
0.3079 #: 0.0246
0.1879 = 0.0150
= 0.0068
0.1
0.8846
0.3610 1. 0,0229
0.0304
1:22811000000 0.0154
0.0184
0,0168
0.0064
case
0.1928
0,2302
0,2102
0,0798
Basis Expumsion (best)
Gaussian Process
Kernel Metbod
Model
FPCA
FNN
FAME
29

---

ETDatasct Oil Temp
0.0365 1: 0.0018
0.0355 + 0.0018
0.0355 + 0.0016
0.0380 + 0.0019
0.0355 = 1-0018
0.0264 + 0.0008
0.0271 + 0.0008
0.0286 0.00086
0.0262 0,00080
0,00020 020
Human3. 6M Sitting down
0.0122 0.0006
0.0121 + 0.0007
0.0121 + 0.0006
0.0107 0.0005
0.0121 0.0006
0.0071 Ef 0.0002
0.0075 + 0.00023
0,0083 0.00025
1.000000000
100000000000
0,0070
Table 8: Real- world datasets: average test, MSE (#: standard deviation).
Human3 6M Eating
0.0484 1: 0.0024
0.0099 +: 0.0005
0.0099 + 0.0005
= 0.0004
0.0099 1,00991 + 0.0000 00005
0.0033 = = 10010
0.00.3 00.38 = 0.00011
13
0,004 = #0,0001
1000000
0.0075 0.0073 =
0,0033
Hawaii Occan Temp Human3 6M Walking
0.0359 0.0018
0.0373 +: 0.0019
0.0373 + 0.0019
0.0360 (0360-+0 = 0,0018
0.0373 E 0.0019
0.0327 00227-444 + + 100010
0.0332 00322-4 + +-000010
0.0344 10344-14 #: 10000
101
= 100000
0.0325 00325
0.0014 1. 0.0001
0.0025 +: 0,0001
0.0025 + 0,0000
0.0022 + 100000
0,0020 + 1.0000000
0.001 24 1 00M00
0.001 30 + 0,00000
0.00145 + S.00000
1,00000 000044
0,0012
Hawaii Occan Salinity
0.0780 0.0039
0.0865 + 0.0043
0.0754 0,0038
0,0931 0,0047
0.0766 0,0038
0.0751 + 0,0023
0.0759 + 0,0023
0.0773 0.0024
0.0748 0.0022
FAME w/o cross-attention
FAME w/o bidir attention
Basis Expansion (best)
Gaussian Process
FAME w/o MoE
Kemel Method
Model
FPCA
FAME
FNN
30


Reviews:
[{"review_id": "Ea5lP6tDPG", "reviewer": "Reviewer_Xe76", "summary": "This paper proposes a method for solving function-on-function regression tasks. The method entitled FAME does this through learning a mixture of experts and has full attention, employing bidirectional NCDE and continuous attention. This approach ensures continuity, sampling invariance, and Lipschitz stability. This method is very flexible, working for input and output data with potentially distinct irregular grids. This paper has very strong quantitative performance across many datasets including both synthetic and real world data.", "questions": "- How do you decide on the number of experts $K$? Is this assigned a priori depending on how complex you think the data will be? How does changing $K$ affect your results?\n- What are the hyperparameters (e.g. $K$, $d$) used for different real data tests and how did they get selected? \n- Why is the cross attention necessary when there is continuous attention?", "limitations": "A limitation that the authors include is that simpler methods may be preferred when the underlying operator varies little or is simpler.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "(+) This paper is really clear and well written.\n\n(+) There is substantial evaluation across various challenging input and output sampling schemes such as irregular grid, different sampled points, and noisy observations. There is also strong quantitative performance on real-world datasets against various baselines.\n\n(-) Limited discussion on how hyperparameters (e.g. $K$ and $d$) are chosen.\n\n(-) Additional ablation studies would be helpful to understand which components of the method are the most impactful. Why is the cross attention necessary when there is continuous attention?", "was_revised": true, "final_justification": "This paper is novel and proposes an interesting solution to a very challenging problem of FOFR. Despite the fact that the paper could use a little more detail in terms of limitations, I believe it should be accepted due to it's novelty and strong performance on real data.", "rebuttal": "We thank the reviewer for the helpful suggestions, which will make our work stronger. We have carefully revised the paper to address all comments to the best of our ability. Below we respond point‑by‑point. Due to the submission format and character limitations of the rebuttal, we have focused here on presenting only the newly added or key sections; the formatting and full content will be adjusted as needed for the final manuscript.\n\n+ How do you decide on the number of experts K? Is this assigned a priori depending on how complex you think the data will be? How does changing K affect results?\n\n  We agree this needs clearer exposition. In a few cases we initially picked $K$ based on the apparent heterogeneity and complexity of the data; however, this choice was preceded by sensitivity checks, and we then selected a value whose performance was stable on a held‑out validation split. We did not present these sensitivity results clearly—sorry for the confusion. In the revision, we will explicitly report this process and add a new synthetic benchmark (**Case‑8**) to isolate the impact of $K$ under high dimensionality and heterogeneity: the number of input functions is $d=10$; the generating functions use RBF kernels with widths $\\sigma_j\\in\\{0.2,0.3,0.5\\}$; and we evaluate $K\\in\\{1,2,3,5,8\\}$. We will report MSE and normalized routing entropy $\\tilde H = H / \\log K$.  The table below presents our experimental results; the final version will also include corresponding visualizations. In our study, $K=3$ and $K=5$ show stable accuracy and healthy expert utilization, while compute cost grows with $K$, so we adopt $K=3$ as the default.\n\n Case‑8: Sensitivity to K \n\n  |      K      |       1       |       2       |       3       |       5       |       8       |\n  | :---------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: |\n  |     MSE     | 0.3753±0.0353 | 0.3589±0.0256 | 0.3530±0.0180 | 0.3518±0.0219 | 0.3601±0.0230 |\n  | $\\tilde H $ |     0.00      |     0.36      |     0.59      |     0.68      |     0.53      |\n\n+ What are the hyperparameters (e.g. , K, d) used for different real data tests and how did they get selected?\n\n  We denote by d the number of input functions, which equals the number of features in each dataset (see Appendix B.1 for detailed descriptions of d on Hawaii Ocean, Human3.6M and ETT‑small. In our original submission we inadvertently omitted the ETT‑small feature count—it should be d = 6, which will be corrected in the revised version. ).  Across our three real‑data benchmarks, d ranges from 2 to 6; based on the sensitivity analysis for K (Question 1), we set K = 3 in all real‑data experiments, since this choice consistently yielded stable accuracy and healthy expert utilization without excessive computational cost. All other architectural and training hyperparameters (e.g. hidden sizes, learning rate, dropout) are described in Appendix B.2. We emphasize that our goal was to demonstrate a general deep‑learning FoFR framework using a relatively simple, unified setup; as a fully extensible architecture, FAME can easily be combined with more complex deep models to suit different datasets.  \n\n+ Why is the cross attention necessary when there is continuous attention?\n\n  From a theoretical perspective, continuous attention operates **within each function** over time, performing a continuous, kernelized weighting of its own trajectory to capture **local–global intra‑function structure** (cf. Eqs. (3)–(5)). Cross‑attention, in contrast, acts **across functions at each time t by aggregating the concurrent states of the *other* functions, thereby modeling inter‑functional dependencies** explicitly (cf. Eqs. (7)–(10) and Sec. 4.3). Under Assumption 4, cross‑attention enjoys a global Lipschitz bound $L_{\\text{cross}}$ and, when combined with continuous attention and the decoder, yields an end‑to‑end Lipschitz, sampling‑invariant mapping (Theorem 4), so removing cross‑attention weakens the model’s ability to represent cross‑channel couplings.\n\n  From an experimental perspective, and in response to the reviewers’ comments, we will add three ablation variants on both generic (Cases 1–2) and real‑data tasks—(1) without bidirectional attention, (2) without MoE (single expert), and (3) without cross‑attention—and report the resulting MSEs side by side to quantify each component’s contribution. The results will be presented in the table below. \n\n  | Model               |    Case 1 (100)    |    Case 1 (200)     |    Case 1 (500)    |   Case 2  (100)    |    Case 2 (200)    | Case 2      (500)  |\n  | :------------------ | :----------------: | :-----------------: | :----------------: | :----------------: | :----------------: | :----------------: |\n  | FAME w/o Bi‑dir     |   0.1832± 0.0150   |   0.0812± 0.0066    |   0.0654± 0.0055   | **0.1530± 0.0125** |   0.0528± 0.0023   |   0.0355± 0.0016   |\n  | FAME w/o MoE        |   0.1870± 0.0161   |   0.0828± 0.0072    |   0.0663± 0.0062   |   0.1578± 0.0132   |   0.0538± 0.0023   |   0.0362± 0.0017   |\n  | FAME w/o Cross‑attn |   0.1902± 0.0158   |   0.0815± 0.0078    |   0.0668± 0.0064   |   0.1602± 0.0143   |   0.0544± 0.0026   |   0.0375± 0.0018   |\n  | **FAME**            | **0.1806± 0.0152** | **0.0783±  0.0064** | **0.0635± 0.0056** |   0.1532± 0.0120   | **0.0511± 0.0023** | **0.0342± 0.0017** |\n\n  | Model                    | Hawaii Ocean Salinity |    Hawaii Ocean Temp |   Human3.6M Walking |     Human3.6M Eating | Human3.6M Sitting down |   ETDataset Oil Temp |\n  | ------------------------ | --------------------: | -------------------: | ------------------: | -------------------: | ---------------------: | -------------------: |\n  | Basis Expansion (best)   |       0.0780 ± 0.0039 |      0.0014 ± 0.0001 |     0.0359 ± 0.0018 |      0.0484 ± 0.0024 |        0.0122 ± 0.0006 |      0.0365 ± 0.0018 |\n  | FPCA                     |       0.0865 ± 0.0043 |      0.0025 ± 0.0001 |     0.0373 ± 0.0019 |      0.0099 ± 0.0005 |        0.0121 ± 0.0007 |      0.0355 ± 0.0018 |\n  | Kernel Method            |       0.0754 ± 0.0038 |      0.0025 ± 0.0001 |     0.0373 ± 0.0019 |      0.0099 ± 0.0005 |        0.0121 ± 0.0006 |      0.0355 ± 0.0016 |\n  | Gaussian Process         |       0.0931 ± 0.0047 |      0.0022 ± 0.0001 |     0.0360 ± 0.0018 |      0.0075 ± 0.0004 |        0.0107 ± 0.0005 |      0.0380 ± 0.0019 |\n  | FNN                      |       0.0766 ± 0.0038 |      0.0020 ± 0.0001 |     0.0373 ± 0.0019 |      0.0099 ± 0.0005 |        0.0121 ± 0.0006 |      0.0355 ± 0.0018 |\n  | FAME w/o bidir attention |       0.0751 ± 0.0023 |    0.00124 ± 0.00004 |     0.0327 ± 0.0010 |     0.0035 ± 0.00010 |        0.0071 ± 0.0002 |      0.0264 ± 0.0008 |\n  | FAME w/o MoE             |       0.0759 ± 0.0023 |    0.00130 ± 0.00004 |     0.0332 ± 0.0010 |     0.0038 ± 0.00011 |       0.0075 ± 0.00023 |      0.0271 ± 0.0008 |\n  | FAME w/o cross‑attention |       0.0773 ± 0.0024 |    0.00145 ± 0.00005 |     0.0344 ± 0.0011 |     0.0044 ± 0.00013 |       0.0083 ± 0.00025 |     0.0286 ± 0.00086 |\n  | **FAME**                 |   **0.0748 ± 0.0022** | **0.0012 ± 0.00004** | **0.0325 ± 0.0010** | **0.0034 ± 0.00010** |   **0.0070 ± 0.00020** | **0.0262 ± 0.00080** |\n\n\n  As the table shows, compared to the full FAME model, removing cross‑attention degrades performance, highlighting its role in capturing inter‑functional dependencies.\n\nFinally, we again thank you for your careful reading and hope you will enjoy the revised manuscript."}, {"review_id": "dmiflz2xaC", "reviewer": "Reviewer_pq9z", "summary": "The paper introduces FAME, a continuous‑time architecture for function‑on‑function regression that couples bidirectional Neural Controlled Differential‑Equation encoders with a per‑function mixture‑of‑experts router and a cross‑attention fusion block. This design preserves the ability to query predictions at arbitrary points on irregular input and output grids while allowing each input function to adapt its vector field to idiosyncratic scale, frequency, and noise characteristics. A theoretical analysis establishes existence, uniqueness and global Lipschitz stability for every differential block, yields an overall Lipschitz constant for the end‑to‑end operator, and converts that constant into a Rademacher‑complexity generalisation bound. Experiments on seven synthetic regimes built from Gaussian‑process trajectories and on three real‑world datasets show lower test‑set MSE than classical FDA baselines, Gaussian processes, and recent deep‑learning competitors.", "questions": "See Weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n+ The paper’s main conceptual novelty lies in replacing discrete self‑attention with an NCDE‑based continuous attention mechanism, thereby treating both inputs and outputs as genuine functions rather than large vectors. The mixture‑of‑experts router is carefully scoped, one set of weights per function instead of per time step, so it captures inter‑function heterogeneity without incurring prohibitive computational cost. \n\n+ The theoretical component is complete for work: all assumptions are stated explicitly, constants are propagated through the proofs and a non‑trivial Rademacher bound is obtained. On the empirical side, the study is broad, covering diverse sampling patterns, noise regimes, and application domains; the model wins almost every column of the main tables and converges reliably in the training curves shown in the appendix. \n\n+ The paper is well organised; implementation details, hyperparameters, and hardware are documented in the supplementary material, and the authors commit to releasing code and checkpoints upon acceptance.\n\n\nWeaknesses\n\n- The comparison set mainly covers classical FDA techniques. Apart from a functional‐neural-network baseline, every competitor is either a fixed-basis regression, FPCA, a Gaussian-process regressor or an operator-valued kernel; no recent continuous-time deep architectures such as transformer-style sequence models or neural operators are evaluated, even though the related-work section cites them. Because these models are designed to exploit long-range dependencies and irregular grids, omitting them leaves open whether FAME’s margin would persist against the current state of the art.\n\n- For synthetic data, the authors average over only five random train/test splits and quote the mean MSE without standard deviations; for all real-world tasks, the results come from a single split and single seed, with no confidence intervals or significance tests. Without variability measures, it is hard to judge whether the observed gains are robust.\n\n- The supplementary material removes the mixture-of-experts router in one heterogeneity test, but there is no experiment that isolates the benefit of bidirectionality, the cross-attention module, or the continuous-attention block itself across the full benchmark suite. As a result the individual contribution of each architectural component remains unclear.\n\n- Although the theory rests only on deterministic Lipschitz and boundedness conditions, these are assumed to hold globally for every expert vector field and decoder. In real high-dimensional applications global Lipschitz constants can be large or undefined; the paper does not explore how sensitive the Lipschitz stability or the Rademacher bound is to mild violations of these assumptions.", "was_revised": true, "final_justification": "All my concerns are resolved in the rebuttal.", "rebuttal": "We thank the reviewer for the helpful suggestions, which will make our work stronger. We have carefully revised the paper to address all comments to the best of our ability. Below we respond point‑by‑point. Due to the submission format and character limitations of the rebuttal, we have focused here on presenting only the newly added or key sections; the formatting and full content will be adjusted as needed for the final manuscript.\n\n+ **Question  (1)**: Functional‑on‑Functional Regression (FoFR)—where both inputs and outputs are functions—is a central yet challenging FDA problem that arises in practice, e.g., mapping upstream process signals to thickness/roughness curves in manufacturing or force/torque trajectories to end‑effector motion in robotics. Unlike state‑space time‑series that focus on a single evolving state, here **each whole function is one sample**. In real industrial settings, **each function is often sampled irregularly and sparsely**, and **different functions are not aligned** in their sampling grids. Many deep models implicitly assume fixed, aligned grids or pre‑interpolated sequences, which limits their applicability; consequently, practice often relies on basis‑expansion or kernel methods. Our goal is a **fully data‑driven** FoFR framework that preserves the **global structure** emphasized by classical methods while adding **local dynamics modeling**, with both **theoretical guarantees** and **empirical effectiveness**. Different methods’ characteristics on FoFR tasks are summarized in the table below.\n\n  **Comparison of FAME and Existing Methods on FoFR Tasks**\n\n  | Property / Method                            | LSTM | Transformer | DeepONet | FNO  | Basis Expansion | Kernel methods  | FAME |\n  | :------------------------------------------- | :--- | :---------- | :------- | :--- | :-------------- | :-------------- | :--- |\n  | **Discretization invariance**                | ✗    | ✗           | ✗        | ✓    | ~               | ✓               | ✓    |\n  | **Output is a function**                     | ✗    | ✗           | ✓        | ✓    | ✓               | ✓               | ✓    |\n  | **Query output at any point**                | ✗    | ✗           | ✓        | ~    | ✓               | ✓               | ✓    |\n  | **Take input at any point**                  | ~    | ~           | ✗        | ✗    | ~               | ✓               | ✓    |\n  | **Universal approximation (operator level)** | ✗    | ✗           | ✓        | ✓    | ~               | ✓               | ✓    |\n  | **Cross‑resolution train/test**              | ✗    | ✗           | ✓        | ✓    | ✓               | ✓               | ✓    |\n  | **Data‑driven**                              | ✓    | ✓           | ✓        | ✓    | ✗ (fixed basis) | ✗(fixed kernel) | ✓    |\n\n  **Discretization invariance** means that the model’s output remains essentially unchanged when the same underlying function is sampled on different grids. **Universal approximation** refers to the model class being able to approximate general function‑to‑function maps. **Cross‑resolution train/test** indicates that the model can be trained on one sampling density and evaluated on another without retraining or redesign. The symbol “~” denotes partial support via specific preprocessing or variants (e.g., Transformer + Δt), rather than a native property.\n\n  Respecting your suggestion, we will introduce a new Case under the default settings ( $ d = 3, \\quad m = 1, \\quad \\Lambda_i = \\Gamma_i = 20, \\quad N_s = 200, \\quad \\sigma_j = 0.3 $) described in Section 5.1 to compare FAME against LSTM, Transformer, DeepONet, and FNO on  both regular‑ and irregular‑sampling FoFR under a unified protocol. To accommodate the baselines, all functions within each setting will share fixed, equal‑length sampling indices.  The new experimental results are shown in the table below.\n\n  **New Case Results**\n\n  | Method      | Regular‑grid FoFR  (MSE ) | Irregular‑grid FoFR (MSE ) |\n  | ----------- | :-----------------------: | :------------------------: |\n  | LSTM        |       0.1607±0.0285       |       0.2180±0.0302        |\n  | Transformer |       0.0889±0.0202       |       0.0940±0.0232        |\n  | DeepONet    |       0.1260±0.0076       |       0.1791±0.0168        |\n  | FNO         |       0.0797±0.0120       |       0.0944±0.0141        |\n  | **FAME**    |    **0.0775±  0.0062**    |    **0.0783±  0.0064**     |\n\n  From the table, FAME remains stable when switching between regular and irregular sampling and stays competitive against other methods. Thus, in both adaptability to diverse scenarios and actual performance, our model shows clear advantages on FoFR.\n\n+ **Question  (2)**: We fully agree that reporting variability is essential to demonstrate the robustness of our results. In the revision, we will evaluate all methods over 5 random train/test splits × 2 seeds for both synthetic and real‑world tasks and present mean ± std in every table. (Due to time constraints, these represent the experiments we have currently completed; we can increase the number of runs in a future version if needed.)\n\n  | Model               |    Case 1 (100)    |    Case 1 (200)     |    Case 1 (500)    |   Case 2  (100)    |    Case 2 (200)    | Case 2      (500)  |\n  | :------------------ | :----------------: | :-----------------: | :----------------: | :----------------: | :----------------: | :----------------: |\n  | FAME w/o Bi‑dir     |   0.1832± 0.0150   |   0.0812± 0.0066    |   0.0654± 0.0055   | **0.1530± 0.0125** |   0.0528± 0.0023   |   0.0355± 0.0016   |\n  | FAME w/o MoE        |   0.1870± 0.0161   |   0.0828± 0.0072    |   0.0663± 0.0062   |   0.1578± 0.0132   |   0.0538± 0.0023   |   0.0362± 0.0017   |\n  | FAME w/o Cross‑attn |   0.1902± 0.0158   |   0.0815± 0.0078    |   0.0668± 0.0064   |   0.1602± 0.0143   |   0.0544± 0.0026   |   0.0375± 0.0018   |\n  | **FAME**            | **0.1806± 0.0152** | **0.0783±  0.0064** | **0.0635± 0.0056** |   0.1532± 0.0120   | **0.0511± 0.0023** | **0.0342± 0.0017** |\n\n+ **Question  (3)**: We agree and appreciate this point. In response to the reviewers’ comments, we will add three ablation variants on both generic (Cases 1–2) and real‑data tasks—(1) without bidirectional attention, (2) without MoE (single expert), and (3) without cross‑attention—and report the resulting MSEs side by side to quantify each component’s contribution. The results are shown in the tables provided in our response to Question 2 (Due to space constraints, we regret that the full experimental results on real datasets cannot be shown here; please see our detailed response to Reviewer 1 for the complete results). Note that we do not ablate the continuous‑attention block: because FoFR maps **continuous functions to continuous functions**, continuous attention is the minimal mechanism that preserves sampling invariance and supports **function modeling**; removing it would reduce the model to a discrete, grid‑tied surrogate. For further rationale, please see our response to Question 1.   \n\n+ **Question  (4)**: We agree with the concern and will clarify our assumptions and scope. In the main text we stated *global* Lipschitz/boundedness for brevity, but our results can extend to the standard local‑Lipschitz + linear‑growth setting on a data‑dependent compact set $\\mathcal{K}$ that contains the trajectories visited during training/inference. Under this formulation, the NCDE vector fields, cross‑attention map, and decoder each admit local constants $L_{\\mathcal{K}}$, yielding the same existence–uniqueness and Grönwall‑type stability bounds with $L_{\\mathcal{K}}$ in place of global constants. The end‑to‑end bound then follows by composition of these local constants, and the Rademacher‑style generalization statement becomes explicitly data‑dependent (i.e., it involves $L_{\\mathcal{K}}$ and the radius of $\\mathcal{K}$ rather than global quantities). We will add a formal Remark and update the Appendix to state all results under local Lipschitz and linear growth.\n\n  High‑dimensional function‑to‑function regression is relatively rare in real applications and highly challenging. Consequently, global constants are often impractical—they may be large or undefined outside $\\mathcal{K}$, a risk common to many modern models rather than one specific to ours. What matters in practice is that learned trajectories remain inside a bounded region. This is encouraged in our implementation by input normalization, bounded activations (e.g., $\\tanh$ in the vector fields/decoder), softmax with controlled temperature in attention (a Lipschitz map on logits), and optional operator‑norm controls for linear layers (spectral normalization or light‑weight clipping). These standard safeguards keep the effective (local) Lipschitz constants moderate, stabilize the solver, and prevent runaway growth—the regime where our theoretical guarantees apply. We will document these constraints and training guidelines in the revision and explicitly rephrase theorems and assumptions to emphasize local (data‑domain) regularity rather than unrealistic global bounds.\n\n\nOnce again, thank you for your suggestions and careful review; we hope these additional explanations help you better understand our work and appreciate it."}, {"review_id": "6PAWuBGiJE", "reviewer": "Reviewer_EVtp", "summary": "The paper introduces FAME, a novel framework for function-on-function regression (FoFR) that combines bidirectional neural controlled differential equations (NCDEs) with a mixture-of-experts (MoE) architecture and continuous attention mechanisms. The proposed model aims to address challenges such as intra-functional continuity, inter-functional interactions, and feature heterogeneity in functional data. The authors demonstrate FAME's superiority over existing methods through extensive experiments on synthetic and real-world datasets, showcasing its robustness to irregular sampling and feature variability.", "questions": "1. Provide in-depth analysis of the introduced MoE and attention mechanisms.\n2. Analysis on the computational cost of FAME.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Originality: The contribution of this paper lies in introducing MoE and attention mechanisms into NCDEs. This combination is possibly a novel attempt, but its components are all existing techniques.\n\nClarity: The paper is clearly written. The main pipeline is easy to understand. The formulas used seem correct (I didn’t check the lemmas, Theorems, etc. in the paper). \n\nQuality: The paper supports its claims through theoretical analysis and experiments. The combination of NCDEs, MoE, and attention mechanisms is well-justified and effectively addresses the challenges of FoFR.\n\nSignificance: This study solves an important problem in functional data analysis and provides a feasible solution that is superior to existing methods and has potential real-world application value.\n\n## Weaknesses\n\n1. The proposed FAME appears to be a simple combination of NCDEs, MoE and attention mechanisms without improvements in details.\n\n2. The paper lacks in-depth analysis of the introduced MoE and attention mechanisms. How was the result of expert selection in MoE? Was the expert selection sparse, or did it have some regular patterns? Also, what did the attention maps reveal? Which factors (e.g., function similarity, temporal dependency) would lead to high attention scores between functions?\n\n3. The paper did not provide a detailed analysis of the computational cost of FAME, especially compared to simpler baselines. The overhead of MoE routing and attention mechanisms could be prohibitive for large-scale applications.", "was_revised": true, "final_justification": "The rebuttal is strong, detailed, and well-structured, addressing each concern with clear explanations, quantitative evidence, and theoretical backing. The articulation of contributions beyond “stacked” components, inclusion of ablation and sensitivity analyses, and solid complexity comparisons all add credibility. Therefore, I increase my ranking.", "rebuttal": "We thank the reviewer for the helpful suggestions, which will make our work stronger. We have carefully revised the paper to address all comments to the best of our ability. Below we respond point‑by‑point. Due to the submission format and character limitations of the rebuttal, we have focused here on presenting only the newly added or key sections; the formatting and full content will be adjusted as needed for the final manuscript.\n\n+ **Weaknesses 1:**　FoFR is an important problem in practice—yet it remains challenging. We acknowledge that FAME is built from established components (NCDEs, MoE, attention). Our aim, however, was not to refine a particular prior technique nor to “stack” modules for incremental gains, but to design a purpose‑built, fully data‑driven FoFR framework that addresses the core challenges( functions as samples, irregular/misaligned sampling, and continuous outputs ), while preserving global structure and modeling local dynamics. Specifically, our contributions are:  (1) a sampling‑invariant continuous attention mechanism that forms continuous Q/K/V via bi‑directional NCDEs，neither this specific approach nor the underlying problem has been proposed in the literature;  (2)  a shared router with direction‑specific experts so that vector‑field experts specialize to cross‑function heterogeneity without parameter bloat;  (3) cross‑channel attention to aggregate inter‑functional dependencies at each time;  (4) theoretical support—end‑to‑end Lipschitz stability, sampling invariance, and operator‑level expressivity—grounding the design beyond heuristics;  (5) task‑specific design choices for FoFR: the whole function is the sample (motivating bidirectionality), functions differ in shape/amplitude (handled by MoE vector fields), and the output must be a continuous function (realized by an NCDE decoding head).  Due to space limitations, we cannot present complete comparison with existing methods here. If interested, please refer to our response to **Reviewer 2**, where we provide detailed theoretical and experimental comparisons between our method and contemporary Transformer‑based and neural operator approaches.\n\n  In the revised manuscript we will add targeted ablations and hyperparameter analyses to demonstrate the indispensable contribution of each component, substantiating that FAME is not a simple juxtaposition but a principled architecture tailored to FoFR. The detailed results and analyses are reported in our response to **Weaknesses  2**; \n\n+ **Weaknesses  2（Question 1）:**　We agree that our analysis of the MoE needs clearer exposition. Beyond reporting accuracy, we will quantify routing sparsity and patterns using normalized routing entropy $\\tilde H = H/\\log K$ . To isolate how $K$ affects both performance and routing behavior under high dimensionality and heterogeneity, we add **Case‑8** ($d=10$, RBF widths $\\sigma_j\\in\\{0.2,0.3,0.5\\}$) and sweep $K\\in\\{1,2,3,5,8\\}$. The table below presents our experimental results; the final version will also include the corresponding visualizations. In our study, $K=3$ and $K=5$ achieve stable MSE with mid‑range $\\tilde H$ (indicating structured, non‑collapsed expert use), while compute cost grows with $K$, so we adopt **$K=3$** as the default.\n\n**Case‑8: Sensitivity to K** \n\n  |      K      |       1       |       2       |       3       |       5       |       8       |\n  | :---------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: |\n  |     MSE     | 0.3753±0.0353 | 0.3589±0.0256 | 0.3530±0.0180 | 0.3518±0.0219 | 0.3601±0.0230 |\n  | $\\tilde H $ |     0.00      |     0.36      |     0.59      |     0.68      |     0.53      |\n\n  Cross‑attention lets each input function, at each time point, aggregate information from the other functions to capture inter‑functional dependencies, complementing continuous (intra‑function) attention. In the revision, we will report time‑averaged cross‑attention maps and relate attention to function similarity. On Case‑8 (RBF widths $\\sigma_j\\in\\{0.2,0.3,0.5\\}$), pairs of functions sharing the same $\\sigma_j$ show higher and more similar cross‑attention, forming clusters aligned with the $\\sigma_j$ groups. Due to current formatting constraints, we cannot include the figures in this rebuttal; the attention heatmaps and group‑wise summaries will be added in the next revision. To further verify the effectiveness of the attention module, we have also included ablation experiments; because the tables do not fit the rebuttal’s character/format limits, please refer to our response to Reviewer 1 for the complete results.\n\n  | Model               |    Case 1 (100)    |    Case 1 (200)     |    Case 1 (500)    |   Case 2  (100)    |    Case 2 (200)    | Case 2      (500)  |\n  | :------------------ | :----------------: | :-----------------: | :----------------: | :----------------: | :----------------: | :----------------: |\n  | FAME w/o Bi‑dir     |   0.1832± 0.0150   |   0.0812± 0.0066    |   0.0654± 0.0055   | **0.1530± 0.0125** |   0.0528± 0.0023   |   0.0355± 0.0016   |\n  | FAME w/o MoE        |   0.1870± 0.0161   |   0.0828± 0.0072    |   0.0663± 0.0062   |   0.1578± 0.0132   |   0.0538± 0.0023   |   0.0362± 0.0017   |\n  | FAME w/o Cross‑attn |   0.1902± 0.0158   |   0.0815± 0.0078    |   0.0668± 0.0064   |   0.1602± 0.0143   |   0.0544± 0.0026   |   0.0375± 0.0018   |\n  | **FAME**            | **0.1806± 0.0152** | **0.0783±  0.0064** | **0.0635± 0.0056** |   0.1532± 0.0120   | **0.0511± 0.0023** | **0.0342± 0.0017** |\n\n+ **Weaknesses  3（Question 2）:**　Your concern is fully justified, and we will include quantitative measurements of compute and memory usage. Cases 1–2 jointly span the two principal axes of computational load—the number of functions and the number of sampling points per function—so they are well suited for reporting cost, and all results will be produced on the same ordinary desktop (AMD Ryzen 9 5950X CPU) . In brief, basis/FPCA/kernel methods can train faster when grids are fixed and models are small, but they depend on pre‑specified bases or kernels and may fit particular data poorly; by contrast, FAME is fully data‑driven—while training is more costly, it remains lightweight in absolute terms, and inference time is comparable to classical baselines. Even in our most complex setting, training remains tractable (≈ 207 s/epoch × 20 epochs ≈ 69 min on this CPU), and post‑training inference time is on the order of seconds. Memory usage is even less of a concern and varies little across tasks, so we report average usage: the datasets are purely numeric and small on disk (≈ 0.3–1 MB), and the trained FAME model is only on the order of tens of megabytes, making deployment straightforward. From a complexity standpoint, kernel/GP training requires forming and factoring an $n\\times n$ kernel ($O(n^2)$ memory, $O(n^3)$ time) with $O(n)$ per‑test inference, while basis/FPCA costs scale with basis size $p$ (e.g., $O(np^2 + p^3)$); accordingly, classical kernel methods are most practical in the small‑$n$ regime, and basis/FPCA expressivity is ultimately constrained by the chosen basis size $p$—smaller $p$ improves efficiency but limits approximation power; in contrast, FAME processes each sample once per epoch, so the per‑epoch cost is approximately linear in $n$, with runtime dominated by the NCDE solver and only modest per‑sample overhead is required by attention and MoE in FoFR—roughly $O(T d^2)$ for cross‑attention and $O(TK)$ for routing. Moreover, in typical function‑to‑function regression workloads the grids are short (usually $T<100$) and the number of functions is modest (often $d<20$), so these costs are small in practice. Consequently, investing about an hour to train a fully data‑driven model that delivers substantial accuracy gains, handles irregular and misaligned grids, and preserves seconds‑level inference latency is a practical and worthwhile trade‑off.\n\n**Runtime**\n\n| Method           | C1‑100 (Train/Infer s)  | C1‑200 (Train/Infer s) | C1‑500 (Train/Infer s) | C2‑100 (Train/Infer s) | C2‑200 (Train/Infer s) | C2‑500 (Train/Infer s) |\n| :--------------- | :---------------------: | :--------------------: | :--------------------: | :--------------------: | :--------------------: | :--------------------: |\n| Basis expansion  |       0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |\n| FPCA             |       0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |\n| Kernel Method    |       0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |\n| Gaussian Process |       0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |\n| FNN              |      0.40*10 /0.36      |     0.62*10 /0.37      |     0.97*10 /0.36      |     1.02 *10 /0.81     |     1.98*10 /0.82      |     3.32 *10 /0.84     |\n| **FAME**         | 48.79*20  (epoch)/ 1.02 |   57.46*20   / 1.00    |   79.53 *20  / 1.03    |   134.42*20  / 2.60    |   151.47*20  / 2.66    |   207.20 *20  / 2.92   |\n\n**Memory Usage**\n\n|   Method    | Basis Expansion | FPCA | Kernel Method | Gaussian Process | FNN  | FAME  |\n| :---------: | --------------- | :--: | :-----------: | :--------------: | :--: | :---: |\n| Memory (MB) | 0.30            | 0.25 |     2.91      |       12.7       | 3.82 | 15.36 |\n\nOnce again, thank you for your suggestions and careful review; we hope these additional explanations and experiments help you better understand our work and appreciate it."}, {"review_id": "vTfnYd9cHC", "reviewer": "Reviewer_gU2z", "summary": "This paper presents FAME, a deep learning framework for function-on-function regression (FoFR). The model is designed to work with irregularly sampled functional data and produces continuous functional outputs. It combines several ideas: (1) bidirectional neural controlled differential equations (NCDEs) for capturing intra-functional structure, (2) a mixture-of-experts (MoE) router for modeling heterogeneity across functions, and (3) cross attention for learning dependencies across input channels. The authors claim that FAME provides theoretical guarantees such as Lipschitz stability, sampling invariance, and universal approximation. Experimental results on synthetic and real-world datasets show that FAME outperforms various statistical and neural baselines.", "questions": "- Could you provide runtime and memory usage for FAME? Since the model uses NCDEs and attention, it may be more computationally demanding than classical methods. It would be helpful to understand how the training cost compares to baselines like FPCA, kernel methods, or functional neural networks.\n- The proofs in Appendix A don't clearly map to the theorems stated in the main text. For example, Theorem 3 in the main paper doesn't appear to have an explicit label in the appendix. It would improve clarity if each theoretical result in the main paper pointed directly to its proof in the appendix using consistent numbering.\n- Have you tested the model's behavior on longer or more densely sampled functional data? Since NCDEs can be computationally intensive, some indication of the method's scalability would be useful.\n- You mention running five random splits for experiments, but the results are presented without error bars. Including standard deviations or confidence intervals would help readers assess the statistical significance of the improvements.", "limitations": "The authors mention that FAME may not be optimal in low-complexity settings where simpler models suffice. However, the paper does not discuss the method's computational cost or efficiency relative to baselines, which is important for practical deployment. This should be acknowledged more explicitly.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The architecture is thoughtfully designed and well-motivated for the challenges of FoFR.\n- The theoretical analysis includes Lipschitz bounds and approximation results, which help support the method's generality and stability.\n- Extensive experiments on both synthetic and real-world datasets show FAME achieves strong performance, especially in irregular or noisy settings.\n\nWeaknesses:\n- The paper doesn't report any computational metrics. Given that the model relies on NCDE solvers and attention mechanisms, it's important to understand how it compares to lighter models in terms of runtime or memory use.\n- Appendix A contains the theoretical proofs, but it's hard to trace which result in the appendix corresponds to which theorem in the main text. The connection between Theorem 3 and the appendix, for instance, is unclear.\n- No ablation studies are provided. It would be helpful to know how much each component (e.g., MoE, bidirectionality) contributes.", "was_revised": true, "final_justification": "While I appreciate the additional runtime and memory metrics, they confirm that FAME is substantially slower than classical baselines (e.g., ~69 minutes vs. milliseconds), and the justification provided is qualitative rather than supported by a clear cost-benefit analysis or evidence from large-scale experiments. This leaves open the concern about the method's practicality for real-world applications.\n\nAdditionally, the issue with mapping proofs remains unresolved. In Appendix A.1(i), the text begins with \"Theorem 1 in the main text shows that…,\" which suggests that the appendix assumes rather than proves the main result. This contradicts the rebuttal's claim that A.1(i)-(iii) correspond to the proofs of Theorems 1-3, creating confusion about the completeness and correctness of the theoretical support.\n\nGiven these points, the rebuttal improves clarity but does not fully address key concerns, so I maintain my original evaluation.", "rebuttal": "We thank the reviewer for the helpful suggestions. We have revised the paper to address all comments and respond point‑by‑point. Due to the rebuttal’s format and character limits, we present only newly added or key sections; full formatting and content will be finalized in the manuscript.\n\n+ **Question  (1)**: Your concern is fully justified, and we will include quantitative measurements of compute and memory usage. Cases 1–2 jointly span the two principal axes of computational load—the number of functions and the number of sampling points per function—so they are well suited for reporting cost, and all results will be produced on the same ordinary desktop (AMD Ryzen 9 5950X CPU) . In brief, basis/FPCA/kernel methods can train faster when grids are fixed and models are small, but they depend on pre‑specified bases or kernels and may fit particular data poorly; by contrast, FAME is fully data‑driven—while training is more costly, it remains lightweight in absolute terms , and inference time is comparable to classical baselines. Even in our most complex setting, training remains tractable (≈ 207 s/epoch × 20 epochs ≈ 69 min on this CPU), and post‑training inference time is on the order of seconds. Memory usage is even less of a concern and varies little across tasks, so we report average usage: the datasets are purely numeric and small on disk (≈ 0.3–1 MB), and the trained FAME model is only on the order of tens of megabytes, making deployment straightforward. From a complexity standpoint, kernel/GP training requires forming and factoring an $n\\times n$ kernel ($O(n^2)$ memory, $O(n^3)$ time) with $O(n)$ per‑test inference, while basis/FPCA costs scale with basis size $p$ (e.g., $O(np^2 + p^3)$), making kernels practical mainly for small $n$ and basis/FPCA expressivity limited by $p$. In contrast, FAME processes each sample once per epoch, so the per‑epoch cost is approximately linear in $n$, with runtime dominated by the NCDE solver and only modest per‑sample overhead is required by attention and MoE in FoFR—roughly $O(T d^2)$ for cross‑attention and $O(TK)$ for routing. Moreover, in typical function‑to‑function regression workloads the grids are short (usually $T<100$) and the number of functions/channels is modest (often $d<20$), so these costs are small in practice. Consequently, investing about an hour to train a fully data‑driven model that delivers substantial accuracy gains, handles irregular and misaligned grids, and preserves seconds‑level inference latency is a practical and worthwhile trade‑off.\n\n\n### Runtime\n\n| Method           | C1‑100 (Train/Infer s)  | C1‑200 (Train/Infer s) | C1‑500 (Train/Infer s) | C2‑100 (Train/Infer s) | C2‑200 (Train/Infer s) | C2‑500 (Train/Infer s) |\n| :--------------- | :---------------------: | :--------------------: | :--------------------: | :--------------------: | :--------------------: | :--------------------: |\n| Basis expansion  |       0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |\n| FPCA             |       0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |\n| Kernel Method    |       0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |      0.01 / 0.01       |\n| Gaussian Process |       0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |      0.03 / 0.03       |\n| FNN              |      0.40*10 /0.36      |     0.62*10 /0.37      |     0.97*10 /0.36      |     1.02 *10 /0.81     |     1.98*10 /0.82      |     3.32 *10 /0.84     |\n| **FAME**         | 48.79*20  (epoch)/ 1.02 |   57.46*20   / 1.00    |   79.53 *20  / 1.03    |   134.42*20  / 2.60    |   151.47*20  / 2.66    |   207.20 *20  / 2.92   |\n\n**Memory Usage**\n\n|   Method    | Basis Expansion | FPCA | Kernel Method | Gaussian Process | FNN  | FAME  |\n| :---------: | --------------- | :--: | :-----------: | :--------------: | :--: | :---: |\n| Memory (MB) | 0.30            | 0.25 |     2.91      |       12.7       | 3.82 | 15.36 |\n\n+ **Question  (2)**：We apologize for the confusion and agree that your suggestion will improve clarity. In the main text, Theorems 1–3 establish Lipschitz properties for different modules. Because concrete module instantiations and computations appear throughout, Appendix A.1 was added to show the end‑to‑end Lipschitz property of the overall framework. Each of Theorems 1–3 corresponds to items (i)–(iii) in Appendix A.1, respectively; however, this mapping was not made explicit. In the revision, we will  give (i)–(iii) explicit titles and numbering (e.g., Appendix A.1(i)), and add explicit pointers in the main text so that Theorems 1–3 directly reference their proofs in Appendix A.1(i)–(iii)(e.g., *“Proof in Appendix A.1(i)”*). \n\n+ **Question  (3)** ： Thank you for the suggestion. As shown in Fig. 2, varying the number of sampling points $T\\in \\{5,10,20,50\\}$ yields MSEs $\\{0.131,0.086,0.078,0.051\\}$. We also tested denser grids with $T=80$ and $T=100$, obtaining MSEs $0.048$ and $0.064$, respectively; these do not change our conclusions and were omitted from Table 1 due to space. We did not pursue substantially longer grids $(T>100)$ for two reasons: first, our task is FoFR, which treats the function as a whole—sampling points are merely a discretization of the underlying curve—whereas long‑sequence interpolation/extrapolation focuses on the state at specific time steps; accordingly, a good FoFR model should be **discretization‑invariant**, i.e., able to represent functions across different numbers of samples and sampling grids, and the fewer‑point regime is actually the more challenging one. Our method remains accurate and stable for $T\\in[5,100]$ (Appendix Figs. 5–6 show that $T\\in[20,50]$ already captures the functions well). Second, real‑world acquisitions are typically **sparse and irregular**, and moving from more data to fewer is straightforward, so demonstrating strong performance with fewer samples is the more informative stress test. In a subsequent version, we will extend Fig. 2(b) to include the $T=80,100$ results for completeness. Finally, from Question  (1)’s analysis and the runtime table, our training time scales approximately linearly with $T$, without sharp growth in compute or memory cost, and the method remains lightweight in absolute terms—hence FAME is practically scalable for FoFR.\n\n+ **Question  (4)**: We fully agree that reporting variability is essential to demonstrate the robustness of our results. In the revision, we will evaluate all methods over 5 random train/test splits × 2 seeds for both synthetic and real‑world tasks and present mean ± std in every table. (Due to time constraints, the table below represents we have currently completed; we can increase the number of runs in a future version if needed.)\n\n  | Model               |    Case 1 (100)    |    Case 1 (200)     |    Case 1 (500)    |   Case 2  (100)    |    Case 2 (200)    | Case 2      (500)  |\n  | :------------------ | :----------------: | :-----------------: | :----------------: | :----------------: | :----------------: | :----------------: |\n  | FAME w/o Bi‑dir     |   0.1832± 0.0150   |   0.0812± 0.0066    |   0.0654± 0.0055   | **0.1530± 0.0125** |   0.0528± 0.0023   |   0.0355± 0.0016   |\n  | FAME w/o MoE        |   0.1870± 0.0161   |   0.0828± 0.0072    |   0.0663± 0.0062   |   0.1578± 0.0132   |   0.0538± 0.0023   |   0.0362± 0.0017   |\n  | FAME w/o Cross‑attn |   0.1902± 0.0158   |   0.0815± 0.0078    |   0.0668± 0.0064   |   0.1602± 0.0143   |   0.0544± 0.0026   |   0.0375± 0.0018   |\n  | **FAME**            | **0.1806± 0.0152** | **0.0783±  0.0064** | **0.0635± 0.0056** |   0.1532± 0.0120   | **0.0511± 0.0023** | **0.0342± 0.0017** |\n\n  | Model                    | Hawaii Ocean Salinity |    Hawaii Ocean Temp |   Human3.6M Walking |     Human3.6M Eating | Human3.6M Sitting down |   ETDataset Oil Temp |\n  | ------------------------ | --------------------: | -------------------: | ------------------: | -------------------: | ---------------------: | -------------------: |\n  | Basis Expansion (best)   |       0.0780 ± 0.0039 |      0.0014 ± 0.0001 |     0.0359 ± 0.0018 |      0.0484 ± 0.0024 |        0.0122 ± 0.0006 |      0.0365 ± 0.0018 |\n  | FPCA                     |       0.0865 ± 0.0043 |      0.0025 ± 0.0001 |     0.0373 ± 0.0019 |      0.0099 ± 0.0005 |        0.0121 ± 0.0007 |      0.0355 ± 0.0018 |\n  | Kernel Method            |       0.0754 ± 0.0038 |      0.0025 ± 0.0001 |     0.0373 ± 0.0019 |      0.0099 ± 0.0005 |        0.0121 ± 0.0006 |      0.0355 ± 0.0016 |\n  | Gaussian Process         |       0.0931 ± 0.0047 |      0.0022 ± 0.0001 |     0.0360 ± 0.0018 |      0.0075 ± 0.0004 |        0.0107 ± 0.0005 |      0.0380 ± 0.0019 |\n  | FNN                      |       0.0766 ± 0.0038 |      0.0020 ± 0.0001 |     0.0373 ± 0.0019 |      0.0099 ± 0.0005 |        0.0121 ± 0.0006 |      0.0355 ± 0.0018 |\n  | FAME w/o bidir attention |       0.0751 ± 0.0023 |    0.00124 ± 0.00004 |     0.0327 ± 0.0010 |     0.0035 ± 0.00010 |        0.0071 ± 0.0002 |      0.0264 ± 0.0008 |\n  | FAME w/o MoE             |       0.0759 ± 0.0023 |    0.00130 ± 0.00004 |     0.0332 ± 0.0010 |     0.0038 ± 0.00011 |       0.0075 ± 0.00023 |      0.0271 ± 0.0008 |\n  | FAME w/o cross‑attention |       0.0773 ± 0.0024 |    0.00145 ± 0.00005 |     0.0344 ± 0.0011 |     0.0044 ± 0.00013 |       0.0083 ± 0.00025 |     0.0286 ± 0.00086 |\n  | **FAME**                 |   **0.0748 ± 0.0022** | **0.0012 ± 0.00004** | **0.0325 ± 0.0010** | **0.0034 ± 0.00010** |   **0.0070 ± 0.00020** | **0.0262 ± 0.00080** |\n\n  Once again, thank you for your suggestions and careful review; we hope these additional explanations help you better understand our work and appreciate it."}]

Author rebuttal:

Meta-review: This paper proposes a functional attention mechanism FAME to perform functional-on-functional regression where both covariates and responses are functional data. A combination of tools (neural controlled differential equations, mixture-of-experts and continuous attention mechanisms) are deployed for this complex problem that involves infinite-dimensional data. Numerical experiments based on synthetic data shows the advantage of FAME over various baselines. The theoretical analysis establishes existence, uniqueness and global Lipschitz stability for every differential block along with an end-to-end Lipschitz bound and generalization guarantees.
Overall, this is a strong and well-structured paper that addresses key challenges in functional data, including intra-functional continuity, inter-functional interactions, and feature heterogeneity.  One weakness in the numerical experiment is that the comparison did not include vanilla transformers with simpler schemes, so we do not know specific kind of attention scheme works. The paper received scores 5-5-5-3.  I recommend it for publication.
Decision: Accept (poster)