End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Authors: Xiaole Zhang
Peiyu Zhang
Xiongye Xiao
Shixuan Li
Vasileios Tzoumas
Vijay Gupta
Paul Bogdan
Venue: ICML 2025 poster (2025)
Keywords: Fractional-order dynamical system
Optimal control
Primary area: optimization
TL;DR: A novel approach for the optimal control of fractional-order linear time-invariant (LTI) systems via the linear quadratic regulator (LQR)

Abstract: Integer-order calculus fails to capture the long-range dependence (LRD) and memory effects found in many complex systems. Fractional calculus addresses these gaps through fractional-order integrals and derivatives, but fractional-order dynamical systems pose substantial challenges in system identification and optimal control tasks. In this paper, we theoretically derive the optimal control via linear quadratic regulator (LQR) for fractional-order linear time-invariant (FOLTI) systems and develop an end-to-end deep learning framework based on this theoretical foundation. Our approach establishes a rigorous mathematical model, derives analytical solutions, and incorporates deep learning to achieve data-driven optimal control of FOLTI systems. Our key contributions include: (i) proposing a novel method for system identification and optimal control strategy in FOLTI systems, (ii) developing the first end-to-end data-driven learning framework, Fractional-Order Learning for Optimal Control (FOLOC), that learns control policies from observed trajectories, and (iii) deriving theoretical bounds on the sample complexity for learning accurate control policies under fractional-order dynamics. Experimental results indicate that our method accurately approximates fractional-order system behaviors without relying on Gaussian noise assumptions, pointing to promising avenues for advanced optimal control.

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Xiaole Zhang" "1
Peiyu Zhang" 1 Xiongye Xiao 12 Shixuan Li"
Vasileios Tzoumas
Vijay Gupta
Paul Bogdan"
Abstract
1. Introduction
Integer-order calculus fails to capture the long- range dependence (LRD) and memory effects found in many complex systems. Fractional calcu- lus addresses these gaps through fractional- -order integrals and derivatives, but fractional-order dy- namical systems pose substantial challenges in system identification and optimal control tasks. In this paper, we theoretically derive the opti- mal control via linear quadratic regulator (LQQ) for fractional-order linear time-invariant (FOLTI) systems and develop end-to-end deep learning framework based on this theoretical foundation. Our approach establishes a rigorous mathematical model, derives analytical solutions, and incorpo- rates deep learning to achieve data-driven optimal control of FOLTI systems. Our key contributions include: (i) proposing novel method for sys- tem identification and optimal control strategy in FOLTI systems, (ii) developing the first end-to- end data-driven learning framework, Fractional- Order Learning for Optimal Control (FOLOC), that learns control policies from observed trajec- tories, and (iii) deriving theoretical bounds on the sample complexity for learning accurate control policies under fractional-order dynamics. Experi- mental results indicate that our method accurately approximates fractional-order system behaviors without relying on Gaussian noise assumptions, pointing to promising avenues for advanced opti- mal control.
Many real-world systemmmmmssttms including biological systems, neu- ronal networks, organ networks, financial markets, and cyber-physical systems, exhibit long-range dependence (LRD) and memory effects that traditional integer-order models struggle to capture (Lundstrom et al., 2008; Ivanov et al., 1999; Ghorbani & Bogdan, 2013; Ghorbani et al., 2018). Fractional-order linear iimeiinariintt (FOLTI) sys- tems extend classical integer-order models through frac- tional calculus, providing more flexible framework for modeling such complex behaviors. Due to their ability to represent non-local dependencies and history namics, FOLTI systems have found broad applications in control systems (Baleanu et al., 201 1; Monje et al., 2010; Reed et al., 2023b), bioengineering (Magin, 2004; 2010; Ghorbani & Bogdan, 2013), neuroscience (Pequito al., 2015; Gupta et al., 2019; Reed et al., 2022; 2023a), cyber- physical systems (Xue & Bogdan, 2017), and financial mod- eling (Granger & Joyeux, 1980). However, the practical deployment of FOLTI systems faces significant challenges in system identification and optimal control. Unlike Marko- vian systems, where the current state depends solely on the immediate past state, FOLTI systems exhibit non-Markovian properties, meaning their evolution is influenced by wide range of past states. This memory effect introduces addi- tional complexities, making it difficult for traditional meth- ods to generalize effectively to FOLTI systems.
dy-
Despite their advantages, FOLTI systems present signifi- cant challenges in both modeling and control. Identifying system parameters for fractional-order dynamical systems remains open research problem. One major difficulty in system identification arises from the non-Markovian nature of fractional order systems, where the state evolution is not solely determined by recent states but depends on an entire history of past observations. Additionally, the fractional derivative operators, such as the Grüinaald-Letnikon frac- tional derivative (Hilfer, 2000; Monje al., 2010; Oldham & Spanier, 1974), introduce combinatorial complexity and nonlinearities, making standard parameter estimation tech- niques ineffective. Existing methods often rely on strong assumptions, such as noiseless data or constrained data gen-
Equal contribution 'Ming Hsieh Department of Electrical and Computer Engineeringg University of Southern California, Los Angeles, CA 90089, USA 'Min H. Kao Department of Electri- cal Engineering and Computer Science, University of Tennessce, Knoxville, TN 37996, USA 'Department of Aerospace Engineer- ing, University of Michigan, Ann Arbor, MI 48109, USA *Elmore Family School of Electrical and Computer Engineering, Purdue University. West Lafayette, IN 47907, USA. Correspondence to: Xiaole Zhang Sitaolezh@us.cduu.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
eration processes, limiting their applicability in practical scenarios.
Zhang al., 2025) and robust control present significant technical difficulties.
Beyond system identification, optimal control of FOLTI systems remains a major challenge. Unlike integer-order systems, where well-established control frameworks exist, fractional-order systems exhibit long-range dependencies and memory effects, making classical control strategies inadequate. In particular, the lack of analytical solutions for optimal control in FOLTI systems requires new approaches that account for their unique dynamics. Addressing these challenges is crucial for enabling real-world applications of FOLTI systems in control systems.
To address these challenges, this work establishes a the- oretical framework for fractional-order LQR, deriving a principled end-to-end learning approach for FOLTI optimal control. Specifically, we formulate the mathematical struc- ture of FOLTI systems, enabling rigorous extension of LQR to the fractional-order setting. Given the diagonal ele- ments of the system matrix, we propose method to estimate the unknown system parameters from multiple observed tra- jectories. Using this estimated model, we derive the optimal control policy using LQR, leveraging least squares optimiza- tion and Lagrange multipliers to obtain analytically tractable solutions despite the inherent complexities introduced by fractional-order dynamics. We further analyze the sample complexity of this learning process, quantifying the number of samples required for the estimated LQQ loss to converge to the true LQR loss under unknown input conditions.
well-established framework for optimal control in integer- order systems is the Linear Quadratic Regulator (LQR), introduced by Rudolf Kalman in 1960 (Kalman et al., 1960). LQR determines optimal control policy that minimizes quadratic cost function defined over the system state and control input, subject to weighing matrices, for systems governed by linear dynamics. The traditional integer-order LQR problem has been extensively studied over the decades (Dean et al., 2018; 2020; Tu & Recht, 2019). For linear time-invariant (LTI) systems with known parameters, the LQR problem has closed-form solution on the infinite time horizon and can be solved efficiently using dynamic programming on finite horizons (Anderson & Moore, 2007). Furthermore, numerous researchers have developed end-to- end guarantees for LQR in the context of LTI systems. These guarantees often involve a two-step process: (i) estimating the unknown system parameters. and (ii) designing robust controllers to account for model uncertainties.
Beyond theoretical derivations, we develop an end-to-end data-driven learning framework, fractional-order learning for optimal control (FOLOC), that jointly performs system identification and optimal control. In contrast to classical control methods that require pre-specified system dynamics, our approach learns both the system parameters and control policies directly from observed trajectories. Unlike classical approaches that assume structured noise distributions, our framework is designed to operate in realistic, noisy envi- ronments. Specifically, we incorporate deep learning- based modeling techniques to ensure that the control policy re- mains robust under non-Gaussian noise, distributional shifts, and real-world uncertainties.
While the LQR problem for integer-order LTI systems is well understood (see related work in Appendix B), far less attention has been devoted to the case of fractional-order dy- namical systems (Monje et al., 2010). These systems, which originate from fractional calculus (Hilfer, 2000; Ionescu et al., 2017), differ significantly from their integer-order counterparts. Unlike Markovian systems, where the current state depends solely on the immediate past state, fractional- order systems exhibit a non- Markovann behavior, with the current state influenced by a memory effect spanning a broad range of past states. This memory effect makes the analysis and control of fractional-order dynamical systems particularly challenging. Estimation of unknown parame- ters in fractional- order dynamical systems remains open problem (Yaghooti& & Sinopoli, 2023; Chatterjee & Pequito, 2022; Zhang et al., 2025), even under the assumption of lin- ear fractional operators. For finite horizons, optimal control problems have been addressed under specific assumptions (Li & Chen, 2008), but no comprehensive results exist for infinite horizons. Moreover, achieving end-to-end guaran- tees for fractional-order systems particularly challenging, as both system identification (Yaghooti & Sinopoli, 2023;
In summary, this work makes the following key contribu- tions: (i) We provide theoretical foundations for optimal control of fractional-order linear time-invariant (FOLTI) sys- tems. (ii) We develop a principled end-to-end data-driven learning framework, FOLOC, that jointly optimizes system identification loss and control policies, ensuring robustness to non- Gaussian noise and real-world uncertainties. (iii) We analyze the thcoretical sample complexity, providing convergence guarantees for learned control policies under unknown system dynamics. (iv) We conduct extensive ex- periments, demonstrating the FOLOO'ss robustness across noise distributions, scalability to different system complexi- ties, and computational efficiency in real-time control tasks.
2. Preliminaries and Problem Formulation
In this section, we introduce key theoretical foun- dations from fractional calculus, focusing on the Grünwald Letnikov fractional-order derivative and its role in modeling FOLTI systems. By extending LQR FOLTI systems, we then formulate the end-to-end learning problem.
2

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
where the objective is to identify system parameters and op- timize control policies directly from data. This formulation integrates system identification and control optimization, addressing challenges arising from memory effects and non- Markovian dynamics in FOLTI systems.
Using the Grünwald-Letnikov difference operator, can write the system as follows:
K+1
Fk+1 AvR + Bu D(a,))aRt1-j
2.1. Fractional-Oree Derivative
The last term in Eq. 7 represents the memory-dependent nature of fractional-ordes dynamical systems, making them suitable for modeling Mar-Maakov and long-rang depen- dent processes.
Definition 2.1 Serünwald-Letmikor fractional-order deriva- tive). The left-side and right-side Grünnalld-Lennkovv fractional-order derivatives of order a on the finite inter-
val / = [a, b] are defined as follows:
2.4. Linear Quadratic Regulator for FOLTI systems
(GL (f = lim Σ (-
1)
k
1
f(r kh),
The linear quadratic regulator (LQR) problem concerns the optimal control of a dynamical system given known and fixed quadratic costs. Formally, the goal of LQR is find the optimal control u, that minimizes the quadratic loss fitting in the FOLTI systems, which in turn solves the fol- lowing optimization problem.
h- ha
(1)
lim
(-1)*
a 1:
((r + kh),
h-0 h
(2)
Definition 2.2 (LQR for FOLTI systems). The LQR prob- lem for FOLTI systems is defined as follows:
where E R. 0 h < b and a 0. Here, (:)
denotes the generalized binomial coefficient, defined as 1(++1) (+) [a+t)](a- where I(-) denotes the gamma
function, which is defined as ((z)= = fo ---le-t dt for 0.
min (T(u) = (ok Qu: + u Rux) + QT2T (w:)i
2.2. Grüinald-LLetiikr Difference Operator
0441 Ar Buk-
(9)
The fractional-order Grünwald-Letnike difference opera- tor allows to discretize the fractional-orde derivative and represent it as a finite difference as follows:
where Q Rexn and Q5 E Tx are positive semi- definite matrices representing the state cost and terminal state cost, respectively, where n is the dimension of the state vector Ty. Similarly, R E mR is a positive semi- definite matrix representing the control cost, where m is the dimension the control input vector uk-
A A rx D(a,))ak-p)
(3)
==
where Th E _2", [an, ....... anl E 1R" represents the order of the difference operator, and D(a,j) E Raxn is an n x n diagonal matrix defined as
2.5. Problem Formulation
Consider a fractional-order linear inme-invariant system de- fined using the Grünwald-Letnikov difference operator. The goal is find optimal control sequence via solving LQR problem based on system identification from observed data.
D(a.j) = diag (v((1, (), ((2,) (... (4)
with
((ax.)):= == T(j an) I(- + 1)' i= 1,2,...,
Specifically, we are given N trajectories, each tra- jectory consisting of T time steps, sampled from a joint distribution IT over initial states, process noise, system parameters and cost matrices, i.e., (5o, [wx) Thus, we can represent the dataset as (f+t) (u Q', R fui (x) A, a,) 1=o Jimi where ut denotes the optimal control input. We aim to solve the following two-step problem:
(5)
2.3. Fractional-Order Linear Time Invariant System
1f-1 {ux ) (A, B, a., (Q,
II. T-1
The state-space representation of the discrete-time fractional-order linear time invariant system reads:
A 1 Ar Buk:
(6)
where Fg E Ra is the state vector, 4y E Rm is the system input, matrices A and B are constant matrices with size n and m, respectively.
(i) System Identification. The fractional-order system dynamics are parameterized by e = (A, a, where A E Raxx and B E R n are system matrices, and
3

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
E (0, 1)2h is the fractional order. The system identifi- cation problem is posed as:
Theorem 3.2 (LQR solution for FOLTI systems). The least-
squares optimal control solution for the L.RR problem of FOLTI system is given by:
N EE xx =
arg min
(10)
U -- QG - R)) -R- G"Q Hoo,
(16)
where #f ()) represents the predicted state at time & for the i-th trajectory, obtained using the identified parameters O,
whereas the Lagrange multiplier optimal control solution is:
(ii) Optimal Control. Once Ô {A, B, à} is estimated, the optimal control problem is formulated as:
U U -I-GA)--HA00
(17)
Here, U, G. G2 H. Hy, R. and Q are defined in Appendix D.3, and @ denotes the Kronecker product. Notably, U [uo
T-1
min fz(u): = Σ (a] oxx + ug Rux) + x [QQATT (mx))-
k=0
01
uT
(11)
subject to the identified system dynamies:
Using the optimal control solution via LQR, a natural ques- tion arises: given a system identification algorithm and a threshold error e, how many samples are required for the system identification process to ensure that the error be- tween the estimated LQR loss J and the true LQR loss (computed with the true system parameters) remains within the threshold d(J - J) S e. where dl--) denotes the metric. We derive the following sample complexity results.
A 244+1
Aza Buk.
(12)
Deep Learning-Basee Reformulation. The optimal con- trol problem can be framed in a deep learning context as learning an operator the set of optimal control inputs. Formally, the operator f: D lf computes the optimal control policy {= directly from the observed trajectory, while simultaneously uncovering the system parameters θ = {A, B, a that en- capsulate the dynamics of the fmctional-ordee system.
D + U. where U represents
Theorem 3.3 (Sample Complexity for FOLTI systems). Consider a FOLTI system with a known matrix A, fractional order a, and an unknown matrix B. The following sample complexity bounds hold using the system identification in Appendix D.I: Let
3. Methodology
3.1. Mathematical Foundations for Fractional-Order Learning and Control
K
- N
10 O0)
Knolo -
Lemma 3.1 (Discrete-time FOLTI system solution). The solution to the discrete-time FOLTI system is given by (Guer- mah et al., 2012):
(a) Least- solution.
j J - | (L (12111121
k-1
(Tr(KB) B) + 2| B1| Tr(K (18)
Ik = GE +
Σ
Gk-1-BBuj,
(13)
==0
(b) Lagrange multiplier solution. Assume LQg (I - Lu) t
where the matrices G3: are defined recursively as:
0,
for k- 0,
j - J s |b2le|//4scoll21111111111
G Ej= AjGu-15 for kork? 2 1,
(14)
|B
|| || (Tr(K))
and the matrices Aj are given by:
2 2| Tr(Kn)),
(19)
A, = A - D(a,j+
an)
i = 0, i > 1.
(15)
where 2 = GJQ Hæo. S = GTQGG, |||| = I| (LQq) |22| LQallz, and Ga. Q, and H are system matri- ces as defined in Appendix D.4.
The proof is provided in Appendix D.2. The first compo- nent of the FOLTI system solution represents the unforced response of the system. The term G2 exhibts the particular- ity of being time-varying, attributed to the fractional-order a which inherently accounts for all the past states. The second component takes the role of the convolution sum corresponding to the forced response.
Corollary 3.4 (Simplified Sample Complexity for FOLTI systems). Let Kn, = 0 and u/ ~ NNO, I). For p > m m+ L, then:
nmo? 2 A
Tr(KB) N 1)
(20)

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
where p denotes the number of initial conditions, and N denotes the number of trials adding Gaussian noise to fractional-order dynamics from these p initial conditions, and m is the dimension of control inputs (see Appendix D.5).
system is often presumed known in advance. In the context of fractional-order dynamical systems, estimating the sys- tem order-i.e., the fractional derivatives-is substantially more challenging, and to the best of our knowledge, no existing method provides theoretical guarantees in a gen- eral setting. To address this limitation, we propose FOLOC framework, which can give accurate optimal control policies without this assumption.
The sample complexity results demonstrate that the overall convergence rates of the least-squares solution and the La- grange multiplier solution are both O vx) ), differing only in their constant factors.
3.3. Fractional-Order Learning for Optimal Control Framework
3.2. Theoretical Approach for Fractional-Order System Identification and Optimal Control
Our end-to-end deep learning framework FOLOC is guided by the mathematical derivation of the optimal control pol- icy for FOLTI systems (see Methodology 3.1). Generally, FOLOC framework can be structured into two main compo- nents.
Our end-to-end theoretical learning method follows a similar strategy to those used in Markovian processes, specifically for LTT systems. The learning procedure can summarized in two main steps. Given N noisy trajectories, each of T time steps, and assuming the diagonal elements of the sys- tem matrix A are known, first formulate a least-squares problem to identify the system parameters by using the lin- carity of the Grünwald-Letniko difference operator. The state evolution with noise can be represented as
System Identification Module: Inspired by traditional LTI and our proposed FOLTI system identification, we use deep learning based time-series models estimate system param- eters and approximate intermediate variables. The objective is to learn the underlying function that maps trajectories to the variables required for solving the LQR problem.
x = Ark + Bug- + Wk:
Theoretical system identification for both LTI and FOLTI systems predominantly relies on least-squares methods, which can be viewed as operators learning a function that maps from the trajectory and control input space X to the system parameter space S. Consider the temporal nature of the trajectory, we choose a recurrent neural network to estimate the system parameters. The mapping is defined as Rs X E [T + S E R" = +mm such that vectorize( (Ao, (o) = Ro u, To") From Lemma 3.1. the function mapping between the system parameters to the time-varying matrices Aş and then to the matrices G₁ (23) can be represented as follows respectively
(21)
where (w T-1 is an independent and identically dis- tributed (i.i.d) Gaussian noise process. Using the linearity of the fractional difference operator, the state evolution for each trajectory can be expressed as:
T Aro + Buo + C2 + wo
(22)
Using this formulation, we define the following optimization problem across N trajectories
i arg min
@ ||X
{0 E0
with the closed-form solution
A& = I)),,;; G. = fA ... AR-1);
(25)
Ô TT
(24)
where I)): obtained as a function of the Gamma function, which governs the fractional-order dynamics. To model the first mapping, use MLPs with injecting time embeddings to approximate the function Ry S + H e T7 such that {A,, 1T = R. R. âg (ei)T= where h can be chosen based the dynamies complexity and {e: [ei)e represents embedding of specific timestamp i. The second mapping is approximated by an Encoder- only Transformer R : H + g E TR'xx such that [T= T-1 R, JTo'), where G.,, represents the learned approximation of the intermediate parameters Gt-
where system parameters Ca. 0 and feature matrices X,ç are detailed in Appendix D.I.
In the second step, the derived system parameters are used to compute the optimal control solution as described in The- orem 3.2. For a given identified FOLTI system and a time horizon T, the optimal control sequence can be derived for any cost matrices Q and R, along with the initial condition Trp
Remark. Knowing the diagonal elements of the system matrix A is strong assumption and represents a limitation of the theoretical system identification approach. It serves a similar role to assumptions commonly made in classical system identification, where the exact (integer) order of the
Optimal Control Module: Building upon the mathematical derivation of the LQR problem using Lagrange multipliers, we use a neural operator to learn the mapping between the
5

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
SEQ Model
SEQ, Encoder
Total Loss
Tokens A
7-3
Time Encoding
Identification Loss
Tokens G
Optimal Control Loss
A
B
B
io
RNN
MLP
Stack
NO
Input
Q
Figure 1. Overview FOLOC framework architecture. The pipeline first infers fractional-order system parameters (A, a, B) from input X' using an RNN+MLP based system identification module. These parameters are then encoded as embeddings of sequential tokens A; for dime-dependent modeling. An attention based Sequence Encoder processes these embeddings to obtain latent representations, which along with cost matrices Q, R. estimated system matrix B are fed to the Stack MLPs (see Appendix E.1.4) with residual connection input X' for Fourier Neural Operator to predict optimal control signals. Finally, a composite loss function unifies system identification and control prediction, enabling end-to-end training of both system parameter estimation and control law synthesis.
input space (variable required for solving the LQR problem) and the output space (
iteratively applying several Fourier layer to update the repre- sentation U++1 we have the final vr E Rd,xe which has the same dimensionality as Up. The FNO outputs opti- mal control sequence U = ((vr) through the projection of er by the transformation operator Q Rd. xe - Tx This design enables efficient computation of the optimal control sequence by leveraging the special properties of the block Toeplitz linear system and the expressive capabilities of the FNO.
7-1 Ja=0 ( (optimal control sequence).
The core of optimal control module mirrors the process of solving the linear system derived from the LQR problem through Lagrange multipliers. The mathematical formula- tion of the linear system expressed as:
- T
(26)
where À is defined in Appendix D.3 and I- G. E RuxxTn is a block Toeplitz matrix (see Definition D.2). Using the solution of the Lagrange multipliers, the optimal control sequence {u: T- can given by:
The loss function used in the training process is defined as follows:
=,00 = (0) + (1 - Aw) ()),
(28)
where C. quantifies the loss due to system identification errors, and L, quantifies the loss due optimal control errors. Specifically,
U == U @ (I -I-GN) G1-
(27)
Luo et al. (2024) has demonstrated that solving linear sys- tems via Krylov iterations can be significantly accelerated by using the Fourier Neural Operator (FNO) (Li et al., 2021). Specifically, FNO learns the mapping from the linear system to its corresponding invariant subspace, us- ing the predicted subspace to guide iterative solvers. In- spired by this approach, we use a neural operator directly leam the mapping between two Hilbert spaces, denoted as T: A + U E T 2 Due to the block Toeplitz structure of the linear system, the input to FNO is the reduced relevant function variables a E R xxd,, The transformation opera- tor C first lifts the input a to higher dimensional channel space Up E Rd where the number of channels. Then
(. 14 10-A-l1: A + ||B B + ||a à
(.) = - - -Ūg
(29)
Note that A E [o, 1] is a user-defined weighting parameter that determines the balance between system identification and optimal control errors during the learning process.
Joint optimization pipeline: The system identification loss L. is a physical constraint for FOLOC framework that en- courages the model to accurately identify system dynamics, further ensuring accurate control- the ultimate goal of our method. During training, we use system parameters as guid- ance to ensure accurate system identification, and we do not
6

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
need these parameters in testing. Some methods, such as PINN (Raissi al., 2019), couple multiple loss terms by incorporating physical constraint losses. This helps the sys- tem learn representations that are both physically grounded and task-relevant, reducing error propagation and avoiding reliance on purely minimizing prediction error. For real- world datasets where the system parameters are unknown, we can first pretrain the model on synthetic datasets with known parameters (A, a)). Then, we do not need sys- tem parameters (set Aw 0) and finetune the model on the real-world dataset. We provide results for Aue 0 for comparison in Appendix H.2.
We also compare our method with the traditional end-to-end learning framework for LTT systems (see Appendix I). The mean MSE of FOLTI system learning framework decreases by 81.36% compared to LTI system learning framework.
0.045
0.010
Mean L1 Loss (LQR Loss)
Mean L1 Loss (LQR Loss))
0.035
0.008
0.025
0.006
0.015
0.004
0.01
0.02
0.03
0.01
0.02
0.03
1
1
Remark. The fractional-order dynamical system param- eters A, B and a can be extracted from an intermediate layer in the network because of the underlying loss func- tion. Though our end-to-end deep learning framework is guided by the theoretical optimal control solution derived by using Lagrange multipliers, certain learned parameters do not directly correspond one-to-one with their theoretical counterparts (see Appendix 1.1 for details).
vp
VN
Figure 2. Lagrange multiplier based sample complexity simula- tion.
5.0
5.0
Xu:
Rui
2.5
Aram
Xire
MSE
x
0.0
2.5
-2.5
4. Experiments
5.0
0.0
10
30
10
20
30
In this section, we empirically evaluate the proposed sys- tem identification algorithm and FOLOC framework using both synthetic and real- world system dynamics. The syn- thetic data is categorized into two primary types: (i) data generated from a single FOLTI system with fixed system pa- rameters A, B, and a and (ii) data generated from multiple FOLTI systems, where A, B are fixed, and a vary across systems. The real-world data is simulated from cart-pole and quadrotor dynamical systems.
Figure 3. Numerical simulation of theoretical approaches for learn- ing fractional- dynamics.
FOLOC framework evaluation. We evaluate our proposed data-driven approach by varying different variables. Our main findings are as follows: (i) FOLOC framework ef- fectively captures the underlying system dynamics with high accuracy and remains robust to variations in fractional orders, system dimensions, noise types, noise scales, and training samples. (ii) The model's ability to learn the sys- tem dynamics improves as the time horizon increases. (iii) FOLOC framework can be trained using a small number of samples.
4.1. Performance Evaluation on Synthetic Data
We evaluate our theoretical end-to-end learning approach and FOLOC framework using synthetic data. Due to the strict assumptions required for the theoretical learning pro- cess, we test it using synthetic data generated from a single underlying FOLTI system, with data corrupted by Gaussian noise. Our empirical analysis examines the performance of FOLOC framework from different perspectíves.
Varying fractional orders. By varying the fractional order a, we observe that FOLOC framework effectively captures long-term dependencies. The results of our experiments commensurate FOLTI systems (see Appendix C) with different fractional orders are shown in Table 1. As the fractional order increases, we observe that the MSE and MAE remain approximately the same, with an overall mean MSE of 0.40% (standard deviation: 0.21%) and an overall mean MAE of 0.95% (standard deviation: 0.55%).
0.1, 0.3, 0.5, 0.7, 0.9
The underlying fractional-order dynamics used to generate the data for our model are defined as follows:
1+1
T6+1 = Ark + Buk
D(a. + wk. (30)
Varying time horizons. We show the results of learn- ing fractional-order dynamics with different time horizons T = 8, 16, 32, 64, 128 in Table 1. As the time hori- zon increases, FOLOC framework demonstrates improved ability to capture the fractional-order dynamics. Compared with at T = 8, the MSE and MAE at T = 128 of FOLOC framework decrease by 97.37% and 92.23% respectively.
where wa represents the process noise.
Theoretical approach validation. By varying the number of trajectories used in the system identification process, we empirically validate Lagrange multiplier based sample com- plexity results presented in Theorem 3.3 as shown in Fig. 2.
7

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Table 1. Experimental results on synthetic FOLTI system under varying input conditions. All reported values are in units of 10
Bolded
values represent minimum in each row.
Metries
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
Fractional
alpha 0.1
alpha 0.3
alpha 0.5
alpha 0.7
alpha 0.9
Order
3.39+0.528.0444..22 50441.82/1..551.11
5.61+0.84/16.5400..1
5.18+0.0/9222
4.94-0.091005--0.00
State
Dimension
1.981..06.6001.1333
001=:0.4/16.551006
58410.71/13441055
4.8810.341134410.10
4.26=0.22// 2.0.0/22/2.33....11
Time
T=8
T= 16
T=
T=
T 128
Horizon
101114.4/1111.40
202218..0//9.111111
14.2112.6633..11..77
566110..8//6.55..000
2.66- 0.64/8.62....... 0.70
Cauchy
Gamma
Sine-squared
Uniform
Poisson
Noise Type
6.98+ 89/17.0-00.71 5.84:1.15/16,71...6 4.87+1.46/77666..77.
1.29/1..777 1.30 6.59 1.59-11..111---11 0.90
Noise Scale
a 0.0001
a= 0.001
0.01
a= 0.1
(Gaussian)
5.85:10.55/1..210..66
6.09:0.74/1.4420.5
5.7410.83/6.710..6
5.65-0..7//16.100..33
11/2.58111005-7..3
Training Size
Nr = 1000
Nr = 2000
Nr = 4000
Nr 6000
Nr = 8000
5.18
3.98 1.89109..9110. 1.79-22.11/0044....8 5.22-2.331/1011..22
Varying state dimensions. We vary the system state di- mension = 2, 4, 6, 8 to evaluate its impact the learning process (assume
0.10
MSE
0.10
MAE
m). Increasing the state di- mension significantly raises the learning complexity due to the larger number of variables and the interactions or couplings between the states. Despite this, FOLOC frame- work demonstrates consistent performance, with an overall mean MSE of 0.45% (standard deviation: 0.15%) and overall mean MAE of 1.24% (standard deviation: 0.36%). 2, the MSE (MAE) of FOLOC
MSE
MAE
0.05
0.05
0.00
0.00
2
24
2
26
27
Time Steps
Figure 4. Vary time horizons.
Compared with framework decreases by 13.01% % (18.79%) at = 6 and 24.06% (25.45%) at n = 8. This experiment demonstrates the scalability of our model on moderate- dimensional sys- tems. We further evaluate the scalability of the FOLOC framework in high-dimensional settings (n = m), as shown in Table 2. The results verify its ability capture inter- actions within the fractional dynamics of the state the system dimensionality increases.
process noise, we demonstrate the robustness and effective- ness of FOLOC framework. We evaluate the model's per- formance by simulating the noise from the following prob- ability distribution functions: Gaussian, Poisson, Uniform, Gamma, Sinc-squared (Adigun & Kosko, 2023; Zhang et al., 2024), and Cauchy. Compared with using Gaussian noise, the maximal increases of MSE/MAE by 24.42% % Seachhy/Sinssquureeee
The empirical results (Table 1) show that FOLOC framework can not only predict an ap- proximately accurate optimal control sequence but also ef- fectively estimate the system parameters.
Table 2. Experiments on nigh-dimensiones systems.
Metries
MSE/MAE
Dimension 16
3.85 + 0.05/12.7 + 0.08
Dimension 32
2.8 + 0.04/11.0 + 0.07
Varying noise scales. FOLOC framework shows the abil- ity to learning the fractional-order dynamics with data cor- rupted by different Gaussian noise scales. As data is cor- rupted by different scales of Gaussian noise, FOLOC frame- work demonstrates consistent performance, with an overall mean MSE of 0.67% (standard deviation: 0.19%) and an overall mean MAE of 1.75% (standard deviation: 0.18%)
Dimension 64
1.8 + 0.001/8.39 + 0.007
Dimension 128
1.43 0.01/7.22 0.03
Varying training samples. We show the model perfor- mance with different number of training samples N 1000, 2000, 4000, 6000, 8000. Compared with N 8000, the MSE only decreases by 1.65% at N = 1000. This demonstrates that FOLOC framework is sample-efficient and capable of learning the underlying fractional-order dy- namics with a limited number of samples.
The additional results also demonstrate that the FOLOC framework is computationally efficient (see Appendix J.3) and capable of learning from multiple FOLTI systems (see Apendix J.2). We also evaluate the performance by vary- ing the weight parameter X in the loss function (28) (see
Varying noise distributions. By varying the distribution of
8

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Appendix H.2).
Table 6. Baseline Model Evaluation quadrotor dynamics.
Bolded values represent the minimum in each column.
4.2. Performance Evaluation on Real- World Systems
Metrics
MSE/MAE
Run Time (s)
To demonstrate that our deep learning model can effectively capture complex system dynamics, we evaluate it two practical scenarios: cart-pole dynamics and quadrotor dy- namics (Zhou & Tzoumas, 2024). The details of the system dynamics are given in the Appendix E.3.
MBRL
0.489/0.546
0.385
CMA-ES
0.613/0.590
115.151
MPC
0.612/0.552
71.943
PID
1.143/0.909
0.0053
iLQR
0.628/0.570
3.387
Cart Pole. The empirical experiments show that FOLOC framework can correctly capture the underlying system dy- namies even when we add some large-scale Gaussian noise. The overall mean MSE is less than 3.99 10-5 and the overal mean MAE is less than 3.39 10-3
FOLOC
0.211/0.127
0.00027
highest accuracy and the lowest computational cost across both tasks. Notably, FOLOC framework reduces the mean squared error (MSE) by up to 99.99% and the mean abso- lute error (MAE) by up to 99.32% compared to existing baselines on the cart-pole dynamies task. All experimental code is available at the link provided in Appendix A.
Table 3. Model performance the cart pole experiment.
Noise Scale (r)
No noise
Metrics
10-1
10-2
10
3
MSE (10 ) MAE (10
4.0
3.9
4.1
3.9
3.9
Remark. The FOLOC framework, end-to-end learning- based LQR in FOLTT systems, offers several key advantages: (i) FOLOC framework generalizes across different FOLTI systems (see Appendix J.2), predicting system parameters and optimal control policies directly from trajectories. No- tably, system parameters are not used during testing on synthetic data, and are not used entirely during both training and testing on real-world dynamics. (ii) FOLOC framework enables fast and accurate real-time control. While closed- form solutions (Eqs. 16 and 17) are available given full sys- tem knowledge, they require large-scale matrix inversions and become computationally infeasible for long horizons or high-dimensional systems. MPC, though an alternative, also struggles with the memory effects in fractional order dynamics. FOLOC framework addresses these challenges by learning direct mapping from observed trajectories and cost matrices to optimal control solutions.
3,4
3.4
3.4
3.3
3.3
Table Baseline Model Evaluation on cart-pole dynamics. Bolded values represent the minimum in each column.
Metrics
MSE/MAE
Run Time (s)
MBRL
0.195/0.411
0.045
CMA-ES
0.237/0.472
16.405
MPC
0.237/0.473
0.674
PID
0.254/0.499
0.001
iLQR
0.238/0.474
0.263
FOLOC
0.00004/0.0034
0.0004
Quadrotor. The empirical experiments show that FOLOC framework is robust to different Gaussian noise scales when learning the Quadrotor dynamics. The overall mean MSE 0.13 (standard deviation: 0.052), and the overall mean MAE 0.21 (standard deviation: 0.086).
5. Conclusion
Table 5. Model performance in the quadrotor experiment.
This paper introduces theoretical learning framework for optimal control of FOLTI systems, addressing key chal- lenges in system identification, control design, and sample complexity analysis. By extending LQR to the fractional- order setting, we derive analytical solutions and provide sample complexity guarantees. Based on this theoretical foundation, we propose FOLOC, an end-to-end framework that jointly estimates system parameters and learns opti- mal control policies from noisy trajectories. Unlike clas- sical methods, FOLOC framework is designed operate effectively in real-world noisy environments, ensuring ro- bustness against non -on-saasii noise and limited training samples. Extensive simulations and real-world applications demonstrate the scalability, robustness, and computational efficiency of our approach.
Noise Scale (a)
No
Metrics
10-1
10-2
10 3
noise
MSE (10-1)
1.27
1.27
1.25
1.32
1.23
MAE (10-1)
2.10
2.14
2.08
2.13
2.06
Baseline Comparisons. We compare the proposed FOLOC framework against several existing control methods. includ- ing model-based reinforcement learning (MBRL), covari- ance matrix adaptation evolution strategy (CMA-ES), model predictive control (MPC), proportional-interre derivaaive (PID) control, and the iterative linear quadratic regulator (iLQR), evaluated on both cart-pole and quadrotor dynam- ics. For each method, we report predictive performance and runtime. FOLOC framework consistently achieves the

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Acknowledgements
control policy learning across various noise distributions and system complexities.
The authors acknowledge the support by the U.S. Army Research Office (ARO) under Grant No. W911NF-23-1 0111, the National Science Foundation (NSF) under the Career Award 1936775,
This research lays the groundwork for future advance- ments in learning-based control of fractional-order dynami- cal systems, paving the way for real-world deployment of fractional-order optimal controllers autonomous systems, robotics, and large-scale networks.
Career Award 2337412, MCB- and the NSF Award 2243104 under the Center for Complex Particle Systems (COMPASS), the Defense Advanced Research Projects Agency (DARPA) Young Faculty Award and DARPA Director Fellowship Award under Grant Number N66001-17-1-4044 Intel fac- ulty awards and Northrop Grumman grant. P.B. is also grateful to National Institute of Health (NIH) for the grants RO1 AG 079957 "Interpretable machine learning to syner- gize brain age estimation and neuroimaging genetics" and RFI AG 082201 "Neurovascular calcification and ADRD in two nonindustrial Native American populations". was a wonderful experience designing and writing the grant ap- plication entitled "Neurovascular calcification and ADRD in two nonindustrial Native American populations" and awarded under RF1 AG 082201. The views, opinions, and/or findings in this article are those of the authors and should not be interpreted as official views or policies of the Department of Defense, the National Institute of Health or the National Science Foundation. The authors would like thank Zehao Zhao and Zonglin Zhang for their insightful discussions during the course of this project.
References
Adigun, O. and Kosko, B. Hidden priors for bayesian bidi- rectional backpropagation. 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 445-451. IEEE, 2023.
Anderson, B. D. and Moore, J. B. Optimal control: linear quadratic methods. Courier Corporation, 2007.
Aronszajn, N. Theory of reproducing kernels. Transactions of the American mathematical society, 68(3):337-404 1950.
Baleanu, D., Machado, A. T., and Luo, A. C. Fractional dynamics and control. Springer Science & Business Media, 2011.
Chan, R. H.-F. and Jin, X.-Q. An introduction iterative Toeplitz solvers. SIAM, 2007.
Impact Statement
Chandrasekaran, S. and Sayed, A. H. A fast stable solver for nonsymmetric toeplitz and quasi-toeplitz systems of linear equations. SIAM Journal on Matrix Analysis and Applications, 19(1):107-13, 1998.
Fractional-order dynamical systems can model complex dynamical behaviors characterized by long-range dependen- cies and memory effects, yet their practical implementation remains challenging due the lack of system identifica- tion and control strategies. This work advances the field by formulating both theoretical and deep learning frameworks for solving fractional-order optimal control via the LQR. For theoretical learning, we provide a new fractional-order system identification and a new sample complexity analysis that gives guarantees on the number of samples required for achieving reliable control performance. Beyond theoretical contributions, develop novel end-to-end data-driven learning framework that jointly estimates system parame- ters and optimizes control policies directly from observed trajectories. Unlike traditional methods that rely on strong assumptions (e.g., noiseless data or structured disturbances), our approach remains robust under realistic, noisy environ- ments with non-Gaussir uncertainties.
Chatterjee, S. and Pequito, S. On learning discrete-time fractional-order dynamical systems. In 2022 American Control Conference (ACC), pp. 4335-4340 IEEE, 2022.
Chen, S., Billings, S. A., and Grant, P. Non-linear sys- tem identification using neural networks. International journal of control, 51(6): 191-1124,,, 1990.
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. Learn- ing phrase representations using rnn encoder-decoder for statistical machine translation, 2014. URL https: / //arxiv.. org/ httpg//bb///.///// 1406.1078
Cho, Y. and Saul, L. Kernel methods for deep learning. Advances in neural information processing systems, 22, 2009.
By bridging theoretical foundations with deep learning tech- niques, work enables scalable and efficient optimal con- trol for fractional-order dynamical systems in domains such as biomedical engineering, neuroscience, and financial mod- eling. Experimental results validate the effectiveness of our approach, demonstrating accurate system identification and
Dean, S., Mania, H., Matni, N., Recht, B., and Tu, S. Regret bounds for robust adaptive control of the linear quadratic regulator, 2018. URL https: /arxiv. org/ /ba/ 1805.09388.
10

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Dean, S., Mania, H., Matni, N., Recht, B., and Tu, S. On the sample complexity of the linear quadratic regulator. Foundations of Computational Mathematics, 20(4))633- 679, 2020.
Hochreiter, S. Long short-term memory. Neural Compula- tion MIT-Press, 1997.
lonescu, C., Lopes, A., Copot, D., Machado, J. T., and Bates, J. H. The role of fractional calculus in modeling biological phenomena: A review. Communications in Nonlinear Science and Numerical Simulation, 51:141- 159, 2017.
Dinuzzo, F. and Schölkopf, B. The representer theorem for hilbert spaces: necessary and sufficient condition. Advances in neural information processing systems, 25, 2012.
Ivanov, P. C.,, Amaral, L. A. N., Goldberger, A. L., Havlin, S., Rosenblum, M. G., Struzik, Z. R., and Stanley, H. E. Multifractality in human heartheat dynamics. Nature, 399 (6735):461-463) 1999.
Faradonbeh, M. K. S., Tewari, A., and Michailidis, G. Finite time identification in unstable linear systems, 2018. URL https: /ttp:///aavvvv.aa/.aaaee..ea.ava org/ /ts//b//////// 171011111 01852.
Gedon, D., Wahlström, N., Schön, T. B., and Ljung, L. Deep state space models for nonlinear system identification. IFAC-PapersOnLine, 54(7):481-48) 2021.
Jelich, C., Karimi, M., Kessissoglou, N., and Marburg, S. Efficient solution of block toeplitz systems with multiple right-hand sides arising from a periodic boundary ele- ment formulation. Engineering Analysis with Boundary Elements, 130: 13::135-144, 2021.
Ghorbani, M. and Bogdan, P. A
system ap- proach to artificial pancreas design. In 2013 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS), pp. 1-10. IEEE, 2013.
Kalman, R. E. et al. Contributions to the theory of optimal control. Bol. soc. mat. mexicana, 5(2):102-119, 1960.
Ghorbani, M., Jonckheere, E. A., and Bogdan, P. Gene expression is not random: scaling, long-range cross- dependence, and fractal characteristics of gene regulatory networks. Frontiers in physiology, 9: 9:1446, 2018.
Kalouptsidis, N., Carayannis, G., and Manolakis, D. Fast algorithms for block toeplitz matrices with toeplitz entries. Signal Processing, 6(1):77-81, 1984.
Granger, W. and Joyeux, R. An introduction to long- memory time series models and fractional differencing. Journal of time series analysis, 1(1):15-29, 1980.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Guermah, S., Djennoune, S., and Bettayeb, M. Discrete- time fractional-order systems: Modeling and stability issues. Advances in Discrete Time Systems, pp. 183-212, 2012.
Li, Y. and Chen, Fractional order linear quadratic regu- lator. In 2008 IEEE/ASME International Conference on Mechtronic and Embedded Systems and Applications, pp- 363-368. IEEE, 2008.
Gupta, G., Pequito, S., and Bogdan, P. Learning latent fractional dynamics with unknown unknowns. In 2019 American Control Conference (ACC), pp. 217-222. IEEE, 2019.
Li, Z., Kovachki, N., Azizzadenesheli, K.. Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equa- tions, 2021. URL https: /aap::///axiv... wrw.evorrrrrooo /ba//
Gupta, G., Xiao, X., and Bogdan, P. Multe operator learning for differential equations. Advances in neural information processing systems, 34:24048-24062, 2021.
Lundstrom, B. N., Higgs, M. H., Spain, W. J., and Fairhall, A. L. Fractional differentiation by neocortical pyramidal neurons. Nature neuroscience, 11(11)11335-1342 2008.
Gupta, G., Xiao, X., Balan, R., and Bogdan, P. Non-linear operator approximations for initial value problems. In International Conference on Learning Representations (ICLR), 2022.
Luo, J., Wang, J., Wang, H., huanshuo dong, Geng, Z., Chen, H., and Kuang, Y. Neural krylov iteration for accelerating linear system solving, In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024. URL https: http:////peeeeiieeeeeee nnt/ forum? Fd
Hendriks, J. N., Gustafsson, F. K., Ribeiro, A. H., Wills, A. G., and Schön, T. B. Deep based narx mod- els, 2020. URL /tR../tttt///rrrvi..rr////////// wrtp:////////a........ 04136.
Lusch, B., Kutz, J. N., and Brunton, S. L. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications, 911):455,, 2018.
Hilfer, R. Applications of fractional calculus in physics. World scientific, 2000.

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Magin, R. Fractional calculus in bioengineering, part Critical ReviewSIM in Biomedical Engineering, 32(1), 2004.
Research, 22(26):1-61, 2021. URL http: httt:://mmrr org/papert /ttpeers/v2//// 9-01119---22222 -2mmmmml..
Simchowitz, M., Mania, H., Tu, Jordan, M. I., and Recht, Learning without mixing: Towards sharp analysis of linear system identification, 2018. URL https:// a arxiv, org/ abss//b////// / 802. 08334.
Magin, R. L. Fractional calculus models complex dynam- ics in biological tissues. Computers & Mathematics with Applications, 59(5): 2010.
Masti, D. and Bemporad, A. Learning nonlinear state-space models using autoencoders. Automatica, 129: 109666, 2021.
Strang, G. A proposal for toeplitz matrix calculations. Stud- ies in Applied Mathematics, 74(2):171-176, 1986.
Trench, W. F. Solution of systems with toeplitz matrices
Monje, A., Chen, Y., Vinagre, B. M., Xue, D., and Feliu- Batlle, V. Fractional-onder systems and controls: funda- mentals and applications. Springer Science & Business Media, 2010.
generated by rational functions. Linear algebra and its applications, 74:191-211 211, 1986.
Tu, S. and Recht, B. model-free methods on the linear quadratic regulator: An asymptotic viewpoint. In Beygelzimer, A. and Hsu, D. (eds.), Proceedings of the Thirty-Secoue Conference on Learning Theory, volume 99 of Proceedings of Machine Learning Research, pp. 3036-3083. PMIR, 25-28 Jun 2019. URL http://::/ ptstt:::/prroee http://pocceeeingeeeeeeee mlr press/ v99 tul/tul0 atmm..
The gap between model-based and
Narendra, K. and Parthasarathy, K. Identification and con- trol of dynamical systems using neural networks. IEEE Trunsactions Neural Networks, 1(1):4-27, 1990. doi: 10.1 109/22.80202.
Oldham, and Spanier, J. The fractional calculus the- ory and applications of differentiation and integration to arbitrary order. Elsevier, 1974.
Wishart, The generalised product moment distribu- tion samples from normal multivariate population. Biometrika, pp. 32-52, 1928.
Pequito, S., Bogdan, P., and Pappas, G. J. Minimum number of probes for brain dynamics observability. In 2015 54th IEEE Conference on Decision and Control (CDC), pp- 11.. IEBE, 2015.
Xiao, X., Cao. D., Yang. R., Gupta, G., Liu, G., Yin, C., Balan, R., and Bogdan, P. Coupled multiwavelet oper- ator learning for coupled differential equations. In The Eleventh International Conference on Learning Repre- sentations, 2022.
Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Xue, Y. and Bogdan, Constructing compact causal math- ematical models for complex dynamics. In Proceedings of the 8th International Conference on Systems, pp. 97-107, 2017.
Reed. E., Chatterjee, S., Ramos, G., Bogdan, P., and Pe- quito, S. Fractional cyber-neural systems- brief survey. Annual Reviews in Control, 54:386-408, 2022.
Yaghooti, B. and Sinopoli, B. Inferring dynamics of discrete- time, fractional-order control-affine nonlinear systems. In 2023 American Control Conference (ACC), pp. 935-940. IEEE, 2023.
Reed, E. A., Ramos, G., Bogdan, P., and Pequito, S. Mitigat- ing epilepsy by stabilizing linear fractional-order systems. In 2023 American Control Conference (ACC), pp. 2228- 2233. IEFF, 2023a.
Zancato, L. and Chiuso, A. A novel deep neural network architecture for non non-linear system identification, 2021. URL https:tpp///o /tt///axxxi arxiv. org/ abs abw./b/s....0 206.. 03078.
Reed, E. A., Ramos, G., Bogdan, P., and Pequito, S. The role of long-term power-law memory in controlling large- scale dynamical networks. Scientific Reports, 13(1): 19502, 2023b.
Zhang, X., Zuo, J., and Shao, X. Efficient neural decoder: Mixture-regularizze bidirectional gru with attention. In 2024 International Joint Conference on Neural Networks (IJCNN), pp. 1-7, 2024. doi: 10.1 D0.1109/JCNN60899... 10651174.
Saad, Y. and Schultz, M. H. Gmres: A generalized mini- mal residual algorithm for solving nonsymmetric linear systems. SIAM Journal scientific and statistical com- puling, 733:856-866, 1986.
Zhang, X., Gupta, V., and Bogdan, P. A sampling complexity-awan framework for discrete-time fractional- order dynamical system identification, 2025. URL https: http:ss//errt...rr//as///////
Sarkar, T., Rakhlin, A., and Dahleh, M. A. Finite time Iti system identification. Journal of Machine Learning
12

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Zhou, H. and Tzoumas, V. Simultaneous system identifi- cation and model predictive control with dynamic gret, 2024. URL https: wrg. 2407. 04143.
13

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Limitation:
While our framework is effective for FOLTI systems, its LT1-based design requires further validation in non-LTI settings. Future work will extend the framework to non-LTI dynamics, enhancing its applicability to more complex systems.
A. Code Availability
The source code is available at http::/hhhhbboooooyoy/oe/oeeooot Cow.:piiiinnaottoooooaaeeaennnn
B. Related Work
While research system identification and optimal control for FOLTI systems remains limited, significant progress has been made in L.TI systems. Estimating unknown parameters of linear dynamical systems is a well-established subfield of system identification in control theory (Sarkar et al., 2021; Faradonbeh et al., 2018; Simchowitz et al., 2018), where the task is to estimate parameters from input-output time series generated by the underlying system. With the increasing interest in machine learning and deep learning, many researchers use different deep learning based modeling techniques to solve general nonlinear system identification. Chen al. (1990) shows single hidden layer neural network can identify discrete time nonlinear systems and derives new parameter estimation algorithms based on prediction error formulation. Narendra & Parthasarathy (1990) demonstrates that neural networks can be used effectively for both identification and control of nonlinear dynamical systems. The hierarchical structures of multilayer feedforward neural networks can include dynamic systems features (Zancato & Chiuso, 2021) and bring extra flexibility for probablistic approaches (Hendriks et al., 2020). Kernel-based methods also have been studied in system identification. Linear system identification can be seen as an application of learning the impulse response function, which can be approximated by kernels (Aronszajn, 1950; Dinuzzo & Schölkopf, 2012; Cho Saul, 2009). Deep state-space models (Gedon al., 2021) like recurrent neural networks (Hochreiter, 1997; Cho et al., 2014) and autoencoders (Masti & Bemporad, 2021: Lusch et al., 2018) are also gaining in popularity for system identification. For Another critical aspect of optimal control via LQR is the design of control inputs, which are typically formulated as linear combinations of disturbance processes (Dean et al., 2020).
Solving the optimal control problem via LQR involves addressing a potentially large linear system with a block Toeplitz matrix of size RTTxTm While solutions for Toeplitz and block Toeplitz. systems have been extensively studied (Trench, 1986; Kalouptsidis et al., 1984; Chandrasekaran & Sayed, 1998), they still present significant computational challenges. Direct solution methods for Toeplitz systems typically exhibit a computational complexity of O(N3), where N represents the degrees of freedom. However, the high memory requirements of these methods often limit their applicability large-scale problems. Iterative solvers (Chan & Jin, 2007; Strang, 1986) are more memory- -fficieet and can achieve complexity of O(N (NV) for a single linear block Toeplitz system. Advanced approaches, such global and block variants of the generalized minimal residual (GMRES) method (Saad & Schultz, 1986). can efficiently address sequences of block Toeplitz systems with multiple right-hand sides (Jelich et al., 2021). Machine learning- based methods, such as neural operators (Li et al., 2021; Gupta et al., 2021; 2022; Xiao et al., 2022), offer a promising alternative for accelerating the solution of linear systems. Neural operator-assistee Krylov iterations (Luo et al., 2024), for example, have demonstrated significant computational advantages, achieving up 5.5x speedup in computation time and 16.1x reduction in the number of iterations. These advancements underscore the potential of machine learning techniques to address computational bottlenecks in optimal control for FOLTI systems.
C. Why Fractional Order?
According to the well-known definition, the first-order derivative of the function f(t), denoted by D f(t), is defined by
dt(t)
f(t)
f(t - h))
D' s(1)
lim h-50
(31)
dl
h
that is, as the limit of backward difference. Similarly,
d2f(t)
D2 D²t((t== d12
lim h
1 [f(t) (((t)-2((t-)) f(t-2h)
(32)
and
d3 dzt(t)
1
Dd f(t = dt3
= lim
[f(t) - 3f - h) 3f( 2h) -f(t 3h)] f(t-3))
-ff))-f(tt-
(33)
h
33
14

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Iterating n-times,
obtain
d (() dt
11 (11)
DD ((1)
lim h-+0 hm
f(t kh),
(34)
k
k=
where
12 n(n 1) 2) (n +1
(35)
k!
Naturally we can extend the common binomial coefficients for any E R+ by letting (n- 1)!= -))==(n).. So the Riemann- Liouville fractional-order integral can be defined as a consequence of Cauchy's formula for repeated integrals.
Definition C.1 (Riemann-Liouvile fractional-order integral). Given a positive real number a, the Riemann-Liouvilee fractional-order integral is defined as
1 T() = T(a) [ f(u (---))-((()) 1>GGER
(36)
Consider an integer-order derivative operator D (n N). The derivative operator DD is only left-inverse of the integral operator I" such that D"I" = and D"D" # 1. In order to generalize integer-order derivatives, the fractionnal- order derivative operator DD should follow the same left-inverse rule. We show the definition of the Riemann-Liouville fractional-ord order derivative as follows.
Definition C.2 (Riemann-Liouville fractional-order derivative). Let f: R - R be a function and let I = [a, bj be a finite interval on the real axis R. The left-side and right-side fractional-ordee derivatives at DD+ and n. f of order a E R are defined by
d
f(t) dt
(RLDD-f)(1)) [[n a)
Ja (x 1)a-n-I'
(37)
dx
(RI.D (x) I'mn - a)
1
d
a f"
f(t) dt
(38)
dx
(t - x) za==-t1
where n - S a S n, x a (37), and x < b (38).
The Grünwald-LLeninee
fractional-order derivative is defined the limit of finite differences and is mathematically equivalent to the Riemann-Liouville fractional-order derivative. The Grünnaldd-Letniko fractional-order derivative plays an important role establishing fractional-order dynamical systems. The equations for continuous-time fractional-order dynamical system can be written as follows:
H (Dog (gnra)))) (y,, y2, ..) G (am0))) (uI, U2; ... .ux),
(39)
where ys, u; are functions of time and H-)) are the combination laws of the fractional-order derivative operator. single-input single-output L.TI fractional-order system can described by a fractional differential equation of the form
O+ Da Σ D u((),
(40)
with corresponding transfer function of the form:
Y(s) Ex S4
G(s)
(41)
U(s) Ek Saa
where ak. by E IR. The system said to be commensurate o = 3k = ko, where a E IR+, otherwise the system is non-commensurate.
We can use the Grünwald Letnikov fractional-order derivative to rewrite the fractional-order differential equation as follows in the discrete-time case:
a (t) =
by 4 8k utt),
(42)
0
Am=
15

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
with corresponding transfer function of the following form:
Et kk= by (w(2 f
G(z)
(43)
2k=o as (w(==1))
where (===1) ) is the Z transform of the operator 4
D. Theorems and Proofs
D.1. System Identification
We aim estimate the parameters (A, of the following FOLTI system:
k++1
Tk+1 = Aa + Bug
Σ
(a,
(44)
To note, not identify θ jointly make some assumptions during the system identification.
D.1.1. DATA GENERATION
In this step, we generate data following the procedure outlined below:
x = Ard + Bug - s(a, 1)zg + wp
Arb + Bug + C.2y + w/
(45)
of Axd Bug ((a, 1)ag / + wi
Axd + Bug + C.2 + w
(46)
of Any Buy a(a; 1) + w
Ard Bug + C. w/
(47)
where zá the i-th initial conditions, æf represents the corresponding state at the second time step generated from the initial condition, and w are independent and identically distributed (i.i.d)) white Gaussian noise for i I,... P, and C. diag(Os, (11 02...... .g ).).
1.1.2. ESTIMATION
We can write (86) as follows
of Aud + Bug + Cact + wó
+
72
7/
31
31
B1
2
2 +
02
87
3
B2
5/ +
+ w
n
7
On
+ an
8T
3
8m
A
0
[3
0
u
0
+
+ wi
(48)
7n
[._
20
16

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Thus, given the generated data, we solve the following least squares problem:
T
Φ
f
+
+
(49)
pm
w
7
X .. 6B
w
+
(50)
Then we have the following optimization problem
θ arg min || {B
(51)
with the solution
0 = ()) 5 XX 5) -1
(52)
where 0 = [,T,,,3T' and = [T, [0]
Follow (Zhang et al., 2025), we can revise the above strategy and formulate the sample complexity problem as follows:
O1+1 02: X, EB; + wt:
(53)
where the state vector Og = T 1TT and wy is a white Gaussian noise vector NN(0, Kw),, where K is a diagonal covariance matrix, and -= 1, ..... N. Thus, the solution 0 has the following Gaussian distribution:
1 0~ 0, N (<TT
(54)
Notice that the matrix A and the fractional order a are coupled. Therefore, if the diagonal elements of A are known, the fractional order a can be identified by subtracting these constant values from the corresponding positions in the estimated parameter vector 0.
D.2. Proof of Lemma 3.1
Proof. By mathematical induction as follows:
(1) When / 0, .0 Go=r(0] x=O).. When /. 1,,[1]] Gi [0] Bu(0)
(2) Assume we have [A] = Gx- [0) ΣØ fe-1-;Bu(j], when k.
When -k+1. 1.
k Σ
zlk + 1] =
Aza(k: - -5] + Bu(k
1=0
k
k-j-1
Σ
(G [[]
Σ
Buli-)- + Bu[k]
1=
1=0
k-j-1
Σ A, GkCk x0 + Σ
Aj(
Σ
Ga-1-i-;Buli) + Bu(k]
1=0
1=0
k-j-
Gk+12(0| + Σ
Σ
((Gk-1-i-BBu(i)) + Bu[k]
(55)
1=
By expanding the second term, thus
&
xlk + 1] = Gk+ + Σ
BuljB
(56)
j=0
17

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
D.3. Proof of Theorem 3.2
Proof of least-squares solution. Rewrite as follows:
20
0
0
0
BoB
0
0
0
0
1/1
G
B
Go
0
0
1/2
G2
G2B
G,B
Go
U13
+
Ga
(57)
Ør
GT-1B
Gr-2B
Gr-3B
Gr-4B
GoB
UT-1
G1
11
x GU + Hao
(58)
Rewrite follows:
JT( min
min (x
QX + U RU)
(59)
where
Q
0
0
0
Q
0
0
Q
Q
0
0
Q
0
R
0
0
0
R
0 R
0
0
R
0
(60)
:
R
Substitute (59) using (58):
min (T() min {GU + Hxo) (QQU) + Hoo) + U RR}
(61)
&
&
The solution for the least squares problem
QGTQG RYU - TTQ QT H.o
U = - QG + R) - Q Hæo
(62)
Proof of Lagrange multiplier solution.
Lemma D.1 (Lagrange multiplier condition). The Lagrangian function given by:
T-1
f(x, u, A)
Σ
(xT Qak + uh Run + A + Bug A (*x+))) + +x aT-
(63)
and the solution is given by:
1 RRR'BB -
Uk =
T
A = 2Qxx + A 1
Σ
D(aii-M))
(64)
i=
18

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
with zg xe and Ar 22T.
Proof of Lemmal 1. Taking the partial derivative with respect to uk, Ths Ag. have
ac
1 2R + B ARt1 =0 = uk =
-
R- Ak+1,
(65)
Ouk
2
0C =Q + A A Ak+1 Σ
D(a, k)); =0
(66)
0.k
d
2Qxx
A Ax+11 Σ
D(a,i-
(67)
0C
Ark + Bux
A xk+1
(68)
Explanation for Eq. (66): From Eq. (3), we have
A I = Σ
D(a,j)=(kk - -]
(69)
j=0
Then, have
(\ = D(a,0)*,
Dz
(A1+1 x D(a, 1)dxt1
Ōik
0 (AT XT) D(a, T -k)Ar
(70)
Ork
Solve the À jointly and then substitute in (65) to get ug.
Note have the following equations by (56), (65) and (66):
U = R- BT Ak+1,
2
A = 2Q + A A Σ
D(a,i k)),
i=#+1
[k] Gkr(0) Σ
Gk-1-j Bulj]
(71)
1=0
Obeserve that
F
A1
;
A [2QG -Q-
QG,BR-IBT AT - D(a,1)
-D(a, T -]]
(72)
A6+1
Xr
19

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
We have the following matrix form, where Gy and I GG are block Toeplitz matrices (see Definition D.2).
A1
BQ
A ((a,))
-D 2) A D(a 1) QG&BR BB"
D(a, - 1)
A1
A2
BR
-QGBBR-B
D(a, T 2)
22
QQ2BBR
-R
D(a, 33
A
Ar
B
-Q BB
QGoBBR
AT
GA
QG1
QG2
QG3
2
20-
(73)
QGT
Hx
X == GxA+ 2H
-1- G^)A = -2H220
== 2(T Gx)
(74)
Definition D.2 (Block Toeplitz matrix). A block Toeplitz matrix is a block matrix where the block structure follows Toeplitz pattern. Specifically, let g mmmxxm be partitioned into blocks, each of size n. The matrix T is defined as:
4
d1
d (m-1) 1) d
(m-2)
g
Am-1
Am-2
d
where each o E nx represents a block matrix.
Using (65), have
= ---R-BBT 2
-
(75)
D.4. Proof of Theorem 3.3
Proof of least-squares sample complexity result. If we know A and a, then 0 will reduce to vectorize(B) and the distribution reduces to the following:
I vectorize(B) N (ectooize(B)), 1
N (TTT))
(76)
Then
[ J|| = || J11 = =E [|la - all2]
=
T|
R)- p P + " -T-PPPTSSP SP + R) -1 PT
PT
21|2
F(P)
F(P)
< E ([(())F(PPll)
||
(77)
20

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
We rewrite F(P) F(P) as follows:
F(P) - ((P) =
+ (P(PT SP P -P)++ R))1) PT
(PT SP SP+ Rt -1 ( PPT
+ ((P-P)( SP [t)-1 + P ( -1 -PPT -(PTSP+R))')) PT
Pp Rt) (p- -PT)
sP ( PT + +P ( RR) - -P SP (PTSP+ pt
-P-
(78)
Then we have
E
|J J|
E [(()) - F(P) 1/ ||:12
P P SP + R)~ -1 (DT-PT)
S
|1:12
(1)
+ EE
(PP - P) SP + Rt) -
11:12
(11)
S ((T
+
E
P
- (P SP + R)
pT
1
(79)
2
(665)
We bound term (i) as follows:
E p PT sp Rt) -1 -1 (PT- -P))
(i)
s E ||| ( SP + R) 111
(PTSP+
P p1, [1]]
<| E || 2 || | P|] p|/]]
<
(80)
We now use the following:
P p P P
< p P/
+ ||
2
P
P
< p- P
+ 1111))
p p|12 === ||0- pl[ + +11p1|2
p--1 P
P P 2 + |P1|
<
|| [ 11 E
P P
2
-|
E
B B
||P||
1 B
||B - B
+ |PP| 2
B B
F
1
(Tr(Kn) + |PI|2 V Tr(Kn)
|
(Ir(KB) + +|BB12 <T((K))
(81)
21

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
We bound (ii) as follows:
(i)
E
[[[ P) (P SP + R) pT
2.
1 11 E
p - P
| 2
||
I1 B
B - B
||B||
< R
11, E
B B
|B112
<
R 1R1 V
(T(KK) |B|12-
(82)
We bound (iii) as follows:
P (T (PTSP+ -1 - -(PT SP + R) - P
(3)
(PTSPP+
(T SP + 1²) - (PT SP + R2) 7/1]
|P 1 E
(PT SP + R) -1 ( S$ - pT
P PPP (pT (PT + R -1
<
E
PTSP PTsP|]
1 ||
E
ST P) (p - PT) |pI|[]]
|1
||
(pT PT) s(P - P) + PT S(P - P) + -PT) SP sP|[1]
< |1
IE
(T PT) S(P 11|| + || s(P - -ppppp + |P - PT) SP [
< [P|| |||||||
[151]] Tr(KB) + =1 ||5/| <T(KK + 115||2 |BB||2 vT((K))
2 |1 (11S112 Tr(Kn) + + B ||8||2 vT(KK))
(83)
Put everything together, we have the following bound:
E j S1 S 1
(Tr(KB) + Bl1B2 (T((K))
|| -122 <Tr(KB) |B1|2
+
+1 || (15l), Tr(KB) + Tr(KK)+ 21IBB|, ||5||, v TT(KB)
(118~11 ||2 |1B1 || R 12 (|S|])) Tr(KB)
=1=12
(2 |B8|| +2 BBB R 11 [|S||a) v Tr(Kn)
1| R 1112
+
(T<(K)) + (T(KK))
(84)
22

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Proof of Lagrange multiplier sample complexity result.
U - @ (1 - G2)-1Haa
R B
(I - 1H200
R-
DRB
-Daan
R
B
- Gx)
-1 HHA0
R-1
R-1
= - -- (1 Gx) 11230
(85)
J =q H QGU + x H QH
= =-x H QGDRB (I - Gx) HxTo + r H QH
o0 H QG, 1pT PT (I Gx) HANo + xo H Q
(86)
E B 5 = [|5 J11 = E [lla- - al12l
(87)
Then
E ||la a|l2] = E HT -1----1111660-------------111111
Gx) -1 Hxmoola
= ||l H QG,
< Ball H QGal2 Hxmol12
G2) 112 1F
(88)
Then
1PR--PT(I-G-A [|
E
P (1--)))"""'1--------"--00 (PT1--G----PP0 l12
[[I(P
= |I(P |I(P- - Rt P (1-
- -1 l12
= E
|I(P
|I(P- - P P -1 -
- (1
l12]
< E
|I(P- - 1TT 112 +E [ - -
- -2)
(8)
(48)
+
E
PR-1PT
11
(89)
(156)
23

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Let us rewrite G2 follows:
G
Tad + Lu
TB B
QGo
QG1
QGo
+Lu
QGT-1
QGT-2
QGo
BR-1B"
DRRB
Q
Go
0
G1
Go
DBпB + La
:
:
Q
Gr-1
Gr-2
Go
Dq
Lo
-DoL D DBBB + Tuu
(90)
where
0
AT
D(a,1) 1)
D(a, 2)
D(a, T 1)
0
A ((a,1)
D(a, T -)
Tu
0
D(a, 33
(91)
0
Then using the psd ussumption Lod( L,) 0,
(1(1 - 1|2 = |[ - L. + DoLcDBRB)
La
Lm)) + D DRRR)] 112
& I((g (PD - L.) + DBRB)
12
< [[())la)'llal (1D ((Dg I.m)) -1 1|2
-
(92)
We bound (f) as follows:
E [I( - PPP P) G2) |12]
< < [ | PT |2a1(11 - G2) -11= 112
$ 112E ||-- - Plle (1I - Pll2 + |PlL2) |I(T - (
< | |2 |1|-- - Plla (11-- Pll2 + [1lll2) (((22))~(12) Lool L 1|2
<
|
1| E ||B | + E
||
B1l2] [BB1
< [
Tr(Kn) 111
+ (T((K))11B12)
(93)
We bound (ti) as follows:
E
- PPT -T
|B||aa| R [ "Ialllallllll -122 TT(KK).
(94)
24

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
We Bound (iii) follows:
|PR PT (1-G)"' 112]
(ii)
||
- - [3
2 [I
|1 12 [[(1-))-1 (Gs-Ga
(1 Gx) Gx)
LQa [2 - L.) IE
|1Gx Galle]
120 [2l(1 L..) -..) 122
LLQ - D DBnB)BlBB
s 1 Lqq (1) (/ I... 11
| BT -BR B3 112
|0l1) -.)
I(B BB B)
+ (B B + BR- B) BTT112 112
< B
Iw)
(Tr(K))+|BBBI) +
(95)
Using |1(I Im))-1le ||| = and putting everything together we have
E [J J1]
$ H [2 (1 "12
T<KK)) + B111B122 SepPly//rr(/ Tr(KB)
where ||L||2 =
(96)
D.5. Proof of Corollary 3.4 Proof. We first expand K3 as follows
KB
(ØT
To
2
(T)
N
Φ)
N
T
(97)
N
i=1
where
ufub T
Φ Φ
(98)
Di= u6
Then we have
25

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
P
Trace(Kr)
0 E
Trace
Σ
p
Oi
N
n2o N
E
Trace
uiuu
no
Trace
B
Σ u "0
(99)
N
up
1)
Σ uhug
W(I,p), Wishart Distribution (Wishart, 1928)
p
(I)
Un u0
W (I, Inverse Wishart Distribution
I
E
Ug ug
for m + 1.
D m
(1) no?
(Tace(KB)
Trace
N
- m
nmo 2
for p > m +
(100)
N(p
m
E. Experiment Details
E.1. Model Architecture
Our proposed Fractional-Orri Learning for Optimal Control Framework (FOLOC) is a deep learning framework designed for fractional-order optimal control tasks, combining the strengths of nerual operater, sequential modeling, and parameter regression. The architecture is structured into four key components, enabling end-to-end learning complex control dynamics from system states and parameters. Fig. illustrates the overall pipeline.
E. SYSTEM IDENTIHICATION
The system identification module is designed to infer fractional-order time-invariant system parameters, including the fractional-order of size and the constant matrices A and B of sizes n x and n x respectively, introduced in Eq. 6. These parameters are directly inferred from the observed control imput space X, which is formed by concatenating and lk. where 2& E IR" the state vector and uy E R is the system input. This component leverages hybrid architecture to accommodate both static and temporally correlated systems. Specifically, we use an MLP with residual skip connections and a Sequence Model (e.g., Recurrent Neural Networks, Gated Recurrent Unit Networks, or Long Short-Term Memory Networks) as the backbone of system identification module.
For Mone-Mrrovvin systems with long-range rang-range temporal temporal dependencies, sequential encoders such as bidirectional LSTMs, GRUs, or RNNs-an are adopted. These models process the input sequence autoregressively, capturing latent temporal patterns through hidden state evolution. Bidirectional processing further enables context aggregation from both past and future observations within the horizon T. The regressor outputs a flattened tensor, which is decomposed into A, and a via learned linear projections, ensuring dimensional consistency with the underlying physical system. We formalize this module as follows
26

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
For non-Markovian systems, bidirectional LSTMs model temporal dependencies:
hgg C2 = LSTM((I= ue], (h4-1, c--1)),
(101)
where h2 E 2dd (bidirectional hidden states). The final state hy aggregates sequence information.
Then apply a stack of residual MLP blocks processes final state hr:
h (0) hr, h() BatchNNrm a (W h(t-1) (y-1) + +b0) h(-
(102)
where a is an activation function (e.g., ReLU) and 1,... L indexes the residual blocks.
The regressor outputs a flattened tensor R²* +umt,, which can be decomposed into A, and a
0 = W (o)ch
(103)
A = Reshapee001p (1n.). = Reshape -1mm=-mmmmmm-mmmm = + 1:a0
(104)
The module minimizes multi-task regression loss:
C. | 1 +
IB
B² + la a²1
B
(105)
This module is trained under a multi-task objective, jointly optimizing the reconstruction of system parameters while preserving their interpretable structure (e.g., enforcing A as state transition matrix). By unifying static and sequential modeling paradigms, the regressor adapts to diverse dynamical regimes, laying a foundation for downstream control synthesis.
E.1.2. TEMPORAL EMBEDDING FOR A MATRIX EVOLUTION
In order to obtain the temporal embedding of matrix A, we propose an embedding estimation module that transforms the estimated fractional order system parameters into time-aware representations. These representations capture both the spectral properties of the dynamics and their temporal evolution. By addressing the challenge of modeling parameter drift in partially observed systems, this approach maintains physical consistency throughout.
Our architecture implements novel parameterization of the system matrix A, explicitly encoding hierarchical damping effects. This allows for both stability guarantees and adaptive temporal evolution. Building Eq. 15, the module constructs time-dependent embeddings Acmb E d through a dual-branch architecture.
Let A E x and E R" denote the estimated constant matrix and fractional-orde from the system identification module. To compute the embedding the first token Ag, we decompose the system matrix A₂ = A diag(a) through spectral analysis:
A, -VAV-! A diag(A).......... An). hapec o (W,
(106)
(107)
where X E are eigenvalues encoding system stability. A residual MLP processes the concatenated real/imaginary components (Re(A),Im(A) to produce base embeddings Ao-
In parallel, time embeddings e generated via embedding layer that encodes sequential time indices. These concatenated with embeddings derived from enabling the model to capture both temporal and parametric variations:
hestate (ex) @ (a),, 2a Oa Linear Projections.
(108)
Next, the adaptive fused embeddings pass through series of stacked residual multi-layer perceptron (MLP) blocks with normalization and an activation function (e.g., ReLU or GELU). This produces the final embeddings that serve as inputs to the temporal processor. This residual architecture enhances the model's capacity to learn complex representations while preserving information from earlier layers:
emb CeeMLP((hsppe hstuneli W1, br(=2)) Ambb E g Xxhiddenn size
(109)
27

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
This embedding mechanism enables the model to generate time-dependent parameter representations without directly relying the input state vector By encoding dynamie properties through eigenvalues and temporal embeddings, the model effectively captures system behavior over time, making it suitable for tasks such time-series forecasting and control parameter prediction.
1.1. SEQUENCE ENCODER PROCESSOR
The Sequence Processor translates ppectral-tempori embeddings 4 into latent states Gt that encode the system's temporal evolution, enabling adaptive control under time-varying dynamics. The sequence processor computes latent states Gt through recursive hierarchical structure, enabling efficient modeling of temporal dependencies while preserving stability, rooted in the governing Eq. 14.
we implements this recurrence through a deep encoder architecture, which projects the raw recursive computation into latent space. For systems requiring global temporal context, we use Transformer as backbone:
G Trans former f EncoderAA"" Pk),
(110)
where Pa is positional encoding encoding the step k. Multi-head self-attention layers implicitly approximate the summation in Eq. 14 by learning attention weights that mirror the hierarchy Ay.
1.4. STACK MLPs FOR FOURIER NEURAL OPERATOR
The final stage of our architecture synthesizes hierarchical system representations into spatiotemporal control signals through two-step process: feature unification via the Stack MLPs and spectral control synthesis via the Fourier Neural Operator (FNO). This combination enables efficient learning of distributed control laws while preserving physical consistency, Given temporal states G system parameters B, LQR cost matrices Q,R. and residual skip connection input [z; u]. the transformation constructs an enriched imput tensor X' E R Tx for the FNO:
X = (G.) ΦB B) (Q( on R) de 4
(111)
where 6 are series stacked residual multi-layer perceptron (MLP) blocks and @ denotes channel-wiss concatenation. Next, The FNO processes X through spectral convolutions and residual connections as we introduced Section 3.3:
(k) W(k) x(k) + b(k), k zd
(112)
where x(k) = F(x) is the Fourier transform of the input, and W, are learnable parameters frequency space. Only low-frequency modes ||k|| kmax are retained, enforeing spectral sparsity.
Then, apply the inverse transformation to predict the optimal control U:
L
F=F1
II [(( + MLP(X),
(113)
where K₁ are local kernel integrations in physical space.
The module minimizes the deviations between predicted U, and optimal control trajectories U/:
C. Tm
1 1
Σ U.
(114)
L.5. COMPOSITE LOss FORMULATION
The training objective combines fractional-orde system parameter regression and control prediction errors through a multi-task loss function, enabling joint optimization of system identification and control synthesis:
Liotal
L. System Inlentiicatiion
C. Control Prediction
(115)
28

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
This unified framework successfully marries physics-guided feature engineering with data-driven spectral learning, enabling real-time optimal control in high-dimensional dynamical systems and the unified loss formulation enables end-to-end learning of control policies while maintaining physically consistent parameter estimates- critical requirement for deployment in safety-critical dynamical systems.
remark: Specifically, the system parameters A, B and a can be extracted from an intermediate layer in the network because of the underlying loss function. The intermediate variable As approximated using a shallow neural network P such that A& (A, a, t) and the variable G is reduced R" instead of "=""" Similarly, the cost matrices Q and transformed into reduced representations via local transformation operators Ni : - The block Toeplitz structure of G indicates that it can be fully represented using the parameters Q, R. A, B, a, Zu and )== Using this property, the input dimension of the FNO is determined as d. 7 x (2n + m + de + dn dn). where the extra terms n and m correspond to the trajectory and control input dimensions, respectively. These additional features empirically enhance the performance of the FNO by providing more comprehensive representations of the underlying dynamics.
IR .uR : gg + R
E.2. Synthetic Data
The synthetic data is generated as follows:
State Transition Matrix (A): Rondomly sampled from uniform distribution and normalized to ensure stability by scaling with its spectral radius.
Control Input Matrix (B): Randomly sampled from uniform distribution.
Initial State (zn):: Randomly sampled from standard normal distribution.
Control Inputs (u): Sequence of uniformly random inputs.
Cost Matrices (Q, R): Symmetric positive remi-definite matrices constructed via M T M for some uniformly random M.
E.3. Real-Worlee System Dynamics
In our experiments, we simulate the system dynamics to generate trajectories and optimal control sequences by randomly choosing an initial state and control inputs and then using model predictive control (MPC) to find the optimal control sequence given the quadratic cost matrices Q and R.
E. CART POLE SYSTEM DYNAMICS
The dynamics of the cart pole as follows:
Mipl 02 sin 0 - 0 coos) + F
me + Ml
i g sin @ + cos -m,,60" sin F-F
(116)
1 my cass 0
mm+mp
where ä is the horizontal acceleration of the cart, Ø is the acceleration of the pole, m. is the mass of the cart, my is the mass of the pole, and g is the acceleration due gravity. The control input is the external force F applied to the center of mass of the cart.
E.3.2. QUADROTOR SYSTEM DYNAMICS
The quadrotor dynamics is as follows:
p U, 1
0
q 2 q
w
mZ mg + f + far
Tw - w +
(117)
29

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Here, p E RR and v R represent the position and velocity in the inertial frame, w R is the angular velocity, and the quaternion q descrihes the orientation with & denoting quaternion multiplication. The mass of the quadrotor is given by m, and J denotes the inertia matrix. Gravity is denoted by g. the aerodynamic force is denoted by fa E IR3, the total thrust and body torques are denoted by f E R and E R²
F. Training Parameters
The experiments utilizing GPU acceleration on an NVIDIA A100 80GB PCIe GPU. The experiments are conducted on a machine running Ubuntu 22.04.5 LTS with Intel(R) Xeon(R) Platinum 8358 CPU (2.60 GHz), featuring 128 cores and support for 256 concurrent threads.
The training process is configured to optimize model performance using the Adam optimizer with a learning rate of 10 The model is trained for 300 epochs with a batch size of 128, Learning rate scheduling is managed by a ReduceLROnP lateau scheduler, which monitors validation loss and reduces the learning rate by factor of 0.1 after 5 epochs without improvement. The scheduler uses a relative threshold of 0.0001 to detect improvement. The minimum learning rate set to zero, and early learning stabilization facilitated by an epsilon value of 1 10-# 8 The loss function based on the Ly-norm.
To ensure robust training, data normalization is applied, with both the input data and control signals (U) included in the modeling process. The architecture supports both encoder-only and sequence-to-sequeee configurations; however, in this setup, the sequence-to-sequenee mode is disabled.
G. Metrics
Mean Squared Error (MSE). The Mean Squared Error is given by
n
CMSE (0. 3) =
(y) - 0.)
(118)
where n denotes the number of samples, y1 E R is the ground truth value, and de E IR is the predicted value for the i-th sample.
Mean Absolute Error (MAE). The Mean Absolute Error is given by
LMAR (ū. v)
My - Del,
(119)
where Vi. and ús are defined as in the MSE formulation.
H. Ablation Study
In this section, we provide further analyses to uncover how the choice of encoder models and weight of identification loss affect the performance of the FOLOC framework.
H.1. Varying Sequence Encoder Module
To investigate the performance of different encoder layers on fractional-order system, we choose four sequence encoder Tiiiant-Trrasfommee RNN, LSTM, and GRU. We also test FOLOC framework by replacing the sequence encoder module with MLP. Our analysis of sequence encoder architectures reveals statistically significant differences in performance and stability across models. The results are given in Table 7. The Transformer encoder demonstrates superior performance, achieving the lowest mean squared error (MSE = 0.00495 + 0.00010) with minimal variance, underscoring its robustness to initialization. These results highlight the ability of our model's self-attention mechanisms to handle tasks involving global temporal dependencies. Based on our experiments, the Transformer architecture mitigates the risk of gradient vanishing or explosion by distributing information through attention, effectively stabilizing the optimization process and demonstrating strong suitability for safety-critical control tasks.
30

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Table 7. Ablation study of different sequence encoders.
Model
MSE (10")
MAE (102)
Transformer
4.95 + 0.10
1.05 + 0.10
RNN
6,00 + 1.46
1.15 0.17
LSTM
5.54 1.49
1.12 0.18
GRU
6.78 2.36
1.29 0.34
MLP
5.91 + 1.57
1.20 + 0.21
Table 8. Varying identification weight. All reported values in units of 10-3
Metries
Aw
0.1
Aw =
Aw
0.3
Au 0.5
MSE
7.54#3.11
4.95+0.10
4.89:10144
5.19
5.84+1.36
5.08=0.29
MAE
14.190.39
10.5+0.10
10.3::007
11.0=0.83
11.4+1.69
11.1=0.54
H.2. Varying Identification Weight
We evaluate the performance of the FOLOC framework by varying the system identification weight Aw the values {0, 1, 0.2, 0.3, 0.4, 0.5]. Experimental results show that FOLOC's performance remains approximately the same, with mean MSE of 0.56% (standard deviation: 0.10%) and a mean MAE of 1.14% (standard deviation: 0.14%). The results are given in Table 8.
I. Baseline
To the best of our knowledge, existing end-to-end deep learning framework addresses the optimal control problem under fractional-order dynamical systems, is there theoretical framework that integrates end-to-end learning with FOLTI system identification and optimal control laws. Due to the principle that fractional-order calculus generalizes integer-order calculus, we did some experiments to compare our proposed methods with the traditional theoretical end-to-end approach for optimal control LTI systems. The end-to-end learning framework for LTI systems involves system identification using ordinary least squares, followed by solving the optimal control problem through dynamic programming (Anderson Moore, 2007).
J. Additional Results
J.1. Test Loss of Different Time Horizons
We present the MSE test loss across different time horizons shown in Fig. 5, reveals critical insights into its temporal generalization capabilities. With performance steadily improving as the horizon extends, it reflects the FOLOC framework capacity to suppress high-frequency errors through spectral filtering while gradually resolving low-frequency dynamics. 128 not only shows a smooth reduction in MSE but also achieves the lowest MSE loss among tested horizons, indicating that the model better captures long-range temporal dependencies when provided with more context. The finding that extending the prediction window enhances both model robustness and accuracy confirms that, when given sufficient historical information, our approach is better equipped to handle complex dynamics of fractional-order systems.
J.2. Results of Different Fractional-Order Systems
To evaluate the model's robustness across diverse fractional-order systems, we tested FOLOC framework on dataset constructed from 10,000 unique systems with varying parameters. Our approach demonstrates consistent performance this heterogeneous evaluation set, achieving a mean squared error (MSE) of (8.0025 + 1.1420) x 10-3 and mean absolute eтror (MAE) of (9.4546 + 0.7698) x 10 10-3, indicating stable generalization despite substantial system diversity. These results reflect the architecture's 's capacity to adapt to different fractional-orda systems without task-specific fine-tuning. Such findings align with thcoretical expectations, as the spectral-temporrs embeddings and the FOLOC framework inherently decouple system-speciffe features from control synthesis, enabling robust distrrbbuutoono operation.
31

---

End-to-End Learning Framework for Solving Non-Markovian Optimal Control
10"
T-16
T-32
T=64
10 1.
Loss
10
50
100
125
150
175
200
Epoch
Figure 5. Test MSE loss under each epochs.
J.3. Comparison of Inference Time
The experimental results demonstrate that FOLOC framework achieves superior computational efficiency compares LTI model. Specifically, FOLOC framework requires only 1.215 per sample for inference, which represents significant 44.6% reduction in processing time compared to LTI model's 2.1935 ms per sample.
Table 9. Model inference time comparison.
Model
Inference time (ms/sample)
LTI Model
2.1935
FOLOC
1.215
32


Reviews:
[{"review_id": "OhUXaJ0a8K", "reviewer": "Reviewer_GKDj", "summary": "This paper develops a new end-to-end learning approach for fractional order linear time-varying systems to perform fractional order LQR. A method is proposed to identify the system dynamics from multiple recorded trajectories. Using the identified model, a fractional order LQR controller is designed to obtain an approximately optimal controllers. The method uses end to end deep learning and learns both the fractional order system dynamics and the fractional order LQR controller.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive feedback. We are especially grateful for the recognition of our theoretical contributions and the clarity of our proofs.  \n\n**Complex benchmark systems**: Thank you for your thoughtful comment regarding the applicability of our method to real-world systems. While our current experiments focus on benchmark systems such as the cart-pole and quadrotor for clarity and reproducibility, our approach is broadly motivated by the need to model **complex real-world phenomena** that have memory effects, non-Markovian behavior, and other complex nonlinearities. Fractional-order dynamical systems (FOLTI systems) are well-suited for capturing such dynamics and have been shown to outperform integer-order models in various application domains. We provide the case study for **turbulence control** [1,2] derived from the incompressible Navier–Stokes equations as follows:\n| Metrics             | MSE               | MAE             | \n|---------------------|--------------------|------------------|\n| PID  | 0.5705  | 0.3311 | \n| MBRL | 1.61  | 0.902 | \n| **FOLOC(ours)** | 0.5119  |  0.2012   |\n\nPrior work has also demonstrated the effectiveness of fractional-order models in: (i) EEG signal [3], where non-Markovian dependencies are critical for accurate representation, (ii) Power system dynamics [4], where Phasor Measurement Unit (PMU) data exhibit long-range temporal correlations better captured by ARFIMA models with non-integer differencing parameters, (iii) Chronic disease modeling, such as chronic obtrusive pulmonary disease (COPD) progression prediction [6], where fractional-order deep learning frameworks improve predictive accuracy, (iv) Biological control systems, including artificial pancreas control and pacemaker regulation of heart rate [5,7], where fractional-order models provide a more faithful representation of physiological dynamics. These examples reflect the broad relevance of fractional-order modeling in scenarios where the underlying physics is inherently complex (memory effects). Our method is the first end-to-end, deep learning-based approach designed to solve optimal control problems in fractional-order dynamical systems, and it can be directly applied to a wide range of real-world applications. We also evaluate our method on very high-dimensional synthetic data and demonstrate its scalability, due to space constraints, we kindly refer the reviewer to our detailed response to **Reviewer cBiX** for evaluation results.  \n\n**Baselines**:  Thank you for highlighting the importance of baseline comparisons. We initially excluded them because standard baselines require system parameters, which our method does not. However, we agree that including them strengthens the evaluation. We have conducted extensive baseline comparisons on the cart-pole and quadrotor systems, detailed in our response to **Reviewer cBiX**. We hope these results clarify the advantages of our approach.  \n\n[1] Holmes, Philip. Turbulence, coherent structures, dynamical systems and symmetry. Cambridge university press, 2012.  \n[2] Li, Zhijin, et al. \"Optimal control of cylinder wakes via suction and blowing.\" Computers & Fluids 32.2 (2003): 149-171.  \n[3] Besançon,  et al. \"Fractional-order modeling and identification for a phantom EEG system.\" IEEE Transactions on Control Systems Technology 28.1 (2019): 130-138.  \n[4] Saadatmand, et al. \"PMU-based FOPID controller of large-scale wind-PV farms for LFO damping in smart grid.\" IEEE Access 9 (2021): 94953-94969.  \n[5] Gupta, Gaurav, et al. \"Non-markovian reinforcement learning using fractional dynamics.\" 2021 60th IEEE Conference on Decision and Control (CDC). IEEE, 2021.  \n[6] Yin, Chenzhong, et al. \"Fractional dynamics foster deep learning of COPD stage prediction.\" Advanced Science 10.12 (2023): 2203485.  \n[7] Bogdan, Paul, et al. \"Pacemaker control of heart rate variability: A cyber physical system perspective.\" ACM Transactions on Embedded Computing Systems (TECS) 12.1s (2013): 1-22.", "claims_and_evidence": "The development, analysis, and experiments support the claims made by the authors in the introduction.", "methods_and_evaluation_criteria": "Yes, the evaluation criteria makes sense for synthetic datasets in the experiments.", "theoretical_claims": "Yes, the theoretical claims appear valid. I checked the proofs of theorems in the Appendix. The proofs are well written and I enjoyed reading them.", "experimental_designs_or_analyses": "Yes. Overall the experimental design is valid. However, I have the following concerns:\n\n1. The experiments section can be improved by including more complex benchmark systems beyond the cart pole and the quadrotor. Using a complicated framework like fractional order calculus and deep learning to model real world phenomena makes sense when the underlying physics involves equally complicated nonlinearities, e.g., friction, turbulent aerodynamics, hysteresis or related memory effects, contact mechanics, fluid structure interaction etc. that may be difficult to model using integer order. Otherwise the method is not as well-motivated for real-world applications.\n\n2. No baselines are provided for comparison in the experimental results. The authors state there is no existing deep learning framework that can be used as a baseline for fractional order systems. However, the cart pole and quadrotor are real-world physical systems and the fractional order approach is just a model of the real world nonlinearities. There are plenty of controllers tried and tested on these real world systems. Having comparisons with some of the standard baselines (e.g., integer order equivalents, Koopman operator-based methods, MPC etc) will highly improve the quality of the paper.", "relation_to_broader_scientific_literature": "The topic of this paper is unique and niche, so there is not a lot of literature surrounding this area. The literature survey is sufficient to put the contribution in context.", "essential_references_not_discussed": "None that I know of", "other_strengths_and_weaknesses": "None, see above comments.", "other_comments_or_suggestions": "None, see above comments.", "questions_for_authors": "None, see above comments.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2xuncoJhw0", "reviewer": "Reviewer_cBiX", "summary": "In summary, the paper contributes a novel end-to-end, data-driven framework for optimal control of non-Markovian (fractional-order) systems by uniting advanced control theory with deep learning techniques, backed by both theoretical guarantees and empirical performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for the recognition of our **novel** theoretical contributions, the FOLOC framework, and the experimental design, as well as highlighting its effectiveness and robustness. We also thank the reviewer for pointing out areas where additional **empirical comparisons** and **runtime benchmarks** could further strengthen our work. We will address these points in detail below and incorporate the suggestions in the revised version.  \n\n**High-dimensional settings**: Thank you for your valuable comment regarding the scalability of our FOLOC framework in high-dimensional settings. In response, we have added a new experiment to specifically evaluate performance in such scenarios. The results, summarized in the table below, demonstrate that our method remains efficient and robust as the dimensionality increases. (All reported values are in units of $10^{-3}$.)  \n\n***Table 1: Experiments on high dimensional datasets***\n| Metrics             | MSE               | MAE             | \n|---------------------|--------------------|------------------|\n| Dimension 16   | 3.85 ± 0.05  | 12.7 ± 0.08 | \n| Dimension 32   | 2.8 ± 0.04    | 11.0 ± 0.07 |          \n| Dimension 64   | 1.8 ± 0.001  |8.39 ± 0.007 | \n| Dimension 128 |  1.43 ± 0.01  |  7.27 ±  0.03   |\n\n**Baselines**: Thank you for your thoughtful comment on runtime comparisons. We conducted detailed benchmarks comparing FOLOC with several popular baselines, including **MBRL**, **CMA-ES**, **MPC**, **PID**, and **iLQR**. For each method, we report both performance and runtime. FOLOC consistently achieves the highest accuracy and lowest computation time on both cart-pole and quadrotor tasks. We found that FOLOC achieves superior performance on cart-pole dynamics, with MSE reduced by up to 99.99% and MAE by up to 99.32%. All experiment code is available at the link provided in Appendix A (Page 12).\n\n***Table 2: Baseline Model Evaluation on cart-pole dynamics***\n| Metrics             | MSE               | MAE             |Run Time (s)     |\n|---------------------|--------------------|------------------|----------------------|\n| MBRL              |  0.1949245466         |   0.4106739015   |         0.0445      |             \n| CMA-ES          |    0.236764904      |  0.4723616957           |      16.405419      |\n| MPC             |   0.2369265688           |    0.4725790307          |   0.673999        |\n| PID                 |      0.2542367969       |   0.498872839          |  0.0012775         |\n| iLQR              |         0.2380454333       |      0.4741794493      |     0.26275     |     \n| **FOLOC(ours)** |         0.000038933    |      0.003350039      |     0.000371     |\n\n---\n\n***Table 3: Baseline Model Evaluation on quadrotor dynamics***\n| Metrics             | MSE               | MAE             |Run Time (s)     |\n|---------------------|--------------------|------------------|------------------|\n| MBRL              |  0.4893634967         |   0.5455600058  |         0.0445      |             \n| CMA-ES          |    0.6133190948       |   0.5897855761         |      16.405419      |\n| MPC             |   0.6117894105           |    0.5521787351        |   0.673999        |\n| PID                 |       1.143392167       |   0.9088121938          |  0.0012775         |\n| iLQR              |         0.6277251605       |     0.570096181     |     0.26275     | \n| **FOLOC(ours)** |        0.211356634    |     0.126771140      |     0.000267     |\n\n**Header**: We appreciate the reviewer for pointing out the heading issue, we will ensure to correct it in the revised version.", "claims_and_evidence": "The paper provides a strong combination of rigorous theoretical derivations, algorithmic formulations, and empirical experiments that support many of its key claims. In particular, the extension of LQR theory to fractional-order systems is well-founded, and the proposed FOLOC framework is validated on both synthetic and real-world benchmarks. The sample complexity analysis also provides theoretical backing to the learning guarantees.\n\nWhile the experiments demonstrate robustness and efficiency on moderate-dimensional systems, further evidence would be valuable to confirm that the framework scales effectively to very high-dimensional settings. Additional experiments or comparisons in such regimes would strengthen this claim. The paper asserts that the end-to-end framework enables computational efficiency and fast, real-time control evaluation. Although the empirical results indicate computational advantages, more detailed benchmarks or runtime comparisons with existing methods would provide clearer support for this claim.", "methods_and_evaluation_criteria": "I think the proposed method makes sense for the scope of problems considered in the paper, i.e. solving for optimal control for fractional-order dynamical systems.", "theoretical_claims": "I checked the corrctness of Lemma 3.1 and Theorem 3.2 but I did not look at the details of the proof for Theorem 3.3 and Colloary 3.4.", "experimental_designs_or_analyses": "I think the experimental designs are nice but it may strengthen the paper if the authors can provide comparisons to some existing methods.", "relation_to_broader_scientific_literature": "I do not know much about the FOLTI systems and the methods for solving for optimal control of such systems. So I am not able to comment on the contributions of the paper related to the broader scientific literature.", "essential_references_not_discussed": "As said before, I am not familiar with the literature of the FOLTI systems, so I cannot provide anything specific about possible missing references. However, I find that the authors provide a great literature review in the supplemental material.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "I am not sure why the header of the paper is \"Submission and Formatting Instructions for ICML 2025\". Maybe it's a formatting misprint and I hope the authors will correct it in the revised draft.", "questions_for_authors": "I do not have any questions for the authors.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2w0LCXUxYE", "reviewer": "Reviewer_2GMe", "summary": "This paper devises a learning-based approach to solve linear quadratic regulator (LQR) problems for fractional-order linear time-invariant systems. Here, the fractional order means the system state transition is not entirely dependent on the current state but also depends on previous states (Eq(7) in the paper). This approach involves two procedures: system identification and controller synthesis. In system-ID, given a set of traces from the plant runs, the key coefficients for the system dynamics (A, B matrices) and the fractional factor ($\\alpha$) are estimated. In the control synthesis step, given the A, B, $\\alpha$, and LQR spec ($Q$, $R$ matrix), the control sequence can be derived using Fourier Neural Operators (FNO). A theorem regarding sample complexity is given with proof in the paper. The authors conduct experiments in both synthetic data and two nonlinear control problems to demonstrate that the proposed method is scalable, robust, and computation-efficient.\n\n## update after rebuttal\nI have read the author's response as well as other reviewers' comments. Overall, I think the work is novel and is okay to be accepted by ICML, though a few flaws exist. I updated my final ratings as \"weak accept.\"", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful reading and constructive suggestions. Below, we provide a detailed response regarding your concerns and questions.  \n\n**Baselines**: As detailed in our response to **Reviewer cBiX**, we have conducted extensive comparisons between FOLOC and several widely used methods, including MBRL, CMA-ES, MPC, PID, and iLQR.  \n\n**Why learning based LQR?** (i) FOLOC generalizes across different FOLTI systems (Appendix J.2), predicting system parameters and optimal controls (OCs) directly from trajectories. **System parameters are not used during testing on synthetic data, and are not used entirely during both training and testing on real-world dynamics**. (ii) It enables fast and accurate real-time control. Although closed-form solutions (Eqs. 16 and 17) exist given full system knowledge, they involve large-scale matrix inversions and become impractical for long horizons or high-dimensional systems. MPC, though an alternative, also struggles with the memory effects in fractional-order systems. FOLOC overcomes these challenges by learning a direct mapping from trajectories and cost matrices to OCs.  \n\n**Figures**: We modified our manuscript to address your concerns about figures: we (i) clarified in Fig. 2’s caption that the sample complexity refers to the Lagrange multiplier-based solution, (ii) updated Fig. 4 by removing the secondary y-axis and aligning both error metrics on a shared axis, and (iii) revised Fig. 1’s notation in alignment with Eq. (29).  \n\n**Joint optimization pipeline**: The sys-id loss is a physical constraint for our model that encourages the model to accurately identify the system, further ensuring accurate control, the ultimate goal of our method. During the training period, we use the sys-id label as guidance to ensure accurate sys-id, and **we do not need these labels in the test period**. Some methods, such as PINN [1], couple multiple loss terms by incorporating physical constraint losses. This helps the system learn representations that are both physically grounded and task-relevant, reducing error propagation and avoiding reliance on purely minimizing prediction error. For real-world datasets where the system parameters are unknown, we can first pretrain the model on synthetic datasets with known parameters(**$A$, $B$, $\\alpha$**). Then, we do not need sys-id labels(set sys-id loss weight $λ=0$) and finetune the model on the real-world dataset. We'll provide results for $λ=0$ for comparison in Appendix H.1.\n|Metrics|MSE(×10⁻³)|MAE(×10⁻²)|\n|-|-|-|\n|$λ=0$|7.54±3.11|1.41±0.39|\n\n**Notations $p$ and $N$**: We have clarified in the revised version that $p$ denotes the number of initial conditions, and $N$ denotes the number of trials adding Gaussian noise to fractional-order dynamics from these $p$ initial conditions.   \n\n**Assumption**: This is a strong assumption and represents a limitation of the theoretical sys-id approach. It serves a similar role to assumptions commonly made in classical sys-id, where the exact (integer) order of the system is often presumed known in advance. In the context of fractional-order systems, estimating the system order—i.e., the fractional derivatives—is substantially more challenging, and to the best of our knowledge, no existing method provides theoretical guarantees in a general setting. To address this limitation, we propose FOLOC framework, which can give accurate OC solutions without this assumption.  \n\n**Performance decrease**: The decrease of the performance may be caused by the **Joint training loss trade-off** and **Stochastic training dynamics and over-regularization**, Despite this the variation in MSE/MAE across training sizes is small (e.g., 5.18 → 5.22 × 10⁻³ in MSE) indicate that the model exhibits both robustness and stability across different settings even with a few sample data\nThe results are as follows, which show a significant drop in performance using fewer training samples.(i.e., 500) (All reported values are in units of 10⁻³.)\n|Metrics|MSE|MAE|\n|-|-|-|\n|$N_{T}$ = 500|10.5±3.17|22.8±1.35|\n\n**Designing of model architecture**: Align with theoretical derivation in Eqs. (13)–(15), The system parameters($A, B, \\alpha$) in physical equations of the fractional-order system reveal a recursive relationship within the sequence. We want to use RNN-based to capture the recursive dependency across time steps. In Eq. (13), the sequence $A_j$ is derived from the system parameters A and $\\alpha$, and we employ a sequential model to estimate this computation. We also experimented with replacing the RNN with MLP to predict sys-id parameters. The performance degradation highlights the necessity of using RNN-based models.  (All reported values are in units of 10⁻³.)\n|Metrics|MSE|MAE|\n|-|-|-|\n|RNN + MLP|4.95±0.09|10.5±0.10|\n|MLP Only|5.91±1.57|12.0±2.12|\n\n[1] Raissi, et al. \"Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.\"", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes. I checked all of them.", "experimental_designs_or_analyses": "Yes. I checked all of them.", "relation_to_broader_scientific_literature": "This paper proposes a novel learning-based approach to control dynamical systems with long-term time dependencies, which can potentially solve non-Markovian problems. The experiments in this paper go beyond simple linear dynamics and consider high-dimension nonlinear systems, which shows potential adaptation to a broader range of applications.", "essential_references_not_discussed": "Not that I am aware of.", "other_strengths_and_weaknesses": "Pros:\n1. The proposed method is novel, and I see a good combination of ML with the control theory domain.\n2. Theoretical guarantees are provided.\n\nCons:\n1. Clarity: The paper is not very easy to understand. I have put some of my comments/questions in the sections below.\n2. Lack baselines: I don't see a clear advantage of the proposed approach - why should we use this method instead of some other existing methods? There are many ways to stabilize a system, especially for systems used in this paper, like the cart pole and quadrotor. I suggest the authors to compare to some of the representatives (I could name a few, model-free/model-based RL, CEM/CMA-ES, MPC, PID, iLQR, Hamilton–Jacobi–Bellman equations; but you are encouraged to try more).\n3. OOD: This approach requires sys-id from trajectories, so the training trajectory distribution should support well around the LQR-trajectory distribution - which might not be possible if you don't start with a good MPC to roll out these demonstration trajectories. But if you can have a good MPC to drive the system to track on target, why bother to use this learning-based LQR? This fact limits the application to a more general setup.", "other_comments_or_suggestions": "For Figure 2, you might want to have a recap for the complexity you mentioned - and also in Theorem 3.3, you mentioned two complexities (one for least-square fitting; the other one for the Lagrange-multiplier solution). Which one do you refer to here?\n\nFor Figure 4, you don't need two separate y-axis, since they are both errors, and their scale ranges are roughly the same (also I notice that they are actually not aligned well - if you compare the positions of \"0.00\" for the two measures, you will find out they actually not on the same horizontal level - similar things happens for the \"0.10\" value - this is not rigorous when plotting something like this).\n\nThe notation is not very consistent: in figure 1, you use $\\hat{U}^o$ for network output and $U^o$ for precomputed one, but in Eq(29) you use $U$ for network output and $\\hat{U}_\\theta$ for the pre-computed one.", "questions_for_authors": "I am still confused about the general pipeline: so at the beginning, you only have a bunch of trajectories. Then you estimate the A, B, $\\alpha$ using least-square, and then you can compute the optimal control sequence $U^{\\theta}$ using Eq(16) or Eq(17). Based on your loss in Eq(29), you need to have these (the A,B,$\\alpha$ and $U$) ready before learning the NN. Then why do you still need to learn anything using the NN?\n\nLine 212-213: \"where $p$ and $N$ denote the numbers of samples...\" What do you mean here? Are they both denoting for the number of samples?\n\n\"...and assuming the diagonal elements of the system matrix A are known...\" If you are doing sys-id, why can we assume this in real-world applications? Can you comment on how strong/weak this assumption is?\n\nWhy not train the two stages (sys-id; control-synthesis) separately? I don't see they are coupled in any way, so you should be able to first train sys-id and then train the controller. Using $\\lambda$ here in Eq(28) will trade off both performances.\n\nI don't understand in Table1 the last row. Why will increasing the sample size decrease the performance? Can you explain this?\n\nWhy do you need a separate SEQ model to handle tokens since you already have an RNN structure to handle temporal data? I don't understand the design choice of blending both RNN and Transformer-based sequence encoder. Do you have ablation studies to support this design choice? (I saw in appendix-Table4 you compare different SEQ model archs I guess, but I don't the result to support your usage of RNN in the pipeline).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper developed a learning-based approach to solve the optimal control problems for fractional-order linear time-invariant systems. A two-step algorithm was developed, with system identification followed by controller synthesis. Sample complexity results were then established, followed by experimental results on canonical control problems. The results are overall solid and new in the learning for control domain, and the problem being considered is important. There was some common concern regarding the sufficiency of the baselines, which was addressed in part by the rebuttal. There were also some other concerns regarding the motivation, scalability issues of the algorithm, and clarity of the paper. I suggest that the authors incorporate the feedback in preparing the camera-ready version of the paper.
Decision: Accept (poster)