Formal Methods in Robot Policy Learning and Verification: A Survey on Current Techniques and Future Directions
Authors: Anastasios Manganaris
Vittorio Giammarino
Ahmed H Qureshi
Suresh Jagannathan
Venue: Accepted by TMLR ()

Abstract: As hardware and software systems have grown in complexity, formal methods have been indispensable tools for rigorously specifying acceptable behaviors, synthesizing programs to meet these specifications, and validating the correctness of existing programs. In the field of robotics, a similar trend of rising complexity has emerged, driven in large part by the adoption of deep learning. While this shift has enabled the development of highly performant robot policies, their implementation as deep neural networks has posed challenges to traditional formal analysis, leading to models that are inflexible, fragile, and difficult to interpret. In response, the robotics community has introduced new formal and semi-formal methods to support the precise specification of complex objectives, guide the learning process to achieve them,and enable the verification of learned policies against them. In this survey, we provide a comprehensive overview of how formal methods have been used in recent robot learning research. We organize our discussion around two pillars: policy learning and policy verification. For both, we highlight representative techniques, compare their scalability and expressiveness, and summarize how they contribute to meaningfully improving realistic robot safety and correctness. We conclude with a discussion of remaining obstacles for achieving that goal and promising directions for advancing formal methods in robot learning.

Published in Transactions on Machine Learning Research (12/2025)
Formal Methods in Robot Policy Learning and Verification: A Survey on Current Techniques and Future Directions
Anastasios Manganaris
amangraaSpurdue..dl
Deportment of Computer Science
Purdue Universily
Vittorio Giammarino
vyiammmrr@yrrdue.c. eduu
Department Computer Science
Purdue University
Ahmed H. Qureshi
ahqureshii@purruee.om edu
Department of Computer Science
Purdue University
Suresh Jagannathan
suresh Cs. www.due.ppp......oo.m edu
Department of Computer Science
Purdue Universily
Reviewed on OpenReview: https: // openrevie.. net/ forum? id= DZks DZka 5s
Abstract
As hardware and software systems have grown in complexity, Formal Methods FMs have been indispensable tools for rigorously specifying acceptable behaviors, synthesizing pro- grams meet these specifications, and validating the correctness of existing programs. In the field of robotics, similar trend of rising complexity has emerged, driven in large part by the adoption of Deep Learning (LL While this shift. has enabled the development of highly performant robot policies, their implementation as deep Neural Networks (NN-) has posed challenges to traditional formal analysis, leading to models that are inflexible, fragile, and difficult to interpret. In response, the robotics community has introduced new formal and semi-formal methods to support the precise specification of complex objectives, guide the learning process to achieve them, and enable the verification of learned policies against them. In this survey, we provide comprehensive overview of how FM have been used in recent robot learning research. We organize our discussion around two pillars: policy learning and policy verification. For both, we highlight representative techniques, compare their scalability and expressiveness, and summarize how they contribute to meaningfully improving realistic robot safety and correctness. We conclude with a discussion of remain- ing obstacles for achieving that goal and promising directions for advancing [M in robot learning.
1 Introduction
Both in natural and artificial settings, it has been repeatedly observed that surprising levels of complex- ity and capability can emerge from systems built by massively scaling and composing simple components (Anderson 1972). The "bitter lesson" learned by artificial intelligence researchers one instance of this phenomenon: methods that scale effectively with data and compute tend to outperform those that rely on explicitly incorporating human knowledge (Sutton 2019). The emergent complexity of such scalable systems is one the most significant factors behind the success of DL (Kaplan et al. 2020). When combined with well-developed theoretical frameworks for general decision-making and control. DI techniques have scaled

---

Published in Transactions on Machine Learning Research (12/2025)
from achieving superhuman performance in game playing (Mnih et al. 2015). to enabling effective robotic manipulation (Levine et al. 2016 2018), and now to powering increasingly general embodied agents and foundational models for robotics (Reed et al. 2022 Hu et al. 2024 Firoozi et al., 2024). With this flexibility, it is becoming viable to deploy robots alongside humans in unstructured environments and with open-ended goals fulfilling roles such as household assistants (IX Technologies 2025
autonomous vehicle controllers
Favaro et al. 2023), and medical assistants (Empleo et al. 2025).
However, these new use cases demand higher levels of confidence in deployed robot policies and, consequently, greater levels of policy interpretability, robustness to uncertainty, and compliance with behavioral and safety specifications. Alarmingly, achieving such confidence appears to be fundamentally constrained by the very increase in policy complexity that has enabled these systems" viability. Learned robot policies, typically parameterized by deep NNF, are well known for their lack of interpretability (Zhang et al. 2021). poor generalization to out-of-distribution settings (Ross & Bagnell 2010 Park et al.] 2024), and vulnerability to adversarial inputs (Szegedy et al., 2014 Xiong & Jagannathan, (024). These limitations are particularly concerning in safely-critical or high-stakes environments, where failures can lead to catastrophic outcomes. Conventional approaches to specifying robot behavior, such as reward functions or behavioral demonstrations, have so far failed to address these issues of policy trustworthiness. Without any guarantees, they can lead to unsafe, under-performing, or misaligned behaviors due to phenomena such as reward hacking in Reinforcement. Learning (RL) (Skalse et al. 2022) or overfitting in imitation learning (Goodfellow et al., 2016). Even when safety is not the primary concern, these approaches also lack the expressive power to formally and concisely define complex, compositional, or temporally extended tasks.
In the world of software, similar safety concerns emerged as increasingly complex programs were deployed in safety-critical settings [Baase] 2008), and these concerns have motivated the development of Formal Methods (FM}) to mitigate or eliminate these risks. Central to these and subsequent [FM frameworks is the capability to rigorously specify system behavior, for which modal logics like Linear-time Temporal Logic [LTL] (Pnucli 1977) have been used with great success. After formally specifying some behavior, FM can allow a user to directly senthesize high-level, correct-by-connstuuciion prograis from a Formal Solar-Lezama 2009 Srivastava et al.) 2010). Closely related FMM, such as Model- Checkl-hheciing, provide ways to formally verify that complex programs satisfy their specifications (Baier) Murray et al. 2013). Formal specification, synthesis, and verification now constitute complete toolkit for developing highly dependable software systems.
Specification
FS (Church 1963
et al. 2008 Kästner et al.
2018
In robotics, these FM have also been applied and extended to ensure the correctness and safety of controllers without any learning components. For example, specification languages such as Signal Temporal Logic (TL have been developed to express requirements for continuously-evolingg real-valued properties of dynamical systems (Maler & Nickovic, (2044), and probabilistic and hybrid-system model-checking techniques have been introduced well (Alur et al., ) These advancements have enabled the synthesis and verification of robotic policies based on traditional planning algorithms (Kress- Gazitt et al. 2009), as well controllers derived from optimization (Sun et al. 2022b) and control theory (Abate et al. 2019b. Such formal approaches offer guarantees not only for immediate control actions but also for long-term behavioral trajectories, directly addressing the predictability and robustness of robotic systems.
With the risc of learning-based methods in robotics, more recent rescarch efforts have sought to use to address the critical weaknesses of purely learning-based approaches mentioned above. These new FM
FM
have the potential to systematically provide interpretability, behavioral guarantees, and safety assurances for
based robot policies. Moreover, specifications used in FM allow for more expressive and more concise behavior definitions than reward functions or demonstrations, casily capturing complex, compositional, and temporally extended requirements. This combination promises not only robust performance but also strict conformance to intended behaviors and reduced susceptibility to adversarial exploitation. Consequently, FM represent a promising approach for transforming learned robot policies from impressive but opaque models into transparent, reliable, and trustworthy solutions ready for real- world deployment. In this survey, we contribute thorough overview of these recently developed methods.
Scope and Contribution This survey aims to provide for robotics researchers an accessible introduction and comprehensive overview for recent work at the intersection of FM and DD for robot control. Unlike
2

---

Published in Transactions on Machine Learning Research (12/2025)
Policy Learning
B30
with
S
Imitation Learning with
Offiiee
with
Formal Methods (2)
Temporal Logics and Automata 1:3
Program Synthesis
Program Verification
Policy Verification (84)
Environment- betrrccioos
Reachability Analysis
π
Certificate Functions
Rustime Monitoring Falsification
Figure 1: Our survey first provides background information on FM and their requisite specifications, denoted by db We then survey recent work on [F in robolics in two categories: work on FMfinfrrmed learning of robot policies (T), and work on verifying that learned robot policies satisfy specifications (T Φ).
prior surveys that focus on applying learning methods to FM (e-g., Wang et al. 2020
instead focus
methods. Specifically, this is survey of FMs learning methods for robot policies, rather than to general Machine Learning
the application of FMs to
applied to
ML systems [Larsen et al.
2022
Urban & Miné 2021
Several surveys have reviewed the use FM in robotics. For instance, Luckeuck et al. 2019 examine a range of FM applications autonomous robotic systems, with a focus on model checking for robotics software and specification languages. However, since that survey, DL based systems have gained greater prominence in robotics and remain largely unexamined in that context. Similarly, Belta. & Sadraddini (2019) survey formal synthesis methods for optimizattion-esse robot controllers but does not address learned controllers. Some existing surveys have examined subfields of FM relevant to ensuring the safety of learned controllers such as learning certificate functions (e.g., Lyapunov functions, barrier functions, and contraction metrics) DDwssn et al. 2023). set-propagationn-baee based reachability analysis (Althoff et al.) 2021). Hamilton-Jacobi [HA]
reachability analysis (Bansal et al.] 2017), cyber-physical system verification (Tran et al. 2022). and adjacent safe learning methods (Brunke al. 2022). We also include the most relevant above subfields in this survey, provide an up-to-date overview of each, and refer readers to existing works for more in-depth details where appropriate. To our knowledge, no prior survey has provided such integrated perspective on how FM: have been developed and adapted specifically for DL robot policy learning, and we expect this to be valuable resource for guiding future research in this area.
Outline We consider the use of FMss in robot learning to be naturally summarized with the three stage flowchart shown in Figure [ The root of the flowchart represents the existing work on [MM, from which existing techniques for obtaining and representing FS are used as a starting point for every other FM The next nodes representing learning a policy informed by specifications and the verification of learned policy against specifications. We therefore have adopted the same structure organize our survey. Section 2 covers the preliminaries for understanding(FF.. and the existing methods of constructing Formal Specifications (FS)) which are used in the more recent work applied to robotics. Section g surveys methods for informing policy learning techniques with bchavioral specifications to enable more complex tasks, accolerate learning, improve safety. Section We clarify for each method the underlying verification approach, the specifications they support, and the assumptions they make on the environment and policy. Finally, Section 5 discusscs notable gaps in the existing work clarify directions for future work, and Section concludes the survey with a summary of the above content.
methods focused on verifying that learned policies conform to specifications.
3

---

Published in Transactions on Machine Learning Research (12/2025)
2 Preliminaries for Formal Methods and Specifications
In this section, we introduce the fundamentals of FM to provide background for the methods discussed in our survey. We first describe the standard model used in FM for describing the potential behavior of discrete systems. We then describe the most. widely used represenlations of ESS that describe behaviors in that model. Methods for formal policy learning and verification should operate with a precise description of the behavioral roquirements, so these specifications serve the starting point for every roboticist secking to apply FM.. We conclude with a brief summary of formal synthesis and verification, which have also influenced the development Ff in robotics.
FM aim to ensure that systems behave as desired (Baier et al. 2008), but behaving "as desired" is fraught with ambiguity. The first step in eliminating that ambiguily considering the syslem under an appropriate formal model, and the classical choice a discrete transition system. We provide an illustrative example of such a system and an BS of complex and temporally extended behavior for the system in Figure 2 In general, discrete transition system is a tuple (S,A, -, AP, L), where S is set of states, A is set of actions, - C S x A S is transition relation (i.e., a set of possible transitions (s, a,,,)))) between pairs of states s, E S given actions a E A), I S' is a set of initial states, AP is set of atomic propositions, and L : S + 24P is a labeling function that maps each state to the set of true propositions for that state [Baier et al. 2008). , Under this model, as well as any other model that supports a similar notion of states and propositional labeling, the behavior of a system can be formally defined as (possibly infinite) sequence of states (so, 81....) E 5 where S denotes the set of all possible finite and infinite sequences over S. The possible sequences are further restricted to those starting with states so E I, and those that can be induced by another sequence of actions (ao, 01,...) E A under the system's transition relation - As behaviors under this model are formally sequences of states, it is natural to use tools from formal languages to express specifications of desirable behavior. While there are many such tools with the expressiveness necessary to describe most behaviors relevant to robotics applications, we introduce here two categories of particularly useful and relevant formalisms:
Temporal Logics and Finite State Automata [SA)))
Temporal Logic LTL Pnueli 1977 is an extension of propositional logic. Logical expressions for some system, typically denoted by Φ and belonging to a set of expressions D, are defined over the atomic propo- sitions (AP) for that system. Atomic propositions can be thought of as the simplest Boolean observations one can make from the state of the system. These can then be combined in more complex expressions using standard Boolean operators such as "not" (-), "and" (A), "or" (V), and "implies" (+). To describe behaviors over time, LTL introduces additional "temporal" operators referred to as "next", denoted by O or X, "eventually" (or "finally"), denoted by O or F, "always" (or "globally"), denoted by O or G, and "until" denoted by u. The syntax of INN is given by the following grammar:
Φ === true false p mi 99 A p2 P1 V 92 p1 + p2 OOP! I 0o Dg 11
Baier et al. (2008) provide a complete treatment INI but we provide a brief summary of the temporal operator semantics here. The first three operators O 9, and O are unary, meaning they apply to only one sub-expression, and respectively express the requirement that the sub-expression is true al the immediate next moment in time, at any moment beyond and including the current timestep, or for all time beyond and including the current timestep. The until operator 4 is binary, mcaning that it applics to two sub- expressions, and expresses the requirement that the first sub-expression is true at all times until the second sub-expression becomes true at some future time. By composing and nesting these operators applied to atomic propositions p E AP, one can quickly express complex behaviors with simple LTL specifications like the example in Figure 2
Signal Temporal Logic STL Maler & Nickovic 2004) is popular formalism that extends HNE in order
to specify behaviors for real-valued signals defined over continuous time intervals. Using STL requires that each atomic proposition p E AP is associated with a real-valued function of the system state μ S + R, and the boolean value of the proposition in a state is determined by the value of (us) 2 c for some threshold value e E TR. Furthermore, the temporal operators are extended to consider the value of sub-expressions over a specific future time interval [a, b] with a E R, b E RU The resulting grammar for STL expressions is
:= = m(s) 2 el map w/ A t/2
1/1 V the 1h = 2/2 0/o,)) | (o.111 1/24,0001/2
4

---

Published in Transactions on Machine Learning Research (12/2025)
1 A A00 A
[Cbusy 7 -C A
O1
C'
O2
Recurrence
Safety
Guarantee
(b) An PTTE expression describing language (i.e., set of be- haviors) that requires the two offices are visited infinitely often, the conference room is never visited when it is occupied, and the break-room is eventually visited at least once. According to
1m10,30...
the standard classification in Manna Pnueli (1990), these recurrence, safety, and guarantee sub-expressions.
R
.... 26
36
t
R RURO O VO O2 V Coueyd (Cussy Cousy (Cussy) (busy)
L(a:)
(Cbas)
(a) The office building grid-world environ- ment with two office rooms (O, and O2), conference room (C), a storage room (R), and break-room (B).
(c) The labeling for each state the example behavior, through which one can determine if the behavior satisfies the LTL sion.
expres.
Figure 2: typical discrete domain modeling a single agent in a building with several rooms. The model is defined with atomic propositions corresponding to the agent's presence in each room and the occupancy status of the conference room (AP {R. O2, O2, C, B, Obuay) using the symbols in 2ah. The example behavior in blue depicts the agent exiting the starting room, cycling infinitely between visiting the offices, and at some point visiting the break-room. An example LTL expression (2b) can be used to specify complex behaviors and is evaluated over sequences of labels (2c obtained via the labeling function L S 24P.
with timing parameters a, where b a and threshold parameter c. Expressions in STL addition to supporting the true-or-false semantics of TTA expressions, support a quantitative semantics defined by a "robustness" function p that measures how much a sampled signal satisfies violates the property specified by an STL formula V. We refer the reader to works by Maler & Nickovic (2004) and Dawson & Fan 2022 for the original definition and more recent summary of STL semanlics.
Automata A class of abstract machines termed FSA provide a very useful alternative formalism for specifying behaviors. There are two broad classes of automata meant to represent languages with finite- length elements (i.e., sets containing only finite behaviors) and w-automata, which are capable of representing languages with infinite elements. More specifically, these are called regular and w-regular languages. The former class of automala has a single, well-established representation, complex w-automata.
elaborate here the
An w-automaton is, like BTT used to represent sets of sequences composed of clements in the set 2 However, we note that w-automata are strictly more expressive than LTE and that every LTE expression can be translated to an w-automaton (Gastin & Oddoux 2001). An automaton is able to represent such sets (i.e., behaviors) by defining a procedure for reading one of these sequences element-by-element through which the sequence determined to be member of the intended set or "accepted". An example automaton for [ии expression from the domain in Figure 2 is shown in Figure This automaton is always in some state {90. 92,92); and starts in the state qu- As states function yields corresponding labels L(s) E 24P that then induce some change in the automaton's internal state as determined by the automaton's edges. The resulting sequence of internal automaton states then, based on some acceptance condition, determines the membership of the input sequence the set. Thore are multiple different kinds of w-automata that diller in their acceptance condition (Hahn al. 2022 Baier et al. 2008), of which the Rabin and Büchi acceptance conditions are the most prevalent.
encountered in the system, the labeling
5

---

Published in Transactions on Machine Learning Research (12/2025)
Formal Synthesis and Verification Formal synthesis is the process of generating correct-yy-cconstruciion programs that satisfy semantic correctness requirements expressed by ESt (Solar-Lezama, 2023). Classically, these methods have been applied to single-input, single-oulput programs ex- pressod in digital circuits (Church. 1963), functional pro- grams (Manna & Waldinger, 1980), and general modern
00,
q2
O1
O1
11
40
-01 A02
programming languages (Solar-Lezama 2009; Alur et al.) 2013 2018 Srivastava et al.] 2010 Feser et al. 2015). Reac- tive synthesis, developed for applications to concurrent pro- grams, instead focuses on programs that continuously receive inputs from their environment and produce corresponding outputs (Pnucli Rosner 1988 Bloem et al., 2012). Nat- urally, these reactive synthesis techniques were adapted to produce robotic controllers due to their similar computa- tional requirements (Kress-Gatt et al. 2009). Alternatively, formal verification seeks to prove that eristing programs sat- isfy a formal specification. A central class of these techniques is model checking, which operates on discrete model the program. By exhaustively exploring possible execu- tions of the model and checking them against an automa- ton encoding the specification, model checking either confirm universal satisfaction produce counterexample trace that violates the specification (Baier et al., 2008). One often performs reachability analysis as part of model checking, which focuses on determining if the system can ever reach an undesirable or target state from another state. For example, can be used to verify that a concurrent program never enters a deadlock state. Adjacent to formal verification are other validation methods that provide weaker guarantees. Falsification attempts find a counterexample input under which the system violates the specification. While effective identifying bugs, falsification does not offer any guarantce of correctness in the absence of countercxample, unlike formal verification, which aims to provide a proof of correctness across all executions.
O0
qI
O2
Figure 3: A Deterministic Büchi Automaton (DBA) for Ø = 0$0; A 0602- Intuitively, as the agent periodically encounters states satisfying O1 E L(s) and O2 E L(s), the au- tomaton moves from go, to qı, to q2, and back qo- As a result, the accepting state go F is visited infinitely often and the sequence is decmed to satisfy ø.
Although there are many more specification formats and techniques within FM we conclude our background summary here as the majority of work seeking to bridge FM with DL for robotic control has focused on the above fundamental methods.
3 Formal Methods in Robot Policy Learning
In this section, we survey the current research integrating FM into policy learning, grouped by the major policy learning paradigm they belong to. This overall categorization and constituent subcategories are depicted in Figure We also note that these categories largely align with the different communities working on robotics, each with different motivations for introducing
FM to existing methods. Online methods
in particular introduce DL to generalize more traditional approaches (Fainekos al. 2005 Ktress-Gazit al.,
2009 Wongpiromsarn et al. 2010: Smith et al. 2011 Ding et al. 2011 Chu (Dennis) Ding et al. 2011 Belta & Sadraddini 2019) to settings where they are otherwise inapplicable. These include settings with unknown and complex system models, partial observability, and stochasticity, settings where standard optimization approaches are loo slow, or settings where complex decisions must. be made that consider longer, coarser horizons or partially satisfiable specifications. Offline methods, however, typically come from the robotics or DL for control communiiics and use FM gain support for more complex tasks and improve their safety and reliability. These research trends and their associated challenges are described in the following subsections.

---

Published in Transactions on Machine Learning Research (12/2025)
Online Methods
Offline Methods
Model Frode--Free RL 2-1.2.1 1-4))
Inverse RL (63.2.2 11)
Automsta- Reorel
Learning Reward Automata
Automita-- Guided RL
Learning STI. Rewards
Automata- Free RL.
Programmatie R.
Behavior Cloning (53.2.2 42)
STL-Derived Losses
Logical-Hieree Hierarchy Methods
Guided Difiusion Policies
Model-Basee RL (83.2.1
Learned STT-MPC
Offline RL (33.2
Imitating STL-MPC
Offline Automata- Theoretic RL
Sequence Modeling
The research informedd policy learning can be organized into categories corresponding to five major robot policy learning frameworks. The first two, model-froe and model-based RL. methods that rely on enviromment. interaction during learning. Within model-free RL there are algorithms that learn from antomata-theorest rewards, algorithms that do not rely on automata at all, and algorithms based on programmatic policies. Model-
Figure 4:
methods, are online
rewards, algorithms that use automata for guidance without explicit
primarily extends optimal control techniques, such as Model Predictive Control (MPC) with DL components to enhance performance versatility. The remaining three categories, Inverse Rein- forcement Learning (RL), Behavior Cloning, and offline RL primarily offline methods that learn from pre-collected demonstration datasets. Their constituent subfields include learning FS$ as reward functions for IRL augmenting behavior cloning losses with FS and training sequence models to generate actions conditioned on high FS satisfaction.
3.1 Online Policy Learning
Iilommtaa-Theoeetee Reinforcement Learning The first works using DL to produce policies satisfying
established a connection between automata-theorett verification techniques for probabilistic systems (Couccoubetis & Yannakakis) 1995; Vardi 1985) and model-free RL This resulted in the field of
high-level FS
tutoomit--hhooretic RL where FSF are expressed automata that monitor trajectorics in Markov Deci- sion Process (MPP) and dispense rewards that incentivize satisfying the aulomaton's 's acceptance condition.
Sadigh et al. [2014 initiated this line of research by training policies to act in product MDPp formed from a base MDP and a Deterministic Rabin Automaton (DAA. A product MDP is so named because the state
space the new MDP is formed with the Cartesian product of the plain MDPss s state space and the automa- ton state space. This augmentation can be seen as providing a minimal "memory" necessary for obtaining Markovian rewards and policies for otherwise non-Markovian objectives. They showed that maximizing a Markovian reward function derived from the Rabin acceptance condition of task automaton would imply maximizing the probability of satisfying the IIAA formula. Although this method performed well in practice, subsequent work highlighted its limitations. Specifically, Hahn et al. (2019) demonstrated that for certain MDP: and w-regular objectives, it is not always possible to define a reward function based on the Rabin acceptance condition that yields an optimal strategy. This impossibility was later established in general (Hahn et al. 2022). In place DRA, DBA emerged as an attractive alternative due their simpler and more intuitive acceptance condition. However, simple DBA are not ideal because they lack support for some w-regular objectives. The immediate alternative of using Non- non-Determiiistic Büchi Automata (DBBA)). with completely unrestricted non-determinism, is also infeasible because resolving non-deterministic choices for the automaton may require an unbounded look ahcad into the future (Vardi 1985). which is impossible in context. The solution is to rely on a special class of automata with restricted a non-determinism, in which every non-deteriiiistic choice can be resolved based solely on the history of the exccution. Hasanbeig et al. (2019a) and Hahn et al. (2019) were the first use one of these automata by defining a reward from SimittDeterminittie Büchi Automaton (LBAA) (Sickert et al.) 2016). This class of automata used in model-free RL has been further studied and formalized Mhood-for-MDP.... automata
7

---

Published in Transactions on Machine Learning Research (12/2025)
by Hahn et al. 2020 Independently, ahtooatta-theoreie methods such as Reward Machines (Icarte et al. 2018) were developed for finite regular languages, for which a variety of reward shaping and accelerated RL algorithms have been developed (Camacho et al. 2019: Toro Iearte et al. 2022). Reward Machines have also seen greater integration with deep RL for experiments in continuous robotic domains, in contrast to prior work. Reward shaping lechniques originally developed for Reward Machines in Camnach et al. (2019) have also been extended to w-automata by Bagatella et al. (2024), who achieved success in several LTL specified robotic manipulation and locomotion tasks.
Later work in this direction relaxed assumptions about system observability implicit in the above methods. Hasanbeig et al. (2019b) and Li et al. (2024) both relaxed the assumption that one has access to the exact atomic proposition labeling function, and instead perform learning inside a probabblistically-lbbee
MDF
On top of the former work, Cai et al. (2021 2024) relaxed the assumption that the input FS is always satisfiable and, in the case it isn't, learn policies that minimize task violation.
Automata-Guidoo Reinforcement Learning Instead of dispensing rewards, automata can also serve as high-level structures for guiding the execution of multiple low-level or goal-conditioned policies. For example, Jothimurugan et al. (2021) and Wang & Zhu (025 both adopted an approach in which a separate policy for each edge of the automaton is first trained via RL These edge-specific policies are then composed
at exccution time using a graph search procedure, such as Dijkstra's algorithm, to identify an optimal path
through the automaton. Other approaches, such as those by Nangue Tasse et al. (2024) and Araki et al.) (202I; similarly plan over automata to compose lower-level policies but instead rely value iteration to compute an optimal composition. In recent work, Wu et al. (2025) translate LTL specifications derived from natural language specifications into Büchi automata using an equivalence voting scheme, and then apply constrained decoding techniques to ensure that planned actions adhere to paths that reach accepting states in the automata.
One can trade some generality for computational efficiency by training a single goal-conditioned policy to replace the full library of edge-specific or task-specific policies used in the above methods (Qiu et al., 2023 Manganaris et al. 2025 Guo et al. 2025 Recent works have also proposed to condition policies real-valued embeddings of multi-subgoal sequences induced by FS. (Jackermeier & Abate 2025). or on embeddings of entire FSJ. Fundamental methods for obtaining embeddings of [STL] formulae have been developed by Hashimoto et al. (2022), who proposed a Word2Vec-stylle skip-gram model (Mikolov et al. 2013) for STL specilications, and Saveri et al. (2024), who encoded [ specfficciiiiiioioonn based on the principal components of a Gram matrix defined by two datasets of random behaviors and specifications. Other techniques have also used Graph Neural Networks (GNN:) (Lamb et al.) 2021 Crouse et al., 2020) for this task, using either a specification's automaton representation (Yalcinkaya et al. 2025 or the syntax tree of its temporal logic formula (Kuo et al., 2020 Vaezipoor et al. 2021 as input. These methods enable training agents to generalize to entire new tasks rather than singular goals. However, such methods sacrifice the formal soundness of their imput FS| to gain from the versatility of conditional generative models. The diversity of supported tasks is also sometimes limited by assuming atomic propositions in the specifications exclusively refer to reaching goals in the environment. As a result, the most complex setting used in the evaluation of these methods has often been robotic navigation.
Automata-Free Reinforcement Learning Some methoods bypass the use of automata for rewards or guidance and instead rely directly on other FS formats. Several of these are concerned with contimuous time requirements written in STE which cannot casily use automaton-baser approaches. [Akaraa et al. (2016) focused on STL formulae that can be evaluated over fixed, finite horizon length n and define an MDP that augments the state space to record at all times the past n states, from which Markovian reward function can be defined. Li et al. (0017) proposed defining a non-Markovian reward based on [STL robustness over entire episodes and scarching for the optimal policy based on these complete policy rollouts. Other methods use similar reasoning to provide reward shaping by evaluating STL or other temporal logic formulae over the agent's history (Balakrishnan & Deshmukh 2019 Hsu et al. 2025). Recent works have also explored directly using the gradients provided by differentiable
specifications to train a policy online (Xiong al., 2024 Eappen et al., (2024), although this only supports STL specifications that express requirements of the direct policy outputs (actions) as opposed to the system state. In exchange for reduced scope of supported
8

---

Published in Transactions on Machine Learning Research (12/2025)
FS+, these methods can be applied in many settings and can benefit from dense feedback signals. Their
applications have included real-world robotic manipulation (Li et al. 2017: 2019), quadrotor goal-reaching
[Xiong et al. 2024 Lappen et al.) 2024), and multi-agent control tasks (Eappen et al. 2024 Hsu et al.
2025
Programmatic Reinforcement Learning There is also growing interest in RL algorithms that use pro- grummatic policy representations, which more closely align with traditional program synthesis techniques. Early approaches in this direction focused on approximating trained oracle policies with more inter- pretable, program-like representations that offer benefits such as verifiable correctness (Bastani et al. 2019) and improved generalization (Inala et al.) 2020). Zhhu et al. (2019) in particular took advantage of this verili- ability and adopted a Counter-example Guided Inductive Synthesis (CECIS)-style approach (Solar- Sellr-Lezaaal le al. 2006 2008). Their algorithm first approximates a learned policy with a symbolic program and iter- atively refines it when countercxamples to its correctness are found during verification. Subsequent work sought. to bypass the need for an oracle policy altogether by developing program representations that could be trained end-to-end with policy gradient methods (Qiu & Zhul 2022). More recently, Cui et al. (2024) addressed the limitations of these approaches in long-horizon, sparsc-reward settings by explicitly guiding a tree-scarch over possible programs with encountered rewards, thereby using programmatic policies not only for interpretable execution but also to aid in exploration. However, these methods have only been applied two-dimensional car and quadrotor control (Inala)
to environments including grid-worlds (Cui et al. 20244).
ct al. 2020), and MuJoCo locomotion tasks (Qiu & Zhu 2022 Todorov al. 2012). so application to more complex environments is remaining challenge.
Model-Based Reinforcement Learning Finally, research has also explored online learning of PS satisfying policies with model-based RL Although there are some automata-theoretic approaches in this community (Fu & Topeu, 2014), there are more that draw upon optimization-based optimal control methods [Belta & Sadraddini 2019) and introduce learning to remove dependence on system dynamics knowledge, improve performance, or mimic decisions of experts in situations when specifications can only be partially satisfied. For instance, Cho & Oh (0188) formulated an MPC problem with constraints derived from a soquence of prioritized STL formulac and learned from dataset of expert behavior the degree to which those constraints should be satisfied. Meng & Fan (2023) by bypassed the typical sampling-based or gradient- based optimization used in MPC by directly learning model that predicts optimal action trajectories satisfying STL specifications, enabling real-time performance. Both of these methods are primarily applied to STL constrained autonomous driving, with additional ovaluations being conducted in mobile robot and manipulator goal-reaching lasks.
Several methods simultaneously learn model of the system dynamics along with the control policy. For example, Kapoor et al. (2020) learned deterministic predictive model to enable MPC using objective derived from a given STL formula. Similarly, Liu et al. (2023) learned simple deterministic model but
additionally addressed the history dependence inherent in LTL based objectives by implementing the policy as a Recurrent Neural Network (RNN). Learning the dynamics model can also facilitate formal guarantees for the resulting policy. For instance, Cohen & Belta (202)) decomposed a high-level temporal logic requirement into a sequence of optimal control problems, each solved individually using model-based RL The overall policy then selects among subproblem solutions based on the current state of the task automaton. The learned models and associated value functions support the construction of barrier functions (See Section 4.3)
that formally certify the safety of the synthesized controller. Instead of autonomous driving, the common evaluation environment for these methods was mobile robot navigation, including real-world deployment by Liu et al. 2023). Kapoor et al. (2020) also demonstrated goal-reaching with a manipulator robot.
3.2 Offline Policy Learning
Incorporating logical specifications into oflline policy learning is the most recent and shallowest development of the work we survey, RL methods benefit from the fact that the logical specification can be turned into a form of online feodback a variety of ways, but using a specification strations is more complex. However, these approaches are not burdened by the issues of sample-ineficiinn found with online methods. Consequently, they more easily scale to realistic robot systems. Furthermore,
conjunction with a set of demon-
9

---

Published in Transactions on Machine Learning Research (12/2025)
the incorporation of highly informative, compact FS$ holds the promise of dramatically reducing their initial data requirements.
Inverse Reinforcement Learning Early methods in offline policy learning extended the product MDP
construction developed for online settings and applied [RL Ng & Russell 2000 techniques to learn reward functions defined over the product MDP state space (Wen et al., 2017; Zhou & Li, 2018). Several of these introducing the idea of generating adversarial counterexamples that highlight policy violations of the input specification in unrepresented regions of the
approaches framed the problem as an instance of CEGIS
training data, thereby improving robustness during learning (Zhou & Li, 2018 Puranic et al.) 2021 Ghosh
ct al. 2021 Dang ct al., 2023). More recent work has sought to extend IRL by incorporating Specification
Mining ISM,, which is the automatic generation of LSS from past experience or other informal descriptions
of correctness (Ammons et al. 2002). Many techniques have been developed for this general task (Bartocci ct al.) 2022), of which the most significant for robotics have boen learning temporal logic formulac from
natural language (Liu et al.
2022) or demonnstrations (Shah et al. 2018 Soroka et al.) 2025) and learning
Reward Machines to capture Non-Markovian task rewards (Topper et al. 2022 Dohmen et al. 2022 Gaon
2021 Abate et al. 2023). Generally, these methods allow one to acquire learned reward functions that can be more expressive and significantly more interpretable than
& Brafman 2020 Rens et al.
2020 Xu et al.
N
However, we note that the policies resulting from the majority of these works have only been evaluated in very simple discrete or classic control environments,
parameterized reward functions (Liu et al.
2025b
with the most recent works (Liu et al. b025b Soroka et al., 2025 being applied to 2D robot navigation, MuJoCo locomotion [Todorov et al. 2012). and autonomous driving.
Behavior Cloning Temporal logic specifications have also been leveraged in behavior cloning as additional differentiable terms in loss functions for training policies. [nnes & Ramamoorthy (2020) used this approach to train dynamic motion primitive (Schaal et al., 2007) parameterized policies for pouring and goal-reaching behaviors on a real PR-2 robot. More recent approaches blend offline and online learning: an offline phase first trains a policy to mimic overall movement patterns, followed by an online phase that refines the policy by learning boundaries between discrete logical modes (Wang et al. 2022 2024). Other behavioral cloning methods employ generative models such as diffusion models (Zhong et al. 2023b a Meng & Fan 2024
Feng et al. 2024a) and flow matching models (Meng & Fan 2025) whose inference process can be directly guided by robustness gradients from differentiable STL
2023 Overall, these frameworks (Teung et al.
approaches have seen much more frequent and successful application to complex tasks including simulated maze navigation and manipulation (Meng & Fan 2025 multi-stage real-world manipulation (Wang et al.,
2022 2024 traffic simulation Zhoong et al. 2023b a Meng & Fan 2024), and real-world navigation with a Unitree Go2 quadruped robot (Feng et al. 2024a).
Offline Reinforcement Learning The most recent advances have focused on fully offline RL techniques [Levine et al.) 2020). Due to the complexity of defining reward functions for specificctis, relatively few methods apply offline to product MDP] with the kinds of reward structures discussed in Scction 3.1 One notable exception is the work by Feng et al. (2024b), who developed an approach for learning a state-option value function (Sutton et al.) 1999) from a dataset with rewards based on incremental satisfaction [TL formulae. They demonstrated superior performance to pure behavior cloning methods (Feng et al. 2024aa in 2D-Maze navigation tasks, Push-T tasks (Chi et al., 2025), and indoor navigation with a real quadruped robot. Guo et al. (2024) instead proposed an ITEE rolled decision transformer that conditioned trajectory's robustness at each step, and they showed its successful application in several simulated robot navigation environments. Overall, this direction of offline policy learning remains underexplored and presents significant opportunities to build upon the insights developed in the online setting.
the
4 Formal Methods in Robot Policy Verification
We conclude our survey by examining research aimed at the formal verification of learned robot policies with respect to given ES Compared to the methods discussed in Section 3 these approaches place significantly greater emphasis on establishing formal guarantees of correct behavior, as opposed to just encouraging policies to achieve requirements given by an FS For clarity of discussion, we uniformly frame these techniques
10

---

Published in Transactions on Machine Learning Research (12/2025)
A A (Qq)
General w-regular FSA
Anteratto&bbrrrattioo Methode (81.2.1)
Supports complex behaviors
Assumes known dynamics in discrete environment
nop
O
High computational cost
Obligation FSS
A0p v (q)
Supports common behaviors (e.g., Reach g while avolding
Reachability Analyate (.2.2.2) Propagation, Sensitivity Analysle, Reuckability
Model-based and model-free methods
Moderate computational cost, low approximation error
Guarantee / Safety FS
Op
Dp
Supports stability invariance properties
Lyagpanov Punc-
Hona
Barrier Punctions (94.2.3)
Computable model-free with access sample transitions
Contraction
Computationally Scalable (Parallalizsble), moderate approximation
Metricu
Figure 5: Our survey of robot policy verification methods is organized by three major categories of temporal FSH. visualized here using the temporal property hicrarchy of Manna & Pnueli (1990). The first category (lop), primarily represented by methods using discrete system abstractions, is defined by support for general w-regular FSB. The second category (middle) supports the more limited "obligation" class of FS and is primarily represented by reachability analysis methods. The final category (bottom) supports particular forms of safety (invariance) and guarantee (eventual stability) FS and is represented by certificate function methods. The typical capabilities of these methods are included and approximately rated as either poor red, moderate in orange, or good in green.
all aiming to prove that, for all initial states in some set I, given NN controller π operating within some environment will generate trajectories r that satisfy specification db Under this unified view, the surveyed approaches can be compared along several axes for example, the assumptions made about the policy, the environment, computational scalability, or the specification. We choose to primarily organize our survey around the last axis, as naturally aligns with the others: supporting more complex specifications typically requires tradc-offs in scalability and stronger assumptions on the environment model and policy. Figure 5 summarizes the surveyed work across the three major categories along this axis of DS support.. While this categorization provides useful structure, many methods within each category blur these boundaries, and attempt to describe these liminal methods in the most appropriate location. There are also two categories that fall outside of this structure but are still important to improving the safety of robot policies: run-time monitoring and falsification methods. We also describe these and the additional capabilities they provide in the last subsections.
4.1
Methods
Discrete environment abstractions are necessary for obtaining a model under which classical verification methods are tractable, so some early approaches for verifying A prominent example is the work by [Sun et al. (2019), who proposed a method that, under assumptions of lincar system dynamics and a Rectified Lincar Unit (RLMU -based NN architocture, utilized a discrete system abstraction to verify closed-loop properties of simulated, NNCcontroleed 2D quadrotor equipped with
also relied on such abstractions.
LiDAR perception system. Their approach obtained these guarantees using Satisfiabllity Modulo Convex Programming (SCC) queries (Shoukry et al.] 2017] and their custom system model. A similar strategy, with comparable assumptions but additional support for system stochasticity, was presented by Sun et al.
2022c
While these abstraction-basod methods offer considerable ficxibility the temporal specifications they can handle and provide strong correctness guarantees, the assumption that the environment can be faithfully modeled with discrete abstraction is burdensome and unrealistic in most cases. Moreover, the discrete
11

---

Published in Transactions on Machine Learning Research (12/2025)
abstractions themselves are prone to combinatorial explosions in complexity, which further limits their prac- ticality (Dimitrova & Majumdar, 2014 Lindemann & Dimarogonas 2019). As result, these approaches have attracted less attention compared to methods that offer greater computational scalability.
4.2 Reacha Methods
One class of computationally scalable verification methods is based on reachability analysis. While exhaus- tively checking all possible system behaviors is infeasible for infinite-state systems, reachability analysis provides a method to verify a more restricted set of behaviors in such continuous and hybrid systems (Bert- sekas & Rhodes, 1971; Alur al.) 1995). Specifically, reachability analysis is concerned with computing approximate sets of reachable states. This reachability computation can either be performed forward in time from some initial set of states resuling in forward reachable set-or backward in time from some target set of states-resulttee putation is given in Figure G These computations then can be used to verify safety and liveness properties, often given as "Reach-Avoid specifications" (e.g,, eventually reach region A and always avoid region B). For example, one wishes to prove that a system will always avoid an unsafe set U, one can either show that the forward reachable set. does not intersect with U or that the backward reachable set from U does not include the current system state. Therefore, the forward reachable set is primarily useful for verification of the overall system when considering specific "worst-case" initial set to start. from, whereas backward reachable set lets the agent know in general from what states safety is guaranteed and from which states it is not (Mitchell, 2007p. Due to its scalability and despite some trade-offs in generality, substantial portion of recent work has focused on reachability analysis as a viable means of verifying learned policies.
in backward reachable set. A visualization for these two types of reachability com-
Set- Pet-Propagation Set-propagation methods for systems with NV controllers naturally emerged by combining existing techniques developed independently for N and for dynamical systems without learned components. Several carly works took this integrative approach and adapted methods from both com- munities to analyze closed-loop systems. Early works by Xiang et al. (2018); Xiang Johnson (2018) explicitly computed the bounds of a hypernre that over-approximated the output set of [Re.UU-based [N These bounds could then be passed to external tools for reachability analysis of systems modeled by piecewise-linear or Ordinary Differential Equation (ODE) based dynamics. The Verisig method proposed by Ivanov et al. (209)) similarly repurposed existing system-level reachability tools by translating sigmoid-based NNI into hybrid systems modeled with ODE, leveraging the fact that the sigmoid acti- vation function satisfies quadratic differential equation. This translation enabled the composition of the neural network with the environment. model, allowing the verification ion of the result- ing closed-loop system using tools such as Flow* |Chen et al. 2013). However, these early approaches frequently suffered from significant aprriopproiiiaiion errors during reachability analysis. major factor contributing to this issue was the re- liance on hyper-rectangulat set representations, which poorly capture the often non-convex and non-contiguous output sets produced by neural networks. More generally, directly composing the reachability computations of the NN and the dynamical system often led to rapidly compounding approximation errors due to the well-known wrapping effect (Neumaier 1993). To address these issues, Dutta et al.) (2019) proposed approximating the NN function itself, rather Ihan just its output set, using a Taylor polynomial over a specified input range. Their tool, named Sherlock, used this approxi- mation to create a more accurate composite model with the environment, leading to improved precision the computed reachable set. This Taylor-mode soyllr-moodl approach was later adopted and refined in follow-up versions of Verisig, which introduced techniques such as Taylor-model precondit preconditioning and shrink- (Tvanov)
X1
U
(a) Forward Reachable Set
x1(t)
(b) Backward Reachable Set
Figure 6: Reachability analysis for contin- nous and hybrid systems can verify all pos- sible ways a system may evolve from a set of initial states 1, or all possible ways a sys- tem might possibly evolve toward a set of unsafe states U.
12

---

Published in Transactions on Machine Learning Research (12/2025)
2020 2.21). A related line of work replaced Taylor models with Bernstein polynomials to abstract et al.
the NN was implemented in the ReachNN tool (Huang et al. 2019). This approach reduced dependence on specific activation functions and supported networks with heterogeneous activations.
Several Long-maintained software packages for general reachability analysis, such as CORA (Althoff 2015) and JuliaReach (Schilling et al., 2022), have also been extended over time with implementations of reach- ability algorithms specifically for NNs. Newer packages such as NNV (Tran et al.) 2020b) have also been verification. NNV implements wide range of NN reachability algo-
created with an explicit focus on NN
rithms based on different set representations, including polytopes (Tran et al. 2019b), star sets (Tran et al. 2019a), zonotopes (using CORA) (Singh et al. 2018 Althoff 2015), and ImageStar sets (Tran et al. 20200a When combined with system-level reachability components, NNV supports verification of both feedforward and convolutional NNN, across a variety of activation functions. The follow-up work, NNV 2.0, significantly enhanced the tool's scalability and numerical accuracy, while extending support to neural ODE: semantic
segmentation networks, and RNN
[Lopez ct al. 2023).
The recent work by Hashemi et al. (0233) extended set-propagation methods beyond one assumption common in the work discussed so far: the restriction to handling reach-avoid specifications. Instead, they targeted more general specifications written in STL by constructing an NN that maps initial system states to the robustness of the resulting trajectories with respect to the ISTL formula. Using this network, they applied standard neural network verification techniques to determine whether any initial states lead to negative robustness values. This enabled the verification of policies against a broader class of temporal specifications using only reachability-bass -bsee methods.
Sensitivity Analysis Despite extensive development of set propagation methods, they are fundamentally limited by their reliance on specific formats of highly accurate and deterministic system models. For the most part, they have only been successfully demonstrated double-integrator dynamics, quadrotors, or simplified autonomous cars. A slightly more flexible approach is available through simulating many trajectories and analyzing their sensitivity to their starting conditions
small, simulated robotic domaīns featuring
properties of the system related to trajectory divergence (Girard & Pappas 2006 Donzé & Maler 2007;
Ramdani et al., 2008 Duggirala et al. 2013). One use case for scnsitivity analysis is significantly refining the reachable sets computed by set propagation methods (Ladner & Althoff, 2023). It can also be used on its own, as in one algorithm presented in Hashemi et al. (2023) that supports a more general class of models not necessarily using ReLU activations. Their method involves dense sampling of the initial state space and estimation of the Lipschitz constant of the system's robustness function, which is then used to certify the remaining, non-sampled points. Tn a similar spirit, (Salamati et al. (2020) propose a Bayesian inference- based technique that operates with partially unknown model and dataset of trajectories, constructing a posterior distribution to characterize the satisfaction probability of a given STL specification.
Hamilton-Jacobi Reachability Analysis Finally, a significant body of recent research on verifying NN policies is based on Hamilton-Jacobi (H reachability analysis. At its core, HJ reachability models the interaction between control system and an adversarial disturbance
two-player zero-sum differential game (Tomlin et al., 2000). The goal is to determine the set of states from which the system can be guaranteed to reach (or avoid) a target set despite worst-case disturbances. This problem can be formulated as a HD Partial Differential Equation (DE). whose solution defines a value function over the state space. The zero-superlevel set of this value function then represents the set of states from which one player is able to win, which yiclds the backward reachable set from a given target region (Mitchell et al. 2005).
Unlike set-propagation or sensitivity- methods, traditional HJ reachability solves PDE using grid- based dynamic programming, whose computational cost scales exponentially with the dimension of the state space. Addressing this scalability issue has been a central focus in rescarch efforts aiming to apply HJ reachability to the verification of realistic robot policies. One promising direction is the use of deep-IRLlstyyll approximate dynamic programming techniques, with a contractive operator derived from the reachability PDE to obtain a apppoooiintt reachability value function (Fisac et al., 2019 Hsu et al., 2021). These
methods improve scalability and, crucially, eliminate the need for an explicit model of the system, which has allowed for their application to ensure collision avoidance in more complex robotic navigation domains YYu
13

---

Published in Transactions on Machine Learning Research (12/2025)
et al. 2022 Manganaris et al., 2025]. However, they typically require large amounts of simulated experience and may carry safety risks during training, limiting their applicability in real-world deployments unless reliable simulators or offline data are available. A second major approach, introduced by [annall & Tomlin
(2021), draws inspiration from dbysics-informer N The HJ reachability PDE] is encoded directly into a
loss function that provides guidance for training a sinusoidal NN Sitzmann et al. 2020 to approximale the
PDE solution. While this method therefore assumes known system dynamics and boundary conditions, it benefits by being significantly more data-efficient than RRL-iike sample-based approaches.
Extensions of HJ reachability methods have also been proposed to support several new features. Bansal ct al. (2020) developed a method for the computation of probabilistic reachable sets and applied them to a real-world system for safe navigation around humans, where reasoning about uncertainty and intent is essential. Yu et al. (2022) integrated reachability- based safety methods with RL frameworks to jointly learn safe policies and their associated safety sets. Some work has even bridged H
reachability with ST (Chen ct al. (220)) to support richer classes of safety specifications, similar to some efforts for extending set-propagation methods.
4.3 Certificate- Based Methods
A separate category of verification methods derives proofs of [FS] satisfaction through certificate functions. Certificate function methods rely on the construction of scalar functions, namely Lyapunov functions, barrier functions, and contraction metrics, whose existence imply desirable control properties. The existence of a Lyаршnоv function V certifies that a system will eventually be driven toward a stable equilibrium region since it guarantees that (xx) always decreases along system trajectories to eventually reach zero. In contrast, a barrier function B certifics safety rather than goal-reaching: if the system starts within the zero sublevel set {a : B(x) < 0}. then the system remains in this safe set indefinitely, never crossing into the superlevel set (a : (ar) > 0)- Finally, contraction metries generalize Lyapunov functions to time-varying or trajectory- tracking contexts. Instead of certifying convergence to single point, they prove that a system can be controlled to converge exponentially toward any given reference trajectory, provided it is dynamically feasible (i.e., it is physically realizable by the system). An informative introduction and survey of these techniques is provided by Dawson et al. (2023), so we do not include a full theoretical treatment here. Instead, we focus on how certificate function methods have been developed in the context of robot policy learning and how they compare to other available verification approaches, particularly in terms of their flexibility, scalability, and applicability to learned or black-box systems.
First, the FSB verified by certificate functions are typically limited to properties concerning system invariance (e.g., remaining on one side of a barrier) or stability (e.g., eventually converging to an equilibrium point or trajectory). Therefore, such properties are not often expressed as [TL specifications, but we include them as such in Figure 5 to facilitate comparison with other verification methods. In exchange for their restricted expressiveness, certificale-basse methods offer the advantages of not requiring an explicit system model and achieving favorable computational scalability.
Certificate functions are typically obtained via search over set of differentiable functions, with constraints determined by the particular certificate type (Dawson et al. 2023). Naturally, this function space can be represented using a family of parameterized N enabling the search to be cast as an unconstrained training problem by incorporating the constraints as penalty terms. The objective is then evaluated empirically over a finite set of system trajectories. This forms the basic approach underlying most certificate-leammine methods [Richards et al. 2018 Srinivasan et al.] 2020 Zhao et al.) 2020). Subsequent research has addressed both challenges and downstream applications of this approach, including joint learning of certificates and dynamics models (Kolter & Manek 2019), rapid adaptation of nominal certificates to new systems (Taylor et al. 2019). and new NN architectures tailored for certificate functions (Gaby et al. 2022). One particularly important challenge is that Icarned certificate functions resulting from this procedure are not guaranteed to satisfy their required properties. This can arise for two reasons: the NN search space may be too limited to contain a valid certificate or the empirical loss may fail to enforce constraints across the entire state space. Therefore, new synthesis techniques for certificate functions, based on (EEGI) have been introduced which adaptively refine the NN space (Perfffo al. 2021) and verify certificate constraints using Satisfiability Modulo
Theories -solvers (Abate et al. 2021b a Edwards et al. 2024
14

---

Published in Transactions on Machine Learning Research (12/2025)
ceatffaatt-funcciio-aassss verification also differs from other methods in that it naturally extends to obtaining formally verified controllers. Just as Lyapunov and barrier functions certify the stability and invariance of a closed-loop control system, Control Lyapunov Functions (CLFF) and Control Barrier Functions (CBF)) certify the existence of a controller for an open-loop system that can yield a closed-loop system with the desired properties. This has motivated several lines of research inlo certificate-regulaaiieed imitation learning
CCosner et al.] 2022) and reinforcement learning (Perkins & Barto 2003 Chow ct al. 2018 Chang et al.
2019 Xiong et al. 2022; Marvi & Kiumarsi 2021: Emam et al. 2025
Ma et al. 2021: Ahmad et al. 20255)
In this context, other work has also sought to generalize [CLFs and CBF$ to support broader range of
STL.specified properties, through constructions such as time-varying CBF (Lindemann & Dimarogonas,
2019 For a more comprehensive overview of certificate-based policy learning methods and of certificate functions more broadly, we again refer the reader to Dawson el. al. (2023
4.4 Run- Time Monitoring Methods
Another class of methods that achieve considerable scalability in exchange for weaker guarantees is run-time monitoring. As opposed to the above methods, which are capable of up-front policy verification before they are deployed, these methods continuously check during execution whether policy is encountering error. Once detected, the policy can be halted to allow for user intervention. For run-time monitoring with robot policies, errors can be defined in several ways that can be grouped into two categories: those defined by explicit logical specifications and those defined by hcuristics or implicit specifications.
Run-time Monitoring with Explicit Specifications Run-time monitoring emerged as lightweight alternative to full model checking for complex systems, motivated in part by the combinatorial explosion of model state spaces, and is therefore classically intended to verify formally specified properties. Simulating or recording the behavior of robotic systems and evaluating specifications (e.g., STTL formulae) over recorded behavior similarly allows for qualitatively evaluating otherwise intractable systems. Consequently, several foundational algorithms have been developed for efficiently evaluating the satisfaction and robustness of
temporal logic specifications over system recordings, either online or offline (Donzé et al., 2013 Bartocci
et al. 2018). More recent. work exlends these foundations to more expressive temporal logics (Bakhrrkin
& Basset 2019 Bonnah & Hoque 2022 Chalupa & Henzinger 2023) as well as preemptive detection of
errors. As opposed to purcly retrospective monitoring algorithms that detect specification violation only with measurements preceding the current limestep, predictive moniloring algorithms also forecast. future system behavior during monitoring to detect imminent violations using provided (Pinisetty et al. or learned predictive models {Ferrando & Delzamno 2023 Lindemaann al. 2023 Henzinger al.
2017)
2025
While these algorithms are generally applicable to many systems and specifications, they are, result, less specialized for errors encountered with robotic policies.
Run-time Monitoring with Implicit Specifications Certain errors for robotic systems can be more readily defined without typical FS formats. Instead, they can be heuristically defined by sets of states labeled with known, domain-specific failure modes (Farid et al.) 2022 Daftry et al. 2016 Rabice & Biswas 2019: Inceoglu et al. 2021 Yu et al. 2025 by anornaoou behavior associaled with visiting out-of- states (Marco et al. 2023 Xu et al.] 2024), or by feedback from large vision-language models (Agia et al. 2024). Like specification--ased run-time monitors, these methods can also benefit from learned dynamics models for explicitly computing representations of future states encountered by a policy and subsequently classifying those states as invalid (Tiu et al.; 2024 Marco et al. 2023). The scalability of these methods has translated into their successful deployment across a variety of complex robotics domains, including real-world single-arm and dual-arm manipulation tasks (Agia et al.) 2024 Liu et al. 2024 Xu et al. 2024). However, supporting more general and precise notions of correctness, as in specification- based monitoring, on complex robotic systems is an important objective for future rescarch.
4.5 Falsification
Finally, several notable methods have instead focused on the problem of falsification, which is the process of showing that there exists an initial state from which the system follows trajectory 7 that does not
15

---

Published in Transactions on Machine Learning Research (12/2025)
satisfy specification ø. Although this is logically separated from the normal verification problem by only single negation operator, this form of the problem can be solved more easily and can accommodate fewer assumptions on the system, controller, and specification. One benefit of this formulation is that one can find a solution with a non-cxhaustive search through the set I. Broad approaches to falsification can involve some form of stochastic optimization (Das et al. 2021 or robustness-guided RL Yamagata et al. 2021 More recent approaches have enhanced the falsification process by incorporating notions of NN "coverage, aiming to ensure that the network's activation space thoroughly explored during falsification (Zhang al.
2023 This idea draws inspiration from prior work automated NN testing (Sun et al.) 2018 Pei et al, 2019 Other studies such as Dreossi et al. (2019) address the falsification of autonomous driving systems that combine DL for perception with classical control. Their approach alternates between searching for specification violations in the classical control component, using inpuls from various abstractions of the DL perception component, and identifying errors in the component itself, which then inform and refine those abstractions. These methods, although not providing the same guarantoes as strict verification mothods, reflect the trade-off between guarantees and practicality. They still provide insight into the safety of a learned robot policy but scale better to real-world scenarios and require significantly fewer assumptions the network architecture and model design.
5 Future Research Directions
In this section, we summarize the most significant remaining questions in formally specifying robot behavior, learning policies that obey those specifications, and verifying that existing learned policies do the same. For each specific area of FMs covered in our survey, we describe nascent and hypothetical research directions that we believe are promising. Morcover, we motivate these new directions with links to the larger themes we have identified as already having significance in the work surveyed above. For instance, one theme the struggle optimally balance formality (i.e., the strength of guarantees) and versatility (i.e., the supported specification, policy, and system complexity). Another is the increasing reliance scalable numerical methods and decreasing favor for discrete-bsstraation- based methods for synthesis and verification. We hope that these directions and the underlying trends they belong to will promote future research toward the complete resolution of safety, interpretability, and trustworthiness concerns in the modern wave of robot lcarning research.
5.1 Specifications
Specification Mining for Robotic Systems In robotics and artificial intelligence, comprehensive, cor- rect, and formal specifications become harder to define as the ability to effectively model the system decreases, which is the case for the complex, real-time, continuous, stochastic, and partially-obserrabbee dynamical sys- tems in robotics. Although
[SM has been explored in the context of robotics (Liu et al. 2022 Shah et al.
2018 Soroka et al. 2025 Topper el. al. 2022 Dohmen al., 2022 Gaon & Bralman 2020 Rens et al. 2020
Xu et al. 2021 Abate et al. 2023) there are several ways in which these works are limited. For instance, extracting specifications from the full STL grammar remains intractable due to the immense search space and the limited guidance provided by datasets of observed behaviors alone. A key direction for making this problem more tractable is to incorporate external domain knowledge to constrain and guide the search. This presents a natural opportunity to leverage foundation models, which have already been shown to be highly effective complements to genetic algorithus (Novikov el al. 2025). Incorporating knowledge from a system model either by simulating additional trajectories to enrich the dataset or by analytically reasoning about the model's behavior to rule out irrelevant specifications is another underexplored and promising direction. Current SM methods lypically focus extracting a narrow class of specifications from specific types of datasets, most often learning unconditional formulae from complete system traces. Future work may expand this scope by targeting conditional, input-output specifications, especially from partial or noisy observations. This capability is particularly critical when learning specifications that express infinite-horizon objectives, which cannot be directly observed from finite traces (Bartocci et al. 2022).
More Expressive Specifications Certain robotics tasks will also require fundamentally more expressive logical languages for robotics applications, providing mechanisms to express high-level, geometrically-rich,
16

---

Published in Transactions on Machine Learning Research (12/2025)
abstract concepts that are difficult to represent formally. Several works have already pursued this goal by developing new temporal logics both inside and outside of robotics (Sadigh & Kapoor 2016 Dokhanchi ltt al. 2019 Hekmatnejad 2021 Balakrishnan et al. 2021 Kapoor et al. 2025), but most of these techniques have not yet been applicd to robot policy learning or verification. For this last application, we believe new specification formats hold significant, promise in their ability to represent interpretable, temporally structured representations of goals for artificially intelligent agents.
Partial Observability and Stochasticity Obtaining and representing specifications for systems featur- ing stochasticity and partial observability still faces significant challenges. There are a small number of works that investigate stochasticity (Yoo & Belta 2015 Sadigh & Kapoor 2016) or partial observability [Kapoor et al., (025)) individually, but no methods pursuing them simultancously. Future work may extend the approach taken by Kapoor al. (2025) and incorporate more sophisticated probabilistic models to build formal specifications whose semantics would scamlessly incorporate multi-modal distributions for atomic propositions. This would continue the trend of increasing [S flexibility at the expense of formal specificity, so additional work may be necessary to ensure adequate guarantees are obtained when applying learning and verification methods with these FSJ. Furthermore, future work can extend FS representations and im- plementation frameworks to accommodate these more advanced features. Differentiable SYT frameworks can naturally be extended to support differentiable probabilistic models. Automaton-based representations supporting probabilistic propositions also exist (Rheinboldt & Paz 2014) but may need adaptation for
application to automata-theoretic RL
5.2 Learning
Handling more Complex Specifications A broad limitation in current research on learning with FS. lies in the temporal complexity of the supported FS We believe this gap presents an opportunity for both theoretical and practical advances. On the theoretical side, several fundamental obstacles have been identified in satisfying complex [TL specifications with RL Addressing these challenges remains open and compelling direction for future work. Notably, Yang et al. (2022a) showed that RL for almost all CTL objectives under their default semantics is intractable. That is, no algorithm can guarantee near- optimal performance for anything beyond the simplest [TL objjctive given any finite number of environment interactions. Despite this, the work surveyed in Section 3.1 demonstrated progress by relying on reward structures derived from relaxed LTL semantics. Nonethcless, developing a unified and thcoretically grounded solution to this intractability will be essential for establishing more robust foundation for combining In and RL One promising direction comes from Alur et al. (2023), who propose discounted LTL semantics that avoids the intractability while still supporting expressive specifications. This framework offers a fertile starting point for future rescarch. It is also the case that these more complex specifications quickly increase the sparsity of rewards. heoretial research efforts can investigate practical strategies to deal with this sparsity, such as incorporating additional model-informmaiinn with differentiable simulators (Bozkurt et al., 2025), introducing additional rewards in the experience replay buffer (Voloshin et al., 2023). , or improving exploration with evolutionary RL algorithms (Zhu et al. 2021).
Combining Formal and Informal Requirements A distinct avenue for future research could focus more on combining LTL constraints with standard RL objectives (Voloshin et al. 2022 Shah et al. 2025 as opposed most existing algorithms that focus on solving tasks completely specified with an HTL expression.
This is a critical direction, as many FSS are more meaningful when the agent pursues a competing objective, requiring it to make optimal decisions that balance task performance with formal constraints. For instance, complex specifications imposing recurrence and fairness constraints can have trivial solutions in the absence of another objective. A concrete example would be the specification "always periodically return to the charging station" " in combination with a normal reward for collecting and organizing objects. The agent must then decide when to pursue objects and when to return to the charging station. Without the competing objective, however, the optimal behavior would be to remain permanently at the charging station. A possible starting point for this new direction is generalizing standard constrained MDP constraints (Altman 1999) to accommodate functions of [ии reward or STL robustness.
17

---

Published in Transactions on Machine Learning Research (12/2025)
Better Offline Learning with FS: In the same vein, combining constraints with objectives used in offline policy learning is also fruitful and insufficiently explored direction. Offline policy learning is already considered important for safety critical settings due to circumventing the need for safe exploration, so their combination is natural. A possible broad strategy for this direction could apply innovations in auteoei RL for handling non-Markovian objectives to the oflline RL setting (Levine et al. 2020). Initially, rescarch in this direction may examine enhancing methods for offline RL specifically for datasets with rewards and automaton- an-anneentd states. Later work can then return to using standard datasets and incorporate additional guidance from specifications mined from the available data. The final and most ambitious goal for this line of research would be eventually generalizing the concept of learning optimal goal- reaching policies from suboptimal data to learning general specification- policies from suboptimal data not necessarily following any specification.
5.3 Verification
The final major research direction we identify from the surveyed literature is in continuing to scale verification methods to support modern robot policies deployed in the real world. While recent work has already made substantial progress toward verifying satisfaction Ff under increasingly relaxed assumptions and for more complex, high-dimensions systems, many of these methods still face significant challenges. Among the most scalable and promising lines of research are sampling-basee (Lew & Pavone 2021), synamic-prorramming (Bansal & Tomlin 2021 Edwards et al.) (024) verification methods. We anticipate these will provide a foundation for research into more practical verification pipelines, and that this research will proceed by carefully conceding the currently offered strong guarantees in exchange for computational scalability, fewer assumptions on the environment, and support for significantly more complex models.
based Hsu et al., 2021 Fisac et al., 2019), and general neural-network- neuna--netort--pppoiiaaaioo
Relaxing Guarantees for Scalability Several recent. works already exemplify this trend through prob- abilistic extensions of reachability analysis and certificate-based methods, which allow for reasoning over stochastic system models (Lew & Pavone 2021 Bansal et al. 2020 Castañeda et al.] 2024). This proba- bilistic framing is particularly suitable for unstructured real-world deployment, where full certainly of future events is almost never achievable and models ought to be learned or refined online. Complex stochastic world models will also integrate well with runtime verification methods (Liu et al. (0224). While these meth- ods may only provide probabilistic guarantees for short durations, they are the most feasible approach for long-horizon tasks in realistic, uncertain environments. This integration may also open avenues for applying these runtime verification methods to standard FS
Large Policy Models A significant amount of future work can also be devoted to scalability concerns for more modern, complex policy models, which remains bottleneck for many formal verification pipelines. Many current verification tools operate under restrictive assumptions about policy architectures; they often only target shallow or moderately deep feedforward networks with serewwse--innan activations. However, state-of-the-art policics incrcasingly rely on much more expressive and compositional architectures such as transformers and diffusion-based generative models. These models are often opaque to current analyzers, but initial efforts toward verifying these models using reachability analysis have begun to address these challenges (Shi et al. 22000] Manzanas Lopez et al., 2022 Ladner et al., (022).. Still, future research will have to further extend these techniques or pioneer new ones to handle the complex networks used in real robot policies.
New Formal Methods for Robot Policies Future work may also develop FM that are currently under- utilized for learned robot policies. For instance, repairing NN to eliminate identified faulty behavior is an active rescarch area, with many developed techniques for localizing and efficiently retraining the responsible
network components (Sotoudeh & Thakur 2021 Dong et al.] 2021 Usman et al. 2021 Sun et al. 2022a Yang et al. 2022b Sohn et al.) 2023] Xing et al. 2024 Majd et al. 2024 Tao & Thakur 2025). However, such techniques are most often applied to NN$ used as classifiers, generative models, or discrete-action policios, and they rely on specifications whose expressiveness is tailored to those uses. Additional development is necessary for techniques to repair policies operating within the complex dynamical systems encountered by
18

---

Published in Transactions on Machine Learning Research (12/2025)
real robots, as well as to specifications capable of expressing behavioral requirements in those domains. FM for hierarchical or multi-agent policies are also far less explored than those for single-agent systems. Progress in this direction could draw on compositional verification techniques that decompose and verify systems using a divide-and-conquer strategy (Cobleigh et al. 2003 Alur et al.) 2005). Some recent works achiove this decompasition in multi-agent systems by leveraging cont.acts that express each agent's individual guarantees and assumptions about other agents ("assumo-guarantee contracts") (Saoud et al.) 2021 Liu et al. 2025a] Kazemi et al. 2024). Still, these approaches have not yet been applied to poliey learning or verilication scales needed for practical robot deployment.
6 Conclusion
Throughout this survey, we have shown that FM in robot learning have the potential to substantially increase our confidence in the safe, correct, and reliable behavior of autonomous systems. To make this case, we reviewed current research in two major categories. We first surveyed techniques for learning policies that satisfy formal definitions of correct behavior (Section 3). These include methods that incorporate automata-based rewards and other forms of FS guidance into model-free and model-basod RL techniques in [RL that produce interpretable reward functions in the form of FSS, and behavior cloning and offline RE approaches that. integrate supervision or conditioning based FSS. We then examined a complementary set of verification techniques that aim to prove that learned policy satisfies given specification (Section 4 The verification methods range from the most formal but least scalable methods based on formulating and checking discrete enviromment abstractions, to the more focused, moderately scalable reachability-analye methods, and finally to the most specific and scalable certificate function methods. We also surveyed work in the less formal but practical categories of run-time monitoring and falsification. These summaries provide a comprehensive overview of the current rescarch on FMs applied learned robot policies.
Beyond summarixing the state of the art, our survey provides coherent organizational framework that can serve both new entrants and seasoned researchers in the field. For readers newly approaching formal methods in robot learning, we have included accessible illustrations, examples, and conceptual overviews that clarify foundational ideas and map out the landscape of existing work. For experts, our survey provides a concise taxonomy of techniques, starting points for deeper inquiry into all the surveyed subficlds, and connections between them that are not often discussed. Most importantly, by summarizing current trends FM informed policy learning and verification research, we identify a number of underexplored questions and promising future directions (Section 5). In particular, emphasize the need to increase the expressiveness and scalability of current FM to support learning and verification in real-world robotic systems. We believe that achieving this goal will be essential for ultimately deploying robots that are not just performant, but provably safe and trustworthy.
Acknowledgments
This matorial is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-24-1-0233. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force.
References
1X Technologies. NEO Gamma 1X, 2025. URL htttps///ww...... 1x. tech/neo
Alessandro Abate, Iury Bessa, Lucas Cordeiro, Cristina David, Pascal Kesseli, Daniel Kroening, and Eliza- beth Polgreen. Automated formal synthesis of provably safe digital controllers for continuous plants. Acta Informatica, December 2019. ISSN 1432-0525. doi: 00.10077000033---199003333
Alessandro Abate, Daniele Ahmed, Alec Edwards, Mirco Giacobbe, and Andrea Peruffo. FOSSIL: software tool for the formal synthesis of Iyapunov functions and barrier certificates using neural networks. In International Conference on Hybrid Systems: Computation and Control (HSCC), New York, NY, USA, 2.21a. ACM. ISBN 978-1-450-38339-44 doi: 10. 10.11455/34499288355566
19

---

Published in Transactions on Machine Learning Research (12/2025)
Alessandro Abate, Danicle Ahmed, Mirco Giacobbe, and Andrea Peruffo. Formal synthesis of Lyapunov neural networks. IEEE Control Systems Letlers, 5(3):773-77) 778, July 2021b. ISSN 2475-1456. doi: 10.1109/ lcsys.2020.3005328.
Alessandro Abate, Yousif Almulla, James Fox, David Hyland, and Michael Wooldridge. Learning task aulomata for reinforcement learning using hidden Markov models. In Europewn Conference on Artificial Intelligence (ECAT), pp. 3-10. TOS Press, 2023. doi: 10.3233 10.3233/FATA222 FATA230247.
Christopher Agia, Rohan Sinha, Jingyun Yang, Zi-ang Cao, Rika Antonova, Marco Pavone, and Jeannette Bohg. Unpacking failure modes of generative policies: Runtime monitoring of consistency and progress. In Conference on Robot Learning (CoRL), volune 270 of Proceedings of Machine Learving Research, pp. 689-72 -23. PMLR, 2024.
H M Sabbir Ahmad, Ehsan Sabouni, Alexander Wasilkoff, Param Budhraja, Zijian Guo, Songyuan Zhang, Chuchu Fan, Christos Cassandras, and Wenchao Li. HMARL-CBF Hicrarchical multi-agent reinforce- ment learning with control barrier functions for salety-critical aulonomous systems. In Advances in Newral Information Processing Systems (NeurIPS), 2025. URL https://openreviest www.:eeeeffoffitmmm IcLVgvyiqV
Derya Aksaray, Austin Jones, Zhaodan Kong, Mac Schwager, and Calin Belta. Q-learning for robust satis- faction of signal temporal logic specifications. In IEEE Conference on Decision and Control (CDC), pp. 6565-66 6570, 2016. doi: D0.109/CCDC.2216..779999
Matthias Althoff. An introduction to CORA 2015. In Workshop on Applied Verification for Continuous and Hybrid Systems (ARCH), pp. 120 151. EasyChair December 2015. doi: 10.29007/zbkv. URL https:
//easychair.org http:///p/paaaaaaoo//pppeeeeeeee
Matthias Althoff, Goran Frehse, and Antoine Girard. Set propagation techniques for reachability analy-
sis. Annual Review of Control, Robotics, and Autonomous Systems, 4(Volume 4, 2021):369-38 2021.
ISSN 2573-5144. doi: 10. 10111/////mueee--ooo000000000000
URL attpp://www.amuarreviiee. org/
content/journals//u 10. 1146/ anw.rev /otr://rooooorrr110110000----000
Fitan. Altman. Constrained Markov Decision Processes. Stochastic Modeling Series. Taylor & Francis, 1999.
ISBN 978-0-8493-03822 --
Rajeev Alur, Costas Courcoubetis, and David Dill. Model-checking for probabilistic real-time systems. Javier Leach Albert, Burkhard Monien, and Mario Rodríguez Artalejo (eds.), Automata, Languages and Programming, volume 510 of Lecture Notes in Computer Science, pp. 115-126, Berlin, Heidelberg, 1991. 128.
Springer. ISBN 978-3-540-44516-3, doi:
Rajeev Alur, Costas Courcoubetis, Nicolas Halbwachs, Thomas A. Henzinger, Peihsin H. Ho, Xavier Nicollin, Alfredo M. Olivero, Joseph Sifakis, and Sergio Yovine. The algorithmic analysis of hybrid systems. Theo- retical Computer Science, 138(1):3-31) 34, 1995. ISSN 0304-3975. doi: 10. /000440-97504440002222 URL wwtp:::tttccccccccccceeeeeecccte http://www..tit..............tta /
Rajoev Alur, P. Madhusudan, and Wonhong Nam. Symbolic compositional verification by learning assump- tions. In Kousha Etessami and Sriram K. Rajamani (eds.), International Conference Compuler Aided Verification (CAV), volume 3576 of Lecture Notes in Computer Science, pp. 548-562, Berlin, Heidelberg, 2005. Springer. ISBN 978-3-540-316888 doi: 10.1007 11100111113988 52.
Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo M. K. Martin, Mukund Raghothaman, Sanjit A. Seshia, Rishabh Singh, Armando Solar-Tezama, Emina Torlak, and Abhishek Udupa. Syntax-guided synthesis. In Conference on Formal Methods in Computer-Alle Aided Design (FMCAD), pp. 1 1-8, 2013. doi: 10.1109/
Rajeev Alur, Rishabh Singh, Dana Fisman, and Armando Solar-Lezama Search-based program synthesis. Commanications of the ACM, 61 (12):884 93, November 2018. ISSN 0001-0782. doi: 10.1145/320800
20

---

Published in Transactions on Machine Learning Research (12/2025)
Rajeev Alur, Osbert Bastani, Kishor Jothimurugan, Mateo Perez, Fabio Somenzi, and Ashutosh Trivedi. Policy synthesis and reinforcement learning for discounted LTL. In Constantin Enea and Akash Lal (eds.), International Conference on Computer Aided Verification (CAV), volume 13964 of Lec- ture Notes in Computer Science, pp. 413-435, Cham, 2023. Springer. ISBN 978-3-03--37706-5 doi: 00.1007/998-33033--77006 21.
Glenn Ammons, Rastislav Bodik, and James R. Larus. Mining specifications. In ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL), pp. 4-16, New York, NY, USA, 2002. ACM. ISBN 1-581-13450-9. doi: 10.1145/ 10.1145/503272..55 503272.503271
Philip W. Anderson. More is different. Science, 177(4047):393 396, 1972. doi:
sccince.. 177.4047.390
Brandon Araki, Xiao Li, Kiran Vodrahalli, Jonathan Decastro, Micah Fry, and Daniela Rus. The logical options framework. In Marina Meila and Tong Zhang (eds.), International Conference Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Research, pp. 307 317. PMLR, 2021. URL
https://proceedding mlr. prwss/ /V//ss/vv///// wwwp///aa//9........ attaaiiii.hmml...
Sara Baase. A Gift of Pire: Social, Legal, and Ethicol Issues for Compuling and the Internet. Alvernative Etext Formats. Pearson Prentice Hall, 2008. ISBN 978-0-136-00848-4 URL http::ttt//:::sts.gge google. com/
books"id==Gcaa577744g
Marco Bagatella, Andreas Krause, and Georg Martius. Directed exploration in reinforcement learning from lincar temporal logic, 2024. URL https:://axxi..rr//////////// www./bra///aaeeeeeeeeeee 09495
Christel. Baier, Joost-Pieter. Katoen, and Kim Guldstrand. Larsen. Principles of Model Checking. MIT Press, 2008. ISBN 978-0-262-02649-9.
Alexey Bakhirkin and Nicolas Basset. Specification and efficient monitoring beyond stl. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 11428 of Leclure Noles in Compuler Science, pp. 79 79-97, Cham, 2019. Springer. ISBN 978-3-030-17465-1. doi: 10.1007/978-3-0033 17465-1
Anand Balakrishnan and Jyotirmoy V. Deshmukh. Structured reward shaping using signal temporal logic specifications. In JEEE/RSI International Conference on Intelligent Robots and Systems (IROS), pp. doi: 10.1109/IRRO /R00//RRR009922000000 201...992.....54..
3481-3486, 2019.
Anand Balakrishuan, Jyotirmoy Deshmmuk,, Bardh Hoxha, Tomoya Yamaguchi, and Georgios Fainekos. PerceMon: Online monitoring for perception systems. In International Conference on Runtime Verification (RV), volume 12974 of Lecture Notes in Computer Science, pp. 297 297-308, Berlin, Heidelberg, 2021. Springer- Verlag. ISBN 116-3-030-88449--2 doi: 14.1007/978-3-10 000-3---000-8------ 18.
Somil Bansal and Claire J. Tomlin. DeepReach: deep learning approach to high-dimensional reachability. IEEE International Conference on Robotics and Automation (ICRA), pp. 1817 1824, 2021. doi: 10. 902.99006220199519999
1109/ICRA4850662022
Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J. Tomlin. Hamilton-jacobi reachability: A brief overview and recent advances. In IEEE Conference on Decision and Control (CDC), pp. 2242-2253, 2017. doi:
10. 10110//CDC.2011.826555
Somil Bansal, Andrea Bajesy, Ellis Ratner, Anca D. Dragan, and Claire J. Tomlin. A Hamilton- Jacobi reachabillty-baseed framework for predicting and analyzing human motion for safe planning. IEEE International Conference on Robotics and Automation (ICRA), pp. 7149 7155, 2020. doi: 10.1109/ICRRA44945... 45..4499995.002999975557..
Ezio Bartocci, Jyotirmoy Deshmukh, Alexandre Donzé, Georgios Fainekos, Oded Maler, Dejan Ničković, and Sriram Sankaranarayanan. Specification- based monitoring of wuber-physical systems: A survey on theory, tools and applications. In Ezio Bartocci and Yliès Falcone (eds.), Lectures on Runtime Verification: Introductory and Advanced Topics, volume 10457 of Lecture Notes in Computer Science, pp. 135-175. Springer, Cham, 2018. ISBN 978-3-319--5632-5. doi: 10.1007/976-33311--766322
21

---

Published in Transactions on Machine Learning Research (12/2025)
Ezio Bartocci, Cristinel Matcis, Elconora Nesterini, and Dejan Nickovic. Survey mining signal temporal logic specifications. Information and Compulation, 289:104957, 2022.
Osbert Bastani, Yewen Pu, and Armando Solar-Lezama, Verifiable reinforcement learning via policy extrac- tion, 2019. URL /tt/://tt////xxxxoooo/////////. wwtp:orrraab///...02222
Calin Belta and Sadra Sadraddini. Formal methods for control synthesis: An optimization perspective. Annual Review of Control, Robotics, and Autonomous Systems, 2(Volume 2, 2019):115 -400, 2019. ISSN 2573-5144. doi:
Dimitri P. Bertsekas and Ian B. Rhodes. On the minimax reachability of target sets and target tubes. Aulomalica, 7(2):233 247, 1971. ISSN 0005-1098.
Roderick Bloem, Barbara Jobstmann, Nir Piterman, Amir Pmneli, and Yaniv Sa'ar. Synthesis of (eactive(1) designs. Journal of Computer and System Sciences, 78(3):911 938, 2012. ISSN 0022-0000. doi: 10.1016/j. 20s..2011.88.07., URL https: bttpss:/www.. http:///wwwwsceecedrree..tct www.ssciiiieerccecccccc....eeeee
Ernest Bonnah and Khaza Anuarul Hoque. Runtime monitoring of time window temporal logic. IEEE Robotics and Automation Letters, 7(3):5888-5895, 2022. doi: http://www.................////.
Alper Kamil Bozkurt, Calin Belta, and Ming C. Lin. Accelcrated learning with lincar temporal logic using differentiable simulation, 2025. URL
wrg/ 01167
Lukas Brinke, Mclissa Greeff, Adam W. Hall, Zhaocong Yuan, Siqi Zhou, Jacopo Panerati, and Angela P. Schoellig. Safe learning in robotics: From learning- based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5(Volume 5, 2022):411 2022. ISSN 2573- 5144. doi: 10.1146.11... |001 journals/10, 10. 1146/ cmwrr
URL https::/www..ammaalr http:/rrrviiiiooooooooeeteeeeeee
Mingyu Cai, Shaoping Xiao, Zhijun Li, and Zhen Kan. Optimal probabilistic motion planning with potential infeasible LTL, constraints. IEEE Transaclions on Aulomatic Control, 68(1):301-33) -316, 2021.
Mingyu Cai, Zhangli Zhou, Lin Li, Shaoping Xiao, and Zhen Kan. Reinforcement learning with soft tempo- ral logic constraints using limit-determint generalized Büchi automaton. Journal of Automation and Intelligence, 2024. ISSN 2949-8554. doi: 10.
URL
com/acience/ prtosiaceeaaricaaaaaa///99900000
Alberto Camacho, Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. Mcllraith. LTL and beyond: Formal languages for reward function specification in reinforcement learning. In International Joint Conference on Artificial Intelligence (IJCAT), pp. 0065-6073. International Joint Conferences on Artificial Intelligence Organization, 2019. doi: 1.22496//jccai.2010
Fernando Castañeda, Jason J. Choi, Wonsuhk Jung, Bike Zhang, Claire J. Tomlin, and Koushil Sreenath. Recursively fcasible probabilistic safe online learning with control barrier functions, 2024. URL https:
//arxiv. wtt..rgg//bs//aaaa.....// 12732
Marek Chalupa and Thomas A. Henzinger. Monitoring hyperproperties with prefix transducers. In Panagiotis Katsaros and Laura Nenzi (eds.), International Conference on Runtime Verification (RV), volume 14245 of Lecture Notes in Computer Science, pp. 168-190, Cham, 2023. Springer. ISBN 978-3-031-44267-- doi: 9.
10.1007/9725 0-00779978-3001-445577
Ya- Ya-Chien Chang, Nima Roohi, and Sicun Gao. Neural Lyapunov control. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché Buce E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems (NewrIPS), volume 32. Curran Associates, Inc., 2019. URL /ttp/ttpss///prrceenns..eeerrrr
paper_
Mo Chen, Qizhan Tam, Scott C. Livingston, and Marco Pavone. Signal temporal logic meets reacha- bility: Connections and applications. In Marco Morales, Lydia Tapia, Gildardo Sánchez-Ante, and Seth Hutchinson (eds.), Workshop the Algorithmic Foundations of Robotics (WAFR), volume 14 of
22

---

Published in Transactions on Machine Learning Research (12/2025)
Proceedings in Advanced Robotics, pp. 581-601, Cham, 2020. Springer. ISBN 978-3-030-44051-0 doi: 34.
Xin Chen, Erika Ábrahám, and Sriram Sankaranarayanan. Flow*: An analyzer for non-linear hybrid sys- tems. In Natasha Sharygina and Helmut Veith (eds.), International Conference Compuler Aided Verification (CAV), volume 8044 of Lecture Notes in Computer Science, pp. 258 263, Berlin, Heidelberg, 2013. Springer. ISBN 178-3-642-89799-8 doi: 10.1007 1978-3-642-39791
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffiusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 44(10-11):1184--706) 1704, 2025.
Kyunghoon Cho and Songhwai Oh. Learning-basee based model predictive control under signal temporal logic specifications. In IEEE International Conference on Robotics and Automation (ICRA), pp. 7822-7329, 2018. doi: R01109//CRA.2001 2468811
Yinlam Chow, Ofir Nachum, Edgar Duenez- Guzman, and Mohammad Ghavamzadeh. A Lyapunov-based approach to safe reinforcement learning. In Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (ods.), Advances in Neural Information Processing Systems (NeurIPS), vol- une 31. Curran Associates, Inc., 2018. URL
wwwprocceddngg..urisss..o... httppp/sceeppppppaeeseeeeeeeeeee fiaesppppre/peep//////ee paper/ pdf
files/
Xu Chu (Demnis) Ding, Stephen L. Smith, Calin Belta, and Daniela Rus. LTL control in uncertain environ- ments with probabilistic satisfaction guarantees. IFAC Proceedings Volumes, 44(1):3515-335 2011. ISSN
1474-6670. doi:
Alonzo Church. Application of recursive arithmetic to the problem of circuit synthesis. Journal of Symbolic Logic, 28(4)2289 290, 1963.
Jamieson M. Cobeeghh, Dimitra Ciannakopoulou, and Corina. S. PÂsÄreamu. Learning assumptions for compositional verification. In Hubert Garavel and John Hatcliff (eds.), Tools and Algorithms for the Construction and Analysis of Systems, volume 2619 of Lecture Noles in Compuler Science, pp. Berlin, Heidelberg, 2003. Springer. ISBN 978-3-540-36577-8. doi: 10.1007 111100773-500-0 13------6677---- 24.
Max H. Cohen and Calin Belta. Model-based reinforcement learning for approximate optimal control with temporal logic specifications. In International Conference on Hybrid Systems: Computation and Control (HSCC), New York, NY, USA, 2021. ACM. ISBN 978-1-450-38330-4 doi: 10.1145/344798883458833
Ryan K. Cosner, Yisong Yue, and Aaron D. Ames. End-to-end imitation learning with safety guarantees using control barrier functions. In IEEE Conference on Decision and Control (CDC), pp. 5316 5322, 2022.
doi: 10.1109 CCDD55059.2022.9999
Costas Courcoubetis and Mihalis Yannakakis. The complexity of probabilistic verification. Journal of the ACM, 424)8885790 July 1995. ISSN 0004-5411. doi: 10.1145/210832.210333
Maxwell Crouse, Ibrahim Abdelaziz, Cristina Cornelio, Veronika Thost, Lingfei Wu, Kenneth Forbus, and Achille Fokone. Improving graph neural network representations of logical formulae with subgraph pooling, 2020. URL https:: http://axxvv..///bbbb//a/e 19910 06904
Guofeng Cui, Yuning Wang, Wenjie Qiu, and He Zhu. Reward-guided synthesis of intelligent agents with control structures. Proceedings of the ACM on Programming Languages, S(PLDI), June 2024. doi: 10. 1145/3656447.
Shreyansh Daftry, Sam Zeng, J. Andrew Bagnell, and Martial Hebert. Introspective perception: Learning predict failures in vision systems. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1743 1750, 2016. doi: 0011009/RRS.2016..77
Thao Dang, Alexandre Donzé Inzemamul Haque, Nikolaos Kekatos, and Indranil Saha. Counter-cxample guided imitation learning of feedback controllers from temporal logic specifications. In IEEE Conference on Decision and Control (CDC), pp.
2023. doi: 10.1 202CCCC3332222
23

---

Published in Transactions on Machine Learning Research (12/2025)
Moumita Das, Rajarshi Ray, Swarup Kumar Mohalik, and Ansuman Banerjee. Fast falsification of neural networks using property directed testing, 2021. URL. htttp:://arxiv.orgg//ae wwwg/aba////0a/ 12418
Charles Dawson and Chuchu Fan. Robust counterexample-guded optimization for planning from differen- tiable temporal logic. In JEEE/RS, International Conference on Intelligent Robots and Systems (IROS), pp. 2212, 2022. doi:
Charles Dawson, Sicun Gao, and Chuchu Fan.
Safe control with learned certificates: survey of neural Lyapunov, barrier, and contraction methods for robotics and control. IEFE Transactions on Roboties, pp. 1 19, 2023. doi: 10.1109/TRG, (0000///RO.222...32.. 202..222222244
Rayna Dimitrova and Rupak Majumdar. Deductive control synthesis for alternating- logics. In 2014 In- ternational Conference on Embedded Software (EMSOFT), pp. 1 1-10, 2014. doi: 10.1145/26564452656666
Xu Chu Ding, Stephen L Smith, Calin Belta, and Daniela Rus. MDP optimal control under temporal logic constraints. In IEEE Conference on Decision and Control (CDC), pp. 532-538. IEEE, 2011.
Taylor Dohmen, Noah Topper, George Atia, Andre Beckus, Ashutosh Trivedi, and Alvaro Velasquez. In- ferring probabilistic reward machines from non-Markovian reward signals for reinforcement learning. In- ternational Conference on Automated Planning and Scheduling (ICAPS), 32(1):574 582, June 2022. doi: (0.6699/caps.3/111 19844. URL https:///js aaai www.aaaa.aaa................ www.nnxxx.ppp/IaaaA.e httpp//////////rrraaaa//////////
Adel Dokhanchi, Heni Ben Amor, Jyotirmoy V. Deshmukh, and Georgios Fainekos. Evaluating perception systems for autonomous vehicles using quality temporal logic. In Christian Colombo and Martin Leucker (eds.), International Conference on Runtime Verification (RV), volume 11237 of Lecture Notes in Computer Science, pp. 409-416. Springer- Verlag, 2019. ISBN 978-3-030-03768- doi: 10.1007 00.1007/978-385 23.
Guoliang Dong, Jun Sun, Xingen Wang, Xinyu Wang, and Ting Dai. Towards repairing neural networks correctly. In JEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 714 725, 2021. doi: 10.1109 2000//R5554444.00 2022..0000..02000011.....
Alexandre Donzé and Oded Maler. Systematic simulation using sensitivity analysis. In Alberto Bemporad, Antonio Bicchi, and Giorgio Buttazzo (eds.), Hybrid Systems: Computation and Control, volume 4416 of Lecture Notes in Computer Science, pp. 174-189, Berlin, Heidelberg, 2007. Springer. ISBN 978-3-540- 71493-4. doi:
16.
Alexandre Donzé, Thomas Ferrère, and Oded Maler. Efficient robust monitoring for STL. In International Conference on Computer Aided Verification (CAV), volume 8044 of Lecture Notes in Computer Science, pp. 264-279, Berlin, Heidelberg. 2013. Springer. ISBN 978-3-642-39799-8 doi: 00.1007/978-346423377999
19.
Tommaso Dreossi, Alexandre Donzé, and Sanjit A. Seshia. Compositional falsification of cyber-physice systems with machine learning components. Journal of Automated Reasoning, 13)011133,,, December 2019. ISSN 1573-0670. doi: 10.1007/s10817-018-9955
Parasara Sridhar Duggirala, Sayan Mitra, and Mahesh Viswanathan. Verification of annotated models from executions. In International Conference Embedded Software (EMSOFT), pp. 1-10, 2013. doi: 2PPT.2013.. 2013.6655600
Souradeep Dutta, Xin Chen, and Sriram Sankaranarayanan. Reachability analysis for neural feedback systems using regressive polynomial rule inference. In ACM International Conference on Hybrid Systems: Compulation and Control (HSCC), pp. 157-168, New York, NY, USA, 2019. ACM. ISBN 978-1-450- 36282-5. doi:
Joe Eappen, Zikang Xiong, Dipam Patel, Aniket Bera, and Suresh Jagannathan. Scaling safe multi-agent control for signal temporal logic specifications. In Conference on Robot Learning (CoRL), 2024. URL
https://openrevier
http:://ppauueee......ee/meeaaap net//forumm/ddell44BB000
24

---

Published in Transactions on Machine Learning Research (12/2025)
Alec Edwards, Andrea Peruffo, and Alessandro Abate. Fossil 2.0: Formal certificate synthesis for the verification and control of dynamical models. In ACM International Conference on Hybrid Systems: Computation and Control (HSCC), New York, NY, USA, 2024. ACM. ISBN 979-8-400--0522--9 doi:
10.1145/364151..80 1399.
Yousef Emam, Gennaro Notomista, Paul Glotfelter, 7solt Kira, and Magnus Egerstedt. Safe reinforcement learning using robust control barrier functions. IEEE Robotics and Automation Letters, 10(3))2886 2893, 2025. doi: 10.1109/LRR 2.R..2022.3219996
Wilbert Peter Empleo, Yitack Kim, Hansoul Kim, Thiusius Rajeoth Savarimuthu, and lnigo Iturrate. Safe uncertainly-aware learning of robotic suturing, 2025. URL /tttss///axiiv. wwapziiv.orr/aaee wot.oogg//aaa. wwtgg/bbb//00.... 16596
Georgios E. Fainekos, Hadas. Kress-Gazit, and George J. Pappas. Temporal logic motion planning for mobile robots. In IEEE International Conference Robotics and Automation (ICRA), pp. 2020-2022 2005. doi: N..1009/RBBTT 100.00..005.170441 1570410.
Alec Farid, David Snyder, Allen Z. Ren, and Anirudha Majumdar. Failure prediction with statistical guar- antees for based robot control. In Robotics: Science and Systems (RSS), New York City, NY, USA, 2022. doi: 10.15607
Francesca Favaro, Laura Srnade-Blanar, Scott Schnelle, Trent Victor, Mauricio Peña, Johan Engstrom, John Scanlon, Kris Kusano, and Dan Smith. Building a credible case for safety: Waymo's approach for the determination of absence of unreasonable risk, 2023. URL
Zeyu Feng, Hao Luan, Pranav Goyal, and Harold Soh. LTLDoG: Satisfying lemporally-extended symbolic constraints for safe diffusion-based planning. IEEE Robotics and Automation Letters, 9(10)8857-8877 2024a. doi: 10.1 2ttg:.RR..222.34440
Zeyu Feng, Hao Luan, Kevin Yuchen Ma, and Harold Soh. Diffusion meets options: Hierarchical generative skill composition for temporally-cxtendde tasks. arXiv preprint arXiv:2410.00 023899, 2024b.
Angelo Ferrando and Giorgio Delzanno. Incrementally predictive runtime verification. Journal of Logic and Computation, 33(4):796 817, March 2023. ISSN 0955-792X. doi: 10003//loco///xadiee
John K. Feser, Swarat Chaudhuri, and Isil Dillig. Synthesizing data structure transformations from imput- output examples. SIGPLAN Notices, 50(6):229-23) June 2015. ISSN 0362-1340. doi: 10.1145/2813885 2737977.
Roya. Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, et al. Foundation models in robotics: Ap- plications, challenges, and the future. The International Journal of Robolães Reseurch, 2024. doi:
10.1177/02753349221288551
Jaime F. Fisac, Neil F. Lugovoy, Vicenç Rubies-Royo, Shromona Ghosh, and Claire J. Tomlin. Bridging hamilton-jacobi safety analysis and reinforcement. learning. In IEFF International Conference on Robotics and Automation (ICRA), pp. 8550 8556, 2019. doi: 00.1109/ICRA.2011 8019799771.11..7711000
Jie Fu and Ufuk Topeu. Probably approximately correct MDP learning and control with temporal logic constraints. In Robotics: Science and Systems (RSS), Berkeley, USA, 2014. doi:
Nathan Gaby, Fumin Zhang, and Xiaojing Ye. Lyapunov-net: A deep neural network architecture for Lyapunov function approximation. In JEEE Conference on Decision and Control (CDC), pp. 2022. doi: M.1109/CDC5110 222.09999.......9000.
Maor Gaon and Ronen Brafman. Reinforcement learning with non-Markovian rewards. In AAAT Conference on Artificial Intelligence (AAAT), pp. 3980-3987. AAAI press, June 2020. doi: 10.1609/ v3404..444.
Paul Gastin and Denis Oddoux. Fast LTL to büchi automata translation. In Gérard Berry, Hubert Comon, and Alain Finkel (cds.), International Conference on Computer Aided Verification (CAV), volume 2102 of Lecture Notes in Computer Science, pp. 53-65, Berlin, Heidelberg, 2001. Springer. ISBN 978-3-540-44585- 6.
2. doi:
25

---

Published in Transactions on Machine Learning Research (12/2025)
Shromona Ghosh, Yash Vardhan Pant, Hadi Ravanbakhsh, and Sanjit A. Seshia. Counterexample-guidee synthesis of perception models and control. In American Control Conference (ACC), pp- 3447-3454, 2021.
doi: 10.23919/ | 2.0CC50511.222.9492222
Antoine Girard and George J. Pappas. Verification using simulation. In João P. Hespanha and Ashish Tiwari (eds.), Workshop on Hybrid Systems: Compulation and Control (HSCC), volume 3927 of Lecture Notes in Computer Science, pp. 272-286, Berlin, Heidelberg, 2006. Springer. ISBN 978-3-540-33171-11. doi: 10.1007 10.1007/11730000 0111//17366322221 22.
Tan Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume MIT Press, 2016.
Zijian Guo, Weichao Zhou, and Wenchao Li. Temporal logic speriication-coniiionncc decision transformer for offline safe reinforcement learning. In International Conference on Machine Learning (ICML), 2024. URL https://openrevierr http:/pawwaaaeeee.eoooooo/oaaaaa rett//oroummdddd7gg10333..3
Zijian Guo, mker Işik, H M Sabbir Ahmad, and Wenchao Li. One subgoal at time: Zero-shot, generalization to arbitrary linear temporal logic requirements in multi-task reinforcement learning. In Advonces in Newral Information Processing Systems (NeurIPS), 2025. URL http:///ppnnevve.nnteeeeeee NGgLhJKttI
Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh Trivedi, and Dominik Wojtezak, Omega-regular objectives in model-free reinforcement learning. In Tomáš Vojnar and Lijun Zhang (eds.), International Conference Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 11427 of Lecture Notes in Computer Science, pp. 395-412, Cham, 2019. Springer. ISBN 978-3- 17462-0 27.
030-17462-0, doi:
Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh Trivedi, and Dominik Wojtezak. Good-for-MDPs automata for probabilistic analysis and reinforcement learning. Tn Armin Biere and David Parker (eds.), Tools and Algorithms for the Construction and Analysis of Systems, volume 12078 of Lecture Noles in Compuler Science, pp. 306 323, Cham, 2020. Springer. ISBN 978-3-030-45190-5, doi: 139045110-5511111 17.
Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh Trivedi, and Dominik Wo- jtezak. An impossibility result in automata-theorctic reinforcement learning. In Ahmed Bouajjani, Lukáš Holik, and Zhilin Wu (eds.), Automated Technology for Verification and Analysis, volume 13505 of Lecture Notes in Computer Science, pp. 42-57, 57, Cham, 2022. Springer. ISBN 978-3-031-1999929, doi: 3.
00.1007/978-3-001--9999
Mohammadhosein Hasanbeig, Alessandro Abate, and Daniel Kroening. Logically-consstrineed reinforcement learning, 2019a. URL httph:tttt//atttee org/ wwwp://g//////////e/ee
Mohammadhosein. Hasanbeig, Yiannis. Kantaros, Alessandro. Abate, Daniel. Kroening, George J. Pappas, and Insup. Lee. Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees. In TEEE Conference on Decision and Control (CDC), pp. 5338 5343, 2019b. doi: 10.1109/ CDC40024.2011 201.9902999..9
Navid Hashemi, Bardh Hoxha, Tomoya Yamaguchi, Danil Prokhorov, Georgios Fainckos, and Jyotirmoy Deshmukh. A neurosymbolic approach to the verification of temporal logic properties of learning-enabled control systems. In ACM/ICMMRRA International Conference on Syybe--Physicn Systems (ICCPS), pp. 089109, New York, NY, USA, 2023. ACM. ISBN 979-8-400-1 0-440070006--1.
doi: 0111145/3576441.3355
Wataru Hashimoto, Kazumune Hashimoto, and Shigemasa Takai. STL2vec: Signal temporal logic embed- dings for control synthesis with recurrent neural networks. IEEE Robotics and Automation Letters, 7(2): 5246 5253, 2022. doi: 10.1109/LRA http:/LR..0222315555 2022.3155197.
Mohammad Hekmatnejad. Formalizing Safety, Perception, and Mission Requirements for Testing and Plan- ming in Aulonomous Vehicles. PhD thesis, Arizona State University, USA, 2021.
26

---

Published in Transactions on Machine Learning Research (12/2025)
Thomas A. Henzinger, Fabian Kresse, Kaushik Mallik, Emily Yu, and Dorde Zikelić. Predictive monitor- ing of black-box dynamical systems. In Necmiye Ozay, Laura Balzano, Dimitra Panagou, and Alessan- dro Abate (eds.); Learning for Dynamics and Control Conference (L4DC), volume 283 of Proceedings of Machine Learning Research, pp. 804-816. PMLR, 2025. URL httts::/prroceeddngg mlr mlr. press/v283/ henzinger25a.hrt
Kai-Chich Hsu, Viceng Rubies-Royo, Claire J. Tomlin, and Jaime F. Fisac. Safety and liveness guarantees through reach-avoid reinforcement learning. In Robotics: Science and Systems (RSS), Held Virtually, 2021. doi: 2R..:0//RRSS.22.XIII..
Tzu-Han Hsu, Arshia Rafieioskouei, and Borzoo Bonakdarpour. HYPRL: Reinforcement learning of control policics for hyperpropertics. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
URL httpss//openreeiee..eeee. wwtp/oooumiddd.eeeeeeee.e.
Yafei Hu, Quanting Xie, Vidhi Jain, Jonathan Francis, Jay Patrikar, Nikhil Keetha, Seungchan Kim, Yaqi Xie, Tianyi Zhang, Hao-Shu Fang, Shibo Zhao, Shayegan Omidshafiei, Dong-Ki Kim, Ali akbar Agha- mohammadi, Katia Sycara, Matthew Johnson-Rooone Roberson, Dhruv Batra, Xiaolong Wang, Sebastian Scherer, Chen Wang, 7solt Kira, Fei Xia, and Yonatan Bisk. Toward general- robots via foundation models: A survey and meta-analysis, 2024. URL httpttt/////artttt/ htttpp://arxtt.orrr//bb////// http//abb/231..087///
Chao Huang, Jiameng Fan, Wenchao Li, Xin Chen, and Qi Zhu. ReachNN: Reachability analysis of neural- network controlled systems. ACM Transactions on Embedded Computing Systems, 18(5s), October 2019. ISSN 1539-9087. doi:
Rodrigo Toro Icarte, Toryn Klassen, Richard Valenzano, and Sheila McIlraith. Using reward machines for high-level lask specification and decomposition in reinforcement learning. In Jennifer Dy and Andreas Krause (eds.), International Conference on Machine Learning (ICML), volume 80 of Proceedings of Ma- chine Learning Research, pp. 2107 2116. PMLR, 2018. URL htttps://proceedings mlr. press/v80/ icarte18a.html
Jeevana Priya Inala, Osbert Bastani, Zenna Tavares, and Armando Solar-Lezama. Synthesizing program- matic policies that inductively generalize. In International Conference on Learning Representations (ICLR), 2020. URL
Arda Inceoglu, Eren Erdal Aksoy, Abdullah Cihan Ak, and Sanem Sariel. FINO-Net: A deep multimodal
sensor fusion framework for manipulation failure detection. In TEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 6841-6847, 2021. doi:
Craig Innes and Subramanian Ramamoorthy. Elaborating on learned demonstrations with temporal logic specifications. In Robotics: Science and Systems (RSS), Corvalis, Oregon, USA, 2020. doi: 10. /RSS. 20.0.XV1.004.
Radoslav Ivanov, James Weimer, Rajeev Alur, George J. Pappas, and Insup Lee. Verisig: Verifying safety properties of hybrid systems with neural network controllers. In ACM International Conference on Hybrid Systems: Computation and Control (HSCC), pp. (89-178, New York, NY, USA, 2019. ACM. ISBN 978-1-450-33682-5. doi: 10.1455/3302004:3111866
Radoslav Ivanov, Taylor J. Carpenter, James Weimer, Rajeev Alur, George J. Pappas, and Insup Lee. Veri- fying the safety of autonomous systems with neural network controllers. ACM Transactions on Embedded Compating Systems, 20(1), December 2020. ISSN 1539-9087. doi: 10.1145/3419742
Radoslav Ivanov, Taylor Carpenter, James Weimer, Rajeev Alur, George Pappas, and Insup Lee. Verisig 2.0: Verification of neural network controllers using Taylor model preconditioning. In Alexandra Silva and K. Rustan M. Leino (eds.), International Conference on Computer Aided Verification (CAV), volume 12759 of Lecture Notes in Computer Science, pp. 249-2449290 262, Cham, 2021. Springer. ISBN 978-3-030-81685-5 doi: 10.1007 111077978---0000-2 8165-88 11.
27

---

Published in Transactions on Machine Learning Research (12/2025)
Mathias Jackermeier and Alessandro Abate. DeepLTL: Learning to efficiently satisfy complex LTL specifi- cations for multi-task RT. In International Conference on Learning Representations (TCL.R, 2025. URL
https://openrevier httpp://pppaarrriiv..eeodooooorp wett/forummidd=9pW2J4998
Kishor Jothimurugan, Suguman Bansal, Osbert Bastani, and Rajeev Alur. Compositional reinforcement learning from logical specifications. In Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Newral Information Processing Systems (NewrIPS), volume 34, pp. 10026-10039. Curran Associates, Inc., 2021. URL https://proceeddngg pdf
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020. URL
https://aras http://aaxivv..o /ttpaaaaww...raa/////a....// /tw.a////0..000....
Parv Kapoor, Anand Balakrishnan, and Jyotirmoy V. Deshmukh. Model-based reinforcement learning from signal temporal logic specifications, 2020. URL https:://axxiv.og/abbb////// 04950
Parv Kapoor, Abigail Hammer, Ashish Kapoor, Karen Leung, and Eunsuk Kang. Pretrained embeddings behavior specification mechanism, 2025. URL httpp:://aaxiiv.or/abbbbt.. 2
Milad Kazemi, Maleo Perez, Fabio Somenzi, Sadegh Soudjani, Ashutosh Trivedi, and Alvaro Velasquez. Assume- -uarantte reinforcement learning. In AAAI Conference on Artificial Intelligence (AAAI), pp. AAAI Press, 2024. ISBN 978-1-57735-887-9 doi: 10.1609 /a00//aai.voo v38119.3016..
J. Zico Kolter and Gaurav Manek. Learning stable deep dynamics models. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Buc, E. Fox, and R. Garnett (ods.), Advances in Neural Information Processing Systems (NeurIPS), volume 32. Curran Associates, Inc., 2019. URL http//tttpe//rrrreddngggsee paper
Hadas Kress-Gavit, Georgios E. Fainekos, and George Pappas. Temporal-logic-based reactive mission and motion planning. IEEE Trunsactions on Robotics, 25(6):1370 1381, 2009. doi: 10.1109/TR0.200. 2030225.
Yen-Ling Kuo, Boris Katz, and Andrei Barbu, Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas. In FEEE/RSJ International Conference on Intelligent Robots and doi: 10.1109 TRO///TOO555744200000000
Systems (IROS), pp. 5604-50 5610, 2020.
Daniel Kästner, Ulrich Wünsche, Jörg Bartho, Marc Schlickling, Bernhard Schommer, Michael Schmidt, Christian Ferdinand, Xavier Leroy, and Sandrine Blazy. CompCert: Practical experience on integrating and qualifying formally verified optimizing compiler. In European Congress on Embedded Real Time Software and Systems (ERTS). SEE, 2018. URL TRRLhttp:// htttp::/aaaieeeerryyorrr/////// http://////////r/t2/0aaaaaooooo/ pdf
Tobias Ladner and Matthias Althoff. Automatic abstraction refinement in neural network verification using sensitivity analysis. In ACM International Conference on Hybrid Systems: Compulation and Control (HSCC), New York, NY, USA, 2023. ACM. ISBN 979-8-400-70033-0. doi: 10.1145/357787003587111
Tobias Ladner, Ahmed Rayen Mhadhbi, and Matthias Althoff. Towards formally verifying 1ms:: Taming the nonlinearity of the transformer, 2025. URL http:///pppppreeeeeeeeeees//ottr
Luis C. Lamb, Artur d'Avila Garcez, Marco Gori, Marcolo O.R. Prates, Pedro H.C. Avelar, and Moshe Y. Vardi. Graph neural networks meet neural-symbolic computing: survey and perspective. In International Joint Conference on Artificial Intelligence (IJCAI), 2021. ISBN 178-0-999-24116-4
Kim Larsen, Axel Legay, Gerrit Nolte, Maxinilian Schlüter, Marielle Stoelinga, and Bernhard Steflen. Formal methods meet machine learning (F3ML). In International Symposium On Leveraging Applications of Formal Methods, Verification and Validation (ISoLA), volume 13703 of Lecture Notes Computer Science, pp. 393-405, Berlin, Heidelberg, 2022. Springer-Verte ISBN 378-3-031-19955-1. doi: 10.1007/ 978-3-031-19759-8 24.
28

---

Published in Transactions on Machine Learning Research (12/2025)
Karen Leung, Nikos Aréchiga, and Marco Pavone. Backpropagation through signal temporal logic speci- fications: Infusing logical structure into gradient-based methods. The International Journal of Robolics Research, 42(6):356 370, 2023. doi: 10.1177002753644221000111
Sergey Levine, Chelleaa Finn, Trevor Darrell, and Pieter Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Research, 17(39):1-40, 2016. URL htttp::/jnilr..og//papssss
15- 52m
Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. Learning hand-eye coordi- nation for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research, 37(4-5):421-438 2018. doi: 10.1177 021177/02783669117700116
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems, 2020. URL
Thomas Lew and Marco Pavone. Sampling- based reachability analysis: A random set theory approach with adversarial sampling. In Jens Kober, Fabio Ramos, and Claire Tomlin (eds.), Conference on Robot Learning (CoRL), volume 155 of Proceedings of Machine Learning Research, pp. mlr. pwwps//11//ee/22222 html
PMLR, 2021.
URL
Andrew C Li, Zizhao Chen, Toryn Q. Klassen, Pashootan Vaezipoor, Rodrigo Toro Icarte, and Sheila. A. Mcllraith. Reward machines for deep RL in noisy and uncertain environments. In Advances in Newral In- formation Processing Systems (NeurIPS), 2024. URL httpps://openrevier...
Xiao Li, Cristian-loan Vasile, and Calin Belta. Reinforcement learning with temporal logic rewards. In IEEE/RSJ International Conference on Intelligent Robols and Systems (TROS), pp. 3834-388888 2017. doi: (LL.199/IRO..001.......
Xiao Li, Zachary Serlin, Guang Yang, and Calin Belta. A formal methods approach to interpretable rein- forcement learning for robotic planning. Science Robotics, 4(37))cnay6276, 2019. doi: 10.1126 si.11///cirooootcss aay6276.
Lars Lindemann and Dimos V. Dimarogonas. Control barricr functions for signal temporal logic tasks. JEEE Control Systems Letters, 3(1):96-101) 0:1:96101, 2019. doi: 10.1109/LCSYS.2001 2018.
Lars Lindemann, Xin Qin, Jyotirmoy V. Deshmukh, and George J. Pappas. Conformal prediction for stl runtime verification. In ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS), pp. 142-153, 2023. doi: 04444/55576441355555
Huíhan Liu, Shivin Dass, Roberto Martin-Martnn, and Yuke Zhu. Model-based runtime monitoring with interactive imitation learning. In IEEE International Conference on Robotics and Automation (ICRA), 2024.
Jason Xinyu Liu, Ziyi Yang, Benjamin Schornstein, Sam Liang, Ifrah Idrees, Stefanie Tellex, and Ankit Shah. Lang2I.TI. Translating natural language commands to temporal specification with large language models. In Workshop on Language and Robotics at CoRL 2022, 2022. URL https://openreriex net/ forum?id Vxx
Siyuan Liu, Adnane Saoud, and Dimos V. Dimarogonas. Controller synthesis of collaborative signal temporal logic lasks for multiagent systems via assume--uuaanlee contracts. IEEE Transaclions on Aulomatic Control, 70(9):5894 -5899999999, 2025a. doi:
Wenliang Liu, Mirai Nishioka, and Calin Belta. Safe model-based control from signal temporal logic specifi- cations using recurrent neural networks. In IEFE International Conference Robotics and Automation (ICRA), pp. 12416 12422, 2023. doi: 1..1009/CCRAA88812222...0016
Wenliang Liu, Danyang Li, Erfan Aasi, Roberto Tron, and Calin Belta. Interpretable imitation Icarning via generative adversarial STL inference and control. In 2nd International Conference on Neuro-symbolic Systems (NeuS), 2025b.
29

---

Published in Transactions on Machine Learning Research (12/2025)
Diego Manzanas Lopez, Sung Woo Choi, Hoang-Dung Tran, and Taylor T. Johnson. NNV 2.0: The neural network verification tool. In Constantin Enea and Akash Lal (eds.), International Conference on Computer Aided Verification (CAV), volume 13965 of Lecture Notes in Computer Science, pp. 397 412, Cham, 2023. Springer. ISBN 978-3-031-37708-7. doi: 10.1007 10788---01--7773---1
Matt Luckeuck, Marie Farrell, Louise A. Dennis, Clare Dixon, and Michael Fisher. Formal specification and verification of autonomous robotic systems: A survey, ACM Computing Surveys, 52(5), September 2019.
ISSN 0360-0300. doi:
Haitong Ma, Jianyu Chen, Shengbo Eben, Ziyu Lin, Yang Guan, Yangang Ren, and Sifa Zheng. Model-based constrained reinforcement learning using generalized control barrier function. In TEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4552 4559, 2021. doi:
2021.9636468.
Keyvan Majd, Geoffrey Clark, Georgios Fainekos, and Heni Ben Amor. Repairing neural networks for safety in robotic systems using predictive models. In JEEE/RSJ International Conference on Intelligent Robots and Systems (TROS), pp- 11827-11834, 2024. doi: 10. 11110000055599221101010 10802650.
Oded Maler and Dejan Nickovic. Monitoring temporal properties of continuous signals. In Yassine Lakhnech and Sergio Yovine (eds.), Formal Techniques, Modelling and Analysis of Timed and Fault- Toleratt Systems (FTRTFT), volume 3253 of Lecture Notes in Compuler Science, pp. 152 166, Berlin, Heidelberg, 2004. 12.
Springer. ISBN 978-3-540-30206-3. doi: 10.1007 10.1007/978-3-5444
Anastasios Manganaris, Vittorio Giammarino, and Ahmed H Qureshi. Automaton constrained Q-learning. In Advances in Newral Information Processing Systems (NewrIPS), 2025. URL https://openrevier: net net/fforumidd--LL22Bp1533
Zohar Manna and Amir Pnueli. A hierarchy of temporal properties. In ACM Symposium on Principles of Distributed Computing (PODC), pp. 377-410, New York, NY, USA, 1990. ACM. ISBN 0-897-91404-X. doi: 10.1145, 01145/33355.933
Zohar Manna and Richard Waldinger. A deductive approach to program synthesis. ACM Transactions Programming Languages and Systems, 2(1):90-1 121,,111111111111111110111, January 1980. ISSN 0164-0925. doi: 041145/357008 357090.
Diego Manzanas Lopez, Patrick Musau, Nathaniel P. Hamilton, and Taylor T. Johnson. Reachability analysis of general class of neural ordinary differential equations. In Sergiy Bogomolov and David Parker (eds.), Formal Modeling and Analysis of Timed Systems, volume 13465 of Lecture Noles in Computer Science, pp. 258-277, Cham, 2022. Springer. ISBN 978-3-031-15888 doi: 10.1007/998-3-0011 15839--11191111111 15.
Alonso Marco, Elias Morley, and Claire J. Tomlin. Out of distribution detection via domain-informn gaussian process state space models. In IEEE Conference on Decision and Control (CDC), pp. 5487- 2023. doi:
Zahra Marvi and Bahare Kiumarsi. Safe reinforcement learning: A control barrier function optimization approach. International Journal of Robust and Nonlinear Control, (1(6):1933-1940 2021. doi: 10.1002/ rnc.5132.
Yue Meng and Chuchu Fan. Signal temporal logic neural predictive control. IEEE Robotics and Automation Lellers, 8(11):7719-777 2023. doi: 10.1109/LLB (.0099//R..2000 20A.202333315536.
Yue Meng and Chuchu Fan. Diverse controllable diffusson policy with signal temporal logic. IEEE Robolics and Automation Letters, 910)):8354 8354--361, 2024. doi: 00.1099LRR..204..00
Yue Meng and Chuchu Fan. TeLoGraF:: Temporal logic planning via graph-encoded flow matching. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (ods.), International Conference on Machine Learning (ICML), volume 267 of Proceedings of Machine Learning Research, pp. 43754-4544 43780. PMLR, 2025. URL https://proceedd:: mlr. wwwppp/sssse/////pppppppp http///22mmmmmmmm5552 meng25b. html
30

---

Published in Transactions on Machine Learning Research (12/2025)
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionalityy In C. J. Burges, I. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 26. Cur- nww.ips. cappaper
ran Associates, Inc., 2013. URL httpp::/pproceet
pdf
Tan M. Mitchell. Comparing forward and backward reachability as tools for safety analysis. In Alberto Bemporad, Antonio Biechi, and Giorgio Buttazzo (eds.), International Conference on Hybrid Systems: Computation and Control (HSCC), volume 4416 of Lecture Notes in Computer Science, pp. 428 443, Berlin, Heidelberg, 2007. Springer. ISBN 978-3-540-71493-4 doi: 1111007/978-3-540-7111 34.
Ian M. Mitchell, Alexandre M. Bayen, and Claire J. Tomlin. A timc-dependent hamiltnnjacohii formulation of reachable sets for continuous dynamic games. IEEE Transactions on Aulomalic Control, 50(7):947 957, 2005. doi: (0.1109/TAC.20055 2TAC.2005.851433
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Mare G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattic, Amir Sadik, Toannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, February 2015. ISSN 1476-4687. doi: 10.1038/ (a1008/nature// maturrl4233
Toby Murray, Daniel Matichuk, Matthew Brassil, Peter Gammie, Timothy Bourke, Sean Seefried, Corey Tewis, Xin Gao, and Gerwin Klein. sel.:: From general purpose to a proof of information flow enforcement. In IEEE Symposium on Security and Privacy (SBP), pp. 415-429, 2013. doi: 10.1109/SP.2013.35.
Geraud Nangue Tasse, Devon Jarvis, Steven James, and Benjamin Rosman. Skill machines: Temporal logic skill composition in reinforcement learning. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Representation Leurning (ICLR), volume 2024, pp. 36937-36956, 2024. URL httpss://proceddiin iclr. cctpppper_illlepppppeeeeeeeeeeee ffl.es//apor/2222//EL..
Arnold Noumaier. The wrapping effect, ellipsoid arithmetic, stability and confidence regions. In Al- brecht, G. Alefeld, and H. J. Stetter (eds.), Validation Numerics: Theory and Applications, volume 9 of Computing Supplementum, pp. 175-190. Springer, Vienna, 1993. ISBN 7799---999-6----6 doi: 10.1007 19777799787---691888
14.
Andrew Y. Ng and Stuart J. Russell. Algorithms for inverse reinforcement learning. In International Conference Machine Learning (ICML), pp. 663 670, San Francisco, CA, USA, 2000. Morgan Kaulmann Publishers Inc. ISBN 1-558-60707-2.
Alexander Novikov, Ngån Vü, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mchrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. AlolaEvolve: coding agent for scientific and algorithmic discovery, 2025. URL https: https:///rrx:: //axxivv//x org/ abt///bb///555 13131
Seohong Park, Kevin Frans, Sergey Levine, and Aviral Kumar. Is value learning really the main bottleneck offline RL? In Advances Neural Information Processing Systems (NeurIPS), 2024. URL https:
//openreview.netee http:/eeewe.tootmmmmonssssaaaaaa
Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. DeepXplore: Automated whitebox testing of deep learning systems. Communications of the ACM, 62(11):137 -45,, October 2019. ISSN 0001-0782, doi:
/331116633
Theodore J. Perkins and Andrew G. Barto. Lyapunov design for safe reinforcement learning. Journal of Machine Learning Research, 3(null):803 832, March 2003. ISSN 1532-4435.
31

---

Published in Transactions on Machine Learning Research (12/2025)
Andrea Peruffo, Danicle Ahmed, and Alessandro Abate. Automated and formal synthesis of neural barrier certificates for dynamical models. In Jan Friso Groote and Kim Guldstrand Larsen (eds.), Tools and Algorithms for the Construction and Analysis of Systems, volume 12651 of Lecture Notes in Computer Science, pp. 370-388, Cham, 2021. Springer. ISBN 978-3-030-72016-2. doi: 20.
Srinivas Pinisetty, Thierry Jéron, Stavros Tripakis, Yliès Falcone, Hervé Marchand, and Viorel Preoteasa. Predictive runtime verification timed properties. Journal of Systems and Software, 132:353- 365, October 2017. doi: (0001//..ss.201..000..
Amir Pmmeli. The temporal logic of programs. In TEEE Symposium on Foundations of Computer Science (SFCS), pp. 46 57, 1977. doi: D0.1109/SFCS.197..33
Amir Pnueli and Roni Rosner. A framework for the synthesis of reactive modules. In Frederich H. Vogt (ed.), International Conference on Concurrency, volume 335 of Lecture Notes in Computer Science, pp.
4-17, Berlin, Heidelberg, 1988. Springer. ISBN 978-3-540-45999-6. doi: 10.1007/3-544--54003--- 28.
Aniruddh Puranic, Jyotirmoy Deshmukh, and Stefanos Nikolaidis. Learning from demonstrations using signal temporal logic. In Jens Kober, Fabio Ramos, and Claire Tomlin (eds.), Conference on Robot Learning (CoRL), volume 155 of Proceedings of Machine Learning Research, pp. 2228 2242. PMLR, 2021. URL
https://proceedings mlr. ppwssa/v15/puurnniiii.. htmll
Wenjie Qiu and He Zhu. Programmatic reinforcement. learning without oracles. In International Conference on Learning Representations (ICLR), 2022. URL http:p///ppaareeeeeeeeee/rrrrted
Wenjie Qiu, Wensen Mao, and He Zhu. Instructing goal-conditioned reinforcement learning agents with temporal logic objectives. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 36, pp. 39147-39175. Cur- Associates, Inc., 2023. URL wwweeeeeeeeeeeeeeeeaaeeeeeeeeeee
neurips. cc.ppperrcppppp..ppppp httpc//ppppp..leee//////////////
Sadegh Rabiee and Joydeep Biswas. IVOA: Introspective vision for obstacle avoidance. In IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 1230 1235, 2019. 011-:00897.211189681111
doi: 10.1109/
Nacim Ramdani, Nacim Meslem, and Yves Candau. Reachability analysis of uncertain nonlinear systems using guaranteed set integration. IFAC Proceedings Volumes, (1()):8972-8077, 2008. ISSN 1474-6670. doi: pJ1/814444667116603449
KR-100000001011 URL
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexander Novikov, Gabriel Barth- maron, Mai Giménez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas. A generalist agent. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https://openreviee: wttp://ewww..eeeoooo/o////ddddad
Gavin Rens, Jean-François Raskin, Raphaël Reynouad, and Giuseppe Marra. Online learning of non- Markovian reward models, 2020. URL http:///waxxv..o/bbbb////ss 12600
Werner C. Rheinboldt and Azaria Paz. Introduction to probabilistic automata. Academic Press, 2014.
Spencer M. Richards, Felix Berkenkamp, and Andreas Krause. The Lyapunov neural network: Adaptive sta- bility certification for safe learning of dynamical systems. In Aude Billard, Anca Dragan, Jan Peters, and Jun Morimoto (eds.), Conference on Robot Learning (CoRL), volume 87 of Proceedings of Machine Learn- ing Research, pp. 466-476. PMLR, 2018. URL https://prroeediing. mlr, press/v87 httpa////8///ccarsssss.. /rrr/riicaaaaaa... wtt.a//sss.mi.....//e/e...
html
32

---

Published in Transactions on Machine Learning Research (12/2025)
Stephane Ross and Drew Bagnell. Efficient reductions for imitation learning. In Yee Whye Teh and Mike Titterington (eds.), International Conference. on Artificial Intelligence and Statistics (AISTATS), volume 9 of Proceedings of Machine Learning Research, pp. 661 668, Chia Laguna Resort, Sardinia, Italy, 2010. PMLR. URL httts:://proceedingg http:/cceddnng......rreeeesee/ee
mlr. pttpps/////rssssaaa.sa. html
Dorsa Sadigh and Ashish Kapoor. Safe control under uncertainty with probabilistic signal temporal logic. In Robolics: Science and Systems (RSS), AmnArbor, Michigan, 2016. doi: 10.15607/RSS. 2016.
Dorsa Sadigh, Eric S. Kim, Samuel Coogan, S. Shankar Sastry, and Sanjit A. Seshia. A learning based approach to control synthesis of Markov decision processes for linear temporal logic specifications. In IEEE Conference on Decision and Control (CDC), pp- 1091-1096, 2014. doi: (01109/CCC2000 7039557..
Ali Salamati, Sadegh Soudjani, and Majid Zamani. Data-driven verification under signal temporal logic constraints. IFAC-PapersOnnLine, 53(2):69-77) 2020. ISSN 2405-8963. doi: 10.1016/f..fccol.2200.2222
Adnane Saoud, Antoine Girard, and Laurent Fribourg. Assume-guarante contracts for continuous-time systems. Automatica, 134:109910, 2021. ISSN 0005-1098. doi: 10000///..aaoooatto0000000010100 URL
attps://wwe http://wccccccctc.......ooooocca
Gaia Saveri, Laura Nenzi, Luca Bortolussi, and Jan Kretínský. st|2vec: Semantic and interpretable vector representation of temporal logic. In Ulle Endriss, Francisco S. Melo, Kerstin Bach, Alberto José Bugarín Diz, Jose Maria Alonso-Moral, Senén Barro, and Fredrik Heintz (eds.), European Conference on Arlificial Intelligence (ECAI), volume 392 of Frontiers in Artificial Intelligence and Applications, pp. 1381-1388. IOS Press, 2024. ISBN 978-1-64368-548-9 doi: 10.3233 /2333FAAA240638..
Stefan Schaal, Peyman Mohajerian, and Auke Ijspeert. Dynamics systems vs. optimal control - a unifying view. In Paul Cisek, Trevor Drew, and John F. Kalaska (eds.), Computational Newroscience: Theoretical Insights into Brain Function, volume 165 of Progress in Brain Research, pp. 425-444- Elsevier, 2007. doi:
(06)600077. CRL httpss//www. http::/ww.accccttttc.ccoco//ee/a
50079612306650279
Christian Schilling, Marcelo Forets, and Sebastián Guadalupe. Verification of neural-network control systems by integrating Taylor models and zonotopes. In AAAI Conference on Artificial Intelligence (AAAI), pp. AAAI Press, 2022. doi:
URL https://ooj..aaaaa aaai org/ wwtgg/indxxx//e php/
AMATarrticle/view//000
Ameesh Shah, Cameron Voloshin, Chenxi Yang, Abhinav Verma, Swarat Chaudhuri, and Sanjit A. Seshia. LTL-constrained policy optimization with cycle experience replay. Transactions Machine Learning Research, 2025. ISSN 2835-8856. URL
Ankit Shah, Pritish Kamath, Julie A Shah, and Shen Li. Bayesian inference of temporal task specifications from demonstrations. In Sany Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett ). Advances in Newral Information Processing Systems (NewrIPS), volume 31. Curran Associates, Inc., 2018. http//tttt://rrccednggg/ neurips...pppp.pp wwt.neewwi..pppppe.....e...eeee. forrrrffleess wwwesfllppppppee paper/
.df
Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, and Cho-Jui Hsieh. Robustness verification
for transformers. In International Conference on Learning Representations (ICLR), 2020. URL https: JHFWS)
//opeareviev.ne//formmittt
Yasser Shoukry, Pierluigi Nuzzo, Alberto L. Sangiovanni-
Sanjit A. Seshia, George J. Pappas, and Paulo Tabuada. SMC: Satisfiability modulo convex optimization. In International Conference on Hybrid Systems: Compulation and Control (HSCC), pp. 19-28, 28, New York, NY, USA, 2017. ACM. ISBN 978-1-450-34590-3. doi: 10.1145/3049797...
Salomon Sickert, Javier Esparza, Stefan Jaax, and Jan Křetinský. Limit-deterministic Büchi automata for lincar temporal logic. In Swarat Chaudhuri and Azadch Farzan (ods.), International Conference on Computer Aided Verification (CAV), volume 9780 of Lecture Notes in Computer Science, pp. 312-332, Cham, 2016. Springer. ISBN 978-3-319-41540-6. doi: 10.1007/077-33319-41555 17.
33

---

Published in Transactions on Machine Learning Research (12/2025)
Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin Vechev. Fast and effective robustness certification. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi and Roman Garnett (eds.), Advances in Newral Information Processing Systems (NeurIPS), volume 31. Curran Associates, Inc., 2018. URL htt.:tts:///prooediigge hew.eeeeiinnnn.nnniiips.......pe
neurips cearrps.cc/ppper....
paper/
Vincent. Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward hacking. In Advonces in Neural Information Processing Systems (NeurIPS), Red Hook, NY, USA, 2022. Curran Associates, Inc. ISBN 978-1-713-87108--
Stephen I. Smith, Jana Túmová, Calin Belta, and Daniela Rus. Optimal path planning for surveillance with temporal-logic constraints. The International Journal of Robotics Research, 30(14):1695 1708, 2011. doi: 10.1177/0273364911417999
Jeongju Sohn, Sungmin Kang, and Shin Yoo. Arachne: Search-based repair of deep neural networks. ACM Transactions Software Engineering Methodology, 32(4), May 2023. ISSN 1049-331X. doi: 10.1145/ 3563210.
Armando Solar-Lozama. The sketching approach to program synthesis. In Zhenjiang Hu (ed.), Asian Symposium on Programminng Languages and Systems (APLAS), volume 5904 of Lecture Notes in Com- puter Science, pp. 4-13, Berlin, Heidelberg, 2009. Springer. ISBN 978-3-642-1067244 doi: 10.1007/
978-3-642-10677-9
Armando Solar-Tezama. Introduction to program synthesis, 2023. URL httpss:/peoole..cat csail. wwt....mt.eeeee edu/ aaolar/SSyathessisooorreee//iii htm
Armando Solar-Lezama, Liviu Tancau, Rastislav Bodík, Sanjit A. Seshia, and Vijay A. Saraswat. Combina- torial sketching for finite programs. In International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 404 415, 2006. doi: 10.1145/ 1161117..116990000 URL
0w0.58887..1688999 1168907
Armando Solar-Lczama, Christopher Grant Jones, and Rastislav Bodik. Sketching concurrent data struc- tures. ACM SIGPLAN Conference on Proyramming Language Design and Implementation (PLDI), pp. 136-148, 148, 2008,
doi: 10.1145/ 1151/133551111355500 URL http:://oi..... acm. 145///0..14//1378811111333
Fmi Soroka, Rohan Sinha, and Sanjay Lall. Learning temporal logic predicates from data with statistical guarantees. In Necmiye Ozay, Laura Balzano, Dimitra Panagou, and Alessandro Abate (eds.), Learning for Dynamics and Control Conference (L4DC), volume 283 of Proceedings of Machine Learning Research, pp. 86-98, PMLR, 2025. URL httpps:/pproceedingg mlr, ppwsss/223/soooooaa22222
html
Matthew Sotoudeh and Aditya V. Thakur. Provable repair of deep neural networks. In ACM SIGPLAN International Conference Programming Language Design and Implementation (PLDI), pp. 588-603, New York, NY, USA, 2021. ACM. ISBN 978-1-450-38391-2 doi: (0.1145/3453488.345400000
Mohit Srinivasan, Amogh Dabholkar, Samuel Coogan, and Patricio A. Vela. Synthesis of control barrier func- tions using a supervised machine learning approach. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7139-7144 2020. doi:
Saurabh Srivastava, Sumit Gulwani, and Jeffrey S. Foster. From program verification to program synthesis. SIGPLAN Notices, 45(1):313 326, January 2010. ISSN 0362-1340. doi: 10.1165/17070007003377.
Bing Sun, Jun Sun, Long H. Pham, and Jie Shi. Causality- neural network repair. In International Conference on Software Engineering (ICSE), pp. 338-349, New York, NY, USA, 2022a. ACM. ISBN 978-1-450--9921--1 doi: 00.1145/350000335111
34

---

Published in Transactions on Machine Learning Research (12/2025)
Dawei Sun, Jingkai Chen, Sayan Mitra, and Chuchu Fan. Multi-agent motion planning from signal temporal logic specifications. IEEE Robolies and Aulomation Letters, (03):451134455 2022b. doi: 10.1109/T.RA
2022.3146951.
Shiqi Sun, Yan Zhang, Xusheng Luo, Panagiotis Vlantis, Miroslav Pajic, and Michael M. Zavlanos. Formal verification of stochastic systems with ReLU neural network controllers. In IEEE International Conference on Robotics and Automation (ICRA), pp. 6800-6806, 2022c. doi: 10.1109 10.1109/ICRA46680 /II///AAAA4499.2222 www.6669...2229999 2022.9811866.
Xiaowu Sun, Haitham Khedr, and Yasser Shoukry. Formal verification of neural network controlled au- tonomous systems. In ACM International Conference on Hybrid Systems: Compulation and Control (HSCC), pp. 147-156, New York, NY, USA, 2019. ACM. ISBN 778-1-450-36282-11 doi: 10.1145/3302504 3311802.
Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska, and Daniel Kroening. Con- colic testing for deep neural networks. In ACM/TEEE International Conference on Automated Software Engineering (ASE), pp. 109-119, New York, NY, USA, 2018. ACM. ISBN 978-1-450-35997-5. doi:
10.1145/3338147.323333
Richard Sutton. The bitter lesson 2019.
URL
BitterLesson html
Richard S. Sutton, Doina Precup, and Satinder Singh. Between MDPs and scmi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1):181-22) -211,, August 1999. ISSN 0004-3702. doi: 10. (0.1011/5000443700000))))0
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks, 2014. URL httts:://axxi: wrw. abs/a//s///// 1312.6316... 6199
Zhe Tao and Aditya V. Thakur. Provable gradient editing of deep neural networks. In Advances in Neural In- formation Processing Systems (NeurIPS), 2025. URL http:://pppperreeeeeeeetttrrrrna
Andrew J. Taylor, Victor D. Dorobantu, Hoang M. Le, Yisong Yue, and Aaron D. Ames. Episodic learning with control Lyapunov functions for uncertain robotic systems. Tn IEFF/ //RJJ International Conference doi:
on Intelligent Robots and Systems (IROS), pp. 6878-6884, 2019.
Emanuel Todorov, Tom Erez, and Yuval Tassa. MuJoCo: A physics engine for model-based control. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5026-5033, 2012. doi: 10.1109 10.1109/IROS. IROS. PTROS.201....
C.J. Tomlin, J. Lygeros, and S. Shankar Sastry. A game theoretic approach to controller design for hybrid systems. Proceedings of the IEEE, 88(7):949 970, 2000. doi: 10.1109/5.87133
Noah Topper, George Atia, Ashutosh Trivedi, and Alvaro Velasquez. Active grammatical inference for non- Markovian planning. In International Conference Automated Planning and Scheduling (ICAPS), pp. 647-651, 2022.
doi: 10. 20.1609/icaps.vve wtt.p....32....90.. 19853.
Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. Mcllraith. Reward machines: Exploiting reward function structure in reinforcement learning. Journal of Artificial Intelligence Research. 73:173 208, January 2022. ISSN 1076-9757. doi: 10.1613/jair...1244
Hoang-Dung Tran, Diago Manzanas Lopez, Patrick Musau, Xiaodong Yang. Luan Viet Nguyen, Weiming Xiang, and Taylor T. Johnson. Star-based reachability analysis of deep neural networks. In Maurice H. ter Beek, Annabelle McIver, and José N. Oliveira (eds.), International Symposium on Formal Melhods (FM), volume 11800 of Lecture Notes in Computer Science, pp. 670-686, 1866 Cham, 2019a. Springer International Publishing. ISBN 078-3-030-30942-8. doi: 10.1007/972 7777-3-0030-0944 39.
Hoang-Dume Tran, Patrick Musau, Diego Manzanas Lopez, Xiaodong Yang, Luan Viet Nguyen, Weiming Xiang, and Taylor T Johnson. Parallelizable reachability analysis algorithms for food-forward neural net- works. In IEEE/ACE AACM International Conference on Formal Methods in Software Engineering (FormaliSE), pp. 51 51-60, 2019b. doi: 10. 10.1 Fo0m00l555..22299000222
35

---

Published in Transactions on Machine Learning Research (12/2025)
Hoang-Dung Tran, Stanley Bak, Weiming Xiang, and Taylor T. Johnson. Verification of deep convolutional neural networks using imagestars. In Shuvendu K. Lahiri and Chao Wang (eds.), International Conference on Computer Aided Verification (CAV), volume 12224 of Lecture Notes in Computer Science, pp. 18 42, Cham, 2020a. Springer. ISBN 778-3-030-5328--4 doi: 10.1007 1111007977-3-0000 9907799977--00-288-----88
Hoang-Dung Tran, Xiaodong Yang, Diego Manzanas Lopez, Patrick Musau, Luan Viet Nguyen, Weiming Xiang, Stanley Bak, and Taylor T. Johnson. NNV:
The neural network verification tool for deep neural systems. In Shuvendu K. Lahiri and Chao Wang (eds.), Inter-
networks and learning-enabled national Conference on Computer Aided Verification (CAV), volume 12224 of Lecture Notes in Computer Science, pp. 3 17, Cham, 2020b. Springer. ISBN 978-3-030-53288-8. doi: 10. 1-11077978-3-000 /99799988000--32888--
1.
Hoang-Dum Tran, Weiming Xiang, and Taylor T. Johnson. Verification approaches for learning-enable autonomous cyber physical systems. IEEE Design and Test, 39(1):24 34, 2022. doi: 10.1109 ADA.T 2020.3015712.
Caterina Urban and Antoine Miné. A review of formal methods applied to machine learning, 2021. URL https://arxiv. orgo aa/aab///2200 2214..214.0000 22466
Muhammad Usman, Divya Gopinath; Youcheng Sun, Yannic Noller, and Corina S. Päsăreanu. NNrepair: Constraint-based repair of neural network classifiers. In International Conference on Computer Aided Verification (CAV), volume 12759 of Lecture Noles in Compuler Science, pp. 3 3-, Cham, 2021. Springer. 117179999---00---0
ISBN 978-3-030-81685-8, doi:
Pashootan Vaczipoor, Andrew C Li, Rodrigo A Toro Icarte, and Sheila A. Mcilraith. LTL2Action: Generaliz- ing LTL instructions for multi-task RL. In Marina Meila and Tong Zhang (eds.), International Conference on Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Research, pp. 10497-10508. PMLR, 2021. URL
mlr htt.a///g......ppapae/ee/////ppp htmm
Moshe Y. Vardi. Automatic verification of probabilistic concurrent finite state programs. In JEEE Symposium on Poundalions of Compuler Science (SFCS), pp. 327 338, 1985. doi: N..1109/SFCS.155
Cameron Voloshin, Hoang Minh Le, Swarat Chaudhuri, and Yisong Yue. Policy optimization with linear temporal logic constraints. Tn Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.). Advances in Neural Information Processing Systems (NeurIPS), 2022. URL forum? 2=y2P//dyy2GPR122000C
net/
Cameron Voloshin, Abhinav Verma, and Yisong Yue. Eventual discounting temporal logic counterfac- tual experience replay. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara. Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning (ICML), vol- ume 202 of Proceedings of Machine Learning Research, pp. 35137-35150. PMLR, 2023.
URL https:
//proceedings. wwwpeddiinnnmm......mmmmm.mmmmmm wttpppess//v222/ooooiooiaa htmm
Fujun Wang, Zining Cao, Lixing Tan, and Hui Zong. Survey on learning-based formal methods: Taxonomy, applications and possible future directions. IEEE Access, 8:108561 108578, 2020. doi: N0.1109/ACCSS, 2020.3000907.
Yanwei Wang, Nadia Figueroa, Shen Li, Ankit Shah, and Julie Shah. Temporal logic imitation: Learning plan-satisficing motion policies from demonstrations. In Conference Robot Learning (CoRL), 2022. URL htttpp/sspppeneeeeeeeeeeee//eeep
Yanwei Wang, Tsun-Hsuan Wang, Jiayuan Mao, Michael Hagenow, and Julie Shah. Grounding language
plans in demonstrations through counterfactual perturbations. In International Conference on Learning Representations (ICLR), 2024. URL https://openrreviet http:/ewweeeeeoommmmod/eeeeeeee
Yuning Wang and He Zhu. Deductive synthesis of reinforcement learning agents for infinite horizon tasks. In International Conference on Computer Aided Verification (CAV), 2025. URL https://paat .sf. .ss.gov/ biblio/10593428
36

---

Published in Transactions on Machine Learning Research (12/2025)
Min Wen, Ivan Papusha, and Ufuk Topcu. Learning from demonstrations with high-level side information. In International Joinl. Conference on Arlificial Intelligence (IJCAT); pp. 3055-3061, 2017. doi: 10.24963/ ijcai.
Tichakorn Wongpiromsarn, Ufuk Topcu, and Richard M. Murray. Receding horizon control for temporal logic specifications. In ACM International Conference on Hybrid Systems: Computation and Control (HSCC), pp. 101 110, New York, NY, USA, 2010. ACM. ISBN 978- 178-1-605-58955-8 doi: 01445/1755952.1533966
Yi Wu, Zikang Xiong, Yiran Hu, Shreyash S. Iyengar, Nan Jiang, Aniket Bera, Lin Tan, and Suresh Ja- gannathan. SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models. In JEEE International Conference on Robotics and Automation (ICRA), pp. 2599-2605, 2025. doi: 10.1109 10.110//CRAA5570 200
Weiming Xiang and Taylor T. Johnson. Reachability analysis and safety verification for neural network control systems, 2018. URL https: //arxiv org/abs/18055 09944
Weiming Xiang, Hlang-Dung Tran, Joel A. Rosenfeld, and Taylor T. Johnson. Reachable set estimation and safety verification for piecewise linear systems with neural network controllers. In 2018 Annual American Control Conference (ACC), pp. 1574-1579, 2018. doi: 10. 10.23919/ 000999//ACC.221884311144
Yun Xing, Qing Guo, Xiaofeng Cao, Ivor W. Tsang, and Lei Ma. MetaRepair: Learning to repair deep neural networks from repairing experiences. In ACM International Conference on Multimedia (ACM-MM), pp. 1781-1790, New York, NY, USA, 2024. ACM. ISBN 970-8-400-70686-8. doi: 10.1145/3664647..666
Zikang Xiong and Suresh Jagannathan. Manipulating neural path planners via slight perturbations. IEEE Robotics and Automation Letters, 9(6):5006 5013, June 2024. ISSN 2377-3774. doi: 10.1109/Ira.2000 2024..
3387131
Zikang Xiong, Joe Eappen, Ahmed H. Qureshi, and Suresh Jagannathan. Model-free neural Lyapunov control for safe robot navigation. In IEEE/RS /RSJJ International Conference on Intelligent Robots and Systems (IROS), pp. 5572 5579, 2022. doi: 10. 11100//TR0444661222222998888
Zikang Xiong, Daniel Lawson Joe Eappen, Ahmed H. Qureshi, and Suresh Jagannathan. Co-learning planning and control policies constrained by differentiable logic specifications. In IEEE International Conference Robotics and Automation (ICRA), pp. 14272 14278, 2024. doi: 10.1009/CRAA51147.220 10610942.
Chen Xu, Tony Khuong Nguyen, Patrick Miller, Robert Lee, Paarth Shah, Rares Andrei Ambrus, Haruki Nishimura, and Masha Itkina. Uncertainty-aware failure detection for imitation learning robot policies. In CoRL Workshop on Safe and Robust Robot Learning for Operation in the Real World, 2024. URL
net/ wtwppreeveww.ee//forrrfaaae
Zhe Xu, Bo Wu, Aditya Ojha, Daniel Neider, and Ufuk Topcu. Active finite reward automaton inference and reinforcement learning using queries and counterexamples. In Andreas Holzinger, Peter Kieseberg, A. Min Tjoa, and Edgar Weippl (eds.), International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE), volume 12844 of Lecture Notes in Computer Science, pp. 115-135, Cham, 2021. Springer. ISBN 978-3-030-840660--
doi:
8.
Beyazit Yalcinkaya, Niklas Lauffer, Marcell Vazquez-Chanlatte, and Sanjit A. Seshia. Compositional au- tomata embeddings for goal-conditioned reinforcement. learning, 2025. URL https://arxiv..tt /
2411.00205
Yoriyuki Yamagata, Shnang Liu, Takumi Akazaki, Yihai Duan, and Jianye Hao. Falsification of cyber- physical systems using deep reinforcement learning. IEFE Transactions on Software Engineering, 47(12): 2823 2840, 2021. doi: 10.1109 2S0.0//TTE..2009999919
Cambridge Yang, Michael L. Littman, and Michael Carbin. On the (in)tractability of reinforcement learning for LTL objectives. In Lud De Raodt (od.), International Joint Conference on Artificial Intelligence (IJCAI), pp. 3650-3658. International Joint Conferences on Artificial Intelligence Organization, 2022a. doi: 1144966//ccai222222
37

---

Published in Transactions on Machine Learning Research (12/2025)
Xiaodong Yang, Tom Yamaguchi, Hoang-Dung Tran, Bardh Hoxha, Taylor T. Johnson, and Danil Prokhorov. Neural network repair with reachability analysis. In Sergiy Bogomolov and David Parker (eds.), Formal Modeling and Analysis of Timed Systems (FORMATS), volume 13465 of Lecture Notes in Computer Science, pp. 221-236, Cham, 2022b. Springer. ISBN 978-3-001-15889-.. doi: 13.
Chanyeol Yoo and Calin Belta. Control with probabilistic signal temporal logic, 2015. URL nttps://arxii. org/ wttaaabs.....000
Dongjie Yu, Haitong Ma, Shengbo Li, and Jianyu Chen. Reachability constrained reinforcement learning. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), International Conference Machine Learning (ICML), volume 162 of Proceedings of Machine Learning Research, pp. 186666256555 PMLR, 2022. URL https://prooceeding. mlr mwl..ppess//116///2222.. html.
Emily Yu, undefinedrrde Zikelić, and Thomas A. Henzinger. Neural control and certificate repair via runtime monitoring. In AAAT Conference on Artificial Intelligence (AAAT), pp. 26409-26415 AAAT Press, 2025. ISBN 978-1-57735-597-8. doi: 10.1609
Yu Zhang, Peter Tino, Ales Leonardis, and Ke Tang. A survey on neural network interpretability. IEEE Transactions on Emerging Topics in Compulational Intelligence, 5(5):726 742, October 2021. ISSN 2471-
285X. doi: 1111009/ttti.2221... 202.....22...100111...
Zhenya Zhang, Deyun Lyu, Paolo Arcaini, Lei Ma, Ichiro Hasuo, and Jianjun Zhao. FalsifAl: Falsification of Al-enabled hybrid control systems guided by time-aware coverage criteria. IEEE Transactions on Software Engineering,
2023.
doi: 10. TT1.0//TT..22..1114444....
Hengjun Zhao, Xia Zeng, Taolue Chen, and Zhiming Liu. Syuthesizing barrier cortificates using neural networks. In International Conference Hybrid Systems: Compulation and Control (HSCC), New York, NY, USA, 2020. ACM. ISBN 978-1-450-37018-9 doi: 00.114//336365..33
Ziyuan Zhong, Davis Rempe, Yuxiao Chen, Boris Ivanovic, Yulong Cao, Danfei Xu, Marco Pavone, and Baishakhi Ray. Language-uuided traffic simulation via scene-level diffusion. In Conference on Robot Learning (CoRL), 2023a. URL https://openrevieet
Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Tong Che, Baishakhi Ray, and Marco Pavone. Guided conditional diffusion for controllable traffic simulation. In IEEE International Conference on Robotics and Automation (ICRA), pp. 3560 3560-3566, 2023b. doi: 10.1109 10.1109/ICRA48BS ICRAA8891.2022..000
Weichao Zhou and Wenchao Li. Safety-aware apprenticeship learning. In Hana Chockler and Georg Weis- senbacher (eds.), International Conference Computer Aided Verification (CAV), volume 10981 of Lec- ture Notes in Computer Science, pp. 662-680, Cham, 2018. Springer. ISBN 978-3-319-96145-3, doi:
38.
He Zhu, Zikang Xiong, Stephen Magill, and Suresh Jagannathan. An inductive synthesis framework for verifiable reinforcement learning. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pp. 686 701, New York, NY, USA, 2019. ACM. ISBN 978- 778-1-450--37112-7 doi: 0111455/331422.33146233
Shibei Zhu, Francesco Belardinelli, and Borja González León. Evolutionary reinforcement learning for sparse rewards. In Genetic and Evolutionary Computation Conference (GECCO), pp. 1510-111, New York, NY, USA, 2021. ACM. ISBN 978-1-450-38351-6, doi: 34445//3449766.4444
A Acronyms
DL
Deep Learning
ML
Machine Learning
38

---

Published in Transactions on Machine Learning Research (12/2025)
FM
Formal Method
Formal Specification
SM
Spocification Mining
RL
Reinforcement Learning
Inverse Reinforcement Learning
Linear-time Temporal Logic
STL
Signal Temporal Logic
MDP
Markoy Decision Process
FSA
Finite State Automaton
DBA
Deterministic Büchi Automaton
LDBA
Limit- Deterministic Büchi Automaton
NDBA
Non- Deterministicc Bicch Automaton
DRA
Deterministic Rabin Automaton
CEGIS
Counter-example Guided Inductive Sysis
RcLU
Rectified Lincar Unit
NN
Noural Network
RNN
Recurrent Neural Network
GNN
Graph Neural Network
ODE
Ordinary Diflerential Equation
PDE
Partial Difterential Equation
MPC
Model Predictive Control
CLF
Control Lyapunov Function
CBF
Control Barrier Function
HJ
Hamilton-Jacobi
SMC
Satishibilitty Modulo Convex Programming
SMT
Satislibiility Modulo Theories
39


Reviews:
[{"review_id": "C5Wxoo4nt3", "reviewer": "Reviewer_ZusR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper is a survey on the use of Formal Methods (FMs) in robot policy learning. The authors argue that while deep learning has enabled highly capable robot policies, these models are often fragile, difficult to interpret, and lack safety guarantees.\n\nThe survey organizes the field into three key parts:\n\n- Specification: How to rigorously define complex, temporally-extended tasks and safety requirements for robots using formal languages like temporal logic.\n- Synthesis: How to create or \"learn\" robot policies that are guaranteed to meet these formal specifications.\n- Verification: How to formally prove that an already-learned policy correctly and safely adheres to a given specification.\n\nThe paper provides a comprehensive overview of techniques within each category, compares their scalability and expressiveness, and concludes by discussing open challenges and future research directions for developing provably safe and trustworthy robots.", "claims_and_evidence_yn": "Yes", "claims_explanation": "As a survey paper, this paper's claims are mostly about the state of a research field, its structure, and its future trajectory. And they are generally supported by extensive citations and logical organization.\n\nI listed a few \"claims\" in this paper below:\n\n- About problem: While deep learning has led to highly performant robot policies, it has simultaneously introduced critical weaknesses. These learned policies are often \"inflexible, fragile, and difficult to interpret,\" lack formal guarantees, and are vulnerable to issues like reward hacking and poor generalization, making them untrustworthy for safety-critical applications.\n\n\n- About solution: Formal Methods (FMs), which have been indispensable for ensuring correctness in complex software systems, represent a promising approach to address the weaknesses of DL-based robot policies and transform them into \"transparent, reliable, and trustworthy solutions.\n\n- About framework: The work at the intersection of FMs and robot policy learning can be naturally and comprehensively organized into a three-stage pipeline: Specification, Synthesis, and Verification.\n\n- About future directions: The paper identifies specific, significant gaps and promising directions for future research, such as handling more complex specifications in RL, developing better offline learning methods with FMs, and scaling verification to larger, more modern policy models.", "audience_yn": "Yes", "audience_explanation": "Formal methods can potentially bridge the gap between the high performance of modern AI and the high stakes of real-world robotics. So I believe both AI and robotics researchers will be interested in this paper.", "requested_changes": "Overall, I think this paper is in pretty good shape. If possible, I’d love to see a few examples of how formal methods are applied in real-world robotic systems.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ALrnGjcYhQ", "reviewer": "Reviewer_T8XK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides an overview of the integration of Formal Methods (FMs) into robot policy learning, focusing on Deep Learning (DL)-based approaches in robotics. It organizes the paper following a three-part pipeline: \n- specification: (acquiring and representing formal behavioral requirements, such as using temporal logics like LTL and STL, or mining them from data), \n- synthesis: (learning policies that satisfy these specifications, including FM-guided reinforcement learning, imitation learning, and inverse reinforcement learning),\n-  verification: (proving or falsifying that learned policies meet specifications, via methods like reachability analysis, certificate functions, and falsification techniques). \n\nThe paper highlights representative techniques in each area, compares their scalability (e.g., handling high-dimensional systems) and expressiveness (e.g., support for stochastic or partially observable environments), and discusses how they enhance robot safety, interpretability, and trustworthiness. Additionally, it identifies key gaps (e.g., handling complex LTL specifications, partial observability, and scalability for modern architectures like transformers) and proposes promising future directions (e.g., combining LTL with offline RL, probabilistic extensions for real-world deployment, and leveraging foundational models for specification mining). Overall, it serves as a valuable resource for the research community, aiming to guide research toward provably safe and reliable robots by integrating formal methods.\n\npros:\n- Formal methods in robot policy are an important topic that addresses the growing need for safety and trustworthiness in DL-based robotics.\n- This paper provides a broad range of works related to formal methods in robot policy learning, which is useful to the community.\n\nCons\n- Writing should be largely improved. I find it hard to follow the paper. And it took me three times of reading to figure out the major structure of the paper.\n- As a survey focusing on formal methods in robot policy, this paper puts too much effort into introducing the formal specifications methods (whole section 2), for example, different formal methods such as TLT, SLT, and Automata. I strongly recommend that the authors shorten Section 2 as preliminaries or put Section 2 in the appendix to avoid hampering the reading of the whole story. Since Section 2 serves as the preliminary but not part of the robot policy.\n- As two methods which are also formal methods, First Order Logic (FOL) and STRIPS planners are missing in part 2. And correspondingly, the related works are missing in both aspects, such as [1] and [2]\n- I believe this is a typo on page 2, 4th paragraph, 4th line. It should be ‘’moreover , FMs allow…..’’ instead of  ‘’moreover , FSs allow…..’’\n\n\n[1] Ye, Z., Arenz, O., & Kersting, K. (2025). Learning from Less: Guiding Deep Reinforcement Learning with Differentiable Symbolic Planning. arXiv preprint arXiv:2505.11661.\n\n[2] Kienle, C., Alt, B., Arenz, O., & Peters, J. (2025). LODGE: Joint Hierarchical Task Planning and Learning of Domain Models with Grounded Execution. arXiv preprint arXiv:2505.13497.", "claims_and_evidence_yn": "Yes", "claims_explanation": "as a survey paper it cites the corresponding paper and explain each paper accordingly", "audience_yn": "Yes", "audience_explanation": "this paper includes many related work that can be beneficial for the research community and the topic the paper addresses is an important  topic", "requested_changes": "- minor typo on page 2, 4th paragraph, 4th line should be corrected\n- Section 2 should be largely improved and also shortened or maybe moved to the appendix\n- The writing should be largely improved", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZN3SVAzyrK", "reviewer": "Reviewer_NSyM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper surveys formal and semi-formal methods for specifying behavior, training deep learning models, and performing quality assurance (QA), that may be relevant for robot policy learning. The review is structured along these columns: specifications, training, and QA. Further, it provides suggestions for future work at the end. \n\nStrengths:\n 1. The paper covers many relevant applications of formal methods.\n 2. The paper provides a wealth of pointers to existing work on various applications of formal methods.\n\nWeaknesses:\n 1. The survey is not targeted. \n 2. The paper is poorly structured and written.\n 3. The paper contains factual errors.", "claims_and_evidence_yn": "No", "claims_explanation": "The paper claims to perform a survey of formal methods in robot policy learning. Many of the surveyed papers are not about robot policy learning but general formal methods papers. For example, section 2.2.1 contains at best three citations to papers about robotics from 34 citations on about two pages. While the other papers are about general techniques that may be useful for robot policy learning, this does not match the claim that \"this is a survey of FMs as applied to learning methods for robot policies\". \n\nOverall, this is weakness 1, the survey is not targeted. \n\nIf the purpose is to also survey formal methods more generally, it is unclear how the presented approaches were selected, as there is certainly a wealth more of formal method papers that might be broadly interesting for robot policy learning. Some examples under \"Requested Changes\".", "audience_yn": "Yes", "audience_explanation": "Formal methods are interesting for robotics in safety-critical systems, as is motivated well in the paper.", "requested_changes": "Critical Changes.\n\n### Structure & writing needs to be improved (Weakness 2)\nFor example,\n- The preliminaries section 2.1 is clearly also important for sections 3 and 4 but the structure makes it seem like it is only important for section 2. \n- Section 3 is structured along different RL technologies. This might be helpful for a review on RL but it is not clear to me why this differentiation is insightful for robotics or formal methods.\n- Paragraph \"Languages\" in section 2.1 is about transition systems and not formal languages. \n- The paragraph just above \"Temporal Logic\" claims that the paper focuses primarily on LTL, while many of the papers are actually on STL.\n- The text does not say that \"p\" and phi are in the syntax equation for LTL. In contrast, the text specifies this for STL but after LTL was introduced.\n- The discussion of DRAs vs DBAs vs LDBAs is duplicated on page 8 and 12. \n- The paragraph on falsification on page 14 is severely misplaced, as it does not relate to discrete systems or hybrid systems. \n- The MILP-based approaches discussed on page 17 do not employ set propagation but are placed in a paragraph titled \"set propagation\".\n... and more. \n\nOverall, it seems like the paper did not undergo proofreading. This is also evident from formatting errors, such as the many citations that are not set in parentheses.  \n\n### Factual Errors (Weakness 3)\n- The presented LTL semantics are incorrect. For example, the always operator requires phi not only for all times in the future but also the current point in time. \n- The discussion of STL forgets to mention that punctual intervals with a = b are disallowed, although this is essential for the decidability of STL.\n - The discussed finite automata are more expressive than LTL. They do not recognize *the same* languages. \n\nAn adjacent point is the usage of the term \"synthesis\" in the paper. It correctly gives the standard definition of synthesis as generating a system that *provably* obeys a specification in section 3.1. However, in the remainder of the paper, \"synthesis\" is used much more liberally. The first paragraph in section 3 even equates synthesis and learning, which are two very different things from a formal methods point of view. As such, also the fundamental organization as depicted in figure 1 does not make sense from a formal methods point of view, since synthesis does not require verification afterwards. A more apt title for section 3 would be \"learning with formal specifications\". \n\n### Relevant Directions not Covered\n- When looking at learning and verification, repair is a key technique for obtaining a satisfactory pipeline. See, for example \"Provable repair of deep neural networks\" by Sotoudeh and Thakur (2021).\n- There are other recent tools for verifying neural-network-controlled cyber-physical systems besides NNV, such as Cora https://tumcps.github.io/CORA/ or JuliaReach https://juliareach.github.io/ that are not discussed, while NNV is. \n\n### Further Issues\n- The reference (Fang et al., 2024) does not support the claim on page 1. Rather, Fang et al. state that NNs can learn out-of-distribution detection.\n- In what regard are formal specifications \"richer\" than arbitrary reward functions? (4th paragraph on page 2)\n- What is understood by \"value misalignment\"? (same paragraph)\n- What is meant by \"temporally extended behaviors\"? (1st paragraph on page 5)\n- The \"Many followup works in SMs\" in the 3rd paragraph on page 7 are not cited.\n- What is a \"plain\" FSA? (1st paragraph on page 12)\n- What is meant by \"approximate methods\"? (2nd paragraph on page 15)\n- What is meant by \"dynamically feasible\"? (3rd paragraph on page 15)\n\n---\n\nOther changes that would strengthen the paper.\n\n### Length\n- It is unclear if all of the included material is vital for the paper. For example the section on automata could be removed. \n\n### Further Directions\n- Runtime monitoring is another formal method that can be relevant for robot policy learning.", "broader_impact_concerns": "As this is a survey of existing work, I have no concerns about broader impact.", "additional_comments": "A few important formatting and more minor writing issues:\n- A reference for abbreviations should not be an excuse for not introducing abbreviations properly. Abbreviations such as \"HJ\" should be introduced in the text. \n- In figure 1, the figure and the caption use different phi symbols. \n- Figure 7 contains references for some sections but not for others\n- The bibliography is poorly formatted. For example, abbreviations, such as STL and SpaTeL are frequently not capitalized correctly. Also, some conferences are abbreviated, while others are not, and DOIs and URLs sometimes duplicate themselves. \n\nThere are also a number of typos in the manuscript. Most notably the last sentence on page 7 just ends abruptly.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)