SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A quadrotor is tasked to follow a piecewise-linear reference trajectory via bang-bang control (see Figure 1(a)). The six-dimensional state space consists of two 3D vectors for position and velocity. The action space consists of a 3D vector representing possible combinations of pitch  $(\theta)$ , roll  $(\phi)$ , and thrust acceleration  $(\alpha)$ . Using the gravity constant  $g = 9.81$ , the continuous dynamics are:
$$
\dot {p _ {x}} = v _ {x} \qquad \dot {p _ {y}} = v _ {y} \qquad \dot {p _ {z}} = v _ {z} \qquad \dot {v _ {x}} = g \tan (\theta) \qquad \dot {v _ {y}} = - g \tan (\phi) \qquad \dot {v _ {z}} = \alpha - g
$$
The policy's task is to minimize the difference between the reference and the actual trajectory. The set of initial states is  $\mathcal{X}_0 = [-0.05, 0.05]^2 \times \{0\}^4$ . The control period is 0.2 time units and the number of control cycles is 30. The decision-tree policy  $\mathcal{T}$  has depth 10 and is too large to depict here.
B.2 Cart-Pole System
A pole is vertically attached on top of a cart that can be moved left or right along a frictionless track. The goal is to move the cart such that the pole is kept in an upright pose (i.e., the pole angle is small).
The state consists of the cart position  $p$ , cart velocity  $v$ , pole angle  $\theta$ , and angular velocity  $\omega$ . The action space is  $\mathcal{U} = \{-1, 1\}$  ("left", "right") with control variable  $u$ . The continuous dynamics are:
$$
\dot {p} = v \qquad \qquad \dot {v} = \psi - \frac {1}{2 2} \phi \cos (\theta) \qquad \qquad \phi = \frac {9 . 8 \sin (\theta) - \cos (\theta) \psi}{2 / 3 + 5 / 1 1 \cos (\theta) ^ {2}}
$$
$$
\dot {\theta} = \omega \qquad \dot {\omega} = \phi \qquad \psi = \frac {1 0 u + 0 . 0 5 \omega^ {2} \sin (\theta)}{1 . 1}
$$
The set of initial states is the union of two hyperrectangles  $\mathcal{X}_0 = [-0.05, 0.05]^4$  and  $\{0\}^2 \times [-0.025, 0.025] \times [-0.2, 0.2]$ . (We added the second hyperrectangle to make the problem more challenging.) The control period is 0.02 time units. The decision-tree policy  $\mathcal{T}$  is shown in Figure 7(a) and works as follows. If the pole is leaning to the left ( $\theta \leq -0.014$ ) or moving in the left direction ( $\omega \leq -0.201$ ), the cart is moved to the left (action  $-1$  in the leaves), and otherwise to the right.
The goal is to keep the pole angle  $\theta$  within a small range around 0. Figure 8 shows simulations from the corners of  $\mathcal{X}_0$  plus 100 random initial states inside  $\mathcal{X}_0$  for 2 time units. From these simulations, we can see that  $\theta$  seems to stay in the interval  $[-0.06, 0.06]$ , but some trajectories come close to the upper bound. In the evaluation (Figure 5(a)) we prove this property for  $\mathcal{R}_{[0,0.5]}$ .
The fixpoint check allows to conclude that the policy is able to balance the pole forever. The fixpoint is only found in the  $\theta/\omega$  projection, but this is sufficient because the cart's position and velocity are irrelevant for the decisions of the tree.
B.3 Acrobot System
The acrobot system consists of two links connected by a joint to form a chain. One end is fixed while the joint and the other end can swing freely. There are four state dimensions: angles  $\theta_{1}$  and  $\theta_{2}$ , and their velocities  $\dot{\theta}_{1}$  and  $\dot{\theta}_{2}$ . The action space is  $\mathcal{U} = \{0,1,2\}$  ("left", "none", "right") with control variable  $u$ . The continuous dynamics of the velocities are:
$$
\ddot {\theta} _ {1} = - \frac {d _ {2} \psi + \phi_ {1}}{d _ {1}} \quad \ddot {\theta} _ {2} = \psi
$$
$$
d _ {1} = m _ {1} l c _ {1} ^ {2} + m _ {2} \left(l _ {1} ^ {2} + l c _ {2} ^ {2} + 2 l _ {1} l c _ {2} \cos (\theta_ {2})\right) + I _ {1} + I _ {2}
$$
$$
d _ {2} = m _ {2} \left(l c _ {2} ^ {2} + l _ {1} l c _ {2} \cos (\theta_ {2})\right) + I _ {2}
$$
$$
\phi_ {2} = m _ {2} l c _ {2} g \cos \left(\theta_ {1} + \theta_ {2} - \frac {\pi}{2}\right)
$$
$$
\begin{array}{l} \phi_ {1} = - m _ {2} l _ {1} l c _ {2} \dot {\theta} _ {2} ^ {2} \sin (\theta_ {2}) - 2 m _ {2} l _ {1} l c _ {2} \dot {\theta} _ {2} \dot {\theta} _ {1} \sin (\theta_ {2}) \\ + \left(m _ {1} l c _ {1} + m _ {2} l _ {1}\right) g \cos \left(\theta_ {1} - \frac {\pi}{2}\right) + \phi_ {2} \\ \end{array}
$$
$$
\psi = \frac {u + \frac {d _ {2}}{d _ {1}} \phi_ {1} - m _ {2} l _ {1} l c _ {2} \dot {\theta} _ {1} ^ {2} \sin (\theta_ {2}) - \phi_ {2}}{m _ {2} l c _ {2} ^ {2} + I _ {2} - \frac {d _ {2} ^ {2}}{d _ {1}}}
$$
The goal is that the free end reaches a desired height, expressed as the condition  $-\cos(\theta_1) - \cos(\theta_2 + \theta_1) > 1$ , where  $\theta_1$  is the joint angle and  $\theta_2$  is relative to the angle of the first link.
The initial condition  $\mathcal{X}_0$  is a union of a point and a hyperrectangle  $\mathcal{X}_0 = (\{-0.0005\} \times \{0\} \cup [-0.0705, -0.0695] \times [-0.0005, 0.0005]) \times \{0\}^2$ . The control period is 0.2 time units and the time
(a) A decision tree for the cart-pole system.
(b) A decision tree for the mountain-car system.
Figure 7: Decision trees for the cart-pole and the mountain-car systems.
Figure 8: Simulations for the cart-pole system.
Figure 9: Two alternative decision trees for the acrobot system.
horizon is 100 time units. For this system we obtained two different decision-tree policies of depth 2 resp. 3, shown in Figure 9.
The system applies a transformation to the state before being passed to the decision-tree policy. Concretely, the state vector  $(\theta_{1},\theta_{2},\dot{\theta}_{1},\dot{\theta}_{2})$  is mapped to the six-dimensional vector  $(\cos (\theta_1),\sin (\theta_1),\cos (\theta_2),\sin (\theta_2),\dot{\theta}_1,\dot{\theta}_2)$ . (This transformation is technically irrelevant for the two policies we obtained, because they only consider  $\dot{\theta}_{1}$  and  $\dot{\theta}_{2}$ , which can be read from the transformed vector; but we do not exploit that.) We explain how to modify our algorithm in order to cope with transformations in Appendix A.2.
B.4 Mountain-car system
The mountain-car system consists of a sinusoidal valley between two mountains and a car in between. The goal is to bring the car to the top of the higher mountain on the right. The engine is not strong enough to drive up the mountain by itself, so the car must drive up the left mountain and gain momentum to reach the top of the right mountain.
The control actions are  $\mathcal{U} = \{0,1,2\}$  ("left", "none", "right") with control variable  $u$ . The system state consists of the car's position  $x$  and velocity  $v$ . With constants  $F = 0.001$  and  $g = 0.0025$ , the discrete-time dynamics are:
$$
v _ {k + 1} = v _ {k} + (u - 1) F - \cos \left(3 x _ {k}\right) g
$$
$$
x _ {k + 1} = x _ {k} + v _ {k + 1}
$$
The set of initial states is  $\mathcal{X}_0 = [-0.505, -0.445] \times \{0\}$ . The decision-tree policy  $\mathcal{T}$  is given in Fig. 7(b). The maximum number of discrete steps is 200, but as shown in Fig. 5(b), all trajectories reach the goal region within at most 132 steps.
C Proofs of the Theorems
Proof. We show (i) termination and (ii) soundness and relative completeness of Algorithm 1.
(i) The inner loop in line 9 terminates by assumption. The outer loop in line 3 removes one element at time point  $t_0$  from  $Q$  and adds only elements at time point  $t_1 = \min(t_0 + \tau, T_{\max}) > t_0$  to  $Q$  (line 8). Elements after time point  $T_{\max}$  are not processed (line 6).
(ii) We only show the equality case; the inclusion case follows because, in each step, all sets are overapproximated. The algorithm starts with  $\mathcal{X}_0 = \mathcal{R}_0$ . In the first iteration, by assumption,  $post_{\mathcal{T}}$  only returns states  $x_0 \in \mathcal{X}_0$  together with the corresponding action  $act(x_0, \mathcal{T})$ . Similarly,  $post_f$  only returns states reachable within  $[0, \tau]$ . By the end of the first while-loop iteration, we thus obtain  $\overline{\mathcal{R}} = \mathcal{R}_{[0, \tau]}$ , and the queue  $Q$  contains pairs  $(\mathcal{X}_u, \tau)$  such that the union of the sets  $\mathcal{X}_u$  is the set  $\mathcal{R}_{\tau}$ .
Further loop iterations work similarly. While the iteration order of  $Q$  (line 4) is not prescribed, all policies lead to the same result. Let us choose the implementation of  $Q$  as a LIFO (last-in, first-out) queue. This corresponds to a breadth-first search where on level  $k + 1$  of the search tree we have elements  $(\mathcal{X}_i, k\tau)$ ; the union of these sets  $\mathcal{X}_i$  is again  $\mathcal{R}_{k\tau}$ . The claim follows by induction.
We first prove PSPACE-completeness in bounded time. For the membership, we sketch a nondeeterministic algorithm with polynomial space requirements for the complement of the reach-avoid problem, which is sufficient by Savitch's theorem [Savitch, 1970] and because PSPACE is closed under complementation. The algorithm guesses an initial state and simulates the DTCS. In continuous time, the trajectory in one control cycle is a line segment. In each step, the algorithm checks whether (i) the current line segment intersects with the error states  $\mathcal{E}$  and does not intersect with the goal set  $\mathcal{G}$ , or (ii) the final state in the last iteration  $k_{\mathrm{max}}$  is outside the goal set  $\mathcal{G}$ ; these checks can again be performed efficiently. The number of iterations may be exponential, but we can maintain a counter with logarithmically many bits.
For the hardness, we show a polynomial-time reduction from the word problem for deterministic linear bounded automata (LBA). Here we consider the LBA model with accepting state, total transition function, and the input tape containing the input word plus two bound markers for simplicity.
For better intuition, we provide an example after the proof, which we recommend reading in parallel.
Given is an input word  $w \in \{0,1\}^n$  and an LBA  $\mathcal{A} = (Q,A,\delta,q_0,q_{\mathrm{acc}},\#_l,\#_r)$  with states  $Q = \{q_0,q_1,\ldots\}$ , tape alphabet  $A = \{a_0,a_1,\ldots\} \supseteq \{0,1,\#_l,\#_r\}$ , transition function  $\delta \subseteq Q \times A \to Q \times A \times \{L,R\}$  (where  $L$  and  $R$  denote moving to the left resp. right), initial state  $q_0$ , accepting state  $q_{\mathrm{acc}}$ , and bound markers  $\#_l,\#_r$ .
We define two bijections  $N_q: Q \to \{0, \dots, |Q| - 1\}$  and  $N_a: A \to \{0, \dots, |A| - 1\}$  to associate a natural number with each state and tape symbol such that  $q_{\mathrm{acc}}$  is associated with  $|Q| - 1$ .
We construct a DTCS with  $n + 4$  state variables  $x_0, \ldots, x_{n+1}, x_q, x_p, n + 4$  control actions  $u_0, \ldots, u_{n+1}, u_q, u_p$ , control domain  $\mathcal{U} = \mathbb{Z}^{n+4}$  (and control period  $\tau = 1$  for continuous systems). The environment dynamics  $f$  are state-independent, i.e.,  $f(x, u) = u$ .
The decision tree is more complex, and we first establish some intuition. The state variables encode the LBA configuration, where the first  $n + 2$  dimensions  $(x_0, \ldots, x_{n+1})$  encode the tape contents (i.e., the value of  $x_i$  encodes the tape symbol at position  $i$ ),  $x_q$  encodes the current LBA state, and  $x_p$  encodes the current position on the tape. The execution of one control cycle corresponds to taking one transition. For that we construct the tree from four gadgets: a gadget to select the current LBA state, a gadget to select the current position on the tape, a gadget to select the current tape symbol, and a gadget to apply the transition function to the current configuration.
The first gadget (LBA state selection) is a degenerated tree only expanding to the right with  $|Q| - 1$  decision nodes. On the  $k$ -th level the predicate is  $x_{q} \leq k - 1$ . The left leaf node at level  $k + 1$  is the sub-tree  $\mathcal{T}_{k-1}$  corresponding to the second gadget. On the last layer, the right leaf node is  $\mathcal{T}_{|Q| - 1}$ .
(a) Gadget 1: Outer tree spine  $\mathcal{T}$  with three states.
(b) Gadget 2: Sub-tree  $\mathcal{T}_k$  with three tape positions.
(c) Gadget 3: Sub-tree  $\mathcal{T}_{k,\ell}$  with four tape symbols.
Figure 10: Decision-tree structure from Example 4.
The second gadget (position selection) is an analogous tree  $\mathcal{T}_k$  over  $x_{p}$  with  $n + 1$  decision nodes. On the  $\ell$ -th level the predicate is  $x_{p} \leq \ell - 1$ . The left leaf node at level  $\ell + 1$  is the sub-tree  $\mathcal{T}_{k,\ell -1}$  corresponding to the third gadget. On the last layer, the right leaf node is  $\mathcal{T}_{k,n + 1}$ .
The third gadget (symbol selection) is an analogous tree  $\mathcal{T}_{k,\ell}$  over  $x_{\ell}$  with  $|A| - 1$  decision nodes. On the  $m$ -th level the predicate is  $x_{\ell} \leq m - 1$ . The left leaf node at level  $m + 1$  is the sub-tree  $\mathcal{T}_{k,\ell,m-1}$  corresponding to the fourth gadget. On the last layer, the right leaf node is  $\mathcal{T}_{k,\ell,|A|-1}$ .
The fourth gadget (application of the transition function) is a single leave node  $\mathcal{T}_{k,\ell,m}$ . The indices encode that the LBA is currently in state  $N_{q}^{-1}(k)$  at position  $\ell$  and reads tape symbol  $N_{a}^{-1}(m)$ . Let us assume that the transition function  $\delta$  prescribes to switch to state  $q'$ , write tape symbol  $a'$ , and move to the right. Correspondingly, we want to choose the control actions such that in the next step the state of the system is  $x_{q} = N_{q}(q')$ ,  $x_{p} = \ell + 1$ ,  $x_{\ell} = N_{a}(a')$ , and all other state variables remain unchanged. This is achieved by choosing
$$
u _ {q} = N _ {q} \left(q ^ {\prime}\right) - k, \quad u _ {p} = 1, \quad u _ {\ell} = N _ {a} \left(a ^ {\prime}\right) - m, \quad u _ {i} = 0 (\text {f o r a l l} i \neq \ell).
$$
(For moving to the left, choose  $u_{p} = -1$  instead.) Observe that the state variables keep track of the LBA configuration and that we have corresponding leaves in  $\mathcal{T}$  for each transition in  $\delta$ . For a fixed transition  $(q, a, q', a', d)$  we actually have multiple leaves; they only differ in the choice of the control action  $u_{\ell}$  (corresponding to the current cell index on the tape).
Finally, let  $w_{i}$  be the  $i$ -th symbol of  $w$  (starting with  $i = 1$ ). The initial state is  $x_{q} = N_{q}(q_{0}), x_{p} = 1$  (first symbol of  $w$ ),  $x_{0} = N_{a}(\# l)$ ,  $x_{n + 1} = N_{a}(\# r)$ , and  $x_{i} = N_{a}(w_{i})$  for all other  $i$ . The set of goal states  $\mathcal{G}$  consists of all states where  $x_{q} = N_{q}(q_{\mathrm{acc}}) = |Q| - 1$ . The set of error states  $\mathcal{E} = \emptyset$  is not needed. The step bound is  $k_{\max} = |Q||A|^{n + 2}(n + 2)$  (encoded in binary).
The above construction is polynomial in the size of the LBA and the word (the decision tree has  $O(|Q| + |A| + n)$  nodes), and the following equivalence shows that it is a reduction.
$w$  is accepted by  $\mathcal{A}$
$\Longleftrightarrow$  the configuration  $\langle q_0, \#_{l}w\#_{r}\rangle$  leads to a configuration with state  $q_{\mathrm{acc}}$
$\stackrel{(*)}{\Longleftrightarrow}$  the DTCS reaches a state with  $x_{q} = N_{q}(q_{\mathrm{acc}})$  within  $k_{\mathrm{max}}$  steps
$\Longleftrightarrow$  the DTCS satisfies the reach-avoid specification
The critical step is  $(\ast)$ , for which we rely on the intuition established above. Note that in the continuous-time case we have to make sure that a goal state is not reached accidentally between two control cycles. This is ensured by letting  $N_{q}(q_{\mathrm{acc}}) = |Q| - 1$ .
This concludes the proof of PSPACE-completeness. Before we continue with the undecidability in unbounded time, we give an example to better illustrate the previous reduction.
Example 4. Consider the LBA with states  $q_0, q_s, q_{acc}$ , tape symbols  $0, 1, \#_l, \#_r$ , and the only explicit transition  $\delta(q_0, 1) = (q_{acc}, 0, R)$  (all other transitions lead to the sink state  $q_s$  with appropriate direction  $L / R$ ). The LBA accepts words starting with a 1 (and changes it to 0). We use the mappings
$$
N _ {q} (q) = \left\{ \begin{array}{l l} 0 & q = q _ {0} \\ 1 & q = q _ {s} \\ 2 & q = q _ {a c c} \end{array} \right.
$$
$$
N _ {a} (a) = \left\{ \begin{array}{l l} 0 & a = 0 \\ 1 & a = 1 \\ 2 & a = \# _ {l} \\ 3 & a = \# _ {r}. \end{array} \right.
$$
Now consider the input word  $w = 1$ , which means we have the initial configuration  $\langle q_0, \#_l 1 \#_r \rangle$ . We construct a state-independent DTCS with state variables  $x = (x_0, x_1, x_2, x_q, x_p)$  and control actions  $u = (u_0, u_1, u_2, u_q, u_p)$ . The initial state is  $(\underbrace{2,1,3}_{\#_l 1 \#_r}, \underbrace{0}_{q_0}, \underbrace{1}_{position})^T$ .
The decision tree  $\mathcal{T}$  is quite large (36 leaves), and we only depict its structure in Fig. 10. The first cartoon shows the outer structure. Each leaf node follows the structure in the second cartoon, and each leaf node shown there follows the structure in the third cartoon. Finally, we give one example leaf  $\mathcal{T}_{0,1,1}$  of the whole tree, which stands for state  $q_{0}$ , tape position 1, and reading symbol 1. Observe that this is the leaf the decision tree chooses in the initial state. The control-action vector in that leaf is  $(\underbrace{0}_{cell0}, \underbrace{0 - 1}_{cell1}, \underbrace{0}_{cell2}, \underbrace{2 - 0}_{state}, \underbrace{1}_{position})^{T}$ . Thus, in the next time step, the new state will be
$(\underbrace{2,0,3}_{\#_l0\#\_r}, \underbrace{2}_{q_{acc}}, \underbrace{2}_{position})^T$ , which is a goal state. This concludes the example.
Now we prove undecidability in unbounded time. For that, we reduce from the termination problem for deterministic two-counter machines (2CMs). We consider the model from [Shepherdson and Sturgis, 1963, Theorem 4.1] with two counters  $c_1, c_2 \in \mathbb{N}$  and instruction set  $\mathsf{INC}(c_z)$  (increment counter  $c_z$ ),  $\mathsf{DEC}(c_z)$  (decrement counter  $c_z$  with zero lower bound),  $\mathsf{JZ}(c_z, \ell)$  for  $z \in \{1, 2\}$  (jump to location  $\ell$  if counter  $c_z$  is zero), and STOP (terminate). Each instruction  $I$  comes with a position  $k$  in the program, which we write " $k : I$ ". The initial value of the counters is  $c_1 = c_2 = 0$  and the program starts at instruction 1.
Given is a 2CM with  $n$  instructions. We construct a DTCS with 3 state variables  $x_0, x_1, x_{pc}, 3$  control actions  $u_1, u_2, u_{pc}$ , control domain  $\mathcal{U} = \mathbb{Z}^3$  (and control period  $\tau = 1$  for continuous systems). The environment dynamics  $f$  are state-independent, i.e.,  $f(x,u) = u$ .
We first establish some intuition. The state variables  $x_{1}$  and  $x_{2}$  correspond to the counters  $c_{1}$  and  $c_{2}$ , and the state variable  $x_{pc}$  corresponds to the program counter. After each control cycle, these variables will have the corresponding values in the simulated 2CM. For the decision-tree policy, we construct a gadget for each instruction in the 2CM, and one more gadget to select the current instruction based on the value of  $x_{pc}$ .
We start with the latter gadget (instruction selection), which works precisely as most gadgets in the previous proof. It is a degenerated tree only expanding to the right with  $n - 1$  decision nodes. On the  $k$ -th level, the predicate is  $x_{pc} \leq k$ . The left leaf node at level  $k + 1$  is the sub-tree  $\mathcal{T}_k$  corresponding to one of the other gadgets, depending on the  $k$ -th instruction. The last layer's right leaf node is  $\mathcal{T}_n$ .
The other gadgets are trees substituted for  $\mathcal{T}_k$  depending on the  $k$ -th instruction. We only describe the instructions for the counter  $c_1$ ; for  $c_2$  the construction is completely analogous by swapping 1 and 2 in the indices. For instruction " $k: \mathsf{INC}(c_1)$ " we set  $\mathcal{T}_k$  to the leaf node with  $u_1 = 1, u_2 = 0, u_{pc} = 1$ . For instruction " $k: \mathsf{DEC}(c_1)$ " we set  $\mathcal{T}_k$  to the tree with decision node  $x_1 \leq 0$  and two leaves; the left leaf node is  $u_1 = 0, u_2 = 0, u_{pc} = 1$ , and the right leaf node is  $u_1 = -1, u_2 = 0, u_{pc} = 1$ . For instruction " $k: \mathsf{JZ}(c_1, \ell)$ " we set  $\mathcal{T}_k$  to the tree with decision node  $x_1 \leq 0$  and two leaves; the left leaf node is  $u_1 = 0, u_2 = 0, u_{pc} = \ell - k$ , and the right leaf node is  $u_1 = 0, u_2 = 0, u_{pc} = 1$ . For instruction " $k: \mathsf{STOP}$ " we set  $\mathcal{T}_k$  to the leaf node with  $u_1 = 0, u_2 = 0, u_{pc} = -k$ .
Finally we define the (singleton) set of initial states  $\mathcal{X}_0 = \{(0,0,1)\}$ , the goal region  $\mathcal{G} = \{(x,y,0) \mid x,y \in \mathbb{R}\}$ , and the set of error states  $\mathcal{E} = \emptyset$  (not used).
It is easy to see that each of the gadgets implements the 2CM instructions. Executing the action corresponding to the STOP instruction is the only way that  $x_{pc}$  can reach the value 0. Thus the above construction is a reduction.
the 2CM terminates
$\Longleftrightarrow$  the STOP instruction is reached from the configuration  $c_{1} = c_{2} = 0, pc = 1$
$\Longleftrightarrow$  the DTCS reaches a state with  $x_{pc} = 0$
$\Longleftrightarrow$  the DTCS satisfies the reach-avoid specification
D Modeling Decision-Tree Controlled Systems as Hybrid Automata
Here we expand on the discussion from Section 4.3 and explain the encoding.
Systems with mixed continuous and discrete dynamics can be modeled as a hybrid automaton [Henzinger, 1996]. As such, DTCS can also be encoded as a hybrid automaton. The reach-avoid problem has been studied extensively for various subclasses of hybrid automata [Doyen et al., 2018, Althoff et al., 2021]. One could thus argue that solutions to the reach-avoid problem for DTCS exist. However, when modeling a DTCS as a hybrid automaton, the problem structure gets lost, and thus tools analyzing this hybrid automaton will not scale. Hence we argue that our algorithm is the first feasible solution to the reach-avoid problem for DTCS.
Below we present one possible encoding of a DTCS as a hybrid automaton, for which we assume that the reader is familiar with the terminology of hybrid automata. We encoded the systems from our evaluation (Section 4) accordingly and applied the state-of-the-art reachability tool JuliaReach [Bogomolov et al., 2019] (which also implements an algorithm based on Taylor models [Benet et al., 2019]) to them. The result was that the tool got stuck as soon as it had to bisect the set of states for the first time, and we had to terminate the tool after several minutes of no progress.
We encode a DTCS  $(f, \mathcal{T}, \tau)$  as a hybrid automaton as follows. First we introduce a fresh variable  $t$  for time. We use one location  $\ell_{u}$  for each control action  $u \in \mathcal{U}$  that occurs in  $\mathcal{T}$ . The continuous dynamics in  $\ell_{u}$  are given by  $f(x, v) \wedge \dot{t} = 1$ , where we substitute the value  $u$  for the second argument  $v$ . Each location  $\ell_{u}$  has an invariant  $t \leq \tau$ , restricting time to one control period. For each location there is a transition to every location  $\ell_{u'}$  with a guard condition consisting of  $t = \tau$  and a big disjunction of conjunctions of the predicates along the paths in  $\mathcal{T}$  to leaves with action  $u'$ . For instance, for  $\mathcal{T}$  given in Figure 7(a), the guards on the transitions leading to  $\ell_{-1}$  are  $t = \tau \wedge (\theta \leq -0.014 \vee (\theta > -0.014 \wedge \omega \leq -0.201))$ . Finally, each transition has a reset  $t := 0$ .