OoD-Control: Out-of-Distribution Generalization for Adaptive UAV Flight Control
Authors: Yuxiao Duan
Jundong Zhou
Zhaoyu Zeng
Haoqi Zeng
Nanyang Ye
Venue: Submitted to ICLR 2023 (2023)

Abstract: Data-driven control methods have demonstrated precise and agile control of Unmanned Aerial Vehicles (UAVs) over turbulence environments. However, they are relatively weak at taming the out-of-distribution (OoD) data, i.e., encountering the generalization problem when faced with unknown environments with different data distributions from the training set. Many studies have designed algorithms to reduce the impact of the OoD problem, a common but tricky problem in machine learning. To tackle the OoD generalization problem in control, we propose a theoretically guaranteed approach: OoD-Control. We provide proof that for any perturbation within some range on the states, the control error can be upper bounded by a constant. In this paper, we present our OoD-Control generalization algorithm for online adaptive flight control and execute it on two instances. Experiments show that systems trained by the proposed OoD-Control algorithm perform better in quite different environments from training. And the control method is extensible and pervasively applicable and can be applied to different dynamical models. OoD-Control is validated on UAV dynamic models, and we find it performs state-of-the-art in positioning stability and trajectory tracking problems.

Under review as a conference paper at ICLR 2023
OOD-CONTROL: OUT-OF-DISTRIBUTION GENERAL-
IZATION FOR ADAPTIVE UAV FLIGHT CONTROL
Anonymous authors Paper under double-blind review
ABSTRACT
Data-driven control methods have demonstrated precise and agile control of Un- manned Aerial Vehicles (UAVs) over turbulence environments. However, they are relatively weak taming the out-of-distribution (OoD) data, i.e., encountering the generalization problem when faced with unknown environments with differ- ent data distributions from the training Many studies have designed algorithms to reduce the impact of the OoD problem, a common but tricky problem in ma- chine learning. To tackle the OoD generalization problem in control, we propose a theoretically guaranteed approach: DoD-Contron We provide proof that for any perturbation within some range on the states, the control eπror can be upper bounded by a constant. In this paper, we present our DoD-Control generalization algorithm for online adaptive flight control and execute in two instances. Experi- ments show that systems trained by the proposed OoD-Control algorithm perform better in quite different environments from training. And the control method is extensible and pervasively applicable and can be applied to different dynamical models. OoD-Control is validated on UAV dynamic models, and we find it per- forms state-of-the -heetf---t--ttt- in positioning stability and trajectory tracking problems.
INTRODUCTION
UAV's have gained considerable attention and are widely used for various purposes because of their high manoeuvrability and flexibility. For example, quadrotors are widely deployed for inspection, reconnaissance, and rescue. As control strategies evolve, novel scenarios for UAVs, such as aerial grasping. transporting, and bridge inspection (Ruggiero al. 2018). require more precise trajectory tracking. Especially in the outdoor environment, unpredictable and changing wind field conditions pose substantial challenges to the stability of UAVs. Rotor blades are affected by induced airflow caused by the wind, which creates complex and non-stationary aerodynamic interactions (see Ap- pendix B.6.3). From security and policy perspectives, demonstrating that UAVs can operate safely and reliably in unpredictable environments with various distributions is essential requirement. It also the premise for future medical robots, autonomous cars, and manned aerial vehicles to be widely accepted.
Many areas have benefited from data-driven approaches. However, they are susceptible to perfor- mance degradation after generalization. And the majority of deep learning algorithms heavily rely on the I.I.D assumption for data, which is generally violated in practice due to domain generalization (hhu al. 2022). Nevertheless, neural networks may lose their robustness when confronted with OoD data. Many cases of failure in DNN originate from shortcut learning in the learning process Geirhos al. 2020). The damage to the UAV undoubtedly considerable if the UAV cannot adjust to the changing environment, i.e., it is unstable or even crashes in an OoD situation. One significant objective of this paper to propose control algorithm to enable UAVs to maintain accurate control even in the case of environment domain shifts.
Our Contributions. UAVs interact with the changing environment, resulting in complex environment dependeen uncertain aerodynamics, called unknown dynamics, that are tricky to model and significantly impact precise control. Previous data-driven controllers attempt to solve the prob- lem by estimating the unknown dynamics, while the estimation accuracy and the performance of the controllers are limited by the environment domain shifts in tests. This paper presents methodology

---

Under review as a conference paper at ICLR 2023
for adaptive flight control problems, focusing enabling UAVs to fly under unknown environ- ments. Compared with previous works, our proposed OoD-Control algorithm can provide perfor- mance guarantees under domain shifts of the environment distribution. Compared with previous state-of-the-art work (Shi et al. 2021), the proposed OoD-Control method does not require strong assumptions, for example, e- ISS stability and fully actuated system. Additionally, our algorithm has greater capacity for generalization. For different distributions of the environment, we show theoretically that the bound on the prediction error of the unknown dynamies remains constant over a certain range of perturbations. Besides, simulated results under challenging aerodynamic condi- tions indicate that the OoD-Control algorithm achieves better control performance than the SOTA deep learning algorithms.
2
RELATED WORK
2.1 FLIGHT CONTROL ALGORIHTMS
UAVs have found broad applicability in variety of fields and have attracted the attention of several researchers. Many published studies describe the significance and efficiency of flight control algo- rithms, including PID Control (Szafranski & Czyba 2011), LQR Control (Priyambodo al., 2020),
Sliding Mode Control (Chen et al.] 2016). Backstepping Control (Labbadi & Cherkaoui 2019),
Robust Control (Hasseni & Abdou 2021). etc. However, most of the previously mentioned con-
trol methods suffer from limitations. Imprecise system modelling and non-modelled environmental
disturbances may result in unacceptable performance or instability.
Today, artificial intelligence triggered new wave of research in many fields (Jumper et al. 202 Sil-
ver al.] 2017). The data-driven control methods can directly learn the corresponding control strat- egy from the interaction process of the controlled system so that it can adapt to new environments. [Bansal et al. (2016) validates their proposed deep learning algorithm on a quadrotor testhed. On the other hand, reinforcement learning is model-free algorithm widely used for control problems. Koch et al. (2019) present an intelligent nigh-precision flight control system using the reinforcement learning algorithm for UAVs. Moreover, the performance and accuracy of the internal control loop for quadrotor attitude control are analyzed and compared. Results indicate that the neural network has good generalization abilities and can learn the quadrotor dynamics accurately and apply them to the control system. Underwooo & Husain (2010) propose an online parameter estimation, and the experimental results validate the effectiveness of the adaptive control method. O'Connell et al. (2022) have combined online adaptive learning with representation learning and adapted DNN to learn a nonlinear representation. However, the environment's 's diversity is not considered this work. Adapting to an environment completely different from the training set is challenging. Inspired models with learnable dynamics and DNNs are constructed in this study for their interpretability and stability. We further investigate in this paper whether the robustness of the algorithm can be improved with OoD generalization methods.
by (Shi et al. (202T), mechanical-base
2.2 OUT-OF-DISTRIBUTION GENERALIZATION
Out-of-Distribuut (OoD) generalization, which involves generalizing under data distribution do- main shifts, is an active research area in the community. Generalizing a prediction model under distribution shifts is the process of generalizing its performance. Many algorithms have been pro- posed to achieve the OoD generalization, including meta-learning (Li et al., 2019 Zhang et al.)
2020), prototypical learning (Dubey et al. 2021), gradient alignment (Rame et al.) 2022), domain adversarial learning (Akuzawa et al. 2019 Xu et al. 2020) and kernel methods (Li et al. 2018
Ghifary et al. 2016) etc.
Literature has extensively discussed how to deal with domain shift, and the OoD generalization problem is extensively studied in computer vision (Hsu et al. 2020), natural language processing (Hendrycks al. 2020). speech recognition (Shankar et al. 2018), and other fields, but seldom in the context of online control. SníS et al. (2021). a multi-task learning method for nonlinear systems was presented that can withstand disturbances and unknown environments.
Previous studies have suffered from shortcomings in lacking a discussion about the misspecifica- tion of dynamics systems and neglecting the gap between the simulation experiments and reality.
2

---

Under review as a conference paper at ICLR 2023
Additionally, the generalization of flight control is plagued by system measurement enrors and un- known environmental parameters like wind modes and air density and resistance. this work, we demonstrate the average control error can be upper-bounded by constant when the environmental disturbances are within some range. Moreover, experimental results indicate that the Ool-CCotrool robust shifts in the environmental domain.
3 PROBLEM FORMULATION
Notations We use subscripts (e.g.. in =z,²) to denote the time index and superscripts (e.g-, (z) in 2(()) to denote the dynamics state under the environmental perturbation To. where To the environmental distribution, || denotes the 2-norm of vector.
Note: Superscript (z) in =x()) is used a symbol represent that the state is perturbed by the environmental distribution environment.
and zi + where It E R" is the state in the windless
In this paper, we consider a discrete nonlinear control system whose dynamics are described by the following formula:
((((()),) Lit = fo(x + B(x f(x( .c) + wes 15tST,
(8)
where (((e) R. the state variable. E R. changes with the environmental distribution domain shifts. B(xx²)) R" R= is the state-dependent actuation matrix; E R is the control input on the dynamic system; fo(2;²) R" + R is the known nominal dynamic term that can be modelled with well-defined differential equations; f(xx(=),) c) IR x IR - IR" are the unknown cnviromment-dependeen dynamics that are hard to be modelled and e the unknown environmental parameter, we also use fz((") for short in the following paragraphs; E R is the random noise vector.
We hypothesize that the environmental disturbance and the control u4 are bounded. Due to the structural restrictions of the actuators, there are certain limits the output. For example, the control output of the UAV is constrained by the maximum rotor blades' revolutions per minute (RPM). Here we give the boundedness Assumption.
Assumption (Bounded controls and disturbances) Assume that the controller's output has upper bound: Vt, |[ee| S U. Moreover, the environmental noise vectors are also bounded with zero expectations: Vt, wolll < W, E(w:) O.
Definition (Average Control Error under disturbances) The control error of the system under
disturbance distribution To is calculated as [| (x(²)) xell The average control error
ACE of time steps is defined as the performance metric:
ACE
E B (x)
x
where xtt denotes the desired states at t. Remark In Definition 1, we focus on fixed-point hovering and trajectory tracking. sequence of perturbations matching
is obtained by sampling for N times: (2), 22, ........ =NN And the (22). ,x (Ex)). || (z. )|
perturbed states sequence is also derived at time t: (a) (21) It
can be approximated with Monte Carlo method: Σ/ 22 (21) /N, where the subscripts represent the index of the 4th sample. Compared with the average control error definition in Shi et al. (2021): Aström AC'E, Fr, represents the expectation of the difference between the actual states and the desired states of the dynamical system under environmental perturbations.
& Murray (2008), AC
Interaction protocol. We set the study of the OoD adaptive flight control problem under the fol- lowing interaction protocol:
Stochastically selects an environment for the controller to encounter every time step. which depends on the unobserved variable (e.g., wind condition and air density).
3

---

Under review as a conference paper at ICLR 2023
2. The controller interacts with the environment and observes the state (2) take action ue
3. Optionally changes after short time and repeats from Step 1.
4 METHODOLOGY
We expect the UAV to learn the shared representation the unknown dynamics between different environments so that it can generalize well in unseen areas with few adaptations. This section will introduce the methodology that provides the guaranteed upper bound for the prediction errors of the unknown dynamics.
Notations and settings. For modelling unknown euvironment-dependemt dynamics, we use f(x) 2) F( 2) inspired by (Shi et al. 2021). That means we consider the unknown cnvironment-depenee deeendddnn dynamics typically consist of two coupling parts: the irregular higher-order aerodynamics of UAVs that caused by the complex streamlined design, and the environment- dependent variables which encode wind field information. (xx({); () represents
former, which is deep neural network (DNN) with I layers parameterized by O è is the latter, which is par- ticular for a certain environment. This model enables us to consider the joint higher-order effeets beyond nominal dynamics and enable agile control of UAVs. We use f(x()) paragraphs.
(*) for brevity in following
Assume environmental distribution No. the perturbed unknown dynamics and the predictable value denoted as the expectation of the states under environmental disturbances: fra (x(()) (2) _)]. In OoD-control, the objective to minimize the unknown dynamics prediction loss for controller inputs: E, -
[f (2) c)] and (x) = Ez~ra [F(((=) θ), é)] E Es~ao [f()
[f
Il (x, f ) (a]( The loss is mapped to [o. 1] by h(lls=, (xt ) fro (()
and we also use T(x))) for short the following paragraphs. k(()) satisfies: 1) 1)()) E [0: 2): h(.)) is monotonically
decreasing and the inverse (-) exists. In the next section, we will propose framework that
provides guaranteed upper bound for the ACE under OoD-Control.
4.1 THE PROOF OF ACE'S UPPER BOUND
Next, we will introduce methodology to tackle the problem of generalization and give proof of ACE's upper bound. We want verify that for any perturbation in B {8 R" |0ā|2 r) with radius r, the lower bound of the prediction error maintains constant under unpredictable disturbance, still holds hnd (a/²) +8) p. This conclusion is significant for the calculation of the upper bound for ACE. For a perturbation radius r, the expectation of the prediction of the unknown dynamics remains the same.
i.., v|| 3p 0, An (2. (x(-))
Assume H. is function class, which includes h (-) and satisfies H. = {h- h(z) E [0, 12, Yz ]" Performing the following optimization will result in guaranteed lower bound. If H. includes only hag (-), the bound is exact:
min Ti (x.()) + 5) min min { hna (zr + 5) s.t. Ti (2x Pexoo (o(=))) BEB
(3)
bEH 8EB
Theorem (Lagrangian) (HL, B) is denoted as the lower bound in equation Lagrangian methods can be adapted to solve inequality:
Lo (H, min mii max L(h, 6, X) A min min max (i (2) + Al (c(()) Tnoo (1( hEH REB XER
hEW BEB XER
(4)
Exchanging the min and max yields the following dual form:
Cap (H, 2 mar min min L(h, 8, A) = max X20
Auooo (x( max De( (mas) (5) BEB
X20 hEH BEB
where T represents the distribution of 2 + when (h(a) [+([{())]] ['(AT((2) [z(2))
wo and (=((rool
mar (AE [+(] NEN
4

---

Under review as a conference paper at ICLR 2023
Corollary (Gaussian noise) With Gaussian noise No == {6: ||6|| S r). the lower bound in equation 5 satisfies:
-N(0,021) and bounded disturbance
Cro (H, B) max Ahaa (z( max Di (Ara (T)) 2 2 - BEB
(6)
X20
where (.) represents the Gaussian Cumulative Density Function (CDF). For the case p=0.5, i.e., T (s) (3( >>>>>>>>>>)) > 0.5, the radius satisfies r o a (x, ()()))) As a side note, the Monte Carlo method
for perturbation radius calculation is also given by Algorithm 2 in Appendix B.3
4.2 AVERAGE CONTROL ERROR BOUND
The selection of Given a sequence of state variables at under perturbation & ~ no = (0, o²): (21)
X (I)(a)) It
(32) (2N)). Moreover, the predicted and unknown dynamics sequences
de-
fined as F = (f(z) (11)) fare )..... (a/(=))) and F. = (())) (, [(r (x)(())) )...)))...,,)))))) (1,
),
Let
Dp be the discrepancy sequence between F2 and Fu. D, - (1) (1) f(x] Ib (z, (
f(x f(x(+)))| )l.... (z](w( ((](=))]]]) Denote by ý the successful rate for the prediction error under given threshold Et. By simulating with a large sample size N, á calculated as:
p A ((1((()) f(x( <<)== = R4a N
where na the number of elements in D, less than Eg. Recalling the h function's requirements, we can instantiate as follows:
p ky 1p11- (0(1-)
(
n
where k = -1-(0--111-) 9) is the 1- quantile of standard normal distribution. Moreover, this equa- tion represents the lower confidence bound estimation of the error under given confidence level a. As noise increases, the predicted value will deviate from the actual value by a more significant amount. Therefore, the will decrease monotonically. Besides, the range of the lower confidence bound lies in [o, 1], which satisfies both requirements for h as discussed in the previous section.
Let & denote the lower confidence bound in equation In section 4.1 proof is given that h (x(²)) and hng (zr(²) + 5) has the same lower bound under the disturbances ||ö|| < r. The radius ensuring an equal lower bound under the perturbation in this paper is:
p(1 p) n
r = oΦ (b)) = σ (p - k
Controlling, The control term u consists of three parts: feedback, feedforward, and residual, where the feedback part gets information from sensors and minimizes the gap between (x)) and xY, the feedforward part offsets the nominal term fo(a(()), and the residual part counterweights unknown enviromment-depeedeent term f f(x(")). Bt (x( is the pseudo-inverse B(x(()).
The controller of model-based control is 24 = = (-fo)) + +fxx()))
(10)
Thus, the equation I
becomes:
3++1 =+11= f(x()) -f(((() f(x())) ) + we.
(11)
Lemma (ACE bound in ideal case) For any perturbation in B {ō: :|01112 S r)) the theoretical average control error is bounded as:
1 T
AC
Σ
| [f (e) - Ez (x(((())) < h hp)
(12)
T
1=1
5

---

Under review as a conference paper at ICLR 2023
Average control error is related the prediction enror of unknown dynamics and envi- ronmental perturbations in the ideal case. Compared with the upper bound calculated in Shi et al. (202T), the derived bound is more general. Our calculation method does not require the system to be fully actuated and does not require the nominal dynamics to be exponentially input-to-state stable (e-ISS). The assumption of e-ISS is too strong. And many dynamic systems are under-actuated in the real world, such as quadrotors.
Remark 2
Corollary (ACE bound under control actuation misspecification) AB(x(") is the parametric misspecification in the actuation matrix, and the ACE,, ng satisfies:
T
ACE = T
1
Σ
- -h-
1 T
R RILT
AB
(2) B
(3) (3) (1 (x)
1=1
where e ey(x! (o((
Lemma (Trajectory tracking ACE of quadrotor) In this puper, for environmental disturbances in B = (d) |1|12 S r], the quadrotors' trajectory tracking error formula is given as:
T
ACFF.
T Σ
BZaNs (x) (x(=))
x²91
1 T
Σ | (Cier + +Cze -
-
(p)/K,ll
(13)
where C1 = C2
eriat) and = 13) (())),)), which the prediction ervor of the unknown dynamics. f() (c) f(x)
(Kuz ))
Remark Note that Lemma 2 gives the trajectory tracking error that can be calculated. Based on the previous description, it was demonstrated that the errors for unknown dynamics under perturba- tion have the same bound. The detailed proof can be found in Appendix A.5 A.S
4.3 OoD GENERALIZATION ALGORITHM
Based on the theoretical analysis above, we propose an srrrriimm--ot-o---iisiriitttiiii generaliza- tion for adaptive flight control named OwD-Control. We focus on minimizing the prediction loss and learning @ during the simulation. We intend to design an OoD-controller with lower ACEs that converges the estimated unknown dynamics (xx(+)) faster to the true dynamics f(x(()) under environment distribution domain shifts.
The proposed OoD-Control algorithm is shown in Algorithm (see Appendix 1b. Given a set of distribution functions X, E X are picked for each iteration. The wind velocity is series of random variables sampled from X. (Specifically, we use X for the training distribution set with each member denoted X. For the testing set, we use Ω and instead.) Each time-series simulation begins with random noise 6j being introduced to the structural parameters of the system. At each iteration, the predicted loss, which measures the error between the unknown dynamics and its predic- tion is minimized. After the unknown dynamic predictor is trained, can be used for model-basee control discussed in Section 4.2.
EXPERIMENTS AND RESULTS
In this section, numerical experiments the inverted pendulum and quadrotors will be conducted to demonstrate the effectiveness of the proposed OoD-control algorithmic framework. To better understand the proposed OoD-Control algorithm and environment setting, choose uncoupled dynamics model, the inverted pendulum, as the introductory example before the quadrotor instance.
5.1 DYNAMICS MODELING
Inverted Pendulum.
Consider an inverted pendulum, the dynamic model of the pendulum is:
ml2 mlgsint ((,,0,c)
(14)

---

Under review as a conference paper at ICLR 2023
where 9 represents the angle away from the center, is the length of the arm, g is gravitational acceleration, is the length of the pendulum's 's arm, and m is the mass. The state variable consists of f and 0, which could be measured by position and inertia sensors. f(-) represents the unknown dynamic term, including air resistance, wind force and modelling misspecification, subject to 0, and environment parameter c. u is the controlling term. Our goal is to keep the pendulum closer to the center, i.c., minimize average control error.
The quadrotor is a plane model where the four rotors are always on the same plane. So the quadrotor adjusts its attitude by setting different rotation speeds for the four rotors. We define the dynamic model of quadrotor as:
Quadrotor:
md mg + R(0)fr +
(15)
Jó J0 +
(16)
where 0 represents the attitude angel of the quadrotor; R((0) E R3x3 is the attitude rotation matrix subject 0: J is the inertia matrix of the quadrotor, fr is the force imposed on the system; the total torque; m is the mass of quadrotor and is the gravitational acceleration; fr and r are subject to the speeds of rotors n, E R1x4, In the experiments, the goal is maintain the quadrotor's position states or follow a given trajectory under turbulent environments.
5.2 COMPARISON METHOD
In the experiments, the proposed adaptive UAV flight control algorithm OoD-Control are compared with OMAC (Shi et al., 202I) and no-adapt (PID) method. OMAC (online meta-learning adaptive control) is the state-of-ther thee-ar data-driven UAV flight control method. In the OMAC paper, three versions OMAC are provided with different model specifications: convex, bi-convex, and deep learning. We illustrate the results of deep learning because it is the best-performing version of OMAC.
Mcanwhile, the no-adapt method and the omniscient method are compared in this paper. No-adapt indicates the controller cannot perceive the environmental domain shifts with f(x{²) = 0 which is just the conventional PID controller. ommiscient is the controller which has access the unknown dynamics perfectly, i.e., f(x((²) fx)(*)) Among all the controllers, no-adapt and omniscient are the two extremes, with no-adapt being unable to predict while omniscient can do so with zero error. We run each simulation ten times with different random seeds to obtain the mean and standard deviation of ACE under perturbation for rigorousness.
5.3
WIND FIELD CONSTRUCTION AND FLIGHT TRAJECTORY DESIGN
Wind fields can be derived according to the Navier-Stokes (N-S) equations and the continumm equation. However, in practice, the N-S equations are generally hard to be solved due to their high computational cost. For turbulent wind field simulations, the Dryden model (Specification, 1980) is widely used. We refer to the Dryden model to simulate turbulent wind fields on quadrotors by generating Gaussian wind disturbances Beall 1993).
To construct realistic situations in the inverted pendulum and quadrotor experiment, we simulated two types of winds: turbulent wind and gust. For turbulent winds, the speed and direction change at any time. In the case of gusts, the wind speed remains constant over a period of time. For further study, we divided the two wind fields turbulent wind and gust into three categories respectively ac- cording to their strength: breeze, strong breeze and gale. The direction and strength of the turbulent winds change continuously and the wind forces are applied to the object. This requires higher ma- noeuvrability to maintain stability. The wind environment setting in the experiment can be found in Appendix B
Quadrotors must also be capable of flying along the desired trajectory and hovering at a fixed point for various applications, such as inspection, patrol, and delivery. In order to meet the requirements of different application scenarios, we design variety of trajectories to test the performance of the proposed OoD-Control under different situations. The designed trajectory can correspond to spe- cific application scenario, hovering for fixed-point photography, figure-8 trajectories for scenarios
7

---

As shown in row 3 of Table T our OoD-Control algorithm performs significantly better than the OMAC in the gale dataset. That means the former generalizes better than the latter when meeting large env invironmental distribution shift. When changing to the less difficult dataset such as the strong breeze, can that the gap between the two algorithms decreases, but OoD-Control still achieves nearly half the score the OMAC. (The breeze dataset is not complicated enough to distinguish the mentioned methods. Under review as a conference paper at ICLR 2023
requiring high manoeuvrabillty, the spiral trajectory for power lines detection, and sin-forward for transporting items in the forests or area scanning. The mathematical forms of the trajectories are shown in Appendix [B6.1
5.4 RESULTS
Table [ (see Appendix (B.2] and Figure F Dillustrate the average and standard deviation of the control errors in different testing environments. We mainly compare the OoD-Contue Control algo- rithm with the OMAC. And for the completeness of the experiments, we also set two control groups: the no-adapt and the omniscient.
Pendulum.
We also show the results of when è is unchanged. In this setting, both OMAC and OoD-Control perform terribly because the variable used fit the ground truth of e frozen.
No-adapt
OMAC
Ool>-Control
ACE E: 2.904
ACE: 0.837
"AC 0.254
position (rad)
5
-5
torque (N m)
20
-20
10
20
30
40
10 20
30
40
0
10
20 30
40
Figure 1: Result of inverted pendulum experiment where the testing environment is Gale. The black dashed line represents the desired states for the inverted pendulum. The objective of this task is to maintain the angle 9 and angular velocity 0 of the inverted pendulum to zero. f the ground truth of the torque, while f the predicted torque. * is given for the best performance. shown in the amplified areas (the black rounded rectangles), our algorithm predicts much better than OMAC.
(see Quadrotor: We show the result of the quadrotor task in Figure and Table (see Appendix B.6.1 6.1 As testing environments differ from training ones, the OoD-Control method maintains good stability and tracking accuracy. In some cases, the performance of OoD- Con-rolt goes close to the omniscient case where the rotor provided with precise wind conditions, which shows that our algorithm is able to predict the wind with acceptable error. Our algorithm achieves lower ACE than baseline methods (60% than OMAC and over 70% than PID) in most difficult cases. Besides, we shorten the training time to test its sample efficiency, and it turns out that our algorithm performs well in few-shot learning. By adding noise during training DNN and fixing a learning rate meta-learning of é, our algorithm gains robustness and adapts quicker in different environment.
We tested our method under several different trajectories, and OoD-Control outperforms the baseline and conventional no-adapt methods when the distribution domain shifts during the testing process. Meanwhile, CoD-control can learn more from changes in the environment and apply it as prior knowledge, thereby improving its adaptability.
8

---

Under review as a conference paper at ICLR 2023
Eri: Breeze
Erw: Gale
No-adapt
OMAC
OoD-Cartrd
No-adapt
OMAC
CoD-Cortrd
0.5
ACE:0.279
ACE-0,302
*ACE:00B
ACE-0,466
ACE:0.297
*AC:0003
Hlver
-0.
ACE:0.33
ACE.0.260
ACF00.500
ACF:0.403
"A
Ficuree B
yim
ACB-0.205
ACE-0.214
*ACE:0001
ACE.0,490
ACE.0.309
"ACE-0120
Spiral-up v(20
ACE-0.319
ACE-0.277
*ACE:0006
A.8.0.500
ACE-0.579
*ACE:0113
Sin- Sin-forwee forward
x(m]
x[m] 3(m]
x(m]
x[m]
2D view of trajectories in different wind conditions and performance comparison of OMAC and OoDD-Control algorithms for trajectory tracking. The goal of the controller is to get closer to the desired trajectory (black line). Different colors demonstrate the distance from the ac- tual location to the desired location, corresponding to the color bar at the bottom. The shade of color indicates the magnitude of the deviation in position. As compared to OMAC and no-adapt, the proposed OoD-Control method provides more accurate results across wide range of wind environ- ments and trajectories. The average ACE of ten independent experiments is marked in the subplot and given for the best performance under the same environment.
Figure 2:
The OMAC and OoD-Control algorithm were tested under hovering, figure-8, spiral upward and sin-for forffowrrd trajectory scenarios. Figure 2 and Table / (see Appendix B.6. show the trajectory tracking experimental results. DoD-Control provides more accurate results across trajectories under wide range of wind environments and achieves state-of-the-arr performance in all these situations compared with the baseline.
Based experiments, it has been demonstrated that systems trained by the proposed OolD-Contool algorithm perform state-of-the-arr. In addition, the control method can be applied to different dy- namical models and is extensible and universally applicable.
6 CONCLUSION
In this paper, we theoretically demonstrate that the average control error is upper-bounded by con- stant when the perturbation on the state variables within certain radius for UAV flight control. Besides, we propose an algorithmic vironmental conditions. Based on the results of our experiments, we can conclude that our algorithm is scalable and pervasively applicable that can be applied to a variety of dynamic models. For future work, we will explore extending our algorithmic framework to more UAV types, such as unmanned helicopters, tilt-rotors, and unmanned fixed-wing aireraft. As far as first papers that theoretically discusses of di-iiiibibtttion problems in the context of online adaptive UAV flight control.
OoD--Contoo that is evaluated under turbulent en-
aware, this is one of the

---

Under review as a conference paper at ICLR 2023
REFERENCES
Kei Akuzawa, Yusuke Iwasawa, and Yutaka Matsuo. Adversarial invariant feature learning with ac- curacy constraint for domain generalization. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pp. 315-331. Springer, 2019.
Karl Johan Aström and Richard M Murray. Feedback systems. Princeton Univer, 2008.
Somil Bansal, Anayo K Akametalu, Frank J Jiang, Forrest Laine, and Claire J Tomlin. Learning quadrotor dynamics using neural network for flight control. In 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 4653-4660. IEEE, 2016.
TR Beal. Digital simulation of atmospheric turbulence for dryden and von karman models. Journal of Guidance, Control, and Dynamics, 16(1):132-138, 1993.
Fuyang Chen, Rongqiang Jiang, Kangkang Zhang. Bin Jiang. and Gang Tao. Robust backstepping sliding-mode control and observer-based fault estimation for a quadrotor uav. IEFE Transactions on Industrial Electronics, 63(8):5044-5056, 2016.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
AT Conlisk. Modern helicopter rotor aerodynamics. Progress in aerospace sciences, 3765)4419-476, 2001.
Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland, and Dhruv Mahajan. Adaptive methods for real-world domain generalization. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 14340-14349, 2021.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665-673, 2020.
Muhammad Ghifary, David Balduzzi, W Bastiaan Kleijn, and Mengjie Zhang. Scatter component analysis: A unified framework for domain adaptation and domain generalization. JEEE transac- tions on pattern analysis and machine intelligence.
2016.
Seif-El-Islam Hasseni and Latifa Abdou. Adaptive nonlinear robust control of an underactuated micro uav. International Journal of Dynamics and Control, 9(3): 1400:1144-16,, 2021.
Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. Pretrained transformers improve out-of-distribution robustness. arXiv preprint arXiv:2004.06100, 2020.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out- of-distribution image without learning from diitt--diitiitttion data. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10951-10960, 2020.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger. Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7773))5883-588)) 2021.
William Koch, Renato Mancuso, Richard West, and Azer Bestavros. Reinforcement learning for uav attitude control. ACM Transactions on Syber-Physite Systems, 3(2):1-21, 2019.
Moussa Labbadi and Mohamed Cherkaoui. Robust adaptive backstepping fast terminal sliding mode controller for uncertain quadrotor uav. Aerospace Science and Technology, 93:1053066 2019.
Ya Li, Mingming Gong, Xinmei Tian, Tongliang Liu, and Dacheng Tao. Domain generalization via conditional invariant representations. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Yiying Li, Yongxin Yang. Wei Zhou, and Timothy Hospedales. Feature-critie networks for heteroge- neous domain generalization, In International Conference on Machine Learning, pp. 3915-3924. PMLR, 2019.
10

---

Under review as a conference paper at ICLR 2023
Michael O'Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, and Soon-Jo Chung. Neural-fly enables rapid learning for agile flight in strong winds. Science Robotics, 7(66):eabm6597, 2022.
Tri Kuntoro Priyambodo, Oktaf Agni Dhewa, and Try Susanto. Model of linear quadratic regulator (lqr) control system in waypoint flight mission of flying wing uav. Journal of Telecoommnicctionn Electronic and Computer Engineering (JTEC), 12(4):43 49, 2020.
Alexandre Rame, Corentin Dancette, and Matthieu Cord. Fishr: Invariant gradient variances for out-of-distribution generalization. In International Conference Machine Learning, pp. 18347- 18377. PMLR, 2022.
Fabio Ruggiero, Vincenzo Lippiello, and Anibal Ollero. Aerial manipulation: literature review.
IEEE Robotics and Automation Letters, 3(3))1s,3 3(3):1957-1964, 2018.
Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, and Sunita Sarawagi. Generalizing across domains via cross-gradient training. arXiv preprint a80iv: 10745, 2018.
Guanya Shi, Kamyar Arizzadenesheli, Michael O'Connell, Soon-Jo Chung, and Yisong Yue. Meta- adaptive nonlinear control: Theory and algorithms, 2021. URL httpss//aaxiv.orr///aa
2106.
David Silver, Julian Schrittwieser, Karen Simonyan, loannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354-399 2017.
Military Specification. Flying qualities of piloted airplanes. Technical report, MIL-F-8785C, 1980.
G Szafranski and R Czyba. Different approaches of pid control uav type quadrotor. In International Micro Air Vehicle conference and competitions 2011 (IMAV 2011),'t 't Harde, The Netherlands, September 12-15, 2011. Delft University of Technology and Thales, 2011.
Samuel J. Underwood and Iqbal Husain. Online parameter estimation and adaptive control of permanent-magnet synchronous machines. IEEE Transactions on Industrial Electronics, 57(7): 2435-2443, 2010. doi: 200.9//...000..003232
Minghao Xu, Jian Zhang, Binghing Ni, Teng Li, Chengjie Wang. Tian, and Wenjun Zhang- Adversarial domain adaptation with domain mixup. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 6502-6509, 2020.
Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Adaptive risk minimization: meta-learning approach for tackling group distribution shift. arXiv preprint 2rXiv:2007.0293/ 1(3), 2020.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.

---

Under review as a conference paper at ICLR 2023
A PROOF OF LEMMA AND COROLLARY
A.1 PROOF FOR THEOREM 1
Theorem (H, B) is denoted the lower bound in equation Lagrangian methods can be
adapted to solve inequality.
Luo (H, B) -min min maz L(h, 6, A) A min min mar (NTTo (az + 6) (x) - hnoo (2t )1 NEW BEB AER
hER BE XER
(17)
Exchanging the min and max yields the following dual form:
Ah (2l ( max Dx (A [1TA)) } AEB
(H, 2 maz min min L(h, 6, X) = maa A20
(18)
X20 hEW BEB
where n6 represents the distribution of 2 + when 2 ~ to and (ie(Amol(Ts) = mar mar(AEz=z (M Ezara [m(a))))] f[ATT((= +z) NEW
Proof.
(i)
CO (H. B)
min min mar hEH BEB
h (z( 8) Alh (z( (1hTo(z
hro (ar
XER
{ NTT (x( + 66 - Altiro (zt ) To (are (2))
(2)
max min min hEH BEB
max( (aboa (2z( - Nomo (co hEH
max
-
X20
(x (2))
max D'de (AaulTll) } GEB
mar X
(ii) We denote the sign function sgm(2) as:
if ATTO()) TT(2)] 2 0
sgn(o) 0, if (ATo(2) TT(2)] <
(19)
Thus we can calculate DD (Azo (ATulI |1T8)
directly as:
Di (Dto (T8))
(x) + +6) max (Ahmo (z( (x)
hEH
{ (XESNTO [1(2)/+)]] EzNz. (((x())]
mar hEH
[XTTO(2) / (XT(() (=))
/ [AmT(()) (z))l+ddz
A.2 PROOF FOR COROLLARY
Corollary 1 (Gaussian noise) With Gaussian noise To = NO0, 021) and bounded disturbance (8 :]]18||2 S r). the lower bound in equation 5 satisfies:
LTo (H, B) max "hhna (x) X20
mar D' (Aroll bEB
2 2 (hooo (2 (z(
(20)
where (0.) represents the Gaussian Cumulative Density Function (CDF). For the case p=0.5, iee.
Tozo (x) (2) > 0.5, the radius satisfies S oD -1 (I)
(2)
Proof.
12

---

Under review as a conference paper at ICLR 2023
L 2 D(Φ (0000 - 1
(21)
2
c min maz Só X20
Au (1 (1))))
[ATO(E) - =6(2]]+dz
(22)
We denote CΛ = fz: (mu(2) a) 2 To (=))=={(2 oT== 2 11812
2 1n2 and F( x) = (o, (+))
[[Amo(z) - Toa +z). Then we will get:
F(6. A) = Anro (a0l)) / [ATO(2)) =o(2]<+d2
Aito (ar( - -/o, [ATT(2)) [T(=)]lzz
Xnzo (()) A
||012
o In
-=110ll2
In
20
||8112
2x
|
It is notable that F(6, A) is concave function w.r. X, thus the maximum value occurs when (F(5,))
1A=As 0. A direct calculation gives Às exp 201l511200 (hea (x(*))) 1-1861
There is
2a
C
min mar Fd A)
Sr A20
-18112
In As
min + S0
2a
11ō112
||ō112
Φ 1 (Nioo (ar (1)
min +b |
σ
(hTo (2, (1((
r
In case p=0.5, the perturbation radius r is calculated as
mén max F66, A)
1
+ ④
(h (x, (3)
- er
1)
1 o ee<co
(1ho0( (23)
[6 X20
2
A.3 PROOF FOR LEMMA
Lemma (ACE bound in ideal case) For any perturbation in B {8: ||ō1|2 S r), the theoretical average control error is bounded as:
T Σ
ACE
| [f(2(*)]] Ex (f( h- =
(24)
T
1=1
Proof.
From equation]
I
consider discrete nonlinear control-affine system:
Ft+1 fo(xx) + B(xx)u. f(x,, c) + wx. 1 151ST,
(25)
The controller of model-based control with the ideal model is
"1 = Bt (=(()) ((a(=)))
(26)
thus, the equation becomes:
- -
f(z (*) cc +
(27)
'The calculation method of the radius is given in Cohen al. (011 for the p=0.5.
13

---

Under review as a conference paper at ICLR 2023
Following straightforward calculation, will obtain the upper ACE bound under disturbance
2 ~ T
ACE n3 T 1=1
1
2111
| Z [f( BEnTT [f(= .c] + EEnnT (wx)
T
T
Σ
|| [f(x EENns [f(a .)]l + | (w))
$ | (p)ll
(28)
A.4 PROOF FOR COROLLARY 2
Corollary (ACE bound under control actuation misspecification) AB(x) is the parametric misspecification in the actuation matrix, and the ACE,, satisfies:
1 T
ACET, = -T2
Σ BBz (x)²) - =0
1
(BBa )i (x) (e
T
1=1
where (1) (+)) = f(x(-), So(x!²1).
Proof.
T++1 1o(x( + (B(a) (2) AB( (2) (u( [at (2) ,c) wt
= -((x( f(x; (2) .c) uh A B (z) Bd (5t [ 1o(2) (2)) + f(x("))
= f(x(-)) - f(x) .c) + We + ( Bt (z+ (2) (x)
(29)
We use e)(x)
(8)) denote ((a( fo(x(()). the ACE upper bound is calculated as follows:
ACEx1
B2nws (x(()) =
T
+
1
Σ
[B ((((t)) [(x) (t ,c) + w. AB(x))) )Bl (es))))s((?))
T
Σ
BEETTI [f(x(a)) 1(1(() .c) + EENTO Ez (A (x)
(x) (x (
Σ
BenT, [f(((a]]] BENty [ - EEnay [ (1) (err))
T
<h-
1 T
B| AB(1) (2) ) B
(x) (2) (e (11)))
(30)
A.5 PROOF FOR LEMMA 2
Lemma 2 (Trajectory tracking ACE of quadrotor) In this paper, for environmental disturbances in {6: |181|2 < T}. the formula of quadrotors' trajectory tracking error given as:
T
ACE,S T t=1
1
Σ
[Eewass (2( - 2121
1
|| ~-x (1 + - -E/K,]]
T
1
Σ
(00-ss(Ciee'i + - h h''((p)//)))))))pp))p) Kill
(p))
T
(31)
14

---

Under review as a conference paper at ICLR 2023
where C1 -2 C2 c/K.. C2 (e + (K,z, - c)e (e enaat) and
f(a .c) - f(x) (3) (), which is the prediction error of the unknown dynamics.
Proof.
Recall the kinetic function in equation 15 mi = mg + Rfr + fc. The desired control force fa is designed as:
fa Rfr fa ft fa = mi, + Ke mg
(32)
where He x2(²) xz. He is the error trajectory tracking and v, is the desired velocity By substituting equation 32 into equation 15 the UAV dynamics becomes:
me mg + Rfr + ft
me mg Ja St -s.
m(ú i,) - KoTe ft - ffe
mš. Ku c 0
K, 1
Σ.
- E 0
(33)
where e = fi-f - - f(x{(),)) Note that equation is is a second- inhomogencous
linear differential equation.
1) General solution: equation:
Making the substitution in the differential equation, r satisfies the auxiliary
K 11,2 = +
K
(34)
m
m
then, we obtain the general solution of the differential function F. as:
He Cje 2 + Cze
(35)
2) Special solution: Consider the standard second order differential equation: + pd qy e the special solution x is obvious:
=
(36)
Therefore, Te can be expressed as:
Te Ir x Cie Cze - k/Kr
(37)
3) Calculation of C1 and Cz: In equation 37 there exist two fixed points:(xx2.0) and (0,-2z)) We have:
0 Cie + Cre E/Kr Ez = C1 + c/Koo
(38)
By solving the simultaneous formulas, will get the answer of C1 and Cz.
[ - C2 e/Ko
(39)
C2 (e (Kaza - /Ke
(e eriad)
Thus the solution of the original derivative function is:
(z) Fe x x Cie x + Cz K/K,
15

---

Under review as a conference paper at ICLR 2023
and we have the average error bound in equation
T
AC
BENns (x) xل)ه1
T
B 1Ce + Cze ee/Koll
T
[B (1
Cye SewNs(E/K))
T
Σ |B | C1e + Cge
BI f(x(") ,0)/ K,ll
B ( + Cze (p)
(40)
B EXPERIMENTAL SETTINGS AND DETAILS
B.1 OOD-CONTROL PSEUDO CODE AND ALGORITHM SETTINGS
Duist--Distributionn comes from the misspecification of systems' components and the systematic enror of sensors and environment models. Our algorithm is able to extrapolate unknown wind dis- turbances after learning model from previous data containing generalized information.
To eliminate the influence caused by other factors, the two models share the same initial state and environmental conditions. Furthermore, in order to make the training process fair for both models, simulate them for the same number of iterations and sustain each iteration for the same period of time.
Algorithm 1 OoD-Control (Out-of-Distribuii) Generalization control for Adaptive Nonlinear Con-
trol)
Set of distribution functions X DNN & with parameter O environment estimation vector è Parameter: Parameters mechanical system and aerodynamics.
Input:
Output: The estimation of unknown force f
1: while picking from do
2: Sample series of independent random variables subject external wind force.
3: Apply external force to the simulation according to equation 43
4: Calculate loss with noise in the state: L || + Ar) 2
5: Update θ θ = e 7/1 V
6: Update è: ē= è m72 VeL
7: return f= (2x; Ta
8: end while
B.2 UPDATE PARAMETERS Ó AND
Based on equation we design discrete-time simulation process to calculate state variables and estimate the unknown term each time interval and by this way, collect data for training db. In a we update Ô during simulation: θ θ - Lg.
Keeping a constant è the inverted pendulum experiment results the inability to update environ- mental parameters. This can result in higher ACE or even failure to control the system (see Table 2 For the quadrotor, when the wind is severe, it cannot maintain its position, as illustrated in Figure 3 It should be noted, however, that algorithm exhibits better control occurs. Generally, updates to ó would be more energy intensive, whereas updating only č would be closer to the actual embedded device.
when position drift
16

---

Under review as a conference paper at ICLR 2023
Table 1: ACE results pendulum experiments with the changed or unchanged
Test Env.
No-Adapt
OMAC
OoD-Control
Omniscient
Changed
Breeze
0.538(0.325))
0.054(0.022)
0.050(0.024)
0.046(0.024)
Changed
Strong Breeze
1.366(0.257)
0.134(0.032)
0.078(0.029)
0.046(0.024)
Changed
Gale
0.592(0.196) 2.277(0.427)
0.163(0.055)
0.045(0.021)
Unchanged
Strong Breeze
1.366(0.257)
1.553(0.262)
1.548(0.265)
0.046(0.024)
Unchanged
Gale
2.277(0.427)
2.276(0.425)
2.275(0.425)
0.045(0.021)
Deep Learning
O0D-Control
10
10
5
Force [N]
0
-5
-5
-10
-10
-15
-15
Time [s]
Time [s]
Figure 3: Results when both è and e keep unchanged in testing
B.3 PSEUDO CODE OF THE PERTURBATION RADIUS CALCULATION ALGORITHM
Algorithm Monte Carlo algorithm for calculation of perturbation radius
Input: x, number of Monte Carlo samples: N, the variance of Gaussian noise: d Parameter: threshold of error:
Output: radius
1: for in range(N) do
2: Add Gaussian noise = I+ (0, o) and append to x_set
3: end for
4: for x_sample in X_set do
5: Obtain the prediction value and calculate prediction error
6: if error < then
7: nA nA+1 //nA is the number of successful predictions
8: end if
9: end for
10: pABar +- calculate the lower confidence bound
11: if pABar < 0.5 then
12: return // means abstention
13: else
14: radius fmpa (pABar)
Sypa is the percent point function of Gaussian distribution.
15: end if
return radius
B.4 DIFFERENTIAL EQUATION SOLVER
The fourth-order Runge-Kute Kutta method is common iterative method to calculate differential equa- tions and approach continuous functions. this paper, we use this method to calculate equation and simulate the process of the mechanical system. We do the simulation by moving tiny A/ each
17

---

Under review as a conference paper at ICLR 2023
time, which corresponds to the step size of the Runge- Kutta method. And the right- side of equation I is the derivative of the state 2.
B.5 THE COMPARISON OF VARIOUS TRAJECTORY TRACKING MODES
OoD generalized model adapts better and gives inputs forces closer to the desired ones illustrated in Figure 4 to Figure 7 which shows the traces under OMAC (Deep Learning), OoD-Control and omniscient models and the desired trajectory, the OoD-Control algorithm clearly goes closer the desired curve.
Cortrol
CHANC:
6.75-
0.20
0:15
810
-150
-159
0.04mm]
0.00
3.05
0.15
0.05
0.00
-04
1000ml
xmm1
-0.1
Toeld
(a)
(b)
Figure 4: Traces when the quadrotor tries to keep still.
DMAC
Cob Contre
[m)
GMAC
-05
OcD Contro
ommiicart
ymml
tralt
05
-2.0
(a)
(b)
Figure 5: Traces when the trajectory is spiral curve.
B.6 WIND CONDITIONS IN EXPERIMENTS
B.6.1 WIND FIELD SETTINGS
Protocol Two sets of distribution functions X and with X = 0 are defined for training and testing. The training distribution x X and the testing distribution w Ω are specified for different experiment tasks. The wind velocity is series of independent random variables sampled subject the distributions (x or w) picked out. The wind brings induced airflow the rotor blades, creating complex and nonstationary aerodynamic interaction All models are simulated with the same wind series and the simulating duration also the same.
Mooe detailed information related the aerodynamics under wind shown in Appendix B.6.3
18

---

Under review as a conference paper at ICLR 2023
DeD-Cartorl
GMANC
OoD-Contral
muth
0.5
4mh]
110
XUTT
-12
-15
(a)
(h)
Figure 6: Traces of OMAC, OoD-Control and omniscient quadrotor models with figure-8 trajectory
OMAC
DeD-Conibrll
OMACC
OcD Contro omnicc e't ground trut
Amm)
2.0
1.5
ylm]
0.5
0.0
(b)
Figure 7: Traces of OMAC, OoD-Control and omniscient quadrotor models with sin-forward trajec- tory.
Table ACE quadrotor experiment with different trajectories and environment
Trajectory
Test Env.
No-adapt
OMAC
OoD-Control
Omniscient
hover
Breeze
0.279(0.056)
0.202(0.035)
0.048(0.007)
0.029(0.003)
hover
Strong Breeze
0.410(0.085)
0.257(0.049)
0.060(0.011)
0.035(0.003)
hover
Gale
0.478(0. 100)
0.297(0.079)
0.074(0.026)
0.040(0.004)
sin-forward
Breeze
0,319(0.058)
0.277(0.047)
0.098(0.011)
0.079(0.010)
sin-forward
Strong Breeze
0.439(0.086)
0.334(0.053)
0.110(0.017)
0.085(0.016)
sin-forward
Gale
0.504(0.101)
0.379(0.067)
0.133(0.045)
0.092(0.021)
figure-8
Breeze
0.334(0.060)
0.268(0.032)
0.094(0.005)
0.079(0.005)
figure-8
Strong Breeze
0.455(0.085)
0.318(0.055)
0.100(0.006)
0.082(0.005)
figure-8
Gale
0.520(0. 100)
0.403(0.216)
0.169(0.195)
0.085(0.006)
spiral-up
Breeze
0.295(0.060)
0.214(0.044)
0.084(0.012)
0.067(0.004)
spiral-up
Strong Breeze
0.422(0.088)
0.265(0.057)
0.095(0.016)
0.071(0.006)
spiral-up
Gale
0.490(0.103)
0.309(0.090)
0.120(0.055)
0.075(0.008)
19

---

Under review as a conference paper at ICLR 2023
Pendulum In this task, use 5 different normal distributions the training set, i.e. X fx+|x* = (0, 0.2k), 1.2.....5), When testing, we set different levels the wind based on the difficulty, that is the breeze, strong breeze, and gale. And each level has 6 or different uniform distributions. We show the details of the testing set f as follows:
Breeze: 21 = (w | wf U(0.. 0.54), 1, 1,2, 5,
Strong Breeze: Q2 {wh w U[- - 0.5k), 1,2,.... .........)).
Gale: Q2 {wi w = (-6000 0.5k, 6.0 0.5k), k 1,2,... 7,88
In this instantiation, only use one training distribution, the three diree-dimensional stan- dard normal distribution, to train the models. Mcanwhile, we transform the initial wind data sam- pled from the normal distribution their absolute values, i.e. X (,, E),, f [0, 0, 0]. E diag(1, 1, 1). The reason for this operation is that we want to make the wind come from only one octant, then it differs from the test environment in distribution and direction. Similarly, the test set Ω also has 3 different levels, but the exact distributions are different from the pendulum task. Following are the details:
Quadrotor
Breeze: W1 UDD)), {(x, v, 2, (-3,3)}
Strong Breeze: W2 = -D2)) D2 = (12,V,2) 2, E (-6,6)}
Gale: w3 = U())), D3 = {(v, v, K (-8,8))
Trajectory illustration The three trajectories we used in the quadrotor experiment (see Table mathematically described as:
sin-forward: (z,, U, 2) = (2sin zt 3 0.2t, 0.5t)
figure-8: (z, (2sin( (t 2sin( 51) sin sin(? spiral-up: (z, V. 2) = (sin(22) cos( (2g)) - 0.2t)
B.6.2 DISCUSSION
What's the main problem addressed in this puper and whut's ' the innovation compared with other online adaptive control?
Q1:
A1: We study the online adaptive flight control problem when testing and training environment domain shifts and give a methodology that the upper bound of the predicted error of the unknown dynamics maintains constant under radius of perturbation.
Q2: How does the OwD-Contro algorithm perform the i.i.d. environment, where the train and test environment follow the same distribution?
A2: We run OMAC and DoD-Control algorithm on i.dd environment and put the results in Table 3 (see Appendix B.7). In the i.i.d. environment, the ACE of all algorithms drop naturally, but OoD-Control still performs much better than OMAC.
Q3: How does the size of noise add to the state vectors when training affects the performance of our algorithm?
A3: We rum OoD-Control algorithm with different noises in i..d. and o.o.d. environment respec- tively. Results (see Table 4) show that the algorithm performs worse in the i.i.d. environment as the scale of noise increases. And performance in the OoD environment gets little better in certain range when noise size increases, but becomes worse when noise is too large.
B.6.3 AERODYNAMICS OF THE ROTORS IN WIND
Wind conditions affect the quadrotor mainly in two ways. The first is that alters the aerodynamics of the rotors. The second is change the air resistance of each windward side. The aerodynamics of the rotor in windy conditions is illustrated in Figure g This paper defines several types of winds, namely Breeze, Strong Breeze, and Gale.
20

---

Under review as a conference paper at ICLR 2023
Table 3: ACE in quadrotor experiment in i..d. environment
Trajectory
No-adapt
OMAC
OoD-Control
Omniscient
hover
0.378(0.052))
0. 0.138(0.030)
0.036(0.012)
0.027(0.003)
figure-8
0.436(0.043)
0.230(0.030)
0.087(0.008)
0.075(0.005)
spiral-up
0.403(0.050)
0.158(0.031)
0.076(0.008)
0.065(0.003)
sin-forward
0.432(0.048)
0.209(0.022)
0.092(0.017)
0.076(0.004)
F+
VE
T
Vy-t
V
V/
Figure 8: Sketch of the aerodynamics of the propeller in wind conditions
1) Aerodynamics caused by wind gust disturbances. According to the rotor slipstream theory (Con- lisk 2001), the induced velocity is calculated by:
kyn² 2 pr
||
(41)
V
where ky is the lift coefficient, P is the air density, r is the rotor radius. wind field results a total aerodynamic force on the rotor equal the sum of the lift force T and the additional wind disturbance force F.. The total lift can be calculated as:
[ + Fwll = px Vell + Vell
(42)
Therefore, the wind disturbance force Hw, and moment Mw, on éth rotor is:
Hw: kyn, - pT 0, v kkmn + V km Hww, /ky, when the rotor turns clockwise Mwn { kim Hw: ky; otherwise
(43)
where kim is the anti-torque coefficient related to the shape of the rotors and local air density.
2) The air drag. Air drag can be ignored in hovering or low-speed flights without wind. However, in the presence of wind field, the following equation can be used to calculate air drag.
D, 2 cpSaii V2
(44)
where e represents the air drag coefficient, Sair the windward area, and Vair is the relative speed of the wind to the quadrotor.
B.7 PERFORMANCE IN L.I.D ENVIRONMENT AND DIFFERENT NOISE SETTING
Noise, and Noise, mean the noise scale added to the state vector and the environment representa- tion vector respectively. In this experiment, We double the test duration (compared to the results in Table 2 to enlarge the differences.
21

---

Under review as a conference paper at ICLR 2023
Table 4: ACE in quadrotor experiment with different noise
Trajectory
OoD Env.
Noise,
Noise,
i.i.d.
0.o.d.
figure-8
Gale
0.01
0.01
0.0628
0.5897
figure-8
Gale
0.02
0.02
0.0635
0.5897
figure-8
Gale
0.05
0.02
0.0652
0.5767
figure-8
Gale
0.05
0.05
0.0651
0.5877
figure-8
Gale
0.1
0.1
0.0729
0.5908
22


Reviews:
[{"review_id": "tVXZBNTDaz", "reviewer": "Reviewer_ma1x", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper describes a data-driven control approach that is argued to address the out-of-distribution control problem. The approach models unknown environment-dependent dynamics of a UAV using a deep neural network, and then use conventional control theories for UAV controls.", "summary_of_the_review": "Although the idea of integrating control theories with learning methods is attractive, novelty of the paper is a concern and applicability of the approach to real UAVs is also a concern.", "strength_and_weaknesses": "Strength: \n\n+ Combining strengths of data-driven methods and control methods is attractive, especially for robotics applications such as controlling UAVs.\n\n+ The control approach is mathematically formulated and analyzed.\n\nWeakness:\n\n- It seems the argued novelty focuses on addressing out-of-distribution data. However, the paper simply uses a deep neural network (inspired by Shi et al. 2021) to model the unknown environment-dependent dynamics. Why is this neural network capable of modeling such an unknown dynamic during the execution time? How does this neural network model the environment? If the dynamics in the execution phase is unknown and different from the training phase, how to especially model or address this difference?\n\n- The integration of data-driven methods and control theories is simple. The proposed approach assumes the expectation of the prediction of the unknown dynamics remains the same for a perturbation of radius, and uses conventional control theories to analyze the bound. However, this assumption may not be true in the real-world scenario. Even if it is true, how to deal with the transition between areas that may have different perturbations.\n\n- What is the relationship between out-of-distribution data and perturbations? \n\n- When comparing with previous out-of-distribution generalization methods in related work, the paper argues previous methods suffers from “neglecting the gap between the simulation experiments and reality”. The proposed method is also not evaluated on a real system. It is not convincing that the proposed method can be applied to a real UAV.\n\n- The paper generally argues that it is difficult or impractical to model the environment-dependent dynamics. Then in the experiment, the paper uses the widely used Dryden model to simulate wind field. Given this Dryden model, why is it challenging to model the dynamics?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "TYW-WZUpZLI", "reviewer": "Reviewer_r1g3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper aims for out-of-distribution adaptation of UAV controllers, e.g., in the presence of unexpected wind. The authors derive an upper bound on their version of the average control error and an algorithm (OoD-Control) to adapt the dynamics model from samples of the changed dynamics. Experiments on the inverted pendulum and UAV control show improved performance in comparison to OMAC.\n", "summary_of_the_review": "The paper presents an interesting and promising approach to an important problem. The text is sometimes hard to read, and the theory might have some errors (see questions). However, the reviewer is willing to improve the score if the authors can improve readability and convince me that the math is correct (or fix it).", "strength_and_weaknesses": "**Strength**:\n\nThe problem is very relevant and the experimental results look very good.\n\n**Weaknesses:**\n\nThe paper follows Shi et al. (2021) very closely and does not always explain the terminology and choices. This makes the paper sometimes hard to read, as the theoretical contribution comes across as somewhat unmotivated and require large jumps in reasoning. The problem is compounded by some (in this reviewer's opinion) apparent errors in the math. More thoroughness and more explanation on the assumptions, used transformations and definitions would probably improve the paper a lot. \nThe reviewer would like the authors to explain the following:\n1. Is the defined ACE measure useful for OoD generalization? Note that $|| \\mathbb E[x_t^{(z)}] - x_t^d ||$ is zero for symmetric effects, e.g. wind blows equally often from left and right. A controller with ACE=0 can therefore still perform arbitrarily bad! Shi et al. (2021) seem to optimize $\\mathbb E[||x_t^{(z)} - x_t^d ||]$. Please compare the two definitions and state your reasoning for choosing one.\n2. Section 4.1 introduces an important *required* property; how does Th.1 prove this?\tAssuming the right-hand side of (eq.5) is $p$, how do you guarantee that $\\tilde h_{\\pi_0}(x_t^{(z)}) > p$?\n3. Theorem 1 uses the substitution $h_{\\pi_0}(x_t^{(z)} + \\delta)$ to $h_{\\pi_\\delta}(x_t^{(z)})$, where $\\pi_{\\delta}$ is defined as \"the distribution of $z + \\delta$\". How does the transformation $x_t^{(z)} + \\delta$ to $x_t^{(z+\\delta)}$ work? Don't these $\\delta$'s have different dimensionality?\n4. The current use of $\\tilde h$ is very unintuitive. Could you elaborate how one has to interpret $\\tilde h_{\\pi_0}(x_t^{(z)})$, in particular for the crucial (eq.8)?\n5. (eq.10) and (eq.11) seem to ignore $x_{t+1}^d$. To make sense in (eq.12), (eq.10) should have $x_{t+1}^d$ in the RHS bracket and the LHS of (eq.11) should be $x_{t+1}^{(z)} - x_{t+1}^d$.  Similarly, Corollary 2 seems to miss $x_{t+1}^d$ in the definition of $e_f$.\n6. The $p$ in (eq.12) is not defined, but combining the reviewers interpretation of (eq.8) and (eq.11) would mean $p$ is the rhs of (eq.8). Is this correct?\n7. How important is Corollary 2? Can you make an ablation with and without considering control misspecification? Also, where does $K_v$ come from?\n8. Algorithm 1 is not understandable without reading Shi et al. (2021).\tPlease augment the appendix with a self-contained description.\n9. Please indicate which experimental results are statistically significant. For example, in Table 1 only row 3 (and maybe 2) look as if OoD is significantly better than OMAC, whereas in Table 2 all **but** row 9 look significant. Just claiming \"is higher\" is not meaningful. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "5D1mXjCU1OQ", "reviewer": "Reviewer_7hFa", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The work presents a novel adaptive data-driven UAV flight control method, OoD-Control. Under gaussian noise assumptions and bounded control and disturbances, the authors show that the upper bound of the prediction error remains constant under unpredictable disturbance. The control method (and bound) is also shown to be extensible to different nonlinear dynamical models.", "summary_of_the_review": "The paper proposed a very sound construction of a novel adaptive data-driven UAV flight control method based on the derivation of a prediction error bound result. This seems to offer improved expressiveness in learning simulated wind dynamics, and thus improved performance against earlier work. The authors' claim that the architecture achieves OoD generalisation remains however questionable, or to the least has not been explored to the fullest. A lack of clarity in terms of precise training protocol as well as an absence of connection to possible real world implementations penalise the paper's impact.", "strength_and_weaknesses": "Strengths:\n- The work deals with the very pertinent topic of OoD performance for learned dynamics\n- The theoretical bound shown, without the need for the restrictive eISS assumption is a notable contribution for nonlinear under-actuated systems.\n- The performance of the control protocol is impressive in simulation, not only against previous work but even with respect to omniscient ground truth controllers\n- Simulation work is of good quality, permitting clean comparison of baselines and a clear presentation of the results. \n\nWeaknesses:\nThe work is theoretically sound and relevant. The experimental setup however leaves some claims unexplained and many important questions unanswered.\n-  The authors claim in section 5.4 (quadrotor) that \"As the wind field in the testing environment is quite different from the training environment, our algorithm performs relatively better than the baseline and no-adapt.\" Apart from the poor grammatical phrasing, the only material in the paper that comes to support that claim is in appendix B.6.1 \"The wind comes from only one octant, so it differs\nfrom test environment in not only distribution but also direction. The models are trained in only one\nenvironment. Our OoD-Control model adapts well in testing even though the training material is\nquite little.\" Again neglecting the very poor wording of the last sentence, one fails to see how much OoD testing is really happening, i.e. how dissimilar the settings really are to the DNN.\n- An important study would include pushing the architecture to its limits with perturbations of increasingly different distributions and directions. The out of distribution capacities claimed in the very model name OoD-Control, are not significantly exhibited. \n- Along the same lines, a precise explanation of the data consumed to train the model is missing, \"quite little\" being an unacceptable unit of measure for a machine learning conference paper. This remark generalises to the entire training framework which remains very obscure: what model architecture is used? Of what size? What is the data pipeline setup? etc.\n- What about other control methods to fight the wind ? Why only benchmark against OMAC ? The authors claim \"most of the previously mentioned control\nmethods suffer from limitations. Imprecise system modeling and non-modeled environmental\ndisturbances may result in unacceptable performance or instability.\" It would be worth comparing the performance against state of the art robust control protocols for example to support this claim. \n- The authors fail to discuss any possibility of using this algorithm in the real world. The natural limitations of data collection via Monte Carlo methods make this method trainable in simulation. What about the simulation to reality gap? Have the authors considered training a OoD-Controller in simulation and attempting to measure its performance on real hardware? Any results in this direction would be of great value.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject