Original: Safe Robot Learning in Assistive Devices through Neural Network Repair
Revised: Safe Robot Learning in Assistive Devices through Neural Network Repair

Original: Anonymous Author(s) AffiliationAddress email Abstract: Assistive robotic devices are a particularly promising field of application for neural networks (NN) due to the need for personalization and hard-tomodel human-machine interaction dynamics.
Revised: Abstract: Assistive robotic devices are a particularly promising ﬁeld of application for neural networks (NN) due to the need for personalization and hard-tomodel human-machine interaction dynamics.

Original: However, NN based estimators andcontrollers may produce potentially unsafe outputs over previously unseen datapoints.
Revised: However, NN based estimators and controllers may produce potentially unsafe outputs over previously unseen data points.

Original: In this paper, we introduce an algorithm for updating NN control policiesto satisfy a given set of formal safety constraints, while also optimizing the original loss function.
Revised: In this paper, we introduce an algorithm for updating NN control policies to satisfy a given set of formal safety constraints, while also optimizing the original loss function.

Original: Given a set of mixed-integer linear constraints, we define the
Revised: Given a set of mixed-integer linear constraints, we deﬁne the

Original: NN repair problem as a Mixed Integer Quadratic Program (MIQP).
Revised: NN repair problem as a Mixed Integer Quadratic Program (MIQP).

Original: In extensiveexperiments, we demonstrate the efficacy of our repair method in generating safepolicies for a lower-leg prosthesis.
Revised: In extensive experiments, we demonstrate the efﬁcacy of our repair method in generating safe policies for a lower-leg prosthesis.

Original:  Introduction
Revised:  Introduction

Original: Robot learning has the potential to revolutionize the fieldof wearable robotic devices, such as prosthetics, orthosesand exoskeletons [1].
Revised: Robot learning has the potential to revolutionize the ﬁeld of wearable robotic devices, such as prosthetics, orthoses and exoskeletons [1].

Original: Often, such devices are designedin a “one size fits all” manner using models based on average population statistics.
Revised: Often, such devices are designed in a “one size ﬁts all” manner using models based on average population statistics.

Original: However, machine learningtechniques can help adapt control parameters automatically to the wearer’s individual characteristics, motionpatterns or biomechanics.
Revised: However, machine learning techniques can help adapt control parameters automatically to the wearer’s individual characteristics, motion patterns or biomechanics.

Original: The result is a substantial improvement in ergonomic comfort and quality-of-life.
Revised: The result is a substantial improvement in ergonomic comfort and quality-of-life.

Original: Despite these benefits, the adoption of machine learning and,in particular, deep learning [2] in this field is still limited,largely due to safety concerns.
Revised: Despite these beneﬁts, the adoption of machine learning and, in particular, deep learning [2] in this ﬁeld is still limited, largely due to safety concerns.

Original: In the case of a prosthesis, for example, it may be important to guarantee thatthe generated control values do not exceed a maximumthreshold under a set of testing conditions.
Revised: In the case of a prosthesis, for example, it may be important to guarantee that the generated control values do not exceed a maximum threshold under a set of testing conditions.

Original: Other safetyconstraints include limits on velocities, joint angles and bounds on the change in control inputs.
Revised: Other safety constraints include limits on velocities, joint angles and bounds on the change in control inputs.

Original: In this paper, we derive an algorithm for training neural networks controllers to satisfy given safetyspecifications, in addition to fitting the given training/test data.
Revised: In this paper, we derive an algorithm for training neural networks controllers to satisfy given safety speciﬁcations, in addition to ﬁtting the given training/test data.

Original: In particular, we use our approach toderive controllers for a robotic lower-leg prosthesis that satisfy basic safety conditions, (see Fig. 1).
Revised: In particular, we use our approach to derive controllers for a robotic lower-leg prosthesis that satisfy basic safety conditions, (see Fig. 1).

Original: Our proposed approach inputs a trained network (e.g., a network obtained using backpropagation onthe training data) along with a specification that places restrictions on the possible outputs for a givenset of inputs.
Revised: Our proposed approach inputs a trained network (e.g., a network obtained using backpropagation on the training data) along with a speciﬁcation that places restrictions on the possible outputs for a given set of inputs.

Original: It then generates a modified set of weights that obeys the desired safety constraints onthe output using deterministic global optimization.
Revised: It then generates a modiﬁed set of weights that obeys the desired safety constraints on the output using deterministic global optimization.

Original: We provide theoretical guarantees of optimalityfor this technique (assuming no numerical errors).
Revised: We provide theoretical guarantees of optimality for this technique (assuming no numerical errors).

Original: Furthermore, in real-world robot experimentswe show that the introduced methodology produces safe neural policies for a lower-leg prosthesissatisfying a variety of constraints.
Revised: Furthermore, in real-world robot experiments we show that the introduced methodology produces safe neural policies for a lower-leg prosthesis satisfying a variety of constraints.

Original: Contributions: The proposed method can repair any layer in a network increasing the size of thesolution space and the likelihood of a feasible successful repair.
Revised: Contributions. The proposed method can repair any layer in a network increasing the size of the solution space and the likelihood of a feasible successful repair.

Original: More importantly, it comes withtheoretical guarantees on successfully repairing all discovered unsafe samples.
Revised: More importantly, it comes with theoretical guarantees on successfully repairing all discovered unsafe samples.

Original: When compared toretraining or fine-tuning methods, it also has two distinct benefits: (1) it does not require modificationof the training data so as to satisfy the constraints, and (2) it does not utilize gradient optimizationmethods which do not guarantee constraint satisfaction even for the discovered unsafe data points.
Revised: When compared to retraining or ﬁne-tuning methods, it also has two distinct beneﬁts: (1) it does not require modiﬁcation of the training data so as to satisfy the constraints, and (2) it does not utilize gradient optimization methods which do not guarantee constraint satisfaction even for the discovered unsafe data points.

Original: Finally, when compared to other property driven repair works, i.e., [3, 4, 5], it is applied for the firsttime to a real physical system, i.e., a powered prosthetic device, as opposed to a model.
Revised: Finally, when compared to other property driven repair works, i.e., [3, 4, 5, 6, 7], it is applied for the ﬁrst time to a real physical system, i.e., a powered prosthetic device, as opposed to a model.

Original: Related Work Learning based control for prosthetics is motivated by the challenges in modelinghuman-prosthesis dynamics, which exhibits time varying behavior.
Revised: Related Works. Learning based control for prosthetics is motivated by the challenges in modeling human-prosthesis dynamics, which exhibits time varying behavior.

Original: Several different approaches thatutilize learning based methods for controlling prosthetic devices have been proposed in the literature [6, 7].
Revised: Several different approaches that utilize learning based methods for controlling prosthetic devices have been proposed in the literature [8, 9].

Original: Nevertheless, this short review focuses only on neural network (NN) based methods.
Revised: Nevertheless, this short review focuses only on neural network (NN) based methods.

Original: Gao et al.
Revised: Gao et al.

Original: [8] provide a prosthetic controller based on recurrent neural networks, and show that thistype of controller effectively minimizes the difference between desired and actual trajectories on apowered prosthetic device.
Revised: [10] provide a prosthetic controller based on recurrent neural networks, and show that this type of controller effectively minimizes the difference between desired and actual trajectories on a powered prosthetic device.

Original: In another direction, Keles¸ and Yucesoy [9] and Vonsevych et al.
Revised: In another direction, Keles¸ and Yucesoy [11] and Vonsevych et al.

Original: [10]focus on utilizing electromyography (EMG) signals to predict control parameters for prosthetic ankles and hands.
Revised: [12] focus on utilizing electromyography (EMG) signals to predict control parameters for prosthetic ankles and hands.

Original: Our work follows a similar approach in predicting ankle control parameters with aneural network, which then drives the prosthesis through a PD controller.
Revised: Our work follows a similar approach in predicting ankle control parameters with a neural network, which then drives the prosthesis through a PD controller.

Original: The verification of neuralnetworks has been widely studied in order to check specifications for a given NN as a standalonecomponent [11, 12, 13, 14] (for a survey see [15]).
Revised: The veriﬁcation of neural networks has been widely studied in order to check speciﬁcations for a given NN as a standalone component [13, 14, 15, 16] (for a survey see [17]).

Original: or as part of a closed loop system [16, 17, 18].
Revised: or as part of a closed loop system [18, 19, 20].

Original: Testing techniques can produce counterexamples (adversarial samples) for a variety of NN-basedapplications, e.g., [19, 12, 20, 21, 22, 23].
Revised: Testing techniques can produce counterexamples (adversarial samples) for a variety of NN-based applications, e.g., [21, 14, 22, 23, 24, 25].

Original: One approach to ensuring that a NN satisfies given set ofsafety properties is to use retraining and fine-tuning based on counter-examples [24, 25].
Revised: One approach to ensuring that a NN satisﬁes given set of safety properties is to use retraining and ﬁne-tuning based on counter-examples [26, 27, 7].

Original: However,this approach has a number of pitfalls.
Revised: However, this approach has a number of pitfalls.

Original: First, the labels for the counterexamples need to be available.
Revised: First, the labels for the counterexamples need to be available.

Original: This may involve further data-collection, which is often cumbersome.
Revised: This may involve further data-collection, which is often cumbersome.

Original: Also, gradient descent optimization approaches cannot provide any guarantees that the result satisfies the provided constraints.
Revised: Also, gradient descent optimization approaches cannot provide any guarantees that the result satisﬁes the provided constraints.

Original: Our approach, in contrast, avoids generating adversarial examples or performing gradient descent.
Revised: Our approach, in contrast, avoids generating adversarial examples or performing gradient descent.

Original: The problem of “repairing” a network, which involves modifying weights of the network in a minimal manner to satisfy some safety properties, is thus more involved than simply retraining a networkon some additional data involving counterexamples.
Revised: The problem of “repairing” a network, which involves modifying weights of the network in a minimal manner to satisfy some safety properties, is thus more involved than simply retraining a network on some additional data involving counterexamples.

Original: Three methods in the literature [3, 4, 5] makeprogress toward the goal of repairing NNs.
Revised: Some methods in the literature [3, 4, 5, 6] make progress toward the goal of repairing NNs.

Original: Goldberger et al.
Revised: Goldberger et al.

Original: [3] can only repair the output layer,which drastically reduces the space of possible successful repairs (and in many cases a repair is noteven possible).
Revised: [3] can only repair the output layer, which drastically reduces the space of possible successful repairs (and in many cases a repair is not even possible).

Original: The method of Fu and Li [4] produces patches for certain partitions of the inputspace wherein the corresponding outputs are modified.
Revised: The method of Fu and Li [4] produces patches for certain partitions of the input space wherein the corresponding outputs are modiﬁed.

Original: The direction of adding patches on the NNoutput is very promising; however, it is unclear how the approach will scale for high dimensionalinputs, since it requires partitioning the input space into affine subregions.
Revised: The direction of adding patches on the NN output is very promising; however, it is unclear how the approach will scale for high dimensional inputs, since it requires partitioning the input space into afﬁne subregions.

Revised: Authors in [6] repair the linear regions related to each faulty sample in the faulty network’s weight space using a decoupled

Original: Finally, Dong et al.
Revised: DNN architecture.

Original: [26]primarily focus on counter-example generation, and  the repairs are performed through retraining.
Revised: However, this method causes the repaired network to be discontinuous. Therefore, it cannot be employed in robot learning and control applications which is the target application of this paper. This method is not also applicable for the networks with more than three inputs.

Original:  Problem Formulation
Revised:  Problem Formulation

Original: Without loss of generality, we motivate and discuss our approach using the task of learning saferobot controllers for a lower-leg prosthesis.
Revised: Without loss of generality, we motivate and discuss our approach using the task of learning safe robot controllers for a lower-leg prosthesis.

Original: The goal of this task is to learn a policy π θ whichgenerates control values for the ankle angle of the powered prosthesis given a set of sensor values.
Revised: The goal of this task is to learn a policy π θ which generates control values for the ankle angle of the powered prosthesis given a set of sensor values.

Original: Most critically, however, policy π θ is required to satisfy a set of safety constraints Ψ .
Revised: Most critically, however, policy π θ is required to satisfy a set of safety constraints Ψ .

Original: Fig.
Revised: Fig.

Original: 2 providesan overview of our methodology in addressing this challenge.
Revised: 2 provides an overview of our methodology in addressing this challenge.

Original: For now, we assume that an unsafeprior policy network may exist, see Fig.
Revised: For now, we assume that an unsafe prior policy network may exist, see Fig.

Original: 2 (left).
Revised: 2 (left).

Original: The network parameters θ may be learned withthrough imitation learning [8], reinforcement learning [27] or any of the common machine learningparadigms.
Revised: The network parameters θ may be learned with through imitation learning [10], reinforcement learning [28] or any of the common machine learning paradigms.

Original: The policy may be optimized for task efficiency, e.g., stable and low-effort walkinggaits, but may not yet satisfy any safety constraints.
Revised: The policy may be optimized for task efﬁciency, e.g., stable and low-effort walking gaits, but may not yet satisfy any safety constraints.

Original: Our goal is to find an adjusted set of networkparameters that satisfy any such constraint.
Revised: Our goal is to ﬁnd an adjusted set of network parameters that satisfy any such constraint.

Original: We may now choose, for example, to restrict the controlrate to specific bounds.
Revised: We may now choose, for example, to restrict the control rate to speciﬁc bounds.

Original: Running the network on the input values of a validation data set reveals thatviolations occur at a number of time steps, as seen in Fig.
Revised: Running the network on the input values of a validation data set reveals that violations occur at a number of time steps, as seen in Fig.

Original: Our approach, called NNRepLayer (Layer-wise Neural Network Repair), takes the original network parameters θ
Revised: Our approach, called NNRepLayer (Layer-wise Neural Network Repair), takes the original network parameters θ

Original: and the predicates Ψand yields the updated parameters that generate no violations.
Revised: and the predicates Ψ and yields the updated parameters that generate no violations.

Original: Notation.
Revised: Notation.

Original: We denote the set of variables { a 1 , a 2 , · · · , a N } with { a n } Nn =1 .
Revised: We denote the set of variables { a 1 , a 2 , · · · , a N } with { a n } Nn =1 .

Original: Let π θ be a networkpolicy with L hidden layers.
Revised: Let π θ be a network policy with L hidden layers.

Original: The nodes at each layer l ∈ { l } Ll =0 are represented by x l , where| x l | denotes the dimension of layer l ( x 0 represents the input of network).
Revised: The nodes at each layer l ∈ { l } Ll =0 are represented by x l , where | x l | denotes the dimension of layer l ( x 0 represents the input of network).

Original: The network’s output isalso denoted by y or π θ ( x 0 ) .
Revised: The network’s output is also denoted by y or π θ ( x 0 ) .

Original: We consider fully connected policy networks with weight and biasterms { ( θ lw , θ lb ) } L +1 l =1 .
Revised: We consider fully connected policy networks with weight and bias terms { ( θ lw , θ lb ) } L +1 l =1 .

Original: The training data set of N inputs x 0 n and target outputs t n is denoted by{ ( x 0 n , t n ) } Nn =1 sampled from the input-output space X × T ⊆ R | x 0 | × R | t | .
Revised: The training data set of N inputs x 0 n and target outputs t n is denoted by { ( x 0 n , t n ) } Nn =1 sampled from the input-output space X × T ⊆ R | x 0 | × R | t | .

Original: We use x ln to denotethe vector of nodes at layer l for sample n .
Revised: We use x ln to denote the vector of nodes at layer l for sample n .

Original: In this work, we focus on the policy networks with the
Revised: In this work, we focus on the policy networks with the

Original: Rectified Linear Unit (ReLU) activation function R ( z ) = max { 0 , z } .
Revised: Rectiﬁed Linear Unit (ReLU) activation function R ( z ) = max { 0 , z } .

Original: Thus, given the n th sample,in the l th hidden layer, we have x l = R (cid:0) θ lw x l − 1 + θ lb (cid:1) .
Revised: Thus, given the n th sample, in the l th hidden layer, we have x l = R (cid:0) θ lw x l − 1 + θ lb (cid:1) .

Original: An activation function is not applied to thelast layer, i.e. y =
Revised: An activation function is not applied to the last layer, i.e. y =

Original: θ L +1 w x L + θ L +1 b .
Revised: θ L +1 w x L + θ L +1 b .

Original: Problem Statement (Repair Problem).
Revised: Problem Statement (Repair Problem).

Original: Let π θ be a trained policy network over the training inputoutput space X × T ⊆ R | 

Revised: Let π θ be a trained policy network over the training inputoutput space X × T ⊆ R |

Original: | × R | t | and Ψ( y, x 0 ) be a predicate on the output y of network for a setof inputs of interest x 0 ∈ X r ⊆ X .
Revised: | × R | t | and Ψ( y, x 0 ) be a predicate on the output y of network for a set of inputs of interest x 0 ∈ X r ⊆ X .

Original: The Repair Problem is to modify the weight θ w and bias termsθ b of π θ such that the repaired policy π θ r satisfies Ψ ( y, x 0 ) .
Revised: The Repair Problem is to modify the weight θ w and bias terms θ b of π θ such that the repaired policy π θ r satisﬁes Ψ( y, x 0 ) .

Original: The repair of the policy network should not only satisfy the predicate Ψ( y, x 0 ) but should alsomaintain the performance of original policy.
Revised: The repair of the policy network should not only satisfy the predicate Ψ( y, x 0 ) but should also maintain the performance of original policy.

Original: To satisfy the latter, the method proposed in [3] ensuresthe satisfaction of predicate only with a minimal deviation of weights in the last layer.
Revised: To satisfy the latter, the method proposed in [3] ensures the satisfaction of predicate only with a minimal deviation of weights in the last layer.

Original: However,as we show latter in the experimental results, the repair of last layer is not necessarily feasible orsufficient to satisfy the predicates with a minimal deviation from the original parameters.
Revised: However, as we show latter in the experimental results, the repair of last layer is not necessarily feasible or sufﬁcient to satisfy the predicates with a minimal deviation from the original parameters.

Original: Moreover,the minimal deviation from the original weights is not a sufficient guarantee to maintain the originalperformance of network.
Revised: Moreover, the minimal deviation from the original weights is not a sufﬁcient guarantee to maintain the original performance of network.

Original: It is well-known that subtle changes in the weights may cause the networkto significantly deviate from its original performance [28].
Revised: It is well-known that subtle changes in the weights may cause the network to signiﬁcantly deviate from its original performance [29].

Original: Therefore, it is important for the repairedpolicy π θ r to also minimize the loss w.r.t.
Revised: Therefore, it is important for the repaired policy π θ r to also minimize the loss w.r.t.

Original: its original training data.
Revised: its original training data.

Original: We propose NNRepLayer(Layer-wise Neural Network Repair) that satisfies a predicate Ψ ( y, x 0 ) by repairing a specific layerof the policy network while minimizing the training loss.
Revised: We propose NNRepLayer (Layer-wise Neural Network Repair) that satisﬁes a predicate Ψ( y, x 0 ) by repairing a speciﬁc layer of the policy network while minimizing the training loss.

Original:  NNRepLayer
Revised:  NNRepLayer

Original: We can formulate our framework as the minimization problem of the loss function E ( θ w , θ b ) subjectto ( x 0 , t ) ∈ X × T and Ψ( y, x 0 ) for x 0 ∈ X r .
Revised: We can formulate our framework as the minimization problem of the loss function E ( θ w , θ b ) subject to ( x 0 , t ) ∈ X × T and Ψ( y, x 0 ) for x 0 ∈ X r .

Original: However, the resulting optimization problemis non-convex and difficult to solve due to the nonlinear ReLU activation function and high-ordernonlinear constraints resulted from the multiplication of terms involving the weight/bias variables.
Revised: However, the resulting optimization problem is non-convex and difﬁcult to solve due to the nonlinear ReLU activation function and high-order nonlinear constraints resulted from the multiplication of terms involving the weight/bias variables.

Original: In our approach, we obtain a sub-optimal solution by just focusing on repairing a single layer.
Revised: In our approach, we obtain a sub-optimal solution by just focusing on repairing a single layer.

Original: We therefore modify the weight and bias terms of a single layer to adjust the predictions so as tominimize E ( θ w , θ b ) and to satisfy Ψ ( y, x 0 ) .
Revised: We therefore modify the weight and bias terms of a single layer to adjust the predictions so as to minimize E ( θ w , θ b ) and to satisfy Ψ( y, x 0 ) .

Original: Thus, we solve the following problem
Revised: Thus, we solve the following problem

Original: Problem 1.
Revised: Problem 1.

Original: Let π θ denote a trained policy network with L hidden layers over the training inputoutput space X × T ⊆ R | x 0 | × R | t | and Ψ( y, x 0 ) denote a predicate representing constraints on theoutput y of π θ for the set of inputs of interest x 0 ∈ X r ⊆ X .
Revised: Let π θ denote a trained policy network with L hidden layers over the training inputoutput space X × T ⊆ R | x 0 | × R | t | and Ψ( y, x 0 ) denote a predicate representing constraints on the output y of π θ for the set of inputs of interest x 0 ∈ X r ⊆ X .

Original: NNRepLayer modifies the weights of alayer l ∈ { 1 , · · · , L + 1 } in π θ such that the new network π θ r satisfies Ψ( y, x 0 ) while minimizingthe loss of network E ( θ lw , θ lb ) with respect to its original training set.
Revised: NNRepLayer modiﬁes the weights of a layer l ∈ { 1 , · · · , L + 1 } in π θ such that the new network π θ r satisﬁes Ψ ( y, x 0 ) while minimizing the loss of network E ( θ lw , θ lb ) with respect to its original training set.

Original: Since X r and X are not necessarily convex, we formulate NNRepLayer over a data set{ ( x 0 n , t n ) } Nn =1 ∼ X × T ∪ X r × ˜ T , where ˜ T is the set of original target values of inputs in X r .
Revised: Since X r and X are not necessarily convex, we formulate NNRepLayer over a data set { ( x 0 n , t n ) } Nn =1 ∼ X × T ∪ X r × ˜ T , where ˜ T is the set of original target values of inputs in X r .

Original: The predicate Ψ( x 0 , y ) defined over x 0 ∈
Revised: The predicate Ψ( x 0 , y ) deﬁned over x 0 ∈

Original: X r is not necessarily compatible with the target values in˜ T .
Revised: X r is not necessarily compatible with the target values in ˜ T .

Original: It means that the predicate may bound the NN output for X r input space such that not allowingan inputx
Revised: It means that the predicate may bound the NN output for X r input space such that not allowing an input

Original:  0 ∈ X r to reach its target value in ˜ T .
Revised: x 0 ∈ X r to reach its target value in ˜ T .

Original: It is a natural constraint in many applications.
Revised: It is a natural constraint in many applications.

Original: For instance, due to the safety constraints, we may not allow a NN controller to follow its originalcontrol reference for a given unsafe set of input states.
Revised: For instance, due to the safety constraints, we may not allow a NN controller to follow its original control reference for a given unsafe set of input states.

Original: For a given layer l , we also define E ( θ lw , θ lb )in the form of sum of square loss E ( θ l w , θ lb ) =
Revised: For a given layer l , we also deﬁne E ( θ lw , θ lb ) in the form of sum of square loss E ( θ lw , θ lb ) =

Original: (cid:80) N n =1 ∥
Revised: (cid:80) Nn =1 (cid:107)

Original: y n ( x 0 n , θ l w , θ lb ) − t n ∥ 2 2 , where ∥·∥ 2 denotesthe Euclidean norm.
Revised: y n ( x 0 n , θ lw , θ lb ) − t n (cid:107) 22 , where (cid:107)·(cid:107) 2 denotes the Euclidean norm.

Original: Here, since we only repair the weight and bias terms of target layer l , the lossterm E is a function of θ lw and θ lb , respectively.
Revised: Here, since we only repair the weight and bias terms of target layer l , the loss term E is a function of θ lw and θ lb , respectively.

Original: Hence, the weight and bias terms of all layers exceptthe target layer l are fixed.
Revised: Hence, the weight and bias terms of all layers except the target layer l are ﬁxed.

Original: We define our optimization formulation as follows.
Revised: We deﬁne our optimization formulation as follows.

Original: NNRepLayer Optimization Formulation.
Revised: NNRepLayer Optimization Formulation.

Original: Let π θ be a neural network with L hidden layers,Ψ
Revised: Let π θ be a neural network with L hidden layers, Ψ

Original: ( y, x 0 ) be a predicate, and { ( x 0 n , t n ) }
Revised: ( y, x 0 ) be a predicate, and { ( x 0 n , t n ) }

Original: Nn =1 be an input-output data set sampled from ( X × T ) ∪( X r × ˜ T ) over the sets X , X r , T , and ˜ T all as defined in Problem 1.
Revised: Nn =1 be an input-output data set sampled from ( X × T ) ∪ ( X r × ˜ T ) over the sets X , X r , T , and ˜ T all as deﬁned in Problem 1.

Original: NNRepLayer minimizes theloss (1) by modifying θ lw and θ lb subject to the constraints (2)-(5).
Revised: NNRepLayer minimizes the loss (1) by modifying θ l w and θ lb subject to the constraints (2)-(5).

Original: [Equation]
Revised: (1) min θ lw ,θ lb ,δ,y n , { x in } Li = l , { φ in } Li = l E ( θ lw , θ lb ) + δ , s.t. (2) y n = θ L +1 w x

Original: [Equation]
Revised: Ln + θ L +1 b , (3) x in = R ( θ iw x i − 1 n + θ ib ) , for { i } Li = l (4) Ψ( y n , x 0 n ) , for x 0 n ∈ X r (5) δ ≥ (cid:107) θ lw − θ l,initw (cid:107) ∞ , (cid:107) θ lb − θ l,initb (cid:107) ∞ ≥ 0 .

Original: Here, constraint (2) represents the linearforward pass of network’s last layer.
Revised: Here, constraint (2) represents the linear forward pass of network’s last layer.

Original: Constraint (3) represents the forward pass ofhidden layers starting from the layer l .
Revised: Constraint (3) represents the forward pass of hidden layers starting from the layer l .

Original: Except the weight and bias terms θ w and θ b inthe l th layer,  the weight and bias terms ofthe subsequent layers { ( θ lw , θ lb ) } L +1  l =1 arefixed.
Revised: Except the weight and bias terms of the l th layer, i.e. θ lw and θ lb , the weight and bias terms of the subsequent layers { ( θ iw , θ ib ) } L +1 i = l +1 are ﬁxed.

Original: The sample values of x l − 1 n can beobtained by the weighted sum of the nodesin its previous layers starting from x 0 n forall N samples { n } Nn =1 .
Revised: The sample values of x l − 1 n are obtained by the weighted sum of the nodes in its previous layers starting from x 0 n for all N samples { n } Nn =1 .

Original: The ReLU activation functions R ( z ) = max { 0 , z } aremodeled with a disjunctive inequality constraint [ ϕ = 0 , x  ≤ 0] ∨ [ ϕ = 1  ,s ≤ 0] . Assume ϕ is a Boolean variable andz = x − s , wherex, s ≥ 0  .
Revised: Each ReLU node x l is formulated using Big-M formulation [30, 31] by x l ≥ θ lw x l − 1 n + θ lb , x l ≤ (cid:0) θ lw x l − 1 n + θ lb (cid:1) − lb (1 − φ ) , and x l ≤ ub φ , where x l ∈ [0 , ∞ ) , and φ ∈ { 0 , 1 } determines the activation status of node x l .

Original: We have x = z when ϕ = 1 [-( z ≥ 0 ) and x = 0 when ϕ = 
-]
Revised: The bounds lb, ub ∈ R are known as Big-M coefﬁcients, θ lw x l − 1 [-( z ≥ 0 ) and x = 0 when ϕ = 
-]n + θ lb ∈

Original: ( z = − s < 0 ). Here, xdetermines the output of  ReLU activation function R ( z ) , the disjunctive inequalities are then relaxedas mixed integer algebraic equations by the Big-M method [29].
Revised: [ lb, ub ] , that need to be as tight as possible to improve the performance of MIQP solver. We used Interval Arithmetic (IA) Method [32, 14] to obtain tight bounds for ReLU nodes (read the supplementary materials, Sec. A, for further details on IA).

Original: Constraint (4) is a given predicate on y defined over x 0 ∈ X r .
Revised: Constraint (4) is a given predicate on y deﬁned over x 0 ∈ X r .

Original: NNReplayer addresses the predicatesof the form (cid:87) Cc =1 ψ c ( x 0 , y ) where C represents the number of disjunctive propositions and ψ i is anaffine function of x 0 and y .
Revised: NNReplayer addresses the predicates of the form (cid:87) Cc =1 ψ c ( x 0 , y ) where C represents the number of disjunctive propositions and ψ i is an afﬁne function of x 0 and y .

Original: Finally, constraint (5) bounds the entry-wise max-norm error betweenthe weight and bias terms θ lw and θ lb , and the original θ l,initw and θ l,initb by δ .
Revised: Finally, constraint (5) bounds the entry-wise max-norm error between the weight and bias terms θ lw and θ lb , and the original θ l,initw and θ l,initb by δ .

Original: Considering the quadratic loss function E ( θ lw , θ lb ) and the affine disjunctive forms of Ψ( y n , x 0 n ) and 
Revised: Considering the quadratic loss function E ( θ lw , θ lb ) and the afﬁne disjunctive forms of Ψ( y n , x 0 n ) and R ( θ iw x i − 1 n + θ ib ) , we solve

Original: R ( θ iw x i − 1 n + θ ib ) , we solve NNRepLayer as a Mixed Integer Quadratic Program (MIQP).
Revised: NNRepLayer as a Mixed Integer Quadratic Program (MIQP).

Original: Theorem 1.
Revised: Theorem 1.

Original: Given the predicate Ψ( y, x 0 ) , and the input-output data set { ( x 0 n , t n ) } Nn =1 sampledfrom ( X × T ) ∪ ( X r × ˜ T ) over the sets X , X r , T , and ˜ T as defined in Problem 1, assume that θ lwand θ lb are feasible solutions to NNRepLayer.
Revised: Given the predicate Ψ( y, x 0 ) , and the input-output data set { ( x 0 n , t n ) } Nn =1 sampled from ( X × T ) ∪ ( X r × ˜ T ) over the sets X , X r , T , and ˜ T as deﬁned in Problem 1, assume that θ lw and θ lb are feasible solutions to (1)-(5).

Original: Then, for all input samples x 0 n from { ( x 0 n , t n ) } Nn =1 ,Ψ( π θ r ( x 0 n ) , x 0 n ) is satisfied.
Revised: Then, Ψ( π θ r ( x 0 n ) , x 0 n ) is satisﬁed for all input samples x 0 n .

Original: Proof.
Revised: Proof.

Original: Since the feasible solutions θ lw and θ lb satisfy the hard constraint (4) for the repair data set{ ( x 0 n , t n ) } Nn =1 , Ψ( π θ r ( x 0 n ) , x 0 n ) is satisfied.
Revised: Since the feasible solutions θ lw and θ lb satisfy the hard constraint (4) for the repair data set { ( x 0 n , t n ) } Nn =1 , Ψ( π θ r ( x 0 n ) , x 0 n ) is satisﬁed.

Original: Given Thm.
Revised: Given Thm.

Original: 1, the following Corollary is straightforward.
Revised: 1, the following Corollary is straightforward.

Original: Corollary 1.
Revised: Corollary 1.

Original: Given the predicate Ψ( y, x 0 ) , and the input-output data set { ( x 0 n , t n ) } Nn =1 sampledfrom ( X ×T ) ∪ ( X r × ˜ T ) over the sets X , X r , T , and ˜ T as defined in Problem 1, assume that θ l ∗ w andθ l ∗ b are the optimal solutions to the NNRepLayer.
Revised: Given the predicate Ψ( y, x 0 ) , and the input-output data set { ( x 0 n , t n ) } Nn =1 sampled from ( X ×T ) ∪ ( X r × ˜ T ) over the sets X , X r , T , and ˜ T as deﬁned in Problem 1, assume that θ l ∗ w and θ l ∗ b are the optimal solutions to the NNRepLayer.

Original: Then, for all input samples x 0 n from { ( x 0 n , t n ) } Nn =1 ,Ψ( π θ r ( x 0 n ) , x 0 n ) is satisfied.
Revised: Then, for all input samples x 0 n from { ( x 0 n , t n ) } Nn =1 , Ψ( π θ r ( x 0 n ) , x 0 n ) is satisﬁed.

Original:  Evaluation
Revised:  Evaluation

Original: We explore the applicability of the framework in satisfying the following three types of constraints.
Revised: We explore the applicability of the framework in satisfying the following three types of constraints.

Original: Global constraints that encode global bounds on the network’s output, i.e., y ∈
Revised: Global constraints that encode global bounds on the network’s output, i.e., y ∈

Original: [ y min , y max ] .
Revised: [ y min , y max ] .

Original: Input-output constraints that ensure the network’s output y to stay within a certain bound withrespect to the network’s input x 0 , i.e., { ψ c ( x 0 , y ) ≤ 0 } Cc =0 , where C is the number of constraintsand ψ c is an affine function of x 0 and y .
Revised: Input-output constraints that ensure the network’s output y to stay within a certain bound with respect to the network’s input x 0 , i.e., { ψ c ( x 0 , y ) ≤ 0 } Cc =0 , where C is the number of constraints and ψ c is an afﬁne function of x 0 and y .

Original: Finally, conditional constraints that encode if-thenelse constraints described as { ψ c ( x 0 , y ) ≤ 0 , if x 0 , y ∈ S c } C c =0 , where C specifies the numberof conditions, ψ c is an affine function of x 0 and y , and S c ⊆
Revised: Finally, conditional constraints that encode if-then-else constraints described as { ψ c ( x 0 , y ) ≤ 0 , if x 0 , y ∈ S c } Cc =0 , where C speciﬁes the number of conditions, ψ c is an afﬁne function of x 0 and y , and S c ⊆

Original: X × T .
Revised: X × T .

Original: We designed a numberof experiments to validate that our repair framework can successfully apply these constraints tothe policy network.
Revised: We designed a number of experiments to validate that our repair framework can successfully apply these constraints to the policy network.

Original: Following our motivation, all experiments were performed on the prostheticwalking gait generation task introduced in Fig.
Revised: Following our motivation, all experiments were performed on the prosthetic walking gait generation task introduced in Fig.

Original: Through these experiments we aim to answerthe following questions: (1) Does our method enable the Prosthetic device to address all the threetypes of aforementioned constraints?
Revised: Through these experiments we aim to answer the following questions: (1) Does our method enable the Prosthetic device to address all the three types of aforementioned constraints?

Original: (2) Can the repaired controller be employed in a real walkingscenario successfully?
Revised: (2) Can the repaired controller be employed in a real walking scenario successfully?

Original: (3) How robust is the policy repaired through our technique against theunseen constraint-violating samples?
Revised: (3) How robust is the policy repaired through our technique against the unseen constraint-violating samples?

Original: Experimental Setup.
Revised: Experimental Setup.

Original: In our experiment, we train a policy network π θ forcontrolling a prosthesis, which then undergoes the repair process to ensurecompliance with the safety constraints.
Revised: In our experiment, we train a policy network π θ for controlling a prosthesis, which then undergoes the repair process to ensure compliance with the safety constraints.

Original: To this end, we first train the modelusing an imitation learning [30] strategy.
Revised: To this end, we ﬁrst train the model using an imitation learning [33] strategy.

Original: For data collection, we conducteda study approved by the Institutional Review Board (IRB), in which werecorded the walking gait of a healthy subject without any prosthesis.
Revised: For data collection, we conducted a study approved by the Institutional Review Board (IRB), in which we recorded the walking gait of a healthy subject without any prosthesis.

Original: Walking data included three inertial measurement units (IMUs) mounted via strapsto the upper leg (Femur), lower leg (Shin), and foot.
Revised: Walking data included three inertial measurement units (IMUs) mounted via straps to the upper leg (Femur), lower leg (Shin), and foot.

Original: The IMUs acquired boththe angle and angular velocity of each limb portion in the world coordinateframe at 100Hz.
Revised: The IMUs acquired both the angle and angular velocity of each limb portion in the world coordinate frame at 100Hz.

Original: Ankle angle α a was calculated as a post process from thefoot and lower limb IMUs.
Revised: Ankle angle α a was calculated as a post process from the foot and lower limb IMUs.

Original: We then trained the NN to generate the ankleangle from upper and lower limb IMU sensor values.
Revised: We then trained the NN to generate the ankle angle from upper and lower limb IMU sensor values.

Original: More specifically, the NN model receives the angle and velocity from the upper and lower limbsensors, α ul , ˙ α ul , α ll , ˙ α ll , respectively, to predict the ankle angle α a  which is, later, used as thecontrol parameter for a PD controller on the prosthetic.
Revised: More speciﬁcally, the NN model receives the angle and velocity from the upper and lower limb sensors (network inputs x 0 ), α ul , ˙ α ul , α ll , ˙ α ll , respectively, to predict the ankle angle α a (network output y ) which is, later, used as the control parameter for a PD controller on the prosthetic.

Original: See Fig. 3 for a visualization of the individualsensor readings.
Revised: See Fig.for a visualization of the individual sensor readings.

Original: We used a sliding window of input variables, denoted as dt ( dt = 10 in all ourexperiments), to account for the temporal influence on the control parameter and to accommodatefor noise in the sensor readings.
Revised: We used a sliding window of input variables, denoted as dt ( dt = 10 in all our experiments), to account for the temporal inﬂuence on the control parameter and to accommodate for noise in the sensor readings.

Original: Therefore, the input to the network is dt ×| x 0 | , or more specificallythe current and previous dt sensor readings.
Revised: Therefore, the input to the network is dt × | x 0 | , or more speciﬁcally the current and previous dt sensor readings.

Original: In all experiments, we trained a three-hidden-layer deeppolicy network with 32 ReLU nodes at each hidden layer.
Revised: In all experiments, we trained a three-hidden-layer deep policy network with 32 ReLU nodes at each hidden layer.

Original: After the networks were fully trained weassessed the policy for constraint violations and collected samples for NNRepLayer.
Revised: After the networks were fully trained we assessed the policy for constraint violations and collected samples for NNRepLayer.

Original: We tested NNRepLayer on the last and the second to the last layer of network policy to satisfy the constraintswith a subset of the original training data including both adversarial and non-adversarial samples.
Revised: We tested NNRepLayer on the last and the second to the last layer of network policy to satisfy the constraints with a subset of the original training data including both adversarial and non-adversarial samples.

Original: Inall experiments, we used 150 samples in NNRepLayer and a held out set of size 2000 forthe testing.
Revised: In all experiments, we used 150 samples in NNRepLayer and a held out set of size 2000 for testing.

Original: Finally, the repaired policies to satisfy global and input-output constraints are tested on a prostheticdevice for 10 minutes of walking, see Fig.
Revised: Finally, the repaired policies to satisfy global and input-output constraints are tested on a prosthetic device for 10 minutes of walking, see Fig.

Original: More specifically, the same healthy subject was fittedwith an ankle bypass; a carbon fiber structure molded to the lower limb and constructed such that a prosthetic ankle can be attached to allow the able-bodied subject to walk on the prosthesis.
Revised: More speciﬁcally, the same healthy subject was ﬁtted with an ankle bypass; a carbon ﬁber structure molded to the lower limb and constructed such that a prosthetic ankle can be attached to allow the able-bodied subject to walk on the prosthesis, as shown in Fig.

Revised: The extra weight and off-axis positioning of the device incline the individual towards slower, asymmetrical gaits that generates strides out of the original training distribution [34, 10].

Original: Theparticipant is then asked to walk again for 10 minutes to assess whether constraints are satisfied.
Revised: The participant is then asked to walk again for 10 minutes to assess whether constraints are satisﬁed.

Original: To evaluate if the repair can be generalized to the unseen adversarial samples, we analyze the violation degree .
Revised: To evaluate if the repair can be generalized to the unseen adversarial samples, we analyze the violation degree .

Original: The violation degree is measured as the distance of the network output with respectto the constraint set.
Revised: The violation degree is measured as the distance of the network output with respect to the constraint set.

Original: For each experiment, we explain how this distance between the output andthe constraint set is calculated.
Revised: For each experiment, we explain how this distance between the output and the constraint set is calculated.

Original: We compared our framework with fine-tuning [24] and retraining.
Revised: We compared our framework with retraining, ﬁne-tuning [26, 35, 27, 7], and the patch-based repair method in [4] (REASSURE).

Original: Adversarial samples in the repair data set are hand-labeled for fine-tuning and retraining so that thetarget outputs satisfy the given predicates.
Revised: Adversarial samples in the repair data set are hand-labeled for ﬁne-tuning and retraining so that the target outputs satisfy the given predicates.

Original: In fine-tuning, as proposed in [24, 31], we used the collected adversarial data set to train all the parameters of the original policy by gradient descent usinga small learning rate ( 10 − 4 ).
Revised: In ﬁne-tuning, as proposed in [26, 35], we used the collected adversarial data set to train all the parameters of the original policy by gradient descent using a small learning rate ( 10 − 4 ).

Original: To avoid over-fitting to the adversarial data set, we trained the weightsof the top layer first, and thereafter fine-tuned the remaining layers for a few epochs.
Revised: To avoid over-ﬁtting to the adversarial data set, we trained the weights of the top layer ﬁrst, and thereafter ﬁne-tuned the remaining layers for a few epochs.

Original: The samehand-labeling strategy is applied in retraining, except that a new policy is trained from scratch forall original training samples.
Revised: The same hand-labeling strategy is applied in retraining, except that a new policy is trained from scratch for all original training samples.

Original: In both methods, we trained the policy until all the adversarial samplesin the repair data set satisfy the given predicates on the network’s output.
Revised: In both methods, we trained the policy until all the adversarial samples in the repair data set satisfy the given predicates on the network’s output.

Revised: Our code is available on GitHub: https://github.com/k1majd/NNRepLayer.git .

Original: 4.1 Experiments and Results
Revised: 4.1 Experiments and Results

Original: Global Constraint.
Revised: Global Constraint.

Original: The global constraint ensures that the prosthesis control, i.e., α a , stayswithin a certain range and never outputs an unexpected large value that disturbs the user’swalking balance.
Revised: The global constraint ensures that the prosthesis control, i.e., α a , stays within a certain range and never outputs an unexpected large value that disturbs the user’s walking balance.

Original: Additionally, the prosthetic device we utilize in these scenarios contains aparallel compliant mechanism.
Revised: Additionally, the prosthetic device we utilized in these scenarios contains a parallel compliant mechanism.

Original: As such, either the human subject or the robotic controllercould potentially drive the mechanism into the hard limits, potentially damaging the device.
Revised: As such, either the human subject or the robotic controller could potentially drive the mechanism into the hard limits, potentially damaging the device.

Original: imposing the global constraint of [ − 14 , 24] to the control (shown in red) and bounding the control rate by 2 [deg/s] (shown in black).

Original: The color bar represents the normalized L 2 -distance of each test input to its nearest neighbor in the repair set.

Original: In our walking tests, we therefore specifiedglobal constraints such that the ankle anglestays within the bounds of [ − 14 , 24] [deg] regardless of whether it is driven by the humanor the robot.
Revised: In our walking tests, we therefore speciﬁed global constraints such that the ankle angle stays within the bounds of [ − 14 , 24] [deg] regardless of whether it is driven by the human or the robot.

Original: In simulation experiments, we enforced artificially strict bounds on the ankle angle α a to never exceed α a = 10 [deg] which isa harder bound to satisfy.
Revised: In simulation experiments, we enforced artiﬁcially strict bounds on the ankle angle α a to never exceed α a = 10 [deg] which is a harder bound to satisfy.

Original: We defined the degreeof violation as 0 if α a ∈ [ α mina , α maxa ] , andmin {| α a − α maxa | , | α a − α mina |} , otherwise.
Revised: We deﬁned the degree of violation as 0 if α a ∈ [ α mina , α maxa ] , and min {| α a − α maxa | , | α a − α mina |} , otherwise.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 4 (a), the repaired networkcan successfully satisfy the constraints in thepreviously violating regions while maintainingthe tracking performance of the controller inthe unconstrained regions.
Revised: 4 (a)-(b), the repaired network successfully satisﬁes the constraints in the original faulty regions while maintaining the tracking performance of the controller in the unconstrained regions.

Original: Figure 4 (c) demonstrates that the violation degreeafter repairingthe mid layer stays almost zero for even distantoriginally violating data points.
Revised: Figure 4 (c) demonstrates that the violation degree stays almost zero for even distant originally violating data points after repairing the mid layer.

Original: The red control signal in Fig.
Revised: The red control signal in Fig.

Original: 5 also shows that our method successfully imposes the control bounds [ − 14 , 24] to the  prosthesis walking.
Revised: 5 also shows that our method successfully imposes the control bounds [ − 14 , 24] to the actual prosthesis walking test.

Original: Input-output Constraint.
Revised: Input-output Constraint.

Original: Deep neural networks as highly non-linear function approximators have the ability to change the outputs more rapidly than what is feasible for the robotic prosthesis or for the human subject to accommodate.
Revised: Deep neural networks as highly non-linear function approximators have the ability to change the outputs more rapidly than what is feasible for the robotic prosthesis or for the human subject to accommodate.

Original: Therefore, we propose an additional constraint overthe possible change of control actions from one time-step to thenext.
Revised: Therefore, we propose an additional constraint over the possible change of control actions from one time-step to the next.

Original: This constraint should act to both smooth the control action in the presence of sensor noise, as well as to reduce hardpeaks and oscillations in the control action.
Revised: This constraint should act to both smooth the control action in the presence of sensor noise, as well as to reduce hard peaks and oscillations in the control action.

Original: To capture this constraint as an input-output relationship, we trained the policy network by adding the previous dt control actions { α a ( i ) } t − 1 i =
Revised: To capture this constraint as an input-output relationship, we trained the policy network by adding the previous dt control actions { α a ( i ) } t − 1 i =

Original: t − dt asinputs to the policy network along the values of upper and lowerlimb sensors.
Revised: t − dt as inputs to the policy network along the values of upper and lower limb sensors.

Original: Imposed constraints in this example follow the form| α a ( t ) − α a ( t − 1) | ≤ ∆ α maxa .
Revised: Imposed constraints in this example follow the form | α a ( t ) − α a ( t − 1) | ≤ ∆ α maxa .

Original: In prosthetic walking tests, webounded the control rate by ∆ α maxa = 

Revised: In prosthetic walking tests, we bounded the control rate by ∆ α maxa =

Original: [deg/s], and in our simulations we tested ∆ α maxa = 1 . 5 and ∆ α maxa = 

Revised: [deg/s], and in our simulations we tested ∆ α maxa = 1 . 5 and ∆ α maxa =

Original: [deg/s].
Revised: [deg/s].

Original: Oursimulation results in Fig.
Revised: Our simulation results in Fig.

Original: 7 demonstrate that NNRepLayer satisfies both bounds on the control rate which subsequently results ina smoother control output.
Revised: 6 demonstrate that NNRepLayer satisﬁes both bounds on the control rate which subsequently results in a smoother control output.

Original: It can also be observed that NNRepLayeralso successfully preserves the tracking performance ofcontroller.
Revised: It can also be observed that NNRepLayer successfully preserves the tracking performance of controller.

Original: In this experiment, we defined the violation degree as 0 if the | ∆ α a ( t ) | ≤ ∆ α maxa and| ∆ α a ( t ) − ∆ α maxa | , otherwise.
Revised: In this experiment, we deﬁned the violation degree as 0 if the | ∆ α a ( t ) | ≤ ∆ α maxa and | ∆ α a ( t ) − ∆ α maxa | , otherwise.

Original: Figure 6 (a) demonstrates that the violation degree of NNRepLayeris almost zero by increasing the distance of the test samples from the repair set.
Revised: Figure 7 (a) demonstrates that the violation degree of NNRepLayer is almost zero by increasing the distance of the test samples from the repair set.

Original: Same results are obtained in the  prosthetic walking tests as shown in Fig.
Revised: Same results are obtained in the actual prosthetic walking tests as shown in Fig.

Original: The satisfaction of input-output constraints(black signal) is achieved even for the samples that are distant from the repair set.
Revised: The satisfaction of input-output constraints (black signal) is achieved even for the samples that are distant from the repair set.

Original: Finally, in thisexperiment, we observed that applying NNRepLayer to the last layer does not obtain even a feasiblesolution to the optimization problem (1).
Revised: Finally, in this experiment, applying NNRepLayer to the last layer does not obtain even a feasible solution to the optimization problem (1)-(5).

Original: Conditional Constraint.
Revised: Conditional Constraint.

Original: Depending on the ergonomic needs and medical history of a patient, theattending orthopedic doctor or prosthetist may identify certain body configurations that are harmful,e.g., they may increase the risk of osteoarthritis or musculoskeletal conditions [32, 33].
Revised: Depending on the ergonomic needs and medical history of a patient, the attending orthopedic doctor or prosthetist may identify certain body conﬁgurations that are harmful, e.g., they may increase the risk of osteoarthritis or musculoskeletal conditions [36, 37].

Original: Followingthis rationale, we define a region S of joint angles space that should be avoided.
Revised: Following this rationale, we deﬁne a region S of joint angles space that should be avoided.

Original: An example of sucha region is demonstrated in Fig.
Revised: An example of such a region is demonstrated in Fig.

Original: 8 as a grey box S
Revised: 8 as a grey box S

Original: =
Revised: =

Original: { ( α ul , α a ) | α ul ∈ [ − 2 , − 0 .
Revised: { ( α ul , α a ) | α ul ∈ [ − 2 , − 0 .

Original: 5] , α a ∈ [1 , 3] }in the joint space of ankle and femur angles.
Revised: 5] , α a ∈ [1 , 3] } in the joint space of ankle and femur angles.

Original: To satisfy this constraint the control rate shouldbe tuned such that the joint ankle and femur angles stay out of set S .
Revised: To satisfy this constraint the control rate should be tuned such that the joint ankle and femur angles stay out of set S .

Original: This constraint can bedefined as an if-then-else proposition α ul ∈ [ − 2 , − 0 .
Revised: This constraint can be deﬁned as an if-then-else proposition α ul ∈ [ − 2 , − 0 .

Original: [Equation] [Equation]
Revised: = ⇒ (cid:0) α a ∈ ( −∞ , 1] (cid:1) ∨ (cid:0)

Original: ∈[3 , ∞ ) (cid:1) which can be formulated as the disjunction of linear inequalities on the network’s output.
Revised: α a ∈ [3 , ∞ ) (cid:1) which can be formulated as the disjunction of linear inequalities on the network’s output.

Original: For each given test input and its corresponding output α a , the degree of violation is defined as0 if the corresponding output is inside thebox and its distance  to the box  otherwise.
Revised: For each given test input and its corresponding output α a , the degree of violation is deﬁned as the distance of α a to the box if α a is outside the box, and 0 otherwise.

Original: [Table] Figure 8 demonstrates the output of new policy after repairing with NNRepLayer.
Revised: Figure 8 demonstrates the output of new policy after repairing with NNRepLayer.

Original: As it is shown, our methodavoids the joint ankle and femur angles to enter the unsafe region S .
Revised: As it is shown, our method avoids the joint ankle and femur angles to enter the unsafe region S .

Original: Figure 6 (b) also illustrates low output violation degree for the distant test input samples from therepair input set.
Revised: Figure 7 (b) also illustrates low output violation degree for the distant test input samples from the repair input set.

Original: Finally, wealso observed that repairingthe last layer does not result in a feasible solution.
Revised: Finally, we observed that repairing the last layer does not result in a feasible solution.

Original: Comparison w \ Fine-tuning and Retraining.
Revised: Comparison w \ Fine-tuning, Retraining, and REASSURE.

Original: In eachexperiment, we demonstrated the violation degree  of ourmethod compared with fine-tuning and retraining.
Revised: In each experiment, we demonstrated the violation degree and the control signals of our method compared with ﬁne-tuning, retraining [26, 35, 27, 7], and REASSURE [4]. Comparing to [4], while REASSURE guarantees the satisfaction of constraints in the local repaired linear regions, we showed that this method signiﬁcantly reduces the performance of network in the repaired local regions, see Figures 4 and 6. This method cannot address the input-output constraints given the faulty samples, and it introduces 500 times more faulty samples compared to our technique. REASSURE cannot also accommodate the conditional constraints.

Original: Table1 better illustrates the success of  our method in satisfying the  constraints while maintaining the control performance.
Revised: Unlike REASSURE that guarantees the [Table] satisfaction of constraints for the samples in the same linear region as the repaired samples, our technique only guarantees the satisfaction of constraints for the repaired samples. While we empirically showed the generalizability of our technique in a local neighborhood of the repaired samples, our method does not theoretically guarantee the satisfaction of constraints for the unseen adversarial samples.

Revised: We proposed a sound algorithm in the supplementary materials, Sec.

Original:  As shown in Table 1, retraining and NNRepLayerboth perform well in maintaining the minimum absoluteerror and the generalization of constraint satisfaction tothe unseen testing samples for global and input-outputconstraints.
Revised: B, that guarantees the safety for all other unseen samples. Table 1 better illustrates the success of our method in satisfying the constraints while maintaining the control performance. As shown in Table 1, retraining and NNRepLayer both perform well in maintaining the minimum absolute error and the generalization of constraint satisfaction to the unseen testing samples for global and input-output constraints.

Original: However,according the Table 1 the satisfaction of if-then-else constraints is challenging for retraining and fine-tuning as the repair efficacy isdropped by almost 30% using these techniques.
Revised: However, the satisfaction of if-then-else constraints is challenging for retraining and ﬁne-tuning as the repair efﬁcacy is dropped by almost 30% using these techniques.

Original: It also shows the power of our technique in generalizing the satisfaction of conditional constraints to the unseen cases.
Revised: It also highlights the power of our technique in generalizing the satisfaction of conditional constraints to the unseen cases.

Revised: For further details on the comparison results, read the supplementary materials, Sec.

Original: Limitations & Conclusions
Revised: C.  Conclusion & Discussion

Original: In this paper, we introduced an algorithm for training safe neural network controllers that satisfy aformal set of safety constraints.
Revised: In this paper, we introduced an algorithm for training safe neural network controllers that satisfy a formal set of safety constraints.

Original: Our approach, NNRepLayer, performs a global optimization stepin order to perform layer-wise repair of neural network weights.
Revised: Our approach, NNRepLayer, performs a global optimization step in order to perform layer-wise repair of neural network weights.

Original: In real-robot experiments, we haveshown that the introduced methodology produces safe neural policies for a lower-leg prosthesissatisfying a variety of constraints.
Revised: In real-robot experiments, we have shown that the introduced methodology produces safe neural policies for a lower-leg prosthesis satisfying a variety of constraints.

Original: We argue that this type of approach is critical for human-centricand safety-critical applications of robot learning, e.g., the next-generation of assistive robotics.
Revised: We argue that this type of approach is critical for human-centric and safety-critical applications of robot learning, e.g., the next-generation of assistive robotics.

Original: Limitations: The introduced approach does not generally ensure that for any input the constraintswill be satisfied.
Revised: Discussion. The introduced approach does not generally ensure that for any input the constraints will be satisﬁed.

Original: Instead it guarantees this property for all data points provided at  time of repair.
Revised: Instead it guarantees this property for all data points provided at the time of repair.

Original: Hence, proper care has to be taken to ensure that the repair process involves representative samples ofthe variety of inputs seen in the application domain.
Revised: Hence, proper care has to be taken to ensure that the repair process involves representative samples of the variety of inputs seen in the application domain.

Original: From a computational vantage point, solving the MIQP underlying NNRepLayer is a demanding process which scales with the size of the network.
Revised: From a computational vantage point, solving the MIQP underlying NNRepLayer is a demanding process which scales with the size of the network.

Original: Inour experiments, we successfully repaired NN layers with up to 64 neurons, with global optimizationtaking between multiple minutes and up to 6 hours.
Revised: In our experiments, we successfully repaired NN layers with up to 256 neurons, with global optimization taking between multiple minutes and up to 10 hours (read the supplementary materials, Sec. D, for the detailed experimental results). Moreover, we showed in the supplementary materials,

Revised: Sec.

Revised: E, that the repair of randomly selected sub-nodes of a hidden layer can accurately repair the network in much shorter time (more that 12 times faster than the full repair).

Original: Similarly, our approach is limited to repairingindividual layers in a network.
Revised: Finally, our approach is limited to repairing individual layers in a network.

Original: Early results on iteratively repairing multiple layers are promisingand will be reported in the future.
Revised: Early results on iteratively repairing multiple layers are promising and will be reported in the future.

Original: However, our approach cannot simultaneously repair multiplelayers or the entire network. 
Revised: However, our approach cannot simultaneously repair multiple layers or the entire network. Acknowledgments

Original: This restricts its applicability, at the moment, to control tasks withlow- to medium-sized dimensionality, i.e., repair cannot be performed for layer with thousands ofneurons.
Revised: This work was partially supported by the National Science Foundation under grants CNS-1932068, IIS-1749783, and CNS-1932189.