# Semiparametric Deep Learning Manipulator Inverse Dynamics Modeling Method for Smart City and Industrial Applications

Authors: Nan Liu; Liangyu Li; Bing Hao; Liusong Yang; Tonghai Hu; Tao Xue; Shoujun Wang; Xingmao Shao
Journal: Complexity
 (2020)

---

## Abstract

In smart cities and factories, robotic applications require high accuracy and security, which depends on precise inverse dynamics modeling. However, the physical modeling methods cannot include the nondeterministic factors of the manipulator, such as flexibility, joint clearance, and friction. In this paper, the Semiparametric Deep Learning (SDL) method is proposed to model robot inverse dynamics. SDL is a type of deep learning framework, designed for optimal inference, combining the Rigid Body Dynamics (RBD) model and Nonparametric Deep Learning (NDL) model. The SDL model takes advantage of the global characteristics of classic RBD and the powerful fitting capabilities of the deep learning approach. Moreover, the parametric and nonparametric parts of the SDL model can be optimized at the same time instead of being optimized separately. The proposed method is validated using experiments, performed on a UR5 robotic platform. The results show that the performance of SDL model is better than that of RBD model and NDL model. SDL can always provide relatively accurate joint torque prediction, even when the RBD or NDL model is not accurate.


---

## Body

## 1. Introduction

Smart cities and factories contain “intelligent” things that can autonomously and collaboratively enhance the quality of living and working conditions, save human lives, and act as a sustainable resource ecosystem. To implement these advanced collaborative technologies, such as drones, robots, artificial intelligence, and Internet of Things, it is required to increase the “intelligence” of smart cities and factories, by improving the connectivity, energy efficiency, and quality of services [1]. There have been many excellent application cases, such as [2–4]. Particularly, intelligent robotic platforms are a technology, increasingly used, in smart cities and factories, where the constantly changing applications scenarios also place higher demands in robot control. Specifically, in motion control systems, there is a time delay in the transmission of feedback information, making smooth motion impossible to achieve by feedback control alone. Therefore, feedforward control becomes particularly important. In robotics, feedforward control usually refers to model-based control, involving the dynamics of the robotic platform. The accuracy of such dynamical models is critical to the development of control laws that are compliant, energy efficient, and safe [5].There are two major approaches for modeling robot dynamics: parametric and nonparametric. Parametric approaches rely on parameterized Newtonian physics models of the robot dynamics. Common methods for physics-based dynamics modeling can be found in the literature. These methods require the mechanical parameters of the rigid bodies, composing the robot, to be identified [6–9] and then employed in model-based control and state estimation schemes [10]. The advantage of these models is that they represent a global and unique relationship between the joint trajectory q,q˙,q¨ and the torques τRBD. This type of inverse dynamics model can be computed efficiently and employed in real time. Thus, a great deal of prior knowledge is acquired, without the need of data. For example, it is well known that robots are subject to gravitational forces, viscous forces, and joint constraints, making it wasteful to have to go through a laborious data-gathering and machine learning process, to discover these well-known constraints. The disadvantage of parametric models is that they are only crude idealizations of the actual system dynamics, such as rigidity of links or a simple analytical form of friction, which may not be accurate in real systems. In the case of traditional industrial robots, these unmodeled dynamics can often be ignored. However, for modern robotic platforms, these omittances and simplifications result in significant control inefficiencies.Alternatively, the model can be obtained from experimental data, using machine learning techniques, resulting in a nonparametric model. Nonparametric methods, based on algorithms such as Support Vector Regression (SVR) [11–13], Neural Network (NN) [14–16], Local Weighted Projection Regression (LWPR) [17–19], Independent Joint Learning (IJL) [20–22], or Gaussian Processes Regression (GPR) [23–27], can model dynamics by extrapolating the input-output relationship directly from the available data. If a suitable kernel function or learning architecture is selected, then the nonparametric model is a universal approximator which can account for the dynamics factors, not considered by the parametric model. Therefore, nonparametric methods can be more flexible to use and are powerful in capturing higher order nonlinearities, resulting in faster model approximation and higher learning accuracy. When learning inverse dynamics, the nonparametric methods will approximate a function describing the relationship q,q˙,q¨⟶τ, including all nonlinearities encoded by the sampled data.Nonparametric methods attempt to learn the model from scratch and, thus, do not make use of any knowledge available from analytical robotics. Nevertheless, nonparametric learning methods also exhibit several drawbacks. First, very large amounts of data are necessary for obtaining a sufficiently accurate model and predictions on the entire input space [28]. Second, since nonparametric models rely on local neighborhood training data to make predictions, they do not generalize well to unexplored state regions, where little or no training data are available. Covering the entire state space becomes exponentially harder, as the complexity and number of degrees of freedom in the robot system increase. Thus, if only small and relatively poor data sets are available, nonparametric models will not be able to generalize well for unknown data. Third, it is indeed wasteful to have to go through a laborious data-gathering and machine learning process to discover such well-known prior knowledge as Rigid Body Dynamics.Thus, it appears quite desirable to combine the benefits of parametric and nonparametric approaches to improve on the aforementioned issues. However, doing so, in an efficient way, is not trivial. A reasonable approach would be to first fit a parametric model and then fit a nonparametric model to the errors made by the parametric model. Nguyen-Tuong et al. [29] present a learning technique which combines prior knowledge about the physical structure of the mechanical system and learning from available data using Gaussian Process Regression (GPR) [30]. Similar approaches are presented in [20] and [31]. In [32], an incremental semiparametric robot dynamics learning scheme, based on Locally Weighted Projection Regression (LWPR), initialized using a linearized parametric model, is presented [33]. However, this approach uses a fixed parametric model that is not updated, as new data become available. Moreover, LWPR has been shown to underperform with respect to other methods (e.g., [34]). These semiparametric methods, as described above, could not benefit from simultaneous optimization of parametric and nonparametric models. Instead, the nonparametric model is applied, after parametric identification, which may result in a suboptimal model. In addition, as far as it can be known, there is no semiparametric method based on deep learning methods.Deep learning is a new approach in machine learning, which has been widely applied in smart cities and factories [35]. Deep learning has turned out to be very good at discovering intricate structures in high-dimensional data and is therefore applicable to many domains of science, business, and government. Since it requires very little engineering by hand, it can easily take advantage of increased amount of available computation resources and data [36]. In this work, a method that is based on deep learning and semiparametric approach is presented. The method is formalized in the framework of what is called Semiparametric Deep Learning (SDL), designed for optimal inference using combinations of parametric RBD and Nonparametric Deep Learning models. Key properties of this method are (1) appropriate deep learning frame for a semiparametric approach and (2) features that can be optimized simultaneously for parametric and nonparametric models. The proposed method is validated using experiments performed on a UR5 robot. The article is organized as follows. In Section 2, a complete description of the proposed Semiparametric Deep Learning framework is introduced. Section 3 presents the validation of the proposed method on the UR5 robotic platform. Finally, Section 4 summarizes the content of the presented work.


## 2. Methodology

The parametric modeling method and NDL, as the basis of the SDL method, have been elaborated in previous research publications [37, 38]. Therefore, this section only briefly reviews the above two methods, while it analyzes the SDL modeling method, proposed in this paper.


### 2.1. Parametric Robot Dynamics Model

It is well known that the robot dynamics can be modeled according to the following [39]:(1)τq,q˙,q¨=Mqq¨+Cq,q˙+Gq+εq,q˙,q¨,in which q,q˙,q¨∈ℝm×1 are joint positions, velocities, and accelerations of the robot, respectively, τ∈ℝm×1 denotes the joint torques, Mq is the generalized inertia matrix of the robot, Cq,q˙ are the Coriolis and centripetal forces, and Gq is gravity. As shown in equation (1), the robot dynamics equation contains Rigid Body Dynamics (RBD) model:(2)τRBD=Mqq¨+Cq,q˙+Gq.The model errorsεq,q˙,q¨ are caused by unmodeled dynamics (e.g., hydraulic tubes, actuator dynamics, and flexibility and dynamics of the cable drives), ideal-joint assumptions (e.g., no friction and clearance), and inaccuracies in the RBD model parameters. The RBD model of a manipulator is well known to be linear regarding the parameters β [39], i.e.,(3)τRBD=φq,q˙,q¨β,in which φ is a matrix containing nonlinear functions of joint angles, velocities, and accelerations, often called basis functions. Modeling the robot dynamics, using the RBD model in equation (3), requires the identification of the dynamics parameters β. For the 6 Degree-of-Freedom (DoF) UR5 robot, for example, 60 dynamics parameters are to be identified (for each DoF, there are 10 parameters that could ideally be obtained directly from the CAD data).


### 2.2. Nonparametric Deep Learning Model

#### 2.2.1. NDL Formulation

The inverse dynamic model, in model-based control, is described as the mapping from joint positions, velocities, and accelerations to torques, as shown in equation (1). The aim of nonparametric learning model is to predict the torque value of the ith joint, τi∈ℝ, as the response of the query point at q,q˙,q¨∈ℝ3m×1, by using the given n training data, qj,q˙j,q¨j,τijj=1n, in which qj,q˙j,q¨j∈ℝm×1 and τij∈ℝ. Since the problem can be considered as a supervised learning problem, any supervised learning technique can be employed for the learning process, as shown in(4)τq,q˙,q¨∼Dℒq,q˙,q¨.Deep learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but nonlinear modules, each transforming the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. With the composition of enough such transformations, very complex robotic dynamics functions can be learned [36, 40]. Therefore, the application cases of deep learning are too numerous, such as [41–45].The sequential nature of manipulator inverse dynamics suggests that, to predict the joint torque, it is important to model the relationship among sequential data points [39]. RNNs, a type of deep learning network, can be seen as very deep feedforward network, where all the layers share the same weights. Although their main purpose is to determine long-term dependencies, theoretical and empirical evidence shows that it is difficult to learn to store information for very long time [46]. LSTM networks have subsequently proved to be more effective than conventional RNNs, especially when there are several layers at each time step [48], enabling an entire speech recognition system that goes all the way from acoustics to the sequence of characters in the transcription. LSTM networks or related forms of gated units are also currently used for the encoder and decoder networks, performing very well in machine translation [47–49]. Moreover, studies presented in [38] and [50] confirmed the validity of applying LSTM to the prediction of manipulator inverse dynamics. Therefore, in this section, the LSTM network is proposed as the nonparametric learning technique for modeling the inverse dynamics of manipulator.


#### 2.2.2. NDL Model Architecture

In this paper, the proposed architecture of the Nonparametric Deep Learning network has one input layer, one LSTM layer, one full-connected layer, one dropout layer, and one output layer, as shown in Figure1. Only 1 LSTM layer is used here, because it has been verified in our previous studies that, with the same number of neurons, the fewer the layers, the better the prediction performance [38].Figure 1
Proposed deep learning architecture.The input layer has 18 neurons (manipulator’s 6 joint positions, 6 velocities, and 6 accelerations, as shown in Figure2).Figure 2
Inverse dynamics learning architecture of manipulator with 21 input data sets.The state activation functions of LSTM cells are set to “tanh,” while the gate activation functions are set to “sigmoid.” The input weights are initialized according to the Glorot initializer. The forget gate bias is initially set to 1 and the remaining biases are set to 0. The training algorithm adopts back-propagation through time.


### 2.3. Proposed Semiparametric Deep Learning (SDL) Model

In this section, the proposed semiparametric model, based on deep learning and RBD, is described in detail. The method is formalized in the framework of the so-called Semiparametric Deep Learning (SDL), which can be used to predict the joint torque of a robotic arm more accurately. First, formulation of SDL is introduced in Section2.3.1, while the specific model architecture of SDL is described in Section 2.3.2.


#### 2.3.1. SDL Formulation

Using the Nonparametric Deep Learning (DL) framework, the robot dynamics can be modeled byτ∼Dℒq,q˙,q¨, in which q,q˙,q¨ is the input and τ is the output of the Deep Learning Model. Consequently, the DL model does not make use of any prior knowledge, which allows reproducing arbitrary functions. One way to include the RBD model, as shown in equation (5), is to set the τRBD as input to the DL model. This approach is equivalent to a semiparametric model:(5)τq,q˙,q¨∼SDℒq,q˙,q¨∼DℒτRBD,q,q˙,q¨∼Dℒφq,q˙,q¨β,q,q˙,q¨.The resulting dynamics model, as described in equation (5), is a semiparametric frame which consists of a parametric part, i.e., the RBD model. When comparing equation (5) to the robot dynamics in equation (1), it becomes evident that the main purpose of the nonparametric term is absorbing the unmodeled dynamics εq,q˙,q¨. In order to approximate the unmodeled dynamics with an appropriate DL, a model can be used, such as DNN or LSTM. Key properties of this semiparametric learning method are (1) employment of appropriate deep learning frame as nonparametric part and (2) features that can be optimized simultaneously for parametric and nonparametric model; i.e., the value of τRBD will also be updated by weight changes as the learning process evolves.If the RBD model perfectly describes the robot dynamics, the errorεq,q˙,q¨ in equation (1) will disappear and the prediction will depend only on the RBD part; i.e., it is very easy to train deep learning networks. Equation (5) also shows that if the query point is far away from the training data, the resulting torque prediction will mainly depend on the RBD part. This property is important, as the complete state space can never be completely included using finite (and possibly small) training data sets. If the robot moves to the regions of the state space, not considered by the sampled data (i.e., the learned nonparametric models may not generalize well in these state space regions), the torque prediction will rely on the parametric RBD part.


#### 2.3.2. SDL Model Architecture

The effectiveness of the NDL model has been verified in previous work presented in [40]. The SDL model proposed in this article adds an RBD term to the NDL architecture. Its specific model architecture is shown in Figure 3. The input of the SDL architecture is still q,q˙,q¨. However, unlike NDL, the input passes through the RBD term, forming the new input vector τRBD,q,q˙,q¨. Next, the new input vector enters the LSTM hidden layer. The advantage of this architecture is that the parametric and nonparametric parts can be optimized simultaneously; i.e., the weight of τRBD in the feature vector will also be updated during network training. The grid search method is used to optimize the hyperparameters of the SDL model.Figure 3
Proposed deep learning architecture.


## 2.1. Parametric Robot Dynamics Model

It is well known that the robot dynamics can be modeled according to the following [39]:(1)τq,q˙,q¨=Mqq¨+Cq,q˙+Gq+εq,q˙,q¨,in which q,q˙,q¨∈ℝm×1 are joint positions, velocities, and accelerations of the robot, respectively, τ∈ℝm×1 denotes the joint torques, Mq is the generalized inertia matrix of the robot, Cq,q˙ are the Coriolis and centripetal forces, and Gq is gravity. As shown in equation (1), the robot dynamics equation contains Rigid Body Dynamics (RBD) model:(2)τRBD=Mqq¨+Cq,q˙+Gq.The model errorsεq,q˙,q¨ are caused by unmodeled dynamics (e.g., hydraulic tubes, actuator dynamics, and flexibility and dynamics of the cable drives), ideal-joint assumptions (e.g., no friction and clearance), and inaccuracies in the RBD model parameters. The RBD model of a manipulator is well known to be linear regarding the parameters β [39], i.e.,(3)τRBD=φq,q˙,q¨β,in which φ is a matrix containing nonlinear functions of joint angles, velocities, and accelerations, often called basis functions. Modeling the robot dynamics, using the RBD model in equation (3), requires the identification of the dynamics parameters β. For the 6 Degree-of-Freedom (DoF) UR5 robot, for example, 60 dynamics parameters are to be identified (for each DoF, there are 10 parameters that could ideally be obtained directly from the CAD data).


## 2.2. Nonparametric Deep Learning Model

### 2.2.1. NDL Formulation

The inverse dynamic model, in model-based control, is described as the mapping from joint positions, velocities, and accelerations to torques, as shown in equation (1). The aim of nonparametric learning model is to predict the torque value of the ith joint, τi∈ℝ, as the response of the query point at q,q˙,q¨∈ℝ3m×1, by using the given n training data, qj,q˙j,q¨j,τijj=1n, in which qj,q˙j,q¨j∈ℝm×1 and τij∈ℝ. Since the problem can be considered as a supervised learning problem, any supervised learning technique can be employed for the learning process, as shown in(4)τq,q˙,q¨∼Dℒq,q˙,q¨.Deep learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but nonlinear modules, each transforming the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. With the composition of enough such transformations, very complex robotic dynamics functions can be learned [36, 40]. Therefore, the application cases of deep learning are too numerous, such as [41–45].The sequential nature of manipulator inverse dynamics suggests that, to predict the joint torque, it is important to model the relationship among sequential data points [39]. RNNs, a type of deep learning network, can be seen as very deep feedforward network, where all the layers share the same weights. Although their main purpose is to determine long-term dependencies, theoretical and empirical evidence shows that it is difficult to learn to store information for very long time [46]. LSTM networks have subsequently proved to be more effective than conventional RNNs, especially when there are several layers at each time step [48], enabling an entire speech recognition system that goes all the way from acoustics to the sequence of characters in the transcription. LSTM networks or related forms of gated units are also currently used for the encoder and decoder networks, performing very well in machine translation [47–49]. Moreover, studies presented in [38] and [50] confirmed the validity of applying LSTM to the prediction of manipulator inverse dynamics. Therefore, in this section, the LSTM network is proposed as the nonparametric learning technique for modeling the inverse dynamics of manipulator.


### 2.2.2. NDL Model Architecture

In this paper, the proposed architecture of the Nonparametric Deep Learning network has one input layer, one LSTM layer, one full-connected layer, one dropout layer, and one output layer, as shown in Figure1. Only 1 LSTM layer is used here, because it has been verified in our previous studies that, with the same number of neurons, the fewer the layers, the better the prediction performance [38].Figure 1
Proposed deep learning architecture.The input layer has 18 neurons (manipulator’s 6 joint positions, 6 velocities, and 6 accelerations, as shown in Figure2).Figure 2
Inverse dynamics learning architecture of manipulator with 21 input data sets.The state activation functions of LSTM cells are set to “tanh,” while the gate activation functions are set to “sigmoid.” The input weights are initialized according to the Glorot initializer. The forget gate bias is initially set to 1 and the remaining biases are set to 0. The training algorithm adopts back-propagation through time.


## 2.2.1. NDL Formulation

The inverse dynamic model, in model-based control, is described as the mapping from joint positions, velocities, and accelerations to torques, as shown in equation (1). The aim of nonparametric learning model is to predict the torque value of the ith joint, τi∈ℝ, as the response of the query point at q,q˙,q¨∈ℝ3m×1, by using the given n training data, qj,q˙j,q¨j,τijj=1n, in which qj,q˙j,q¨j∈ℝm×1 and τij∈ℝ. Since the problem can be considered as a supervised learning problem, any supervised learning technique can be employed for the learning process, as shown in(4)τq,q˙,q¨∼Dℒq,q˙,q¨.Deep learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but nonlinear modules, each transforming the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. With the composition of enough such transformations, very complex robotic dynamics functions can be learned [36, 40]. Therefore, the application cases of deep learning are too numerous, such as [41–45].The sequential nature of manipulator inverse dynamics suggests that, to predict the joint torque, it is important to model the relationship among sequential data points [39]. RNNs, a type of deep learning network, can be seen as very deep feedforward network, where all the layers share the same weights. Although their main purpose is to determine long-term dependencies, theoretical and empirical evidence shows that it is difficult to learn to store information for very long time [46]. LSTM networks have subsequently proved to be more effective than conventional RNNs, especially when there are several layers at each time step [48], enabling an entire speech recognition system that goes all the way from acoustics to the sequence of characters in the transcription. LSTM networks or related forms of gated units are also currently used for the encoder and decoder networks, performing very well in machine translation [47–49]. Moreover, studies presented in [38] and [50] confirmed the validity of applying LSTM to the prediction of manipulator inverse dynamics. Therefore, in this section, the LSTM network is proposed as the nonparametric learning technique for modeling the inverse dynamics of manipulator.


## 2.2.2. NDL Model Architecture

In this paper, the proposed architecture of the Nonparametric Deep Learning network has one input layer, one LSTM layer, one full-connected layer, one dropout layer, and one output layer, as shown in Figure1. Only 1 LSTM layer is used here, because it has been verified in our previous studies that, with the same number of neurons, the fewer the layers, the better the prediction performance [38].Figure 1
Proposed deep learning architecture.The input layer has 18 neurons (manipulator’s 6 joint positions, 6 velocities, and 6 accelerations, as shown in Figure2).Figure 2
Inverse dynamics learning architecture of manipulator with 21 input data sets.The state activation functions of LSTM cells are set to “tanh,” while the gate activation functions are set to “sigmoid.” The input weights are initialized according to the Glorot initializer. The forget gate bias is initially set to 1 and the remaining biases are set to 0. The training algorithm adopts back-propagation through time.


## 2.3. Proposed Semiparametric Deep Learning (SDL) Model

In this section, the proposed semiparametric model, based on deep learning and RBD, is described in detail. The method is formalized in the framework of the so-called Semiparametric Deep Learning (SDL), which can be used to predict the joint torque of a robotic arm more accurately. First, formulation of SDL is introduced in Section2.3.1, while the specific model architecture of SDL is described in Section 2.3.2.


### 2.3.1. SDL Formulation

Using the Nonparametric Deep Learning (DL) framework, the robot dynamics can be modeled byτ∼Dℒq,q˙,q¨, in which q,q˙,q¨ is the input and τ is the output of the Deep Learning Model. Consequently, the DL model does not make use of any prior knowledge, which allows reproducing arbitrary functions. One way to include the RBD model, as shown in equation (5), is to set the τRBD as input to the DL model. This approach is equivalent to a semiparametric model:(5)τq,q˙,q¨∼SDℒq,q˙,q¨∼DℒτRBD,q,q˙,q¨∼Dℒφq,q˙,q¨β,q,q˙,q¨.The resulting dynamics model, as described in equation (5), is a semiparametric frame which consists of a parametric part, i.e., the RBD model. When comparing equation (5) to the robot dynamics in equation (1), it becomes evident that the main purpose of the nonparametric term is absorbing the unmodeled dynamics εq,q˙,q¨. In order to approximate the unmodeled dynamics with an appropriate DL, a model can be used, such as DNN or LSTM. Key properties of this semiparametric learning method are (1) employment of appropriate deep learning frame as nonparametric part and (2) features that can be optimized simultaneously for parametric and nonparametric model; i.e., the value of τRBD will also be updated by weight changes as the learning process evolves.If the RBD model perfectly describes the robot dynamics, the errorεq,q˙,q¨ in equation (1) will disappear and the prediction will depend only on the RBD part; i.e., it is very easy to train deep learning networks. Equation (5) also shows that if the query point is far away from the training data, the resulting torque prediction will mainly depend on the RBD part. This property is important, as the complete state space can never be completely included using finite (and possibly small) training data sets. If the robot moves to the regions of the state space, not considered by the sampled data (i.e., the learned nonparametric models may not generalize well in these state space regions), the torque prediction will rely on the parametric RBD part.


### 2.3.2. SDL Model Architecture

The effectiveness of the NDL model has been verified in previous work presented in [40]. The SDL model proposed in this article adds an RBD term to the NDL architecture. Its specific model architecture is shown in Figure 3. The input of the SDL architecture is still q,q˙,q¨. However, unlike NDL, the input passes through the RBD term, forming the new input vector τRBD,q,q˙,q¨. Next, the new input vector enters the LSTM hidden layer. The advantage of this architecture is that the parametric and nonparametric parts can be optimized simultaneously; i.e., the weight of τRBD in the feature vector will also be updated during network training. The grid search method is used to optimize the hyperparameters of the SDL model.Figure 3
Proposed deep learning architecture.


## 2.3.1. SDL Formulation

Using the Nonparametric Deep Learning (DL) framework, the robot dynamics can be modeled byτ∼Dℒq,q˙,q¨, in which q,q˙,q¨ is the input and τ is the output of the Deep Learning Model. Consequently, the DL model does not make use of any prior knowledge, which allows reproducing arbitrary functions. One way to include the RBD model, as shown in equation (5), is to set the τRBD as input to the DL model. This approach is equivalent to a semiparametric model:(5)τq,q˙,q¨∼SDℒq,q˙,q¨∼DℒτRBD,q,q˙,q¨∼Dℒφq,q˙,q¨β,q,q˙,q¨.The resulting dynamics model, as described in equation (5), is a semiparametric frame which consists of a parametric part, i.e., the RBD model. When comparing equation (5) to the robot dynamics in equation (1), it becomes evident that the main purpose of the nonparametric term is absorbing the unmodeled dynamics εq,q˙,q¨. In order to approximate the unmodeled dynamics with an appropriate DL, a model can be used, such as DNN or LSTM. Key properties of this semiparametric learning method are (1) employment of appropriate deep learning frame as nonparametric part and (2) features that can be optimized simultaneously for parametric and nonparametric model; i.e., the value of τRBD will also be updated by weight changes as the learning process evolves.If the RBD model perfectly describes the robot dynamics, the errorεq,q˙,q¨ in equation (1) will disappear and the prediction will depend only on the RBD part; i.e., it is very easy to train deep learning networks. Equation (5) also shows that if the query point is far away from the training data, the resulting torque prediction will mainly depend on the RBD part. This property is important, as the complete state space can never be completely included using finite (and possibly small) training data sets. If the robot moves to the regions of the state space, not considered by the sampled data (i.e., the learned nonparametric models may not generalize well in these state space regions), the torque prediction will rely on the parametric RBD part.


## 2.3.2. SDL Model Architecture

The effectiveness of the NDL model has been verified in previous work presented in [40]. The SDL model proposed in this article adds an RBD term to the NDL architecture. Its specific model architecture is shown in Figure 3. The input of the SDL architecture is still q,q˙,q¨. However, unlike NDL, the input passes through the RBD term, forming the new input vector τRBD,q,q˙,q¨. Next, the new input vector enters the LSTM hidden layer. The advantage of this architecture is that the parametric and nonparametric parts can be optimized simultaneously; i.e., the weight of τRBD in the feature vector will also be updated during network training. The grid search method is used to optimize the hyperparameters of the SDL model.Figure 3
Proposed deep learning architecture.


## 3. Evaluation

The proposed SDL method will be verified on a collaborative robot UR5, while the torque prediction results will be compared to the ones provided by NDL methods. The prediction performance for training and for generating predictions in rhythmic motor tasks is evaluated. The joint angles, the joint velocities, the joint accelerations, and the joint torques are recorded using a GUI (Graphical User Interface), i.e., PolyScope, making it easy to program the robot to move the tool along a desired trajectory path.


### 3.1. Experimental Setup

UR5 is a 6-DoF collaboration robot with extruded aluminum tubes and joints. It has six rotary joints, and its structure is shown in Figure4. The UR5 robot has a joint rotation range of [−2π, 2π] (rads) and a joint acceleration range of [0, π] (rads/s2). The UR5 robot is very popular in the robot research field.Figure 4
UR5 robot structure.According to the robot rotation angle and installation restrictions, the robot workspace is selected to be a hemisphere with an approximate radius of 850 mm above the installation plane. The range of joints motion is shown in Table1. In order to best approximate the actual working situation of the robot, within the selected robot workspace, 1000 points are randomly selected. According to the actual use requirements of the robot, the joint running speed range is [0.8–2] rads/s, while the acceleration range is [1–1.8] rads/s.Table 1
Rotation angle range of each joint in the selected robot workspace.
Joint #Joint 1Joint 2Joint 3Joint 4Joint 5Joint 6Range of motion (deg)[−180, 180][−180, 0][−130, 130][−180, 0][−180, 180][−180, 180]The robot is ordered to run in a rhythmic way, according to the set trajectory. The joint position, speed, and servo motor current data, along the robot trajectory, is delivered from the robot controller at a frequency of 100 Hz. Since the UR5 robot is not equipped with a torque sensor, the measured torque is obtained indirectly through the motor current, at each joint. The relationship between torque and current is as follows [51]:(6)τi=NikiIi,in which Ni is the gear ratio, Ni=101,i=1,⋯,6; ki motor constant, ki=0.125Nm/A,i=1,⋯,3, ki=0.0922Nm/A,i=4,⋯,6; and Ii is the motor current (A).During the actual operation, the robot is affected by noise, making the sampling data fluctuate. In that case, there will be large fluctuations and ripples in the actual measured current, which seriously affects the accuracy of the torque prediction. The average data method can increase the signal-to-noise ratio of the data [52], reduce the influence of noise, and improve the prediction accuracy. The average joint position q˙ can be expressed as follows:(7)q¯k=1M∑m=1Mqmk,in which M is the number of running trajectories, qmk is the k-th sampling point of one running trajectory, and q¯k is the position after M times of averaging. The same method is used to deal with speed and current. The zero-phase low-pass Butterworth filter (forward and reverse IIR Butterworth filters) with a cutoff low-pass frequency of 1 Hz is used to process the averaged position and velocity (q¯,q˙¯). Acceleration is obtained by the central difference method [53]. Data processing is important for the accuracy of parameter identification, making the above processing very appropriate, as it avoids large deviations in identification.A total of 1,000 groups (a total of 108,008) of valid samples were obtained. According to the ratio of 80% to 20%, they were divided into training and testing sets, serving inK-fold cross-validation. In addition, the test data is ensured to be sufficiently different from the training data, highlighting the generalization ability of the learned models. The above sample set is used to train and test the NDL model and SDL model, respectively, while it also analyzes and compares the prediction performance.All input and output data are normalized to match the consistency of the learning model. After the prediction of inverse dynamics, the real value is restored. The normalized equation is as follows:(8)xn=xr−xminxmax−xmin,in which xn represents the normalized value; xr denotes the real value; and xmin and xmax are the minimum and maximum real values, respectively.The performance of manipulator inverse dynamics predictions is evaluated using Root Mean Square Error (RMSE), which is defined as follows:(9)RMSE=1N∑i=1Npi−yi2,in which pi and yi represent the i-th predicted value and real value, respectively, and N is the total number of the data sets.


## 3.1. Experimental Setup

UR5 is a 6-DoF collaboration robot with extruded aluminum tubes and joints. It has six rotary joints, and its structure is shown in Figure4. The UR5 robot has a joint rotation range of [−2π, 2π] (rads) and a joint acceleration range of [0, π] (rads/s2). The UR5 robot is very popular in the robot research field.Figure 4
UR5 robot structure.According to the robot rotation angle and installation restrictions, the robot workspace is selected to be a hemisphere with an approximate radius of 850 mm above the installation plane. The range of joints motion is shown in Table1. In order to best approximate the actual working situation of the robot, within the selected robot workspace, 1000 points are randomly selected. According to the actual use requirements of the robot, the joint running speed range is [0.8–2] rads/s, while the acceleration range is [1–1.8] rads/s.Table 1
Rotation angle range of each joint in the selected robot workspace.
Joint #Joint 1Joint 2Joint 3Joint 4Joint 5Joint 6Range of motion (deg)[−180, 180][−180, 0][−130, 130][−180, 0][−180, 180][−180, 180]The robot is ordered to run in a rhythmic way, according to the set trajectory. The joint position, speed, and servo motor current data, along the robot trajectory, is delivered from the robot controller at a frequency of 100 Hz. Since the UR5 robot is not equipped with a torque sensor, the measured torque is obtained indirectly through the motor current, at each joint. The relationship between torque and current is as follows [51]:(6)τi=NikiIi,in which Ni is the gear ratio, Ni=101,i=1,⋯,6; ki motor constant, ki=0.125Nm/A,i=1,⋯,3, ki=0.0922Nm/A,i=4,⋯,6; and Ii is the motor current (A).During the actual operation, the robot is affected by noise, making the sampling data fluctuate. In that case, there will be large fluctuations and ripples in the actual measured current, which seriously affects the accuracy of the torque prediction. The average data method can increase the signal-to-noise ratio of the data [52], reduce the influence of noise, and improve the prediction accuracy. The average joint position q˙ can be expressed as follows:(7)q¯k=1M∑m=1Mqmk,in which M is the number of running trajectories, qmk is the k-th sampling point of one running trajectory, and q¯k is the position after M times of averaging. The same method is used to deal with speed and current. The zero-phase low-pass Butterworth filter (forward and reverse IIR Butterworth filters) with a cutoff low-pass frequency of 1 Hz is used to process the averaged position and velocity (q¯,q˙¯). Acceleration is obtained by the central difference method [53]. Data processing is important for the accuracy of parameter identification, making the above processing very appropriate, as it avoids large deviations in identification.A total of 1,000 groups (a total of 108,008) of valid samples were obtained. According to the ratio of 80% to 20%, they were divided into training and testing sets, serving inK-fold cross-validation. In addition, the test data is ensured to be sufficiently different from the training data, highlighting the generalization ability of the learned models. The above sample set is used to train and test the NDL model and SDL model, respectively, while it also analyzes and compares the prediction performance.All input and output data are normalized to match the consistency of the learning model. After the prediction of inverse dynamics, the real value is restored. The normalized equation is as follows:(8)xn=xr−xminxmax−xmin,in which xn represents the normalized value; xr denotes the real value; and xmin and xmax are the minimum and maximum real values, respectively.The performance of manipulator inverse dynamics predictions is evaluated using Root Mean Square Error (RMSE), which is defined as follows:(9)RMSE=1N∑i=1Npi−yi2,in which pi and yi represent the i-th predicted value and real value, respectively, and N is the total number of the data sets.


## 4. Results

The training and prediction in this paper were performed with MATLAB 2019a, using an ordinary personal computer. Computer hardware has a high influence on training time. In this work, the models are trained on a CPU with a clock speed of 2.7 GHz. The structure and hyperparameters of the NDL and SDL models were initially set according to previous work, while the final settings were determined based on the fivefold cross-validation method.The prediction results of the RBD, NDL, and SDL models are listed in Table2. The data in the table shows the RMSE of the RBD, NDL, and SDL models, on different robot axes and under different cross situations. Based on Table 2, the following conclusions apply:(1)
The results of “cross average” (Figure5) indicate that the prediction accuracy of SDL is generally more accurate than the one of NDL and NDL is generally more accurate than RBD. Also, the prediction accuracy of the SDL model for the first 3 axes of the robot is significantly improved.(2)
The results of “all axes” (Figure6) indicate that the prediction accuracy of the NDL model is sometimes better than the one of the RBD model (cross 3, 4, 5), while sometimes it is worse (cross 1, 2). However, the NDL model is better than the RBD model (mean) in general, while the prediction accuracy of the SDL model is always better than in the case of the other two models.(3)
All the data in Table2 shows that the semiparametric models are able to combine the strengths of both models, i.e., the parametric RBD model and the Nonparametric Deep Learning model. The prediction accuracy of the SDL model is always better than that of the RBD and NDL models.Table 2
Prediction results of RBD, NDL, and SDL models.
Axis 1Axis 2Axis 3Axis 4Axis 5Axis 6All axesCross 1RBD2.28992.34172.36830.65230.58890.52901.7024NDL2.61072.77151.82270.34180.44870.45771.7486SDL1.83101.94601.35040.34500.42200.45241.2560Cross 2RBD2.33162.31622.28930.66420.64980.55251.6937NDL2.33422.54392.54280.38640.51620.47221.7807SDL1.67211.84921.45280.34350.46330.46541.2162Cross 3RBD2.48612.67232.21260.64170.59240.35821.7846NDL2.24772.48201.16890.40810.44870.39771.4779SDL1.40311.55330.96560.32940.40220.34280.9748Cross 4RBD2.73252.73232.51190.66250.63860.37081.9246NDL2.32392.17141.63740.37450.42450.42801.4889SDL1.40091.68161.16130.34350.43990.36961.0478Cross 5RBD2.80112.79462.27410.63260.57950.36201.9015NDL1.67682.29891.66900.43460.38930.33931.3746SDL1.53261.68471.00040.33850.37750.30581.0439Cross averageRBD2.52822.57142.33130.65060.60980.43451.8014NDL2.23872.45351.76820.38910.44550.41901.5741SDL1.56791.74301.18610.34000.42100.38721.1077Figure 5
Cross average prediction results.Figure 6
All axes prediction results.RMSE is not sufficient to fully represent the performance of torque prediction, because the range of torque variation for each joint of the robot varies greatly. Therefore, the ratio of cross average RMSE to the range of measured torque values was used to further analyze the predictive performance of different joints. As shown in Table3, the torque prediction performance of the first three (elbow) joints is better than that of the last three (wrist) joints.Table 3
Comparative analysis of prediction performance of different joint forces.
Axis 1Axis 2Axis 3Axis 4Axis 5Axis 6Cross average RMSERBD2.52822.57142.33130.65060.60980.4345NDL2.23872.45351.76820.38910.44550.4190SDL1.56791.74301.18610.34000.42100.3872Range of measured torque42.181698.120749.31035.77664.49446.6976RatioRBD0.05990.02620.04730.11260.13570.0649NDL0.05310.02500.03590.06740.09910.0626SDL0.03720.01780.02410.05890.09370.0578Part of the measured values of each joint torque, RBD predicted value, NDL predicted value, and SDL predicted value are all plotted in Figure7. Figures 7(a)–7(f) are the predicted moments of axis 1 to axis 6, respectively; Figures 7(g)–7(l) are the predicted errors of moments of axis 1 to axis 6, respectively; the purple curve represents the measured torque value, the blue represents the calculated torque value using RBD method, the red represents the DL predicted torque value, and the yellow represents the SDL predicted torque value. Figure 7 shows that the SDL model combines the advantages of the RBD model and the NDL model. For example, as shown by the red dashed box in Figure 7, when the torque prediction error of the RBD is large, the nonparametric part works, which greatly improves the prediction accuracy of the SDL model. As another example, shown by the green dotted box in Figure 7, when the learned nonparametric models do not generalize well to the state space regions, the torque prediction will rely on the parametric RBD part.Figure 7
Joint torque prediction comparison curves. (a) Axis 1 torque. (g) Axis 1 torque error. (b) Axis 2 torque. (h) Axis 2 torque error. (c) Axis 3 torque. (i) Axis 3 torque error. (d) Axis 4 torque. (j) Axis 4 torque error. (e) Axis 5 torque. (k) Axis 5 torque error. (f) Axis 6 torque. (l) Axis 6 torque error.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)


## 5. Conclusion

In this work, Semiparametric Deep Learning (SDL) method is proposed to model robot inverse dynamics, for smart city and industrial applications. The SDL model takes advantage of the global characteristics of classic RBD and the powerful fitting capabilities of deep learning methods. Moreover, SDL model can be optimized simultaneously for the parametric and nonparametric model, instead of separate optimizations. The results on the UR5 robot show that the SDL models provide higher accuracy and better generalization, compared to RBD and NDL. The essence of the SDL model is to fully utilize the a priori information encoded in the parameterized model, overcome the limitations of the NDL method, and always show good learning performance. As far as future work is concerned, the flexible factors in dynamics will be considered as addition to the SDL model, in order to improve the accuracy prediction performance.



---

*Source: 9053715-2020-06-30.xml*