# Data-Based Control for Humanoid Robots Using Support Vector Regression, Fuzzy Logic, and Cubature Kalman Filter

Authors: Liyang Wang; Ming Chen; Gai Li; Yongqing Fan
Journal: Mathematical Problems in Engineering
 (2016)

---

## Abstract

Time-varying external disturbances cause instability of humanoid robots or even tip robots over. In this work, a trapezoidal fuzzy least squares support vector regression- (TF-LSSVR-) based control system is proposed to learn the external disturbances and increase the zero-moment-point (ZMP) stability margin of humanoid robots. First, the humanoid states and the corresponding control torques of the joints for training the controller are collected by implementing simulation experiments. Secondly, a TF-LSSVR with a time-related trapezoidal fuzzy membership function (TFMF) is proposed to train the controller using the simulated data. Thirdly, the parameters of the proposed TF-LSSVR are updated using a cubature Kalman filter (CKF). Simulation results are provided. The proposed method is shown to be effective in learning and adapting occasional external disturbances and ensuring the stability margin of the robot.


---

## Body

## 1. Introduction

In general, lots of experiences are needed to turn the parameters of the controller for humanoid walking robots [1, 2]. At the same time, the turned parameters could be out of operation once external disturbances occurred [3, 4]. It is still a giant challenge for humanoid robots to walk autonomously in disturbed environments. How to improve the antidisturbance ability of humanoid robots using the data online and offline is an interesting problem to be settled.Traditionally, an accurate dynamic model of the considered robot should be built in order to implement the desired high-quality control [5, 6]. The dynamic model of humanoid robots is a set of strong coupling nonlinear ordinary differential equations about the joint variables. When humanoid robots walk slowly, the coupling among the joints can be treated as disturbances. In this case, proportion integration differentiation (PID) control for each independent joint can be adopted. Additionally, inverse dynamics feed-forward control, decoupling control, and feedback linearization control also can be considered. All the above-mentioned methods have a common feature that they are dependent on the established mathematical model strongly. The control effect could be great if the system model is known exactly. However, these methods need to be improved in practical applications because there always exist external disturbances and the system model cannot be known exactly [7, 8]. Due to the cumulative errors in disturbances, when the robot is tracking the walking pattern planned in advance, the difference between actual motion states and target values will increase rapidly. As a consequence, humanoid robots always fall down after a few steps. To keep stable and sustained humanoid walking, the control systems must be improved to cope with the disturbances.To realize the stable walking of humanoid robots, researchers proposed some effective methods, such as stability controls based on linear inverted pendulum model, stability controls based on ZMP theory, and attitude controls for the upper part of robot. By summarizing the existing humanoid prototypes, ZMP-based methods are the most popular and practical [9–13]. When the humanoid dynamics (the masses of each module, the center of mass, and the moment of inertia) is known exactly, the gait of a humanoid robot can be obtained by solving the ZMP equation. However, it is really difficult to collect all the precise parameters from a physical robot. As a result, several simplified methods are proposed to guarantee the stability criterion of ZMP. For example, Liu et al. designed an effective fuzzy logic (FL) controller for humanoid walking robots using ZMP as one of the antecedents [14]. Inspired by this literature, we try to realize a kind of data-based control with implicit constraints of ZMP.Considering the time-varying external disturbances, it is necessary to deduce the control torques referring to the time-varying states of the humanoid joints. The support vector machine (SVM) is supervised learning methods with associated learning algorithms that analyze data [15–17]. SVMs can be used to solve the classification problems [18, 19] and also the regression problems [20–22]. In this paper, support vector regression (SVR) is used to expresses the scene of solving the regression problems using SVM. The formulation of SVM employs the structural risk minimization (SRM) principle, which has been shown to be superior to the empirical risk minimization (ERM) based on infinite samples. SRM is a technique where nested sets of functions of different complexity, controlled by the regularization term, are considered. One could select then the one which is minimizing the upper bound on the generalization error. This feature makes SVM more efficient in resolving the learning problems with limited training data. To cope with time-varying external disturbances, a time-based fuzzy SVR will be proposed to learn the dynamics between the states and the control torques of each joint in this work.On the other hand, the effectiveness of time-based fuzzy SVR depends on the design of the SVR and the parameters of the fuzzy system. Kalman filter has been used in algorithm studies on the training of neural networks and fuzzy systems. Singhal and Wu [23] demonstrated that the EKF could serve as the basis for training MLP networks by treating the weights of the network as a nonlinear dynamical system. Inspired by the successful application of the Kalman filter for training neural networks [24] and for defuzzification strategies [25], Simon [26] built a nonlinear system to train fuzzy systems using an extended Kalman filter. Recently, there is a research [27] that embedded a third-degree spherical-radial cubature rule into the Bayesian filter to build a kind of new filter named CKF. CKF has demonstrated excellent performance in solving nonlinear filtering problems with minimal computational effort [28–31]. Therefore, we will explore approaches to train the proposed time-based fuzzy SVR using CKF.The contributions of this work can be summarized as follows:(i)
For the first time, a trapezoidal fuzzy SVR is proposed to cope with time-varying external disturbances imposed on humanoid robots.
(ii)
For the first time, a novel approach for training the trapezoidal fuzzy SVR using CKF is presented.The organization of this paper is as follows. In Section2, the backgrounds of humanoid robots, SVR, and CKF are presented. The details for proposed framework are presented in Section 3. Simulation results are provided in Section 4, followed by the conclusions in Section 5.


## 2. Background

### 2.1. Dynamic Balance of Humanoid Robots

The dynamic equations of the single support phase (SSP) can be written as(1)
M
θ
θ
¨
+
C
θ
,
θ
˙
θ
¨
+
G
θ
=
τ
+
τ
d
,where θ
∈
R
n is the generalized coordinate and M
(
θ
)
∈
R
n
×
n is the inertia matrix. C
(
θ
,
θ
˙
)
∈
R
n
×
n denotes the matrix of centripetal acceleration and Coriolis terms, G
(
θ
)
∈
R
n
×
n is the gravity vector, and τ
∈
R
n denotes the input torque vector during the SSP. The external disturbances are represented by τ
d
∈
R
n.The dynamic equations of the double support phase (DSP) can be written as(2)
M
θ
θ
¨
+
C
θ
,
θ
˙
θ
¨
+
G
θ
+
J
T
θ
F
=
τ
+
τ
d
,where J
T
(
θ
)
∈
R
l
×
n is a Jacobian matrix and F
∈
R
l is the force vector of constraints caused by the contact with ground.To analyze the stability of the humanoid motions, the ZMP theory is used as the criterion of dynamic humanoid balance in this work. The concept of zero moment point (ZMP) has been applied to many famous humanoid robots successfully, such as ASIMO [11] of Honda. ZMP is a well-known concept introduced in 1990 [32], which is a point on the ground at which the net moment of the inertial forces and the gravity forces has no component along the horizontal axes. At a given time instant, dynamic balance of legged systems is ensured if the ZMP is inside the support area.


### 2.2. Support Vector Regression

In this section, we briefly review the basis of the theory of SVR.Given a labeled training data setx
i
,
y
i  i
=
1,2
,
…
,
N,  x
i
∈
R
n is the input vector of the system and y
i
∈
R is the output vector. The basic idea of SVR is mapping the input space into a higher dimension feature space using the nonlinear mapping function and searching the optimal linear regression function in this feature space. Objective function of the least squares SVR (LS-SVR) [17] is (3)
m
i
n
1
2
ω
T
ω
+
1
2
C
∑
i
=
1
N
ξ
i
2
s
.
t
.
y
i
=
ω
T
ϕ
x
i
+
b
+
ξ
i
,
i
=
1
,
…
,
N
,where C is a positive real constant for tuning. The error of the regression becomes smaller when the value of C is smaller. ω is a weight vector and b is a bias; ξ
i is the positive slack variable enabling dealing with permitted errors. To solve this optimization problem we construct the Lagrangian(4)
L
ω
,
b
,
ξ
,
α
=
1
2
ω
T
ω
+
C
∑
i
=
1
N
ξ
i
2
-
∑
i
=
1
N
α
i
ω
ϕ
x
i
+
b
+
ξ
i
-
y
i
,and we find the saddle point of L
(
ω
,
b
,
ξ
,
α
), where x
i
∈
R
n is the input vector of the system and y
i
∈
R is the output vector. α
=
[
α
1
,
α
2
,
…
,
α
N
]
T is a vector of the Lagrange multipliers. The parameters must satisfy the following conditions:(5)
∂
L
∂
ω
=
0
⟶
ω
=
∑
i
=
1
N
α
i
ϕ
x
i
,
∂
L
∂
b
=
0
⟶
∑
i
=
1
N
α
i
=
0
,
∂
L
∂
ξ
=
0
⟶
α
i
=
C
ξ
i
,
∂
L
∂
α
=
0
⟶
ω
ϕ
x
i
+
b
+
ξ
i
-
y
i
=
0
.Eliminating ω and ξ, problem (3) can be transformed into(6)
0
A
T
A
Ω
+
C
-
1
I
b
α
=
0
y
.where y
=
[
y
1
,
y
2
,
…
,
y
N
]
T,  A
=
[
1,1
,
…
,
1
]
T,  α
=
[
α
1
,
α
2
,
…
,
α
N
]
T,  Ω is a square matrix, which has elements of Ω
i
=
K
(
x
i
,
x
)
=
ϕ
(
x
i
)
T
ϕ
(
x
), and K
(
x
i
,
x
) is a kernel function. For example, K
(
x
i
,
x
)
=
e
x
p
[
-
(
x
i
-
x
)
2
/
2
σ
2
] denotes a radial basis function (RBF) kernel. Submitting the optimal α and b, the regression function of the least squares SVR [17] is(7)
y
x
=
∑
i
=
1
N
α
i
K
x
i
,
x
+
b
.


### 2.3. Cubature Kalman Filter

To describe the CKF, we consider the filtering problem of a nonlinear dynamic system with additive noise, whose state space model is defined by a process equation and a measurement equation in discrete time:(8)
x
k
=
f
x
k
-
1
,
u
k
-
1
+
w
k
-
1
,
z
k
=
g
x
k
,
u
k
+
v
k
,where x
k is the state of the dynamic system at discrete time k; f
(
·
) and g
(
·
) are some known functions; u
k is the control input; z
k is the measurement; {
w
k
-
1
} and {
v
k
} are independent process and measurement Gaussian noise sequences with zero means and covariances Q
k
-
1 and R
k, respectively.When we deal with a problem of state estimation using a nonlinear filtering, the integrals for the means and variances of the states can be expressed as the form of a Gaussian-weighted integral. Consider a Gaussian-weighted integral of the following form:(9)
I
f
=
∫
U
n
f
x
e
-
x
T
x
d
x
,where f
(
x
) is an arbitrary function and U
n is the region of integration. There are different integration methods to derive different nonlinear filters.For a CKF, the spherical-radial cubature rule is adopted to implement the integration. Letx
=
r
t
(
t
T
t
=
1
,
r
∈
[
0
,
+
∞
)
), and integration (9) can be separated into radial integration and spherical integration. That is,(10)
R
=
∫
0
∞
S
r
r
n
-
1
e
-
r
2
d
r
,
S
r
=
∫
S
n
f
r
t
d
ρ
t
,where S
n is the surface of the sphere defined by ρ
∈
S
n. Using Lagrange integration, the radial integration can be rewritten as (11)
R
≈
1
2
Π
n
2
S
n
2
,where Π
(
n
)
=
∫
0
∞
x
n
-
1
e
-
x
d
x. Using a cubature rule of degree three, the spherical integration can be rewritten as(12)
S
r
≈
∑
i
=
1
2
n
π
n
n
·
Π
n
/
2
f
1
i
,where 1
i denotes the ith column of set 1. For example, 1
=
(
1,0
T
,
-
1,0
T
,
0
,
-
1
T
,
0,1
T
) when n
=
2.Combining (11) and (12), the spherical-radial cubature rule is as follows [27]:(13)
I
F
≈
π
n
2
n
∑
i
=
1
2
n
f
n
2
1
i
.For standard Gaussian distribution, (13) can be rewritten as(14)
I
N
f
≈
∫
R
n
f
x
N
x
;
0
,
I
d
x
=
1
π
n
∫
R
n
f
2
x
e
-
x
T
x
d
x
,where N
(
x
;
0
,
I
) denotes the Gaussian density of x with mean 0 and covariance I. Combining (9) and (14), we get(15)
I
N
f
≈
1
2
n
∑
i
=
1
2
n
f
2
n
2
1
i
=
∑
i
=
1
2
n
β
i
f
η
i
,where n is the dimension of the state vector. The point (
η
i
,
β
i
) is called cubature point here. η
i and β
i can be calculated as follows:(16)
η
i
=
2
n
2
1
i
,
β
i
=
1
2
n
,
i
=
1,2
,
…
,
2
n
.This means that, for the third-degree spherical-radial rule, it entails a total of 2
n cubature points. After calculating the cubature points, we use the cubature-point set to numerically compute the integrals and obtain the CKF algorithm; details of time update and measurement update can be found in the literature [27].


## 2.1. Dynamic Balance of Humanoid Robots

The dynamic equations of the single support phase (SSP) can be written as(1)
M
θ
θ
¨
+
C
θ
,
θ
˙
θ
¨
+
G
θ
=
τ
+
τ
d
,where θ
∈
R
n is the generalized coordinate and M
(
θ
)
∈
R
n
×
n is the inertia matrix. C
(
θ
,
θ
˙
)
∈
R
n
×
n denotes the matrix of centripetal acceleration and Coriolis terms, G
(
θ
)
∈
R
n
×
n is the gravity vector, and τ
∈
R
n denotes the input torque vector during the SSP. The external disturbances are represented by τ
d
∈
R
n.The dynamic equations of the double support phase (DSP) can be written as(2)
M
θ
θ
¨
+
C
θ
,
θ
˙
θ
¨
+
G
θ
+
J
T
θ
F
=
τ
+
τ
d
,where J
T
(
θ
)
∈
R
l
×
n is a Jacobian matrix and F
∈
R
l is the force vector of constraints caused by the contact with ground.To analyze the stability of the humanoid motions, the ZMP theory is used as the criterion of dynamic humanoid balance in this work. The concept of zero moment point (ZMP) has been applied to many famous humanoid robots successfully, such as ASIMO [11] of Honda. ZMP is a well-known concept introduced in 1990 [32], which is a point on the ground at which the net moment of the inertial forces and the gravity forces has no component along the horizontal axes. At a given time instant, dynamic balance of legged systems is ensured if the ZMP is inside the support area.


## 2.2. Support Vector Regression

In this section, we briefly review the basis of the theory of SVR.Given a labeled training data setx
i
,
y
i  i
=
1,2
,
…
,
N,  x
i
∈
R
n is the input vector of the system and y
i
∈
R is the output vector. The basic idea of SVR is mapping the input space into a higher dimension feature space using the nonlinear mapping function and searching the optimal linear regression function in this feature space. Objective function of the least squares SVR (LS-SVR) [17] is (3)
m
i
n
1
2
ω
T
ω
+
1
2
C
∑
i
=
1
N
ξ
i
2
s
.
t
.
y
i
=
ω
T
ϕ
x
i
+
b
+
ξ
i
,
i
=
1
,
…
,
N
,where C is a positive real constant for tuning. The error of the regression becomes smaller when the value of C is smaller. ω is a weight vector and b is a bias; ξ
i is the positive slack variable enabling dealing with permitted errors. To solve this optimization problem we construct the Lagrangian(4)
L
ω
,
b
,
ξ
,
α
=
1
2
ω
T
ω
+
C
∑
i
=
1
N
ξ
i
2
-
∑
i
=
1
N
α
i
ω
ϕ
x
i
+
b
+
ξ
i
-
y
i
,and we find the saddle point of L
(
ω
,
b
,
ξ
,
α
), where x
i
∈
R
n is the input vector of the system and y
i
∈
R is the output vector. α
=
[
α
1
,
α
2
,
…
,
α
N
]
T is a vector of the Lagrange multipliers. The parameters must satisfy the following conditions:(5)
∂
L
∂
ω
=
0
⟶
ω
=
∑
i
=
1
N
α
i
ϕ
x
i
,
∂
L
∂
b
=
0
⟶
∑
i
=
1
N
α
i
=
0
,
∂
L
∂
ξ
=
0
⟶
α
i
=
C
ξ
i
,
∂
L
∂
α
=
0
⟶
ω
ϕ
x
i
+
b
+
ξ
i
-
y
i
=
0
.Eliminating ω and ξ, problem (3) can be transformed into(6)
0
A
T
A
Ω
+
C
-
1
I
b
α
=
0
y
.where y
=
[
y
1
,
y
2
,
…
,
y
N
]
T,  A
=
[
1,1
,
…
,
1
]
T,  α
=
[
α
1
,
α
2
,
…
,
α
N
]
T,  Ω is a square matrix, which has elements of Ω
i
=
K
(
x
i
,
x
)
=
ϕ
(
x
i
)
T
ϕ
(
x
), and K
(
x
i
,
x
) is a kernel function. For example, K
(
x
i
,
x
)
=
e
x
p
[
-
(
x
i
-
x
)
2
/
2
σ
2
] denotes a radial basis function (RBF) kernel. Submitting the optimal α and b, the regression function of the least squares SVR [17] is(7)
y
x
=
∑
i
=
1
N
α
i
K
x
i
,
x
+
b
.


## 2.3. Cubature Kalman Filter

To describe the CKF, we consider the filtering problem of a nonlinear dynamic system with additive noise, whose state space model is defined by a process equation and a measurement equation in discrete time:(8)
x
k
=
f
x
k
-
1
,
u
k
-
1
+
w
k
-
1
,
z
k
=
g
x
k
,
u
k
+
v
k
,where x
k is the state of the dynamic system at discrete time k; f
(
·
) and g
(
·
) are some known functions; u
k is the control input; z
k is the measurement; {
w
k
-
1
} and {
v
k
} are independent process and measurement Gaussian noise sequences with zero means and covariances Q
k
-
1 and R
k, respectively.When we deal with a problem of state estimation using a nonlinear filtering, the integrals for the means and variances of the states can be expressed as the form of a Gaussian-weighted integral. Consider a Gaussian-weighted integral of the following form:(9)
I
f
=
∫
U
n
f
x
e
-
x
T
x
d
x
,where f
(
x
) is an arbitrary function and U
n is the region of integration. There are different integration methods to derive different nonlinear filters.For a CKF, the spherical-radial cubature rule is adopted to implement the integration. Letx
=
r
t
(
t
T
t
=
1
,
r
∈
[
0
,
+
∞
)
), and integration (9) can be separated into radial integration and spherical integration. That is,(10)
R
=
∫
0
∞
S
r
r
n
-
1
e
-
r
2
d
r
,
S
r
=
∫
S
n
f
r
t
d
ρ
t
,where S
n is the surface of the sphere defined by ρ
∈
S
n. Using Lagrange integration, the radial integration can be rewritten as (11)
R
≈
1
2
Π
n
2
S
n
2
,where Π
(
n
)
=
∫
0
∞
x
n
-
1
e
-
x
d
x. Using a cubature rule of degree three, the spherical integration can be rewritten as(12)
S
r
≈
∑
i
=
1
2
n
π
n
n
·
Π
n
/
2
f
1
i
,where 1
i denotes the ith column of set 1. For example, 1
=
(
1,0
T
,
-
1,0
T
,
0
,
-
1
T
,
0,1
T
) when n
=
2.Combining (11) and (12), the spherical-radial cubature rule is as follows [27]:(13)
I
F
≈
π
n
2
n
∑
i
=
1
2
n
f
n
2
1
i
.For standard Gaussian distribution, (13) can be rewritten as(14)
I
N
f
≈
∫
R
n
f
x
N
x
;
0
,
I
d
x
=
1
π
n
∫
R
n
f
2
x
e
-
x
T
x
d
x
,where N
(
x
;
0
,
I
) denotes the Gaussian density of x with mean 0 and covariance I. Combining (9) and (14), we get(15)
I
N
f
≈
1
2
n
∑
i
=
1
2
n
f
2
n
2
1
i
=
∑
i
=
1
2
n
β
i
f
η
i
,where n is the dimension of the state vector. The point (
η
i
,
β
i
) is called cubature point here. η
i and β
i can be calculated as follows:(16)
η
i
=
2
n
2
1
i
,
β
i
=
1
2
n
,
i
=
1,2
,
…
,
2
n
.This means that, for the third-degree spherical-radial rule, it entails a total of 2
n cubature points. After calculating the cubature points, we use the cubature-point set to numerically compute the integrals and obtain the CKF algorithm; details of time update and measurement update can be found in the literature [27].


## 3. Data-Based Control for Humanoid Walking Robots

External disturbance is one of the key issues which influence the stability of humanoid walking robots. On the other side, it is difficult to measure the external disturbances directly. Based on these facts, we turn away to focus attention on the data of the humanoid states and the corresponding control torques because the pattern of the disturbances can be disclosed using the varying data collected from the humanoid robots to some extent.First of all, the states and the control torques of the humanoid joints are collected from a simulated stable walking robot. Then, a data-based controller considering the varying states of the humanoid robot is designed using SVR and fuzzy theory. To optimize the controller, a CKF is designed to train the parameters of the SVR and the fuzzy system. The complete framework for data-based humanoid control using SVR, FL, and CKF is shown in Figure1.Figure 1
Data-based control for humanoid walking robots using support vector regression, fuzzy logic, and cubature Kalman filter.


### 3.1. Data Collecting from the Simulations

Data for training the controller is collected by implementing simulation experiments. Two kinds of data are collected from the simulated humanoid robot, including the joint angles and the driving torques. The way we get the training data is described in detail next.First of all, data of the joint angles are generated from reference trajectory planned offline. The trajectories can be represented as follows:(17)
x
a
i
=
s
π
2
π
i
N
-
sin
⁡
2
π
i
N
,
z
a
i
=
d
π
1
-
cos
⁡
2
π
i
N
,
x
h
i
=
x
a
i
2
+
s
2
,
z
h
i
=
z
a
i
2
+
l
t
h
+
l
s
h
-
d
2
,where x
h, z
h represent the position of the hip and x
a, z
a represent the position of the swinging ankle joint. s denotes the walking step length, and d denotes the height of swinging ankle. N denotes the total number of samples for a step, i denotes the index of the samples, and l
t
h, l
s
h represent the length of lower limbs.Secondly, a proportion integration differentiation (PID) controller is used to obtain the driving torques. In this work, the initial driving torques of all the joints are obtained using this PID controller. Then the key driving torques, including driving torques for the support hip and support ankle in the SSP and driving torques for the knees in the DSP, are improved using SVR, FL, and CKF. The PID controller is as follows:(18)
τ
j
=
P
j
e
j
+
I
j
∫
0
T
e
j
d
t
+
D
j
d
e
j
d
t
,where τ
j  (
j
=
1
,
…
,
n
) is the torque of the joints. e
j denotes the offset of the desired reference trajectories and the actual trajectories. T is the integral period. The proportional gains P
j, integral gains I
j, and differential gains D
j are slightly modified by the trial-error method.


### 3.2. Designing the Controller Using the Collected Data

A TF-LSSVR is proposed in this section to design the controller using the collected data.


#### 3.2.1. Humanoid Controller to Be Built Using the TF-LSSVR

Based on the existing literature [14], when the ZMP criterion is satisfied, the dynamics between the torque control inputs and the joint angles of the DSP can be presented as(19)
τ
left
_
knee
=
h
θ
left
_
knee
,
θ
right
_
knee
,
τ
right
_
knee
=
l
θ
left
_
knee
,
θ
right
_
knee
,where τ
left
_
knee and τ
right
_
knee are the driven torques of the left knee and the right knee. θ
left
_
knee and θ
right
_
knee are the joint angles of the left knee and the right knee. h
(
·
) and l
(
·
) are the nonlinear dynamics in the DSP to be learned using the proposed TF-LSSVR.On the other side, when the ZMP criterion is satisfied, the dynamics between the torque control inputs and the joint angles of the SSP can be presented as [14](20)
τ
s
u
p
_
h
i
p
=
f
θ
s
u
p
_
h
i
p
,
θ
s
u
p
_
a
n
k
l
e
,
τ
s
u
p
_
a
n
k
l
e
=
g
θ
s
u
p
_
h
i
p
,
θ
s
u
p
_
a
n
k
l
e
,where τ
s
u
p
_
h
i
p and τ
s
u
p
_
ankle are the driven torques of the supporting hip and the supporting ankle. θ
s
u
p
_
hip and θ
s
u
p
_
ankle are the joint angles of the supporting hip and the supporting ankle. f
(
·
) and g
(
·
) are the nonlinear dynamics in the SSP to be learned using the proposed TF-LSSVR.For illustration purposes, we will expound the situation in the SSP mainly. The solution for the DSP can be easily deduced in the same way. Then, the humanoid controller to be built can be simply presented as follows:(21)
τ
=
G
Θ
,where G
(
·
) is the nonlinear dynamics that the TF-LSSVR tries to build. τ is the driving torque, and Θ
=
(
θ
s
u
p
_
hip
,
θ
s
u
p
_
ankle
) is a vector of joint angles.


#### 3.2.2. Objective Functions of the TF-LSSVR

When the timeliness of the training data is considered, the collected data from current steps are “more important,” and those from past steps are “less important.” Based on this, in the proposed objective functions of the TF-LSSVR, a TFMF is used to design learning weights of the collected data. Taking the learning of driving torque of the supporting hip as an example and designing a learning weightr
k for each of the training samples, the training sample set can be denoted by {
θ
s
u
p
_
hip
(
k
)
,
θ
s
u
p
_
ankle
(
k
)
,
τ
s
u
p
_
hip
(
k
)
,
r
k
}, 0
≤
r
k
≤
1, k
=
1,2
,
…
,
N. Then the objective function of the TF-LSSVR for training the driving torque of the supporting hip is(22)
m
i
n
1
2
ω
T
ω
+
1
2
C
r
k
∑
k
=
1
N
ξ
k
2
s
.
t
.
τ
s
u
p
_
hip
k
=
ω
T
ϕ
θ
s
u
p
_
hip
k
,
θ
s
u
p
_
ankle
k
+
b
s
u
p
_
hip
+
ξ
k
,
k
=
1
,
…
,
N
,where ω is a weight vector. ϕ
(
·
) is a nonlinear mapping function. C is a penalty coefficient, ξ
k is a positive slack variable, and b
s
u
p
_
hip is the corresponding bias. {
θ
s
u
p
_
hip
(
k
)
,
θ
s
u
p
_
ankle
(
k
)
,
τ
(
k
)
,
r
k
} is the kth sample, and N is the number of the samples. The r
k is the fuzzy learning weight for the training samples, which can be calculated using the fuzzy membership functions designed in the next section. It is noted that in literature [17] also a weighting scheme similar to (22) has been proposed but it is based on robust statistics. In this paper, a different weighting scheme is proposed.


#### 3.2.3. Designing the Learning Weights Using TFMF

The walking data from different time can be evaluated using some linguistic terms such as “more important” or “less important.” When the timeliness of the training data is considered, the collected data from current steps are “more important,” and those from past steps are “less important.”For this reason, a TFMF, which is the left half of a trapezoidal function, is proposed as the membership function of humanoid walking samples. The proposed time-related trapezoidal fuzzy membership function is shown in Figure2. The formula for the trapezoidal fuzzy membership function is as follows:(23)
r
k
=
1
-
h
t
k
-
m
-
t
k
-
m
-
n
t
+
t
k
-
m
h
-
t
k
-
m
-
n
t
k
-
m
-
t
k
-
m
-
n
,
t
∈
t
k
-
m
-
n
,
t
k
-
m
,
1
,
t
∈
t
k
-
m
,
t
k
,where r
k is the trapezoidal fuzzy membership function of time t
k  (
k
=
1,2
,
…
,
N
), r
k
∈
[
h
,
1
], and h
∈
(
0,1
) is a tuning parameter. t
k
-
m
-
n and t
k are the beginning and the ending of the time window, respectively, and t
k
-
m is a time point between the beginning and the ending.Figure 2
The trapezoidal fuzzy membership function for humanoid walking samples.From Figure2 and (23), it can be seen that the time window t
k
-
m
-
n
,
t
k is divided into two parts. The first part t
k
-
m
,
t
k contains the newest walking samples, which are designed to have a full fuzzy membership grade. The second part t
k
-
m
-
n
,
t
k
-
m crosses through the relative old samples, which are assigned descending fuzzy membership grades. As shown in Figure 2, the values of the learning weight at points A and B are larger than that of point C because the states A and B are closer to the current situation. Besides, as the sampling time window moves, the trapezoidal learning-weight function moves along the time axis accordingly. Therefore, reasonable and adaptable learning weights are assigned for all samples in the whole walking process. The deduced learning weights are then used in the learning algorithm of the TF-LSSVR.The nonlinear dynamic modelG
(
·
) in (21) can be built when the learning process of the TF-LSSVR is completed. After this, considering the parameters of the TF-LSSVR, controller (21) can be written as (24)
τ
=
G
Θ
,
ψ
,where τ is the driving torque, Θ is the joint angle, and ψ
=
[
C
,
σ
,
m
,
n
,
h
]
T is a parameter vector. Here, we denote the plenty coefficient by C, the width of the RBF kernel by σ, and the width of the moving time window by m and n, and we denote the minimum value of the membership function by h. To obtain optimized parameters for the controller, a CKF-based training method is proposed in the next section.


### 3.3. Training the Parameters of the Controller Using CKF

#### 3.3.1. Parameters Optimization Problem in a Form Suitable for CKF

In order to cast the parameters optimization problem of the TF-LSSVR in a form suitable for CKF, we let the parameters of the TF-LSSVR constitute the state of a nonlinear system, and we let the output of the TF-LSSVR constitute the output of the nonlinear system to which the CKF is applied.As mentioned above we denote the state of the nonlinear system by(25)
ψ
=
C
,
σ
,
m
,
n
,
h
T
.The vector ψ thus consists of all of the parameters for the TF-LSSVR arranged in a linear array. Let τ
=
G
(
Θ
,
ψ
) be the transfer function of the TF-LSSVR, where τ is the output, Θ is the input, and ψ
=
[
C
,
σ
,
m
,
n
,
h
]
T is a parameter vector. The training of the TF-LSSVR can be formulated as a filtering problem. The model for updating the parameters using CKF can be written as(26)
ψ
k
=
ψ
k
-
1
,
τ
k
=
G
Θ
k
,
ψ
k
.Similar to the approach of training fuzzy systems with the extended Kalman filter [26], in order to execute a stable Kalman filter algorithm, we add some process noise and measurement noise to the above model. That is, (27)
ψ
k
=
ψ
k
-
1
+
w
k
-
1
,
τ
k
=
G
Θ
k
,
ψ
k
+
v
k
,where w
k
-
1 and v
k are artificially added Gaussian noise sequences with zero means and covariances Q
k
-
1 and R
k, respectively.


#### 3.3.2. The CKF Algorithm for Updating the Parameters of the Controller

To update the parameters of the controller, the CKF algorithm [27] is implemented as follows.Step 1 (cubature-points calculation).
The point(
η
i
,
β
i
) is called cubature point here. For the third-degree spherical-radial rule, it entails a total of 2
u cubature points. η
i and β
i can be calculated as follows:(28)
η
i
=
u
1
i
,
β
i
=
1
2
u
,where i
=
1,2
,
…
,
2
u and 1
i denotes the ith column of the set 1. For example, 1
=
(
1,0
T
,
-
1,0
T
,
0
,
-
1
T
,
0,1
T
) when u
=
2. u is the dimension of the state vector.Step 2 (time update).
After calculating the cubature points, we use the cubature-point set to numerically compute the integrals and obtain the time update:(29)
ψ
i
,
k
-
1
∣
k
-
1
=
P
k
-
1
∣
k
-
1
η
i
+
ψ
^
k
-
1
∣
k
-
1
,
ψ
i
,
k
∣
k
-
1
∗
=
ψ
i
,
k
-
1
∣
k
-
1
+
w
k
-
1
,
ψ
^
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
∗
,
P
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
∗
ψ
i
,
k
∣
k
-
1
∗
T
-
ψ
^
k
∣
k
-
1
ψ
^
k
∣
k
-
1
T
+
Q
k
-
1
,
ψ
i
,
k
∣
k
-
1
=
P
k
∣
k
-
1
ξ
i
+
ψ
^
k
∣
k
-
1
.Step 3 (measurement update).
Consider(30)
τ
i
,
k
∣
k
-
1
=
G
Θ
i
,
k
k
-
1
,
ψ
i
,
k
∣
k
-
1
+
v
k
,
τ
^
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
τ
i
,
k
∣
k
-
1
,
P
τ
τ
,
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
τ
i
,
k
∣
k
-
1
τ
i
,
k
∣
k
-
1
T
-
τ
^
k
∣
k
-
1
τ
^
k
∣
k
-
1
T
+
R
k
,
P
ψ
τ
,
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
τ
i
,
k
∣
k
-
1
T
-
ψ
^
k
∣
k
-
1
τ
^
k
∣
k
-
1
T
,
W
k
=
P
ψ
τ
,
k
∣
k
-
1
P
τ
τ
,
k
∣
k
-
1
-
1
,
ψ
^
k
∣
k
=
ψ
^
k
∣
k
-
1
+
W
k
τ
k
-
τ
^
k
∣
k
-
1
,
P
k
∣
k
=
P
k
∣
k
-
1
-
W
k
P
τ
τ
,
k
∣
k
-
1
W
k
T
,where W
k is the Kalman gain, ψ
^
k
∣
k is the updated state, and P
k
∣
k is the corresponding error covariance. P
ψ
τ
,
k
∣
k
-
1 is the cross-covariance matrix and P
τ
τ
,
k
∣
k
-
1
-
1 is the inverse matrix of the innovation covariance matrix.After time update and measurement update, an estimation of the parametersψ for the TF-LSSVR is obtained. That is to say, the initial controller with random parameters is updated to the proposed data-based controller.


## 3.1. Data Collecting from the Simulations

Data for training the controller is collected by implementing simulation experiments. Two kinds of data are collected from the simulated humanoid robot, including the joint angles and the driving torques. The way we get the training data is described in detail next.First of all, data of the joint angles are generated from reference trajectory planned offline. The trajectories can be represented as follows:(17)
x
a
i
=
s
π
2
π
i
N
-
sin
⁡
2
π
i
N
,
z
a
i
=
d
π
1
-
cos
⁡
2
π
i
N
,
x
h
i
=
x
a
i
2
+
s
2
,
z
h
i
=
z
a
i
2
+
l
t
h
+
l
s
h
-
d
2
,where x
h, z
h represent the position of the hip and x
a, z
a represent the position of the swinging ankle joint. s denotes the walking step length, and d denotes the height of swinging ankle. N denotes the total number of samples for a step, i denotes the index of the samples, and l
t
h, l
s
h represent the length of lower limbs.Secondly, a proportion integration differentiation (PID) controller is used to obtain the driving torques. In this work, the initial driving torques of all the joints are obtained using this PID controller. Then the key driving torques, including driving torques for the support hip and support ankle in the SSP and driving torques for the knees in the DSP, are improved using SVR, FL, and CKF. The PID controller is as follows:(18)
τ
j
=
P
j
e
j
+
I
j
∫
0
T
e
j
d
t
+
D
j
d
e
j
d
t
,where τ
j  (
j
=
1
,
…
,
n
) is the torque of the joints. e
j denotes the offset of the desired reference trajectories and the actual trajectories. T is the integral period. The proportional gains P
j, integral gains I
j, and differential gains D
j are slightly modified by the trial-error method.


## 3.2. Designing the Controller Using the Collected Data

A TF-LSSVR is proposed in this section to design the controller using the collected data.


### 3.2.1. Humanoid Controller to Be Built Using the TF-LSSVR

Based on the existing literature [14], when the ZMP criterion is satisfied, the dynamics between the torque control inputs and the joint angles of the DSP can be presented as(19)
τ
left
_
knee
=
h
θ
left
_
knee
,
θ
right
_
knee
,
τ
right
_
knee
=
l
θ
left
_
knee
,
θ
right
_
knee
,where τ
left
_
knee and τ
right
_
knee are the driven torques of the left knee and the right knee. θ
left
_
knee and θ
right
_
knee are the joint angles of the left knee and the right knee. h
(
·
) and l
(
·
) are the nonlinear dynamics in the DSP to be learned using the proposed TF-LSSVR.On the other side, when the ZMP criterion is satisfied, the dynamics between the torque control inputs and the joint angles of the SSP can be presented as [14](20)
τ
s
u
p
_
h
i
p
=
f
θ
s
u
p
_
h
i
p
,
θ
s
u
p
_
a
n
k
l
e
,
τ
s
u
p
_
a
n
k
l
e
=
g
θ
s
u
p
_
h
i
p
,
θ
s
u
p
_
a
n
k
l
e
,where τ
s
u
p
_
h
i
p and τ
s
u
p
_
ankle are the driven torques of the supporting hip and the supporting ankle. θ
s
u
p
_
hip and θ
s
u
p
_
ankle are the joint angles of the supporting hip and the supporting ankle. f
(
·
) and g
(
·
) are the nonlinear dynamics in the SSP to be learned using the proposed TF-LSSVR.For illustration purposes, we will expound the situation in the SSP mainly. The solution for the DSP can be easily deduced in the same way. Then, the humanoid controller to be built can be simply presented as follows:(21)
τ
=
G
Θ
,where G
(
·
) is the nonlinear dynamics that the TF-LSSVR tries to build. τ is the driving torque, and Θ
=
(
θ
s
u
p
_
hip
,
θ
s
u
p
_
ankle
) is a vector of joint angles.


### 3.2.2. Objective Functions of the TF-LSSVR

When the timeliness of the training data is considered, the collected data from current steps are “more important,” and those from past steps are “less important.” Based on this, in the proposed objective functions of the TF-LSSVR, a TFMF is used to design learning weights of the collected data. Taking the learning of driving torque of the supporting hip as an example and designing a learning weightr
k for each of the training samples, the training sample set can be denoted by {
θ
s
u
p
_
hip
(
k
)
,
θ
s
u
p
_
ankle
(
k
)
,
τ
s
u
p
_
hip
(
k
)
,
r
k
}, 0
≤
r
k
≤
1, k
=
1,2
,
…
,
N. Then the objective function of the TF-LSSVR for training the driving torque of the supporting hip is(22)
m
i
n
1
2
ω
T
ω
+
1
2
C
r
k
∑
k
=
1
N
ξ
k
2
s
.
t
.
τ
s
u
p
_
hip
k
=
ω
T
ϕ
θ
s
u
p
_
hip
k
,
θ
s
u
p
_
ankle
k
+
b
s
u
p
_
hip
+
ξ
k
,
k
=
1
,
…
,
N
,where ω is a weight vector. ϕ
(
·
) is a nonlinear mapping function. C is a penalty coefficient, ξ
k is a positive slack variable, and b
s
u
p
_
hip is the corresponding bias. {
θ
s
u
p
_
hip
(
k
)
,
θ
s
u
p
_
ankle
(
k
)
,
τ
(
k
)
,
r
k
} is the kth sample, and N is the number of the samples. The r
k is the fuzzy learning weight for the training samples, which can be calculated using the fuzzy membership functions designed in the next section. It is noted that in literature [17] also a weighting scheme similar to (22) has been proposed but it is based on robust statistics. In this paper, a different weighting scheme is proposed.


### 3.2.3. Designing the Learning Weights Using TFMF

The walking data from different time can be evaluated using some linguistic terms such as “more important” or “less important.” When the timeliness of the training data is considered, the collected data from current steps are “more important,” and those from past steps are “less important.”For this reason, a TFMF, which is the left half of a trapezoidal function, is proposed as the membership function of humanoid walking samples. The proposed time-related trapezoidal fuzzy membership function is shown in Figure2. The formula for the trapezoidal fuzzy membership function is as follows:(23)
r
k
=
1
-
h
t
k
-
m
-
t
k
-
m
-
n
t
+
t
k
-
m
h
-
t
k
-
m
-
n
t
k
-
m
-
t
k
-
m
-
n
,
t
∈
t
k
-
m
-
n
,
t
k
-
m
,
1
,
t
∈
t
k
-
m
,
t
k
,where r
k is the trapezoidal fuzzy membership function of time t
k  (
k
=
1,2
,
…
,
N
), r
k
∈
[
h
,
1
], and h
∈
(
0,1
) is a tuning parameter. t
k
-
m
-
n and t
k are the beginning and the ending of the time window, respectively, and t
k
-
m is a time point between the beginning and the ending.Figure 2
The trapezoidal fuzzy membership function for humanoid walking samples.From Figure2 and (23), it can be seen that the time window t
k
-
m
-
n
,
t
k is divided into two parts. The first part t
k
-
m
,
t
k contains the newest walking samples, which are designed to have a full fuzzy membership grade. The second part t
k
-
m
-
n
,
t
k
-
m crosses through the relative old samples, which are assigned descending fuzzy membership grades. As shown in Figure 2, the values of the learning weight at points A and B are larger than that of point C because the states A and B are closer to the current situation. Besides, as the sampling time window moves, the trapezoidal learning-weight function moves along the time axis accordingly. Therefore, reasonable and adaptable learning weights are assigned for all samples in the whole walking process. The deduced learning weights are then used in the learning algorithm of the TF-LSSVR.The nonlinear dynamic modelG
(
·
) in (21) can be built when the learning process of the TF-LSSVR is completed. After this, considering the parameters of the TF-LSSVR, controller (21) can be written as (24)
τ
=
G
Θ
,
ψ
,where τ is the driving torque, Θ is the joint angle, and ψ
=
[
C
,
σ
,
m
,
n
,
h
]
T is a parameter vector. Here, we denote the plenty coefficient by C, the width of the RBF kernel by σ, and the width of the moving time window by m and n, and we denote the minimum value of the membership function by h. To obtain optimized parameters for the controller, a CKF-based training method is proposed in the next section.


## 3.2.1. Humanoid Controller to Be Built Using the TF-LSSVR

Based on the existing literature [14], when the ZMP criterion is satisfied, the dynamics between the torque control inputs and the joint angles of the DSP can be presented as(19)
τ
left
_
knee
=
h
θ
left
_
knee
,
θ
right
_
knee
,
τ
right
_
knee
=
l
θ
left
_
knee
,
θ
right
_
knee
,where τ
left
_
knee and τ
right
_
knee are the driven torques of the left knee and the right knee. θ
left
_
knee and θ
right
_
knee are the joint angles of the left knee and the right knee. h
(
·
) and l
(
·
) are the nonlinear dynamics in the DSP to be learned using the proposed TF-LSSVR.On the other side, when the ZMP criterion is satisfied, the dynamics between the torque control inputs and the joint angles of the SSP can be presented as [14](20)
τ
s
u
p
_
h
i
p
=
f
θ
s
u
p
_
h
i
p
,
θ
s
u
p
_
a
n
k
l
e
,
τ
s
u
p
_
a
n
k
l
e
=
g
θ
s
u
p
_
h
i
p
,
θ
s
u
p
_
a
n
k
l
e
,where τ
s
u
p
_
h
i
p and τ
s
u
p
_
ankle are the driven torques of the supporting hip and the supporting ankle. θ
s
u
p
_
hip and θ
s
u
p
_
ankle are the joint angles of the supporting hip and the supporting ankle. f
(
·
) and g
(
·
) are the nonlinear dynamics in the SSP to be learned using the proposed TF-LSSVR.For illustration purposes, we will expound the situation in the SSP mainly. The solution for the DSP can be easily deduced in the same way. Then, the humanoid controller to be built can be simply presented as follows:(21)
τ
=
G
Θ
,where G
(
·
) is the nonlinear dynamics that the TF-LSSVR tries to build. τ is the driving torque, and Θ
=
(
θ
s
u
p
_
hip
,
θ
s
u
p
_
ankle
) is a vector of joint angles.


## 3.2.2. Objective Functions of the TF-LSSVR

When the timeliness of the training data is considered, the collected data from current steps are “more important,” and those from past steps are “less important.” Based on this, in the proposed objective functions of the TF-LSSVR, a TFMF is used to design learning weights of the collected data. Taking the learning of driving torque of the supporting hip as an example and designing a learning weightr
k for each of the training samples, the training sample set can be denoted by {
θ
s
u
p
_
hip
(
k
)
,
θ
s
u
p
_
ankle
(
k
)
,
τ
s
u
p
_
hip
(
k
)
,
r
k
}, 0
≤
r
k
≤
1, k
=
1,2
,
…
,
N. Then the objective function of the TF-LSSVR for training the driving torque of the supporting hip is(22)
m
i
n
1
2
ω
T
ω
+
1
2
C
r
k
∑
k
=
1
N
ξ
k
2
s
.
t
.
τ
s
u
p
_
hip
k
=
ω
T
ϕ
θ
s
u
p
_
hip
k
,
θ
s
u
p
_
ankle
k
+
b
s
u
p
_
hip
+
ξ
k
,
k
=
1
,
…
,
N
,where ω is a weight vector. ϕ
(
·
) is a nonlinear mapping function. C is a penalty coefficient, ξ
k is a positive slack variable, and b
s
u
p
_
hip is the corresponding bias. {
θ
s
u
p
_
hip
(
k
)
,
θ
s
u
p
_
ankle
(
k
)
,
τ
(
k
)
,
r
k
} is the kth sample, and N is the number of the samples. The r
k is the fuzzy learning weight for the training samples, which can be calculated using the fuzzy membership functions designed in the next section. It is noted that in literature [17] also a weighting scheme similar to (22) has been proposed but it is based on robust statistics. In this paper, a different weighting scheme is proposed.


## 3.2.3. Designing the Learning Weights Using TFMF

The walking data from different time can be evaluated using some linguistic terms such as “more important” or “less important.” When the timeliness of the training data is considered, the collected data from current steps are “more important,” and those from past steps are “less important.”For this reason, a TFMF, which is the left half of a trapezoidal function, is proposed as the membership function of humanoid walking samples. The proposed time-related trapezoidal fuzzy membership function is shown in Figure2. The formula for the trapezoidal fuzzy membership function is as follows:(23)
r
k
=
1
-
h
t
k
-
m
-
t
k
-
m
-
n
t
+
t
k
-
m
h
-
t
k
-
m
-
n
t
k
-
m
-
t
k
-
m
-
n
,
t
∈
t
k
-
m
-
n
,
t
k
-
m
,
1
,
t
∈
t
k
-
m
,
t
k
,where r
k is the trapezoidal fuzzy membership function of time t
k  (
k
=
1,2
,
…
,
N
), r
k
∈
[
h
,
1
], and h
∈
(
0,1
) is a tuning parameter. t
k
-
m
-
n and t
k are the beginning and the ending of the time window, respectively, and t
k
-
m is a time point between the beginning and the ending.Figure 2
The trapezoidal fuzzy membership function for humanoid walking samples.From Figure2 and (23), it can be seen that the time window t
k
-
m
-
n
,
t
k is divided into two parts. The first part t
k
-
m
,
t
k contains the newest walking samples, which are designed to have a full fuzzy membership grade. The second part t
k
-
m
-
n
,
t
k
-
m crosses through the relative old samples, which are assigned descending fuzzy membership grades. As shown in Figure 2, the values of the learning weight at points A and B are larger than that of point C because the states A and B are closer to the current situation. Besides, as the sampling time window moves, the trapezoidal learning-weight function moves along the time axis accordingly. Therefore, reasonable and adaptable learning weights are assigned for all samples in the whole walking process. The deduced learning weights are then used in the learning algorithm of the TF-LSSVR.The nonlinear dynamic modelG
(
·
) in (21) can be built when the learning process of the TF-LSSVR is completed. After this, considering the parameters of the TF-LSSVR, controller (21) can be written as (24)
τ
=
G
Θ
,
ψ
,where τ is the driving torque, Θ is the joint angle, and ψ
=
[
C
,
σ
,
m
,
n
,
h
]
T is a parameter vector. Here, we denote the plenty coefficient by C, the width of the RBF kernel by σ, and the width of the moving time window by m and n, and we denote the minimum value of the membership function by h. To obtain optimized parameters for the controller, a CKF-based training method is proposed in the next section.


## 3.3. Training the Parameters of the Controller Using CKF

### 3.3.1. Parameters Optimization Problem in a Form Suitable for CKF

In order to cast the parameters optimization problem of the TF-LSSVR in a form suitable for CKF, we let the parameters of the TF-LSSVR constitute the state of a nonlinear system, and we let the output of the TF-LSSVR constitute the output of the nonlinear system to which the CKF is applied.As mentioned above we denote the state of the nonlinear system by(25)
ψ
=
C
,
σ
,
m
,
n
,
h
T
.The vector ψ thus consists of all of the parameters for the TF-LSSVR arranged in a linear array. Let τ
=
G
(
Θ
,
ψ
) be the transfer function of the TF-LSSVR, where τ is the output, Θ is the input, and ψ
=
[
C
,
σ
,
m
,
n
,
h
]
T is a parameter vector. The training of the TF-LSSVR can be formulated as a filtering problem. The model for updating the parameters using CKF can be written as(26)
ψ
k
=
ψ
k
-
1
,
τ
k
=
G
Θ
k
,
ψ
k
.Similar to the approach of training fuzzy systems with the extended Kalman filter [26], in order to execute a stable Kalman filter algorithm, we add some process noise and measurement noise to the above model. That is, (27)
ψ
k
=
ψ
k
-
1
+
w
k
-
1
,
τ
k
=
G
Θ
k
,
ψ
k
+
v
k
,where w
k
-
1 and v
k are artificially added Gaussian noise sequences with zero means and covariances Q
k
-
1 and R
k, respectively.


### 3.3.2. The CKF Algorithm for Updating the Parameters of the Controller

To update the parameters of the controller, the CKF algorithm [27] is implemented as follows.Step 1 (cubature-points calculation).
The point(
η
i
,
β
i
) is called cubature point here. For the third-degree spherical-radial rule, it entails a total of 2
u cubature points. η
i and β
i can be calculated as follows:(28)
η
i
=
u
1
i
,
β
i
=
1
2
u
,where i
=
1,2
,
…
,
2
u and 1
i denotes the ith column of the set 1. For example, 1
=
(
1,0
T
,
-
1,0
T
,
0
,
-
1
T
,
0,1
T
) when u
=
2. u is the dimension of the state vector.Step 2 (time update).
After calculating the cubature points, we use the cubature-point set to numerically compute the integrals and obtain the time update:(29)
ψ
i
,
k
-
1
∣
k
-
1
=
P
k
-
1
∣
k
-
1
η
i
+
ψ
^
k
-
1
∣
k
-
1
,
ψ
i
,
k
∣
k
-
1
∗
=
ψ
i
,
k
-
1
∣
k
-
1
+
w
k
-
1
,
ψ
^
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
∗
,
P
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
∗
ψ
i
,
k
∣
k
-
1
∗
T
-
ψ
^
k
∣
k
-
1
ψ
^
k
∣
k
-
1
T
+
Q
k
-
1
,
ψ
i
,
k
∣
k
-
1
=
P
k
∣
k
-
1
ξ
i
+
ψ
^
k
∣
k
-
1
.Step 3 (measurement update).
Consider(30)
τ
i
,
k
∣
k
-
1
=
G
Θ
i
,
k
k
-
1
,
ψ
i
,
k
∣
k
-
1
+
v
k
,
τ
^
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
τ
i
,
k
∣
k
-
1
,
P
τ
τ
,
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
τ
i
,
k
∣
k
-
1
τ
i
,
k
∣
k
-
1
T
-
τ
^
k
∣
k
-
1
τ
^
k
∣
k
-
1
T
+
R
k
,
P
ψ
τ
,
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
τ
i
,
k
∣
k
-
1
T
-
ψ
^
k
∣
k
-
1
τ
^
k
∣
k
-
1
T
,
W
k
=
P
ψ
τ
,
k
∣
k
-
1
P
τ
τ
,
k
∣
k
-
1
-
1
,
ψ
^
k
∣
k
=
ψ
^
k
∣
k
-
1
+
W
k
τ
k
-
τ
^
k
∣
k
-
1
,
P
k
∣
k
=
P
k
∣
k
-
1
-
W
k
P
τ
τ
,
k
∣
k
-
1
W
k
T
,where W
k is the Kalman gain, ψ
^
k
∣
k is the updated state, and P
k
∣
k is the corresponding error covariance. P
ψ
τ
,
k
∣
k
-
1 is the cross-covariance matrix and P
τ
τ
,
k
∣
k
-
1
-
1 is the inverse matrix of the innovation covariance matrix.After time update and measurement update, an estimation of the parametersψ for the TF-LSSVR is obtained. That is to say, the initial controller with random parameters is updated to the proposed data-based controller.


## 3.3.1. Parameters Optimization Problem in a Form Suitable for CKF

In order to cast the parameters optimization problem of the TF-LSSVR in a form suitable for CKF, we let the parameters of the TF-LSSVR constitute the state of a nonlinear system, and we let the output of the TF-LSSVR constitute the output of the nonlinear system to which the CKF is applied.As mentioned above we denote the state of the nonlinear system by(25)
ψ
=
C
,
σ
,
m
,
n
,
h
T
.The vector ψ thus consists of all of the parameters for the TF-LSSVR arranged in a linear array. Let τ
=
G
(
Θ
,
ψ
) be the transfer function of the TF-LSSVR, where τ is the output, Θ is the input, and ψ
=
[
C
,
σ
,
m
,
n
,
h
]
T is a parameter vector. The training of the TF-LSSVR can be formulated as a filtering problem. The model for updating the parameters using CKF can be written as(26)
ψ
k
=
ψ
k
-
1
,
τ
k
=
G
Θ
k
,
ψ
k
.Similar to the approach of training fuzzy systems with the extended Kalman filter [26], in order to execute a stable Kalman filter algorithm, we add some process noise and measurement noise to the above model. That is, (27)
ψ
k
=
ψ
k
-
1
+
w
k
-
1
,
τ
k
=
G
Θ
k
,
ψ
k
+
v
k
,where w
k
-
1 and v
k are artificially added Gaussian noise sequences with zero means and covariances Q
k
-
1 and R
k, respectively.


## 3.3.2. The CKF Algorithm for Updating the Parameters of the Controller

To update the parameters of the controller, the CKF algorithm [27] is implemented as follows.Step 1 (cubature-points calculation).
The point(
η
i
,
β
i
) is called cubature point here. For the third-degree spherical-radial rule, it entails a total of 2
u cubature points. η
i and β
i can be calculated as follows:(28)
η
i
=
u
1
i
,
β
i
=
1
2
u
,where i
=
1,2
,
…
,
2
u and 1
i denotes the ith column of the set 1. For example, 1
=
(
1,0
T
,
-
1,0
T
,
0
,
-
1
T
,
0,1
T
) when u
=
2. u is the dimension of the state vector.Step 2 (time update).
After calculating the cubature points, we use the cubature-point set to numerically compute the integrals and obtain the time update:(29)
ψ
i
,
k
-
1
∣
k
-
1
=
P
k
-
1
∣
k
-
1
η
i
+
ψ
^
k
-
1
∣
k
-
1
,
ψ
i
,
k
∣
k
-
1
∗
=
ψ
i
,
k
-
1
∣
k
-
1
+
w
k
-
1
,
ψ
^
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
∗
,
P
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
∗
ψ
i
,
k
∣
k
-
1
∗
T
-
ψ
^
k
∣
k
-
1
ψ
^
k
∣
k
-
1
T
+
Q
k
-
1
,
ψ
i
,
k
∣
k
-
1
=
P
k
∣
k
-
1
ξ
i
+
ψ
^
k
∣
k
-
1
.Step 3 (measurement update).
Consider(30)
τ
i
,
k
∣
k
-
1
=
G
Θ
i
,
k
k
-
1
,
ψ
i
,
k
∣
k
-
1
+
v
k
,
τ
^
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
τ
i
,
k
∣
k
-
1
,
P
τ
τ
,
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
τ
i
,
k
∣
k
-
1
τ
i
,
k
∣
k
-
1
T
-
τ
^
k
∣
k
-
1
τ
^
k
∣
k
-
1
T
+
R
k
,
P
ψ
τ
,
k
∣
k
-
1
=
∑
i
=
1
2
u
β
i
ψ
i
,
k
∣
k
-
1
τ
i
,
k
∣
k
-
1
T
-
ψ
^
k
∣
k
-
1
τ
^
k
∣
k
-
1
T
,
W
k
=
P
ψ
τ
,
k
∣
k
-
1
P
τ
τ
,
k
∣
k
-
1
-
1
,
ψ
^
k
∣
k
=
ψ
^
k
∣
k
-
1
+
W
k
τ
k
-
τ
^
k
∣
k
-
1
,
P
k
∣
k
=
P
k
∣
k
-
1
-
W
k
P
τ
τ
,
k
∣
k
-
1
W
k
T
,where W
k is the Kalman gain, ψ
^
k
∣
k is the updated state, and P
k
∣
k is the corresponding error covariance. P
ψ
τ
,
k
∣
k
-
1 is the cross-covariance matrix and P
τ
τ
,
k
∣
k
-
1
-
1 is the inverse matrix of the innovation covariance matrix.After time update and measurement update, an estimation of the parametersψ for the TF-LSSVR is obtained. That is to say, the initial controller with random parameters is updated to the proposed data-based controller.


## 4. Simulation Research

In this section, we test our proposed learning control method on the control of a seven-link robot by simulation experiments. Matlab 7.0 is used to model the humanoid robot and the controller.


### 4.1. Simplified Model of a Seven-Link Humanoid Robot

The simplified model of the robot has two legs and a trunk. Each leg is composed of a thigh, a shank, and a foot. There is one degree of freedom (DOF) in the trunk, one for each hip, one for each knee, and one for each ankle. In this paper, we focus on the sagittal dynamics of the humanoid robot. The simple model of the humanoid is shown in Figure3. The details of the humanoid can be referenced in Table 1.Table 1
Parameters of the humanoid robot.
Link
Length (m)
Mass (kg)
Inertia moment (kg⋅m2)
Trunk
0.175
1.060
0.01623
Thigh
0.130
0.075
(
∗
2
)
0.00037
(
∗
2
)
Shank
0.130
0.075
(
∗
2
)
0.00037
(
∗
2
)
Foot
0.020
0.060
(
∗
2
)
0.00004
(
∗
2
)Figure 3
Simplified model of the simulated humanoid robot.


### 4.2. Sampling for the TF-LSSVR Learning

The whole walking cycle is denoted byT
c, and let T
c
=
1 s with 0.2 s for the DSP and 0.8 s for the SSP. The trunk of the humanoid robot is assumed to be upright during walking. Three different step lengths are implemented (0.16 m, 0.18 m, and 0.20 m) and the step height is 0.02 m for all the three step lengths. The sampling interval is 0.025 s. That is to say, there are 40 groups of samples that are collected in a single walking cycle.To validate the advantage of the proposed method, a simulated time-varying external perturbationτ
d
=
0.08
sin
⁡
(
5
t
) (Nm) is considered here for the tests. It is a horizontal external force with duration of 0.1 s that is applied on the hip at t
=
0.15 s in the DSP and t
=
0.65 s in the SSP, respectively.


### 4.3. Parameters and the Learning Results

#### 4.3.1. Experimental Conditions and the Parameters

Here, we use the universal RBF as the kernel for the TF-LSSVR.The penalty coefficientC
=
1000 and width of the RBF kernels σ
=
0.9 are determined by a 10-fold cross-validation strategy. The other initial parameters for the proposed framework are identified according to experiences, which include the number of the steps for a full weight m
=
1, the number of the steps for a gradient weight n
=
1, and the lower bound of the fuzzy membership function h
=
0.5. Then the initial parameters are updated using CKF.Parameters for the CKF are as follows: process noisew
k
-
1
~
(
0,1
0
-
3
) and the measurement noise v
k
~
(
0,1
0
-
3
), where w
k
-
1 and v
k are both white, zero-mean, uncorrelated noises. The initial covariance of the humanoid state is P
0
=
diag
⁡
{
1
0
-
3
,
…
,
1
0
-
3
}
7
×
7. The initial radians of the humanoid joint angles are shown in Table 2.Table 2
Initial radians of the humanoid joint angles.
Humanoid joint angles (rad)
θ
1
θ
2
θ
3
θ
4
θ
5
θ
6
θ
7
Initial radians when step length is 0.16 m
0.0698
0.2418
0.6811
0
0
0.3550
0.5989
Initial radians when step length is 0.18 m
0.0785
0.1819
0.7036
0
0
0.3550
0.5989
Initial radians when step length is 0.20 m
0.0873
0.1183
0.7210
0
0
0.3550
0.5989


#### 4.3.2. Learning Results of the TF-LSSVR

The learning results of the proposed TF-LSSVR are compared with two other intelligent methods including fuzzy and the traditional LS-SVR. The designing details of the fuzzy control system can be found in the literature [14]. Nonlinear dynamics of the submodels formulated in (19)-(20) are learned using the three different methods (see Figures 4–15).Figure 4
Learning results for Submodel 1 of the SSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 5
Learning results for Submodel 1 of the SSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 6
Learning results for Submodel 1 of the SSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 7
Learning results for Submodel 2 of the SSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 8
Learning results for Submodel 2 of the SSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 9
Learning results for Submodel 2 of the SSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 10
Learning results for Submodel 1 of the DSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 11
Learning results for Submodel 1 of the DSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 12
Learning results for Submodel 1 of the DSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 13
Learning results for Submodel 2 of the DSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 14
Learning results for Submodel 2 of the DSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 15
Learning results for Submodel 2 of the DSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Define control errore
=
τ
^
-
τ, and the integral square error (ISE) is defined as the measure index:(31)
I
S
E
=
1
N
∑
i
=
1
N
τ
^
i
-
τ
i
2
,where i denotes the sampling index and N
=
40 is the number of all the samples. τ
^
(
i
) is the output of the learning methods and τ
(
i
) is the desired output. The integrated square errors (ISEs) criterion is listed in Table 3, which indicates that the proposed TF-LSSVR achieves a better learning result than the other two existing methods.Table 3
Comparisons of the ISE performances.
e
s
u
p
_
h
i
p
e
s
u
p
_
a
n
k
l
e
e
l
e
f
t
_
k
n
e
e
e
r
i
g
h
t
_
k
n
e
e
Fuzzy
8.9
e
-
002
9.6
e
-
002
9.2
e
-
001
9.9
e
-
002
LS-SVR
7.9
e
-
00
2
5.4
e
-
00
2
2.8
e
-
00
1
7.3
e
-
00
2
TF-LSSVR
6.1
e
-
006
7.2
e
-
006
1.0
e
-
004
2.1
e
-
005


### 4.4. Performance Comparisons

ZMP trajectory with disturbance in the SSP can be found in Figures16–18, which indicate that the proposed learning control method produces a larger stability margin relative to the traditional ones. Similar situations appeared in the DSP and the comparisons of humanoid walking when disturbances occur in the DSP are shown in Figures 19–21. As we can see, the ZMP trajectories corresponding to the proposed method have bigger stability margins for the humanoid compared to the other two intelligent methods. That is to say, using the data online and offline with different weights, the proposed learning control method is more effective in learning the external disturbances and increasing the stability margin of humanoid robots.Figure 16
ZMP trajectory with disturbance in the SSP when the step length is 0.16 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 17
ZMP trajectory with disturbance in the SSP when the step length is 0.18 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 18
ZMP trajectory with disturbance in the SSP when the step length is 0.20 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 19
ZMP trajectory with disturbance in the DSP when the step length is 0.16 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 20
ZMP trajectory with disturbance in the DSP when the step length is 0.18 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 21
ZMP trajectory with disturbance in the DSP when the step length is 0.20 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).


## 4.1. Simplified Model of a Seven-Link Humanoid Robot

The simplified model of the robot has two legs and a trunk. Each leg is composed of a thigh, a shank, and a foot. There is one degree of freedom (DOF) in the trunk, one for each hip, one for each knee, and one for each ankle. In this paper, we focus on the sagittal dynamics of the humanoid robot. The simple model of the humanoid is shown in Figure3. The details of the humanoid can be referenced in Table 1.Table 1
Parameters of the humanoid robot.
Link
Length (m)
Mass (kg)
Inertia moment (kg⋅m2)
Trunk
0.175
1.060
0.01623
Thigh
0.130
0.075
(
∗
2
)
0.00037
(
∗
2
)
Shank
0.130
0.075
(
∗
2
)
0.00037
(
∗
2
)
Foot
0.020
0.060
(
∗
2
)
0.00004
(
∗
2
)Figure 3
Simplified model of the simulated humanoid robot.


## 4.2. Sampling for the TF-LSSVR Learning

The whole walking cycle is denoted byT
c, and let T
c
=
1 s with 0.2 s for the DSP and 0.8 s for the SSP. The trunk of the humanoid robot is assumed to be upright during walking. Three different step lengths are implemented (0.16 m, 0.18 m, and 0.20 m) and the step height is 0.02 m for all the three step lengths. The sampling interval is 0.025 s. That is to say, there are 40 groups of samples that are collected in a single walking cycle.To validate the advantage of the proposed method, a simulated time-varying external perturbationτ
d
=
0.08
sin
⁡
(
5
t
) (Nm) is considered here for the tests. It is a horizontal external force with duration of 0.1 s that is applied on the hip at t
=
0.15 s in the DSP and t
=
0.65 s in the SSP, respectively.


## 4.3. Parameters and the Learning Results

### 4.3.1. Experimental Conditions and the Parameters

Here, we use the universal RBF as the kernel for the TF-LSSVR.The penalty coefficientC
=
1000 and width of the RBF kernels σ
=
0.9 are determined by a 10-fold cross-validation strategy. The other initial parameters for the proposed framework are identified according to experiences, which include the number of the steps for a full weight m
=
1, the number of the steps for a gradient weight n
=
1, and the lower bound of the fuzzy membership function h
=
0.5. Then the initial parameters are updated using CKF.Parameters for the CKF are as follows: process noisew
k
-
1
~
(
0,1
0
-
3
) and the measurement noise v
k
~
(
0,1
0
-
3
), where w
k
-
1 and v
k are both white, zero-mean, uncorrelated noises. The initial covariance of the humanoid state is P
0
=
diag
⁡
{
1
0
-
3
,
…
,
1
0
-
3
}
7
×
7. The initial radians of the humanoid joint angles are shown in Table 2.Table 2
Initial radians of the humanoid joint angles.
Humanoid joint angles (rad)
θ
1
θ
2
θ
3
θ
4
θ
5
θ
6
θ
7
Initial radians when step length is 0.16 m
0.0698
0.2418
0.6811
0
0
0.3550
0.5989
Initial radians when step length is 0.18 m
0.0785
0.1819
0.7036
0
0
0.3550
0.5989
Initial radians when step length is 0.20 m
0.0873
0.1183
0.7210
0
0
0.3550
0.5989


### 4.3.2. Learning Results of the TF-LSSVR

The learning results of the proposed TF-LSSVR are compared with two other intelligent methods including fuzzy and the traditional LS-SVR. The designing details of the fuzzy control system can be found in the literature [14]. Nonlinear dynamics of the submodels formulated in (19)-(20) are learned using the three different methods (see Figures 4–15).Figure 4
Learning results for Submodel 1 of the SSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 5
Learning results for Submodel 1 of the SSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 6
Learning results for Submodel 1 of the SSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 7
Learning results for Submodel 2 of the SSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 8
Learning results for Submodel 2 of the SSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 9
Learning results for Submodel 2 of the SSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 10
Learning results for Submodel 1 of the DSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 11
Learning results for Submodel 1 of the DSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 12
Learning results for Submodel 1 of the DSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 13
Learning results for Submodel 2 of the DSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 14
Learning results for Submodel 2 of the DSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 15
Learning results for Submodel 2 of the DSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Define control errore
=
τ
^
-
τ, and the integral square error (ISE) is defined as the measure index:(31)
I
S
E
=
1
N
∑
i
=
1
N
τ
^
i
-
τ
i
2
,where i denotes the sampling index and N
=
40 is the number of all the samples. τ
^
(
i
) is the output of the learning methods and τ
(
i
) is the desired output. The integrated square errors (ISEs) criterion is listed in Table 3, which indicates that the proposed TF-LSSVR achieves a better learning result than the other two existing methods.Table 3
Comparisons of the ISE performances.
e
s
u
p
_
h
i
p
e
s
u
p
_
a
n
k
l
e
e
l
e
f
t
_
k
n
e
e
e
r
i
g
h
t
_
k
n
e
e
Fuzzy
8.9
e
-
002
9.6
e
-
002
9.2
e
-
001
9.9
e
-
002
LS-SVR
7.9
e
-
00
2
5.4
e
-
00
2
2.8
e
-
00
1
7.3
e
-
00
2
TF-LSSVR
6.1
e
-
006
7.2
e
-
006
1.0
e
-
004
2.1
e
-
005


## 4.3.1. Experimental Conditions and the Parameters

Here, we use the universal RBF as the kernel for the TF-LSSVR.The penalty coefficientC
=
1000 and width of the RBF kernels σ
=
0.9 are determined by a 10-fold cross-validation strategy. The other initial parameters for the proposed framework are identified according to experiences, which include the number of the steps for a full weight m
=
1, the number of the steps for a gradient weight n
=
1, and the lower bound of the fuzzy membership function h
=
0.5. Then the initial parameters are updated using CKF.Parameters for the CKF are as follows: process noisew
k
-
1
~
(
0,1
0
-
3
) and the measurement noise v
k
~
(
0,1
0
-
3
), where w
k
-
1 and v
k are both white, zero-mean, uncorrelated noises. The initial covariance of the humanoid state is P
0
=
diag
⁡
{
1
0
-
3
,
…
,
1
0
-
3
}
7
×
7. The initial radians of the humanoid joint angles are shown in Table 2.Table 2
Initial radians of the humanoid joint angles.
Humanoid joint angles (rad)
θ
1
θ
2
θ
3
θ
4
θ
5
θ
6
θ
7
Initial radians when step length is 0.16 m
0.0698
0.2418
0.6811
0
0
0.3550
0.5989
Initial radians when step length is 0.18 m
0.0785
0.1819
0.7036
0
0
0.3550
0.5989
Initial radians when step length is 0.20 m
0.0873
0.1183
0.7210
0
0
0.3550
0.5989


## 4.3.2. Learning Results of the TF-LSSVR

The learning results of the proposed TF-LSSVR are compared with two other intelligent methods including fuzzy and the traditional LS-SVR. The designing details of the fuzzy control system can be found in the literature [14]. Nonlinear dynamics of the submodels formulated in (19)-(20) are learned using the three different methods (see Figures 4–15).Figure 4
Learning results for Submodel 1 of the SSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 5
Learning results for Submodel 1 of the SSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 6
Learning results for Submodel 1 of the SSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 7
Learning results for Submodel 2 of the SSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 8
Learning results for Submodel 2 of the SSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 9
Learning results for Submodel 2 of the SSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 10
Learning results for Submodel 1 of the DSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 11
Learning results for Submodel 1 of the DSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 12
Learning results for Submodel 1 of the DSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 13
Learning results for Submodel 2 of the DSP (the step length is 0.16 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 14
Learning results for Submodel 2 of the DSP (the step length is 0.18 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Figure 15
Learning results for Submodel 2 of the DSP (the step length is 0.20 m). (a) Fuzzy; (b) LS-SVR; (c) the proposed TF-LSSVR.
(a)
(b)
(c)Define control errore
=
τ
^
-
τ, and the integral square error (ISE) is defined as the measure index:(31)
I
S
E
=
1
N
∑
i
=
1
N
τ
^
i
-
τ
i
2
,where i denotes the sampling index and N
=
40 is the number of all the samples. τ
^
(
i
) is the output of the learning methods and τ
(
i
) is the desired output. The integrated square errors (ISEs) criterion is listed in Table 3, which indicates that the proposed TF-LSSVR achieves a better learning result than the other two existing methods.Table 3
Comparisons of the ISE performances.
e
s
u
p
_
h
i
p
e
s
u
p
_
a
n
k
l
e
e
l
e
f
t
_
k
n
e
e
e
r
i
g
h
t
_
k
n
e
e
Fuzzy
8.9
e
-
002
9.6
e
-
002
9.2
e
-
001
9.9
e
-
002
LS-SVR
7.9
e
-
00
2
5.4
e
-
00
2
2.8
e
-
00
1
7.3
e
-
00
2
TF-LSSVR
6.1
e
-
006
7.2
e
-
006
1.0
e
-
004
2.1
e
-
005


## 4.4. Performance Comparisons

ZMP trajectory with disturbance in the SSP can be found in Figures16–18, which indicate that the proposed learning control method produces a larger stability margin relative to the traditional ones. Similar situations appeared in the DSP and the comparisons of humanoid walking when disturbances occur in the DSP are shown in Figures 19–21. As we can see, the ZMP trajectories corresponding to the proposed method have bigger stability margins for the humanoid compared to the other two intelligent methods. That is to say, using the data online and offline with different weights, the proposed learning control method is more effective in learning the external disturbances and increasing the stability margin of humanoid robots.Figure 16
ZMP trajectory with disturbance in the SSP when the step length is 0.16 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 17
ZMP trajectory with disturbance in the SSP when the step length is 0.18 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 18
ZMP trajectory with disturbance in the SSP when the step length is 0.20 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 19
ZMP trajectory with disturbance in the DSP when the step length is 0.16 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 20
ZMP trajectory with disturbance in the DSP when the step length is 0.18 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).Figure 21
ZMP trajectory with disturbance in the DSP when the step length is 0.20 m. Fuzzy controller (green); LS-SVR controller (magenta); the proposed TF-LSSVR controller (black).


## 5. Conclusions

In this work, a TF-LSSVR-based control system is proposed to learn the external disturbances and increase the stability margin of humanoid robots. First, data for training the controller is collected by implementing simulation experiments. Secondly, a TF-LSSVR with a time-related TFMF is proposed to train the controller using the simulated data. Thirdly, the parameters of the proposed TF-LSSVR are updated using a CKF. Simulation results are provided.The proposed method is shown to be effective in learning and adapting occasional external disturbances and ensuring the stability margin of the robot. We believe that the proposed method will be very promising for the development of autonomous humanoid robots.



---

*Source: 1984634-2016-06-30.xml*