# Staring Imaging Attitude Tracking Control Laws for Video Satellites Based on Image Information by Hyperbolic Tangent Fuzzy Sliding Mode Control

Authors: Wenjing Pei
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

This paper studies the staring imaging attitude tracking and control for satellite videos based on image information. An improved temporal-spatial context learning algorithm is employed to extract the image information. Based on this, a hyperbolic tangent fuzzy sliding mode control law is proposed to achieve the attitude tracking and control. Furthermore, the hyperbolic tangent function and fuzzy logic system are introduced into the sliding mode controller. In the experiments, the improved temporal-spatial context learning algorithm is applied for the image information of the space target video sequence captured by Jilin-1 in orbit, where the image information is used as the input of the control loop. Moreover, the proposed method is realized through simulation. Besides, the image change caused by attitude adjustment is achieved successfully, and the target imaging can be located in the center of the image plane to realize the gaze tracking control of the space target effectively.


---

## Body

## 1. Introduction

With the rapid development of remote sensing technology, video satellites have attracted much attention due to their continuous observation ability [1–8]. As a new type of Earth observation satellite, video satellites can employ the payload on the satellite platform to be pushed scan imaging with the help of the orbital motion of the satellite. Note that video satellites are to adjust the attitude real-timely and to obtain the dynamic information of the target area continuously in the process of staring imaging, so the optical axis of the optical load points at the target at all the time. In Ref. [9], Liu et al. introduce the gaze imaging technique, where the process generally refers to the staring attitude control. In addition, video satellites can use agility and attitude control technology to realize continuous imaging of ground targets. Due to this reason, compared with the traditional Earth observation satellites, video satellites have widely applied in many fields, such as vehicle real-time monitoring [10, 11], rapid response to natural disaster emergency [12], and major engineering monitoring [13].Up to now, there are mainly two types of staring imaging satellites in orbit, including the satellites in the geostationary orbit and video satellites in the low orbit [14–16]. Figure 1 shows the schematic diagram of the ground gazing attitude control of video satellites. Furthermore, the attitude control system in the video satellites can adjust the attitude in real-time so that the optical axis of the optical sensor always points to the ground target area for continuous photography. Staring imaging is the main working mode for video satellites [17, 18]. In essence, although the staring imaging control problem is a dynamic attitude tracking problem, it is difficult to ensure the high stability of the optical axis of the satellite optical sensor to the observed object.Figure 1
Schematic diagram of ground gazing attitude control of the video satellite.In the last decades, the staring image attitude control for video satellites can be regarded as a spacecraft attitude tracking and control problem [19–21]. Much research work has been done on the attitude tracking control for the satellites. Meanwhile, some controllers have been employed in satellites attitude control, such as sliding mode controller, robust controller, and intelligent controller [19–34].So far, compared with attitude tracking and control for spacecraft and satellites, several satellite staring attitude control methods have been introduced to achieve real-time tracking [24–32]. For instance, Lian et al. [27] investigated the small satellite attitude problems for staring operation. Liang et al. [28] designed the fuzzy logic control law for the staring imaging satellite attitude problem in LEO, which has a quick response and excellent robustness. In Ref. [29], Chen et al. present a quaternion-based PID feedback tracking controller with the gyroscopic term cancellation, where some desired target can be tracked on the Earth. Chen et al. [30] investigated a staring imaging attitude controller based on double-gimbaled control moment gyroscope (DGCMG), which is simple and effective for agile small satellites. In Ref. [31], Li and Liang proposed a robust finite-time controller aiming at satellite attitude maneuvers to demonstrate the robustness of some typical perturbations such as disturbance torque, model uncertainty, and actuator error. Li et al. [32] implemented the neural network controller for staring imaging, where the real-time performance can be achieved.However, the above staring attitude controller does not consider the image information directly. Moreover, the image information is separated from the attitude tracking controller. Besides, we note that in the satellite staring mode, the optical axis of the camera should point fixedly to the target for a long time. Meanwhile, both the video satellites and the object may be moving in the inertial coordinate system. In this way, the relative velocity and position may be changing over time.In essence, visual information is introduced into the closed-loop control, which is commonly known as visual servo, and is first applied in the field of robots [35–38]. Recently, robot visual servo has achieved numerous results in both theory and practical application. We note that robot visual servo is generally divided into two structures: the position-based visual servo and the image-based visual servo. The position-based servo should calibrate the internal parameters of the camera to determine the relative attitude between the target and the camera coordinate system, which may increase the amount of calculation of the system. On the contrary, the image-based visual servo directly uses the visual feature error of the target in the phase plane and takes the controlled object and the visual system as a whole.Due to the above reasons and advantages, visual information was initially introduced into the control closed loop of satellites and spacecraft. Therefore, an improved temporal-spatial context learning (ISTC) algorithm is employed to extract the image information in this paper. Based on this, a hyperbolic tangent fuzzy sliding mode control law (HTFSMC) for small video satellites is designed to achieve the attitude tracking and control. In special, the related coordinates are defined for attitude transformation. Subsequently, the sliding mode tracking controller is presented based on the image information from satellite videos. Furthermore, the hyperbolic tangent function and the fuzzy logic system are employed in the sliding mode controller.In summary, the contributions of this paper are threefold.(1)
In this paper, an ISTC algorithm is employed to obtain the image information. Hence, the visual information can be employed effectively in the visual tracking control based on spatial moving images, instead of cumbersome, complex camera internal parameter calibration, and accurate information of target and camera motion.(2)
Based on the image information, this paper proposed the HTFSMC, where the hyperbolic tangent function and fuzzy logic system are introduced into the sliding mode controller.(3)
In the experiments, the image information of the space target video sequence captured by Jilin-1 in orbit is used as the input of the controller. The control part is realized through simulation. Besides, the image change caused by attitude adjustment is achieved successfully, and the target imaging can be located in the center of the image plane to realize the gaze tracking control of the space target effectively.The rest of this paper is arranged as follows. In Section2, the staring imaging attitude dynamics model is described in detail. In what follows of this section, the sliding mode controller and fuzzy sliding controller are presented for staring imaging attitude tracking of small satellite videos. In Section 3, the experiment results and some discussion are introduced. Finally, Section 4 concludes this article.


## 2. Materials and Methods

In this paper, in order to extract the image information, the ISTC is employed for moving object tracking. Based on this, an attitude controller based on image information feedback is designed to realize the gaze tracking control of moving targets. The structure diagram is shown in Figure2. In a nutshell, this method is to conform the centroid coordinates and to calculate the position deviation from the image center. Therefore, the cumbersome process of camera calibration and relative pose estimation can be avoided so that the computation cost is reduced.Figure 2
Structure frame of staring imaging attitude tracking control system based on image information.


### 2.1. An Improved Spatio-Temporal Context Learning Algorithm

For moving target video tracking, the local context is the background of the target and an certain area nearby. In fact, there is a strong space-time relationship in the local scene around the target between consecutive frames. According to the space-time relationship between the target and its surrounding area, the spatio-temporal context (STC) learning algorithm is constructed a spatio-temporal context model for the target and the nearby area based on the gray features of the image. Moreover, the confidence map of the target is to be calculated, and the maximum-likelihood probability in the confidence map can be found as the estimated target position. Therefore, the ISTC algorithm is described in detail.The confidence map of the target position can be set asx∗. The luminance feature set of x∗ is defined as(1)Xc=cz=Iz,z|z∈Ωcx∗,where cz and Iz are the luminance feature and the image intensity at position z, respectively. Ωcx∗ denotes the context area of position x∗. In the following, cx can be used to calculate the confidence map as follows:(2)cx=Px|o,=∑cz∈XcPx,cz|o,=∑cz∈XcPx|cz,oPcz|o,where Px|cz,o is the conditional probability, which can represent the spatial relationship between the target position and its the context information. In addition, Pcz|o is the prior probability, which can model the appearance of the local context. Furthermore, Px|cz,o can be defined as(3)Px|cz,o=hscx−z,where hscx−z is the relative distance and direction function between the target position x and its local context information. Subsequently, Pcz|o can be also defined as(4)Pcz|o=Izωσz−x∗=Izae−z/σ2,where ωσz−x∗ is a weight function, a is a normalized constant, which makes Pcz|o range from 0 to 1 in (4), and σ is a scale parameter. Hence, the confidence map cx in (2) can be rewritten as(5)cx=Px|o,=bex−x∗aβ,=∑z∈Ωcx∗hscx−zIzωσz−x∗,=hscx⊗Ixωσx−x∗,where ⊗ is the convolution operator and β is an important shape parameter. Fast Fourier transform is utilized simultaneously on both sides of the equation for (5). Therefore, (5) can be updated as(6)ℱbex−x∗aβ=ℱhscx⊙ℱIxωσx−x∗,where ℱ is the fast Fourier transform (FFT) and ⊙ denotes the element-wise product. Subsequently, hscx can be obtained as(7)hscx=ℱ−1ℱbex−x∗/aβℱIxωσx−x∗,where ℱ−1 is the inverse FFT. In this way, based on (7), the spatio-temporal context model can be derived as(8)Hk−1sc=1−ρHksc+ρhksc,where htsc is the spatio-temporal context model at the k-th frame in (7). Hence, ht+1sc can be also obtained by (7). Thus, the confidence map at the k+1-th frame is expressed as(9)ct+1x=ℱ−1ℱHk+1scx⊙ℱIk+1xωσx−xk∗.The confidence map is maximized, so the location of the target can be obtained as(10)xk+1∗=argmaxX∈Ωcxk∗ct+1x.Since the target attitude may be changed in the process of target movement, the size of the target may be also changed. Besides, the background information may be different in each frame. Therefore, the scale update strategy can be employed for the target, which is given as(11a)st′=ctxt∗ct−1xt−1∗,(11b)st¯=1n∑i=1nst−1′,(11c)st+1=1−λst+λst¯,(11d)σt+1=stσt.However, in Equation (11a), the denominator may be close to zero so that the results of moving object tracking may occasionally lead to overfitting. Due to this reason, an improved scale update strategy is introduced to avoid an abrupt change based on a penalty term ps:(12)ps=ς,log2s<ς,−log2s,log2s≤ς−ς,log2s>ς,,where ς is a constant. In this way, the updated scale can be rewritten as(13)st′=st+1+pst+1.


### 2.2. Staring Imaging Attitude Dynamics Model

#### 2.2.1. The Definitions of the Related Coordinate Systems

In this paper, some related coordinate systems are shown in Figure3. The inertial coordinate system of Earth is defined as Oi−XiYiZi, where the coordinate origin Oi is located at the center of mass of the Earth. The Xi-axis is on the equatorial plane, which is pointing at the vernal equinox of the time. The direction of Zi-axis is the consistent with the Earth rotation axis. The Yi-axis is in the equatorial plane and meets the right-hand orthogonal reference. The satellite body coordinate system is defined as Ob−XbYbZb, where the center of mass of the satellite is the origin of the coordinate system Ob. The image coordinate system is O−XpYp. The camera coordinate system is Oc−XcYcZc. The image pixel coordinate system is I−xy.Figure 3
Schematic diagram of ground gazing attitude control of video small satellite.


#### 2.2.2. The Attitude Solution Based on Satellite Images

In this paper, we assume that the camera coordinate systemOc−XcYcZc coincides with the satellite coordinate system Ob−XbYbZb. The unit vector in the ObZb direction is r=0,0,1T. As shown in Figure 4, the coordinates of target P is set as u,v in the pixel coordinate system I−xy. In the satellite coordinate system Ob−XbYbZb, the target line of sight direction rP can be described as(14)rP=1l2m2+l2n2+f2lulvf=xPyPzP,where xP,yP,zP is the coordinate of the target in the satellite coordinate system Ob−XbYbZb. The focal length of the spaceborne camera is f and the pixel size is l.Figure 4
The diagram of target deviation on image plane.The purpose is to ensure the target can image in the center of the image. Hence, it is necessary to coincide withr and rP. In the process of staring tracking, the images are required to remain stable and no rotation, which is convenient for image observation and analysis. Staring tracking imaging is the process of controlling r to track rP.In the satellite coordinate systemOb−XbYbZb, the following assumptions are given as(15a)Ob=−lu,0,−f,(15b)r1=−ObUObU,=1l2u2+f2lu0f.First, we rotate around theObYb axis so that r is coincided with r1. The rotation angle θ is expressed as(16)θ=arctanluf.Then, we rotate around theObXb axis so that r is coincided with rP. The rotation angle φ is demonstrated as(17)φ=arctan−lvf2+u2.The attitude quaternionsq are defined as(18a)q=q0q¯,(18b)q¯=q1,q2,q3T,(18c)qTq=q02+q12+q22+q32,where q¯ is the vector part, and q0 is the scalar part. The quaternion is obtained as the equation (19) through rotating θ around 0,1,0T.(19)q1=cosθ2,0,sinθ2,0T.Through rotatingφ around 0,1,0T, the quaternion is obtained as(20a)q2=cosφ2,sinφ2,0,0T,(20b)q2⊗q1=q01q02−q¯1q¯2q01q¯2+q02q¯1−q¯2×q¯1.where ⊗ is the rotation multiplication operator of quaternions. Thereby, the expected attitude error quaternion qe is expressed as(21)qe=q2⊗q1−1,=q01q02−q¯1q¯2q01q¯2+q02q¯1−q¯2×q¯1−1,=cosφ2cosθ2−sinφ2cosθ2−cosφ2sinθ2−sinφ2sinθ2.Therefore, the expected attitude quaternionqt is relative to the Earth inertial system can be expressed as(22)qt=qe−1⊗qb,where qb is attitude quaternion of satellite body coordinate system relative to the Earth inertial system.The attitude kinematics equation is shown as(23a)q˙t=12Eqtωt,(23b)Eqt=−q¯Tq¯×+q0I3×3,(23c)q¯×=0−q3q2q30q1q2q10,where q¯× is an antisymmetric matrix, and I3×3 is an identity matrix. Then, the following equation is verified as(24)EqtTEqt=I3×3.Using the equation (23a), the expected angular velocity ωt is inversely solved as(25a)ωt=2ΞTqtq˙t.(25b)Ξqt=−q¯Tq0I3×3−q¯×,The expected attitude error angular velocity is given as(26)ωe=ωb−Aqeωt.Ifq=q0q¯ is any quaternion, we can obtain(27)Aqe=q02−q¯2I3×3−2q0q¯×+2q¯q¯T,where Aqe is the attitude matrix determined by qe.


### 2.3. Sliding Mode Controller

In this paper, based on the actuator of three orthogonal mounted reaction flywheel, the satellite attitude dynamics equation is given as(28)Jω˙b=−ωb×Jωb+h+u+d,where J is the inertia moment of the satellite, ωb is angular velocity of satellite body coordinate system, h is the angular momentum of the flywheel, u is control torque, and d is external disturbance torque.In this paper, the sliding mode function is designed as(29a)ζ=ωe+Kq¯em,(29b)q¯˙e=12Fqeωe,(29c)Fq=q¯×+q0I3×3,(29d)ωe=2ΞTqeqe,(29e)ω˙b=ω˙e+Aqeω˙t−ωe×Aqeωt,where ζ=ζ1,ζ2,ζ3T, K=diagki,i=1,2,3. If ζ⟶0, the angular velocity of the system and the state of the angular velocity can be tracked.The approach function method is used to obtain the sliding mode control law.The exponential reaching law is used as(30)ζ˙=−lζ−εsgnζ,where sgns=sgnζ1,sgnζ2,sgnζ3, l=diagli, and ε=diagεi,i=1,2,3. The control quantity u is obtained as(31)u=JAqeω˙t+ωb×Jωb+h+Jωe×Aqeωt−12JKFqeωe−K1ζ−D1Pζ,where K1 is controller parameters. We notice that the chattering of sliding mode controller is mainly caused by sgnζ and d. Let Pζ=sgnζ. In order to reduce chattering, Pζ is rewritten as(32a)P^s=tanhζ1ε,tanhζ2ε,tanhζ3ε,(32b)tanxε=ex/ε−e−x/εex/ε+e−x/ε,where the inflection point of hyperbolic tangent function is determined through the value of εε>0.


### 2.4. Stability Analysis

In order to ensure that the state of the system move from any initial point tos=0 in a finite time based on the designed sliding mode controller, the following assumptive conditions are given:(1)
Supposedd is bounded, dmax≥D2 is boundary of d.(2)
SupposedminiD1i>d.The stability of the system is proved as follows. Lyapunov function is constructed as(33)V=12ζTJζ.The derivative of the (33) is represented as(34)V˙=ζTJζ˙=ζTd−K1ζ−D1Pζ<0,where V˙=0 if and only if ζ=0. V˙ is a seminegative definite function. Therefore, the system is convergent by using sliding mode control.


### 2.5. Fuzzy Logic System

Fuzzy logic system (FLS) is consisted of fuzzy rule base, fuzzy rule base, fuzzy inference engine, and defuzzifier, as shown Figure5. In this paper, we assume that x1∈X1,x2∈X2,…,xp∈Xp, and y∈Y are p input and an output, respectively. Furthermore, the fuzzy rule base is composed of k rules, expressed as(35)Rl:IFx1isF1l,…,xpisFPl,THENyisGl,where l=1,2,…,k. Thereby, FLS can be employed to simplify fuzzy rules as mapping from fuzzy input sets F1l×⋯Fpl to the fuzzy output set Y, denoted by F1l×⋯Fpl=Al. In this way, (35) can be rewritten as(36)Rl:F1l×⋯Fpl⟶Gl=Al⟶Gl,l=1,2,…,k.Figure 5
The diagram of Fuzzy logic system.The membership functionμRlx,y can be utilized to describe Rl as(37)μRlx,y=μAl⟶Glx,y,where x=x1,x2,…,xpT. Hence, (37) can be rewritten as(38)μRlx,y=μRlx1,x2,…,xp,y,=μAl⟶Glx1,x2,…,xp,y,=μF1lx1⋆μF2lx2⋆⋯⋆μFplxp⋆μGly,=Ti=1pμFilxi⋆μGly,where ⋆ represents that multiple antecedents are juxtaposed and connected with t norm.Ax is a fuzzy set where p input of Rl is given, and the membership function μAxx is defined as(39)μAxx=μX1x1μX2x2⋯μXpxp=Ti=1pμXixi.According to each fuzzy rule, a fuzzy setBl about the set Y is given as(40)Bl=Ax∘Rl.Meanwhile, based on commutativity oft norm, we can obtain the membership function μBly in (41)(41)μBly=μAx∘Rly,=supx∈XμAxx⋆μAl⟶Glx,y,=supx∈XTi=1pμXixi⋆Ti=1pμFilxi⋆μGly,=supx∈XμXixi⋆μFilxi⋆μGly,=μGly⋆supx1∈X1μX1x1⋆μF1lx1⋆supx2∈X2μX2x2⋆μF2lx2⋆⋯⋆supxp∈XpμXpxp⋆μFplxp,y∈Y.Singleton fuzzifier is employed into (41) and (41), which can be rewritten as(42)μBly==μGly⋆μF1lx1′⋆μF2lx2′⋆⋯⋆μFplxp′,y∈Y.Due to centroid defuzzifier, FLS output can be expressed as(43a)ycx=∑i=1NyiμByi∑i=1NμByi,(43b)B=∪l=1MBl,where B is output fuzzy set, and ycx is the clear output.


### 2.6. Fuzzy Sliding Mode Controller

The input and output fuzzy sets of the system are defined as(44a)ζζ˙=NB NM NS Z PS PM PB,(44b)ΔD1=NB NM NS Z PS PM PB,where ζζ˙ is the input, and the variation of D1 is represented ΔD1 as the output. In Equations (44a) and (44b), NB,NM,NS,Z,PS,PM,PB are negative large, negative middle, negative small, zero, positive small, positive middle, and positive large, respectively. Therefore, the following seven rules are designed as(i)
R1 : Ifζζ˙ is PB, THEN ΔD1 is PB.(ii)
R2 : Ifζζ˙ is PM, THEN ΔD1 is PM.(iii)
R3 : Ifζζ˙ is PS, THEN ΔD1 is PS.(iv)
R4 : Ifζζ˙ is Z, THEN ΔD1 is Z.(v)
R5 : Ifζζ˙ is NS, THEN ΔD1 is NS.R6 : Ifζζ˙ is NM, THEN ΔD1 is NM.R7 : Ifζζ˙ is NB, THEN ΔD1 is NB.Besides, Figure6 shows the input/output membership function of fuzzy control system. According to the value of ζζ˙, the centroid defuzzifier is employed and the value of ΔD1 can be obtained as ΔD^1. Meanwhile, D1 can be rewritten as(45)D^1=D1+ΔD^1.Figure 6
The input/output membership function of fuzzy control system: (a) membership function of fuzzy input; (b) membership function of fuzzy output.
(a)(b)Hence, the fuzzy sliding mode controller can be designed as(46)u=JAqeω˙t+ωb×Jωb+h+Jωe×Aqeωt−12JKFqeωe−K1ζ−D^1P^ζ.


## 2.1. An Improved Spatio-Temporal Context Learning Algorithm

For moving target video tracking, the local context is the background of the target and an certain area nearby. In fact, there is a strong space-time relationship in the local scene around the target between consecutive frames. According to the space-time relationship between the target and its surrounding area, the spatio-temporal context (STC) learning algorithm is constructed a spatio-temporal context model for the target and the nearby area based on the gray features of the image. Moreover, the confidence map of the target is to be calculated, and the maximum-likelihood probability in the confidence map can be found as the estimated target position. Therefore, the ISTC algorithm is described in detail.The confidence map of the target position can be set asx∗. The luminance feature set of x∗ is defined as(1)Xc=cz=Iz,z|z∈Ωcx∗,where cz and Iz are the luminance feature and the image intensity at position z, respectively. Ωcx∗ denotes the context area of position x∗. In the following, cx can be used to calculate the confidence map as follows:(2)cx=Px|o,=∑cz∈XcPx,cz|o,=∑cz∈XcPx|cz,oPcz|o,where Px|cz,o is the conditional probability, which can represent the spatial relationship between the target position and its the context information. In addition, Pcz|o is the prior probability, which can model the appearance of the local context. Furthermore, Px|cz,o can be defined as(3)Px|cz,o=hscx−z,where hscx−z is the relative distance and direction function between the target position x and its local context information. Subsequently, Pcz|o can be also defined as(4)Pcz|o=Izωσz−x∗=Izae−z/σ2,where ωσz−x∗ is a weight function, a is a normalized constant, which makes Pcz|o range from 0 to 1 in (4), and σ is a scale parameter. Hence, the confidence map cx in (2) can be rewritten as(5)cx=Px|o,=bex−x∗aβ,=∑z∈Ωcx∗hscx−zIzωσz−x∗,=hscx⊗Ixωσx−x∗,where ⊗ is the convolution operator and β is an important shape parameter. Fast Fourier transform is utilized simultaneously on both sides of the equation for (5). Therefore, (5) can be updated as(6)ℱbex−x∗aβ=ℱhscx⊙ℱIxωσx−x∗,where ℱ is the fast Fourier transform (FFT) and ⊙ denotes the element-wise product. Subsequently, hscx can be obtained as(7)hscx=ℱ−1ℱbex−x∗/aβℱIxωσx−x∗,where ℱ−1 is the inverse FFT. In this way, based on (7), the spatio-temporal context model can be derived as(8)Hk−1sc=1−ρHksc+ρhksc,where htsc is the spatio-temporal context model at the k-th frame in (7). Hence, ht+1sc can be also obtained by (7). Thus, the confidence map at the k+1-th frame is expressed as(9)ct+1x=ℱ−1ℱHk+1scx⊙ℱIk+1xωσx−xk∗.The confidence map is maximized, so the location of the target can be obtained as(10)xk+1∗=argmaxX∈Ωcxk∗ct+1x.Since the target attitude may be changed in the process of target movement, the size of the target may be also changed. Besides, the background information may be different in each frame. Therefore, the scale update strategy can be employed for the target, which is given as(11a)st′=ctxt∗ct−1xt−1∗,(11b)st¯=1n∑i=1nst−1′,(11c)st+1=1−λst+λst¯,(11d)σt+1=stσt.However, in Equation (11a), the denominator may be close to zero so that the results of moving object tracking may occasionally lead to overfitting. Due to this reason, an improved scale update strategy is introduced to avoid an abrupt change based on a penalty term ps:(12)ps=ς,log2s<ς,−log2s,log2s≤ς−ς,log2s>ς,,where ς is a constant. In this way, the updated scale can be rewritten as(13)st′=st+1+pst+1.


## 2.2. Staring Imaging Attitude Dynamics Model

### 2.2.1. The Definitions of the Related Coordinate Systems

In this paper, some related coordinate systems are shown in Figure3. The inertial coordinate system of Earth is defined as Oi−XiYiZi, where the coordinate origin Oi is located at the center of mass of the Earth. The Xi-axis is on the equatorial plane, which is pointing at the vernal equinox of the time. The direction of Zi-axis is the consistent with the Earth rotation axis. The Yi-axis is in the equatorial plane and meets the right-hand orthogonal reference. The satellite body coordinate system is defined as Ob−XbYbZb, where the center of mass of the satellite is the origin of the coordinate system Ob. The image coordinate system is O−XpYp. The camera coordinate system is Oc−XcYcZc. The image pixel coordinate system is I−xy.Figure 3
Schematic diagram of ground gazing attitude control of video small satellite.


### 2.2.2. The Attitude Solution Based on Satellite Images

In this paper, we assume that the camera coordinate systemOc−XcYcZc coincides with the satellite coordinate system Ob−XbYbZb. The unit vector in the ObZb direction is r=0,0,1T. As shown in Figure 4, the coordinates of target P is set as u,v in the pixel coordinate system I−xy. In the satellite coordinate system Ob−XbYbZb, the target line of sight direction rP can be described as(14)rP=1l2m2+l2n2+f2lulvf=xPyPzP,where xP,yP,zP is the coordinate of the target in the satellite coordinate system Ob−XbYbZb. The focal length of the spaceborne camera is f and the pixel size is l.Figure 4
The diagram of target deviation on image plane.The purpose is to ensure the target can image in the center of the image. Hence, it is necessary to coincide withr and rP. In the process of staring tracking, the images are required to remain stable and no rotation, which is convenient for image observation and analysis. Staring tracking imaging is the process of controlling r to track rP.In the satellite coordinate systemOb−XbYbZb, the following assumptions are given as(15a)Ob=−lu,0,−f,(15b)r1=−ObUObU,=1l2u2+f2lu0f.First, we rotate around theObYb axis so that r is coincided with r1. The rotation angle θ is expressed as(16)θ=arctanluf.Then, we rotate around theObXb axis so that r is coincided with rP. The rotation angle φ is demonstrated as(17)φ=arctan−lvf2+u2.The attitude quaternionsq are defined as(18a)q=q0q¯,(18b)q¯=q1,q2,q3T,(18c)qTq=q02+q12+q22+q32,where q¯ is the vector part, and q0 is the scalar part. The quaternion is obtained as the equation (19) through rotating θ around 0,1,0T.(19)q1=cosθ2,0,sinθ2,0T.Through rotatingφ around 0,1,0T, the quaternion is obtained as(20a)q2=cosφ2,sinφ2,0,0T,(20b)q2⊗q1=q01q02−q¯1q¯2q01q¯2+q02q¯1−q¯2×q¯1.where ⊗ is the rotation multiplication operator of quaternions. Thereby, the expected attitude error quaternion qe is expressed as(21)qe=q2⊗q1−1,=q01q02−q¯1q¯2q01q¯2+q02q¯1−q¯2×q¯1−1,=cosφ2cosθ2−sinφ2cosθ2−cosφ2sinθ2−sinφ2sinθ2.Therefore, the expected attitude quaternionqt is relative to the Earth inertial system can be expressed as(22)qt=qe−1⊗qb,where qb is attitude quaternion of satellite body coordinate system relative to the Earth inertial system.The attitude kinematics equation is shown as(23a)q˙t=12Eqtωt,(23b)Eqt=−q¯Tq¯×+q0I3×3,(23c)q¯×=0−q3q2q30q1q2q10,where q¯× is an antisymmetric matrix, and I3×3 is an identity matrix. Then, the following equation is verified as(24)EqtTEqt=I3×3.Using the equation (23a), the expected angular velocity ωt is inversely solved as(25a)ωt=2ΞTqtq˙t.(25b)Ξqt=−q¯Tq0I3×3−q¯×,The expected attitude error angular velocity is given as(26)ωe=ωb−Aqeωt.Ifq=q0q¯ is any quaternion, we can obtain(27)Aqe=q02−q¯2I3×3−2q0q¯×+2q¯q¯T,where Aqe is the attitude matrix determined by qe.


## 2.2.1. The Definitions of the Related Coordinate Systems

In this paper, some related coordinate systems are shown in Figure3. The inertial coordinate system of Earth is defined as Oi−XiYiZi, where the coordinate origin Oi is located at the center of mass of the Earth. The Xi-axis is on the equatorial plane, which is pointing at the vernal equinox of the time. The direction of Zi-axis is the consistent with the Earth rotation axis. The Yi-axis is in the equatorial plane and meets the right-hand orthogonal reference. The satellite body coordinate system is defined as Ob−XbYbZb, where the center of mass of the satellite is the origin of the coordinate system Ob. The image coordinate system is O−XpYp. The camera coordinate system is Oc−XcYcZc. The image pixel coordinate system is I−xy.Figure 3
Schematic diagram of ground gazing attitude control of video small satellite.


## 2.2.2. The Attitude Solution Based on Satellite Images

In this paper, we assume that the camera coordinate systemOc−XcYcZc coincides with the satellite coordinate system Ob−XbYbZb. The unit vector in the ObZb direction is r=0,0,1T. As shown in Figure 4, the coordinates of target P is set as u,v in the pixel coordinate system I−xy. In the satellite coordinate system Ob−XbYbZb, the target line of sight direction rP can be described as(14)rP=1l2m2+l2n2+f2lulvf=xPyPzP,where xP,yP,zP is the coordinate of the target in the satellite coordinate system Ob−XbYbZb. The focal length of the spaceborne camera is f and the pixel size is l.Figure 4
The diagram of target deviation on image plane.The purpose is to ensure the target can image in the center of the image. Hence, it is necessary to coincide withr and rP. In the process of staring tracking, the images are required to remain stable and no rotation, which is convenient for image observation and analysis. Staring tracking imaging is the process of controlling r to track rP.In the satellite coordinate systemOb−XbYbZb, the following assumptions are given as(15a)Ob=−lu,0,−f,(15b)r1=−ObUObU,=1l2u2+f2lu0f.First, we rotate around theObYb axis so that r is coincided with r1. The rotation angle θ is expressed as(16)θ=arctanluf.Then, we rotate around theObXb axis so that r is coincided with rP. The rotation angle φ is demonstrated as(17)φ=arctan−lvf2+u2.The attitude quaternionsq are defined as(18a)q=q0q¯,(18b)q¯=q1,q2,q3T,(18c)qTq=q02+q12+q22+q32,where q¯ is the vector part, and q0 is the scalar part. The quaternion is obtained as the equation (19) through rotating θ around 0,1,0T.(19)q1=cosθ2,0,sinθ2,0T.Through rotatingφ around 0,1,0T, the quaternion is obtained as(20a)q2=cosφ2,sinφ2,0,0T,(20b)q2⊗q1=q01q02−q¯1q¯2q01q¯2+q02q¯1−q¯2×q¯1.where ⊗ is the rotation multiplication operator of quaternions. Thereby, the expected attitude error quaternion qe is expressed as(21)qe=q2⊗q1−1,=q01q02−q¯1q¯2q01q¯2+q02q¯1−q¯2×q¯1−1,=cosφ2cosθ2−sinφ2cosθ2−cosφ2sinθ2−sinφ2sinθ2.Therefore, the expected attitude quaternionqt is relative to the Earth inertial system can be expressed as(22)qt=qe−1⊗qb,where qb is attitude quaternion of satellite body coordinate system relative to the Earth inertial system.The attitude kinematics equation is shown as(23a)q˙t=12Eqtωt,(23b)Eqt=−q¯Tq¯×+q0I3×3,(23c)q¯×=0−q3q2q30q1q2q10,where q¯× is an antisymmetric matrix, and I3×3 is an identity matrix. Then, the following equation is verified as(24)EqtTEqt=I3×3.Using the equation (23a), the expected angular velocity ωt is inversely solved as(25a)ωt=2ΞTqtq˙t.(25b)Ξqt=−q¯Tq0I3×3−q¯×,The expected attitude error angular velocity is given as(26)ωe=ωb−Aqeωt.Ifq=q0q¯ is any quaternion, we can obtain(27)Aqe=q02−q¯2I3×3−2q0q¯×+2q¯q¯T,where Aqe is the attitude matrix determined by qe.


## 2.3. Sliding Mode Controller

In this paper, based on the actuator of three orthogonal mounted reaction flywheel, the satellite attitude dynamics equation is given as(28)Jω˙b=−ωb×Jωb+h+u+d,where J is the inertia moment of the satellite, ωb is angular velocity of satellite body coordinate system, h is the angular momentum of the flywheel, u is control torque, and d is external disturbance torque.In this paper, the sliding mode function is designed as(29a)ζ=ωe+Kq¯em,(29b)q¯˙e=12Fqeωe,(29c)Fq=q¯×+q0I3×3,(29d)ωe=2ΞTqeqe,(29e)ω˙b=ω˙e+Aqeω˙t−ωe×Aqeωt,where ζ=ζ1,ζ2,ζ3T, K=diagki,i=1,2,3. If ζ⟶0, the angular velocity of the system and the state of the angular velocity can be tracked.The approach function method is used to obtain the sliding mode control law.The exponential reaching law is used as(30)ζ˙=−lζ−εsgnζ,where sgns=sgnζ1,sgnζ2,sgnζ3, l=diagli, and ε=diagεi,i=1,2,3. The control quantity u is obtained as(31)u=JAqeω˙t+ωb×Jωb+h+Jωe×Aqeωt−12JKFqeωe−K1ζ−D1Pζ,where K1 is controller parameters. We notice that the chattering of sliding mode controller is mainly caused by sgnζ and d. Let Pζ=sgnζ. In order to reduce chattering, Pζ is rewritten as(32a)P^s=tanhζ1ε,tanhζ2ε,tanhζ3ε,(32b)tanxε=ex/ε−e−x/εex/ε+e−x/ε,where the inflection point of hyperbolic tangent function is determined through the value of εε>0.


## 2.4. Stability Analysis

In order to ensure that the state of the system move from any initial point tos=0 in a finite time based on the designed sliding mode controller, the following assumptive conditions are given:(1)
Supposedd is bounded, dmax≥D2 is boundary of d.(2)
SupposedminiD1i>d.The stability of the system is proved as follows. Lyapunov function is constructed as(33)V=12ζTJζ.The derivative of the (33) is represented as(34)V˙=ζTJζ˙=ζTd−K1ζ−D1Pζ<0,where V˙=0 if and only if ζ=0. V˙ is a seminegative definite function. Therefore, the system is convergent by using sliding mode control.


## 2.5. Fuzzy Logic System

Fuzzy logic system (FLS) is consisted of fuzzy rule base, fuzzy rule base, fuzzy inference engine, and defuzzifier, as shown Figure5. In this paper, we assume that x1∈X1,x2∈X2,…,xp∈Xp, and y∈Y are p input and an output, respectively. Furthermore, the fuzzy rule base is composed of k rules, expressed as(35)Rl:IFx1isF1l,…,xpisFPl,THENyisGl,where l=1,2,…,k. Thereby, FLS can be employed to simplify fuzzy rules as mapping from fuzzy input sets F1l×⋯Fpl to the fuzzy output set Y, denoted by F1l×⋯Fpl=Al. In this way, (35) can be rewritten as(36)Rl:F1l×⋯Fpl⟶Gl=Al⟶Gl,l=1,2,…,k.Figure 5
The diagram of Fuzzy logic system.The membership functionμRlx,y can be utilized to describe Rl as(37)μRlx,y=μAl⟶Glx,y,where x=x1,x2,…,xpT. Hence, (37) can be rewritten as(38)μRlx,y=μRlx1,x2,…,xp,y,=μAl⟶Glx1,x2,…,xp,y,=μF1lx1⋆μF2lx2⋆⋯⋆μFplxp⋆μGly,=Ti=1pμFilxi⋆μGly,where ⋆ represents that multiple antecedents are juxtaposed and connected with t norm.Ax is a fuzzy set where p input of Rl is given, and the membership function μAxx is defined as(39)μAxx=μX1x1μX2x2⋯μXpxp=Ti=1pμXixi.According to each fuzzy rule, a fuzzy setBl about the set Y is given as(40)Bl=Ax∘Rl.Meanwhile, based on commutativity oft norm, we can obtain the membership function μBly in (41)(41)μBly=μAx∘Rly,=supx∈XμAxx⋆μAl⟶Glx,y,=supx∈XTi=1pμXixi⋆Ti=1pμFilxi⋆μGly,=supx∈XμXixi⋆μFilxi⋆μGly,=μGly⋆supx1∈X1μX1x1⋆μF1lx1⋆supx2∈X2μX2x2⋆μF2lx2⋆⋯⋆supxp∈XpμXpxp⋆μFplxp,y∈Y.Singleton fuzzifier is employed into (41) and (41), which can be rewritten as(42)μBly==μGly⋆μF1lx1′⋆μF2lx2′⋆⋯⋆μFplxp′,y∈Y.Due to centroid defuzzifier, FLS output can be expressed as(43a)ycx=∑i=1NyiμByi∑i=1NμByi,(43b)B=∪l=1MBl,where B is output fuzzy set, and ycx is the clear output.


## 2.6. Fuzzy Sliding Mode Controller

The input and output fuzzy sets of the system are defined as(44a)ζζ˙=NB NM NS Z PS PM PB,(44b)ΔD1=NB NM NS Z PS PM PB,where ζζ˙ is the input, and the variation of D1 is represented ΔD1 as the output. In Equations (44a) and (44b), NB,NM,NS,Z,PS,PM,PB are negative large, negative middle, negative small, zero, positive small, positive middle, and positive large, respectively. Therefore, the following seven rules are designed as(i)
R1 : Ifζζ˙ is PB, THEN ΔD1 is PB.(ii)
R2 : Ifζζ˙ is PM, THEN ΔD1 is PM.(iii)
R3 : Ifζζ˙ is PS, THEN ΔD1 is PS.(iv)
R4 : Ifζζ˙ is Z, THEN ΔD1 is Z.(v)
R5 : Ifζζ˙ is NS, THEN ΔD1 is NS.R6 : Ifζζ˙ is NM, THEN ΔD1 is NM.R7 : Ifζζ˙ is NB, THEN ΔD1 is NB.Besides, Figure6 shows the input/output membership function of fuzzy control system. According to the value of ζζ˙, the centroid defuzzifier is employed and the value of ΔD1 can be obtained as ΔD^1. Meanwhile, D1 can be rewritten as(45)D^1=D1+ΔD^1.Figure 6
The input/output membership function of fuzzy control system: (a) membership function of fuzzy input; (b) membership function of fuzzy output.
(a)(b)Hence, the fuzzy sliding mode controller can be designed as(46)u=JAqeω˙t+ωb×Jωb+h+Jωe×Aqeωt−12JKFqeωe−K1ζ−D^1P^ζ.


## 3. Results and Discussion

In this section, in order to verify the performance of the proposed method, the numerical simulations are conducted for the video satellite, where Jilin-1 is selected. The experiments are implemented in Matlab R2018b and NVIDIA GeForce GXT 2080Ti GPU. Firstly, the ISTC algorithm is employed to extract the image information. As shown in Figure7, the results of moving target tracking by ISTC are presented. Meanwhile, based on this, the traditional sliding mode control and the HTFSMC are design comparative experiments. The initial condition of the simulations is demonstrated in Table 1. Besides, we can see that the image size is 4000 pixels × 4000 pixels, which is very large.Figure 7
The results of moving target tracking by ISTC.Table 1
Initial conditions of the simulation.
ParametersParameter valuesInertia matrixdiag(264, 264, 28) kgm2Initial attitude angle[0°,0°,0°]Initial attitude angle angular velocity[0,0,0] rad/sPixel initial position[1100,800]Image size4000 pixels × 4000 pixelsDisturbing torques10−6N mImage processing error5 pixelsPixel size[8.38.3] mmCamera focal length4.2 mmAccording to the above simulation parameters, we assume that the pixel coordinate system of the target located at (0, 510) in the initial time. Meanwhile, the traditional sliding mode control and the proposed controller are applied to obtain the variation curve of output torque, attitude angle, and angular velocity.In Figure8, we can see that Tx and Ty are used to represent the output torque in the x and y directions and can converge after about 40s based on the sliding mode controller. However, in Figure 9, Tx and Ty can converge after 25s based on the HTFSMC. Compared with the sliding mode control, the proposed controller can improve the convergence speed. Besides, Tz is used to represent the output torque in the z direction and can converge faster after about 5s for these two controllers.Figure 8
Variation curve of output torque by the traditional sliding mode controller.Figure 9
Variation curve of output torque by the HTFSMC.In Figure10 and Figure 11, for the traditional sliding mode controller and the HTFSMC, θ can converge after about 30s and 20s, respectively. φ can converge after about 40s and 20s, respectively. Ψ can converge after about 20s.Figure 10
Variation curve of attitude angle by the traditional sliding mode controller.Figure 11
Variation curve of attitude angle by the HTFSMC.In Figure12 and Figure 13, for the traditional sliding mode controller and HTFSMC, ω1 can converge after about 30s and 23s, respectively. ω2 can converge after about 40s and 27s, respectively. ω3 can converge after about 30s and 23s, respectively.Figure 12
Variation curve of angular velocity by the traditional sliding mode controller.Figure 13
Variation curve of angular velocity by the HTFSMC.In Figure14, the image information of the space target video is used as the input of the control loop. The image change caused by attitude adjustment is simulated. The image is scaled to 2000 pixels ∗ 2000 pixels, which represents the satellite visual field and shown in black. Besides, we design that the size of the actual image is 4000 pixels∗ 4000 pixels. Meanwhile, the satellite visual field is embedded the actual image. The red ∗ is demonstrated the target and the green box visual field center. It is can be seen that the attitude control based on image information feedback can be achieved by the proposed controller. Moreover, the target imaging can be located in the center of the image plane to realize the gaze tracking control of the space target effectively. Accordingly, Figure 15 shows the trajectory of the target in image plane based on fuzzy sliding mode control. In addition, we can see that the position of the visual field center is (2000,2000). At first, the target position is not in the visual field center. The target imaging is located in the center based on the feedback control of image information. Figure 16 shows the optical axis pointing error converging to 0 after about 60s based on the proposed controller.Figure 14
Simulation results of moving target gaze tracking based on the HTFSMC.Figure 15
Trajectory of the target in image plane based on the HTFSMC.Figure 16
Optical axis pointing error based on the HTFSMC.Figure17 shows the simulation results of moving target gaze tracking in Jilin-1 video, in which the image changes caused by attitude adjustment are simulated and the moving target is the airplane. The moving airplane is not in the visual field center at the initial moment. The airplane is imaged in the visual field center based on the proposed controller of image information. Accordingly, Figure 18 shows the trajectory of the target in image plane based on fuzzy sliding mode control. Moreover, Figure 19 shows the optical axis pointing error converging to 0 quickly based on the proposed controller. It means that the gaze tracking of space moving target is effectively simulated.Figure 17
Simulation results of moving target gaze tracking based on the HTFSMC.Figure 18
Trajectory of the target in image plane based on the HTFSMC: (a) trajectory of the target in the entire movement process; (b) local enlarged trajectory image of the target.
(a)(b)Figure 19
Optical axis pointing error based on the HTFSMC.


## 4. Conclusions

The staring imaging attitude tracking and control for satellite videos based on image information is studied in this paper. An ISTC algorithm is designed to obtain the image information. Based on this, we introduced a HTFSMC law to achieve the attitude tracking and control. Furthermore, the hyperbolic tangent function and fuzzy logic system are introduced into the sliding mode controller.In the experiments, the image information of the space target video sequence captured by Jilin-1 in orbit is used as the input of the control loop. The control part is realized through simulation. Compared with the traditional sliding mode controller, the image change caused by attitude adjustment is achieved successfully and quickly based on the proposed controller, and the target imaging can be located in the center of the image plane to realize the gaze tracking control of the space target effectively. In the future work, the space target video sequences will be used as the input of the control loop directly.



---

*Source: 8289934-2022-08-10.xml*