# A Memory/Immunology-Based Control Approach with Applications to Multiple Spacecraft Formation Flying

Authors: Liguo Weng; Min Xia; Kai Hu; Zhuhan Qiao
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

This paper addresses the problem of formation control for multiple spacecrafts in Planetary Orbital Environment (POE). Due to the presence of diverse interferences and uncertainties in the outer space, such as the changing spacecraft mass, unavailable space parameters, and varying gravity forces, traditional control methods encounter great difficulties in this area. A new control approach inspired by human memory and immune system is proposed, and this approach is shown to be capable of learning from past control experience and current behavior to improve its performance. It demands much less system dynamic information as compared with traditional controls. Both theoretic analysis and computer simulation verify its effectiveness.


---

## Body

## 1. Introduction

Multiple spacecraft formation flying (MSFF) is an emerging concept which represents the idea of distributing the functionality of large spacecraft among smaller, less-expensive, and cooperative spacecrafts [1–5], as conceptually illustrated in Figure 1. The MSFF approach includes several advantages, such as increased instrument resolution, reduced cost, reconfigurability, and overall system robustness. It bears drastic potentials in the application of scientific and especially military purposes. Both NASA and the Air Force have identified spacecraft formation flying as a key technology for the 21st century.Figure 1
Spacecraft formation flying.Today, there are dozens of missions either flying, under development, or proposed that use spacecraft flying in formation. For example, Terrestrial Planet Finder (TPF) will look for extrasolar, Earth-like planets [6]; XEUS and the Constellation X-Ray Mission will explore high-energy astrophysical sources with unequaled resolution [7]; both EO-1/L-7 and CloudSat/Picasso-Cera will study the Earth [8, 9]; the US Air Force’s TechSat-21 seeks to push the frontier in microscale MSFF to enable global awareness and rapid access to space.The MSFF literature can be divided into two main categories based on the ambient dynamic environment. One is in Deep Space (DS), where the relative spacecraft dynamics could be reduced to double integrator form [1, 10], and the other is in Planetary Orbital Environment (POE), where spacecrafts are subjected to significant orbital dynamics and environmental disturbances [11–13].In this paper, a nonlinear MSFF system model which is based on leader-follower architecture around the earth orbit is introduced. Although there has been a number of studies on leader-follower structure, due to the complex orbital environments, traditional control methods encounter great challenges, such as adaptive and variable structure control methods [5, 14, 15], because under out-space circumstance, it is very hard to get the relative detailed information of the disturbances and uncertainties, even a relatively accurate model for the spacecraft.In consideration of the deficiencies of traditional control approaches, we propose a memory/immunology-based control approach based on our previous studies [16, 17]. This approach is inspired by human memory and immune systems, and it does not rely on precise information about system model and environment uncertainties. Instead, it makes use of past control experience and current system behavior to generate new control actions, steering the vehicles toward the desired locations/positions. Moreover, this control approach demands less computation as compared with most other methods. Both theoretical proof and simulation results confirm the effectiveness of the proposed approach.


## 2. Biological Fundamentals

### 2.1. Human Memory System

Human memory is one of the most intriguing biological phenomena in the nature; it is what makes possible so many of our complex cognitive functions, including communicating and learning.There are close links between learning and memory. The existence of memory depends on previous learning, and learning can most clearly be demonstrated by good performance on a memory test. Learning and memory involve three stages (see Figure2): (1)
encoding, involving the processes occurring during presentation of the learning material;(2)
storage, in which, as a result of encoding, some information is stored within the memory system;(3)
retrieval, which involves recovering or extracting stored information from the memory system.Figure 2
Human memory/learning system.Analogies between the memory system (MS) and the control system (CS) are presented in Table1.Table 1
Analogies between MS and CS.
Memory system
Control system
Past experience
Past control signal
Current (final) response
Current control output
Observations
Available sys. info.
Feedbacks (comparison)
Control errors
Objective
Desired system response


### 2.2. Immune System

The protection system that eliminates foreign substances that invade a living body is called the immune system [18].The human immune system works on two levels with the general goal of pathogen control: a general response mechanism, called innate immunity that does not directly respond to any specific pathogen, and a specific, antibody-mediated response mechanism called acquired immunity.Each type of B-cell in the immune system is genetically programmed to secrete antibodies that are able to recognize one particular antigen. Hence, the immune system as a whole can recognize large numbers of antigens; any type of B-cell is just a tiny proportion of the whole.When one type of antigen is recognized and binds to an antibody, this type of B-cell that secretes such antibodies is induced to proliferate rapidly. In other words, it is stimulated. Hence, more appropriate antibodies are generated to deal with the infection. One type of antigen may be recognized by several types of antibodies, among which the most suitable are stimulated strongly, and the less suitable receive less stimulation (see Figure3).Figure 3
Clone selection.Analogies between the immune system (IS) and the control system are presented in Table2.Table 2
Analogies between IS and CS.
Immune system
Control system
Antigen
System fault
Antibody
Control action
Stimulated
Proper control action
Suppressed
Improper control action


## 2.1. Human Memory System

Human memory is one of the most intriguing biological phenomena in the nature; it is what makes possible so many of our complex cognitive functions, including communicating and learning.There are close links between learning and memory. The existence of memory depends on previous learning, and learning can most clearly be demonstrated by good performance on a memory test. Learning and memory involve three stages (see Figure2): (1)
encoding, involving the processes occurring during presentation of the learning material;(2)
storage, in which, as a result of encoding, some information is stored within the memory system;(3)
retrieval, which involves recovering or extracting stored information from the memory system.Figure 2
Human memory/learning system.Analogies between the memory system (MS) and the control system (CS) are presented in Table1.Table 1
Analogies between MS and CS.
Memory system
Control system
Past experience
Past control signal
Current (final) response
Current control output
Observations
Available sys. info.
Feedbacks (comparison)
Control errors
Objective
Desired system response


## 2.2. Immune System

The protection system that eliminates foreign substances that invade a living body is called the immune system [18].The human immune system works on two levels with the general goal of pathogen control: a general response mechanism, called innate immunity that does not directly respond to any specific pathogen, and a specific, antibody-mediated response mechanism called acquired immunity.Each type of B-cell in the immune system is genetically programmed to secrete antibodies that are able to recognize one particular antigen. Hence, the immune system as a whole can recognize large numbers of antigens; any type of B-cell is just a tiny proportion of the whole.When one type of antigen is recognized and binds to an antibody, this type of B-cell that secretes such antibodies is induced to proliferate rapidly. In other words, it is stimulated. Hence, more appropriate antibodies are generated to deal with the infection. One type of antigen may be recognized by several types of antibodies, among which the most suitable are stimulated strongly, and the less suitable receive less stimulation (see Figure3).Figure 3
Clone selection.Analogies between the immune system (IS) and the control system are presented in Table2.Table 2
Analogies between IS and CS.
Immune system
Control system
Antigen
System fault
Antibody
Control action
Stimulated
Proper control action
Suppressed
Improper control action


## 3. System Model

In this section, we present a nonlinear MSFF system model [11] characterizing the relative position dynamics of follower spacecrafts to a leader spacecraft. In this model, each spacecraft is considered as a point-mass evolving in free space. The leader spacecraft provides a basic reference motion trajectory from which another desired trajectory is identified by the relative distances along {x,y,z} axes for the follower spacecraft to track. Furthermore, we assume that the leader spacecraft is in a circular orbit around the Earth. Schematic drawings of the MSFF system are depicted in Figures 4 and 5.Figure 4
Schematic representation of the MSFF system.Figure 5
Top view of the MSFF system.The inertial coordinate system{x,y,z} is attached to the center of the earth. Let R(t)∈ℝ3 denote the position vector from origin of the inertial coordinate system to the leader spacecraft and qf(t)∈ℝ3 the position vector from origin of the leader spacecraft coordinate system to the follower spacecraft. The leader spacecraft coordinate system {xl,yl,zl} is attached to the leader spacecraft with the xl axis pointing in the opposite direction as the tangential velocity, the yl axis pointing along the direction of vector R, and the zl axis being mutually perpendicular to the xl and yl axes.The mathematical model governing the dynamic behavior of follower spacecraft relative to the leader spacecraft is(1)mfq¨+C(ω)q˙+N(q,ω,R,ul)+Δf=uf
or
(2)q¨=f(·)mf+Δf+1mfuf,f(·)=-C(ω)q˙-N(q,ω,R,ul),
where uf∈ℝ3 is the actual control input vector for the follower spacecraft, mf is the mass of the follower spacecraft, C(ω)∈ℝ3×3 is the Coriolis-like matrix, N(·)∈ℝ3×3 is a nonlinear term defined by
(3)N(·)≜mf[MGx∥R+q∥3-ω2x+ulxmlMG(y+∥R∥∥R+q∥3-1∥R∥2)-ω2y+ulymlMGz∥R+q∥3+ulzml],
with M being the mass of the earth, G being the universal gravity constant, ω being the angular velocity of the leader spacecraft, and ulx, uly, and ulz being the components of the leader control input vector, and Δf is the total disturbance and uncertain forces.


## 4. Control Design

### 4.1. Inspirations from the Human Memory System

Inspired by the process of the human memory system, the so-called memory-based control method was constructed as follows [16, 17]:
(4)u=ω1U+ω2Eg-1t,
for an arbitrary system expressed as
(5)x˙=f(·)+g(·)u,
where u is the control signal, U stands for the past control experiences, E represents the past feedback information, ω1 is the behavior-memory coefficient vector, and ω2 is the feedback-memory coefficient vector. This structure well resembles the human memory and learning system.Once stimulated by a given task, the human brain analyzes its surrounding environment (system working environment) and its self-condition (system states) and then integrates those with past experience (U and E) to generate the desired control action, which is to be saved in the memory for future retrieval.More specifically, depending on how many steps (how long) you trace back in your memory, there are different order memory-based controllers. For 1st-order memory-based controllers, where only short-term memory is involved,ω1=[1]T, ω2=[-2,1]T, U=[uk-1], and E=[ek,ek-1]. The subscription k denotes current time kt, with t being the sampling period. By using Euler approximation and performing a series of mathematical manipulation for 1st-order memory-based control, it is easily shown that as time goes to infinite,
(6)ek+1=t(fk-fk-1).
Therefore,
(7)∥sk+1∥≤t2c0,
where
(8)c0=max||dfdt||,
which denotes the variation rate of the disturbance and uncertainties that cannot be infinite, and the sampling interval t is a very small number, t square is even smaller, and so the tracking error is bound in a very narrow range.The previous analysis is based on first-order or short-term memory; similar analysis can be made for higher order cases.However, the problem with the memory-based control is that, at the very beginning, no previous memory was accessible, and, hence, the controller was very immature and showed lots of chattering [17]. Therefore, we turned to the immune system for solutions.


### 4.2. Inspirations from the Immune System

The human immune system is a dynamic and intelligent system. Once one kind of antigen invades our body, a general immune response is initiated by the immune system. After the invading antigen gets acquainted, a more specific immune response starts to work against the antigen. In other words, because the immune system does not have any knowledge against the antigen, it selects a sort of universal antibody to take effect, and after the antigen is better understood by the intelligent system, a much more matching and effective kind of antibody is activated to fight. Note that the matching antibody could be either already existing or generated from mutation and evolution.Analogies could be made between the immune system and our control system. The immune system is like a big controller [16, 17], and different kinds of antibodies are components and subcontrollers that work together to achieve a designated objective.Therefore, a novel approach called memory/immunology-based control was developed. The controller is constructed as follows (Figure6):
(9)u=(1-σ)uN+σuA,
where u is the overall control signal (overall immune response), u1 and u2 are two candidates (two kinds of antibodies in the pool), and σ is a function adjusting the concentration of each kind of antibody, which is called immune selection function.Figure 6
Memory/immunology-based control.


#### 4.2.1. Immune Selection Function

The selection ofuN is simple, uN is a general and classic proportional subcontroller, and uA is the memory-based subcontroller. However, how to choose an appropriate immune selection function continues to be a problem. In this paper, an intelligent immune selection function was introduced as follows:
(10)σ=1-exp(-μt∥e∥+ε),μ>0,ε>0.
Obviously, this immune selection function incorporates control error. It varies as the magnitude of e fluctuates. When ∥e∥ is large, which means the antigen is not well understood, the function value approaches 0, and therefore the general response tends to function more, whereas when ∥e∥ is small, which means that learned experience has been precise and has been mature, the function value goes to 1, and thus the specific antibody takes role.


### 4.3. Control Design for MSFF System

Now we show the detailed memory/immunology-based control design procedure for the MSFF system.First, we introduce a new variables=e˙+βe, where e=q-q* is the tracking error and β is an arbitrary positive real number. It is easy to prove that if s goes to zero, e will go to zero too.The proposedmth-order memory/immunology-based controller is of the form
(11)u=(1-σ)uN+σuA,uA=ω1U+(ω2mft)S,uN=mf(-ksk+q¨*-βe˙),
with t being the sampling period, U=[uk-1,uk-2,…,uk-m]T storing the past control experiences, S=[sk,sk-1,…,sk-m]Tstoring the past feedback information, ω1∈ℝm being the behavior-memory coefficient vector, and ω2∈ℝm+1 being the feedback-memory coefficient vector. The mth-order here means that the retroactive effect of the controller is to time (k-m)t.Whenm=1, we have
(12)ω1=[1]T,ω2=[-2,1]T,
and by using Euler approximation and performing a series of mathematical manipulation, it is easily shown that as T→∞, sk+1=t(Δfk-Δfk-1). Therefore, ∥sk+1∥≤t2c0, where c0=max||dΔf(·)/dt||, which denotes the variation rate of the disturbance and uncertainties that cannot be infinite, and the sampling interval t is a very small number, t square is even smaller, and so the tracking error is bound in a very narrow range.The previous analysis is based on first-order controller; similar analysis can be made for higher order case. Presumably, higher order control leads to better control precision.


## 4.1. Inspirations from the Human Memory System

Inspired by the process of the human memory system, the so-called memory-based control method was constructed as follows [16, 17]:
(4)u=ω1U+ω2Eg-1t,
for an arbitrary system expressed as
(5)x˙=f(·)+g(·)u,
where u is the control signal, U stands for the past control experiences, E represents the past feedback information, ω1 is the behavior-memory coefficient vector, and ω2 is the feedback-memory coefficient vector. This structure well resembles the human memory and learning system.Once stimulated by a given task, the human brain analyzes its surrounding environment (system working environment) and its self-condition (system states) and then integrates those with past experience (U and E) to generate the desired control action, which is to be saved in the memory for future retrieval.More specifically, depending on how many steps (how long) you trace back in your memory, there are different order memory-based controllers. For 1st-order memory-based controllers, where only short-term memory is involved,ω1=[1]T, ω2=[-2,1]T, U=[uk-1], and E=[ek,ek-1]. The subscription k denotes current time kt, with t being the sampling period. By using Euler approximation and performing a series of mathematical manipulation for 1st-order memory-based control, it is easily shown that as time goes to infinite,
(6)ek+1=t(fk-fk-1).
Therefore,
(7)∥sk+1∥≤t2c0,
where
(8)c0=max||dfdt||,
which denotes the variation rate of the disturbance and uncertainties that cannot be infinite, and the sampling interval t is a very small number, t square is even smaller, and so the tracking error is bound in a very narrow range.The previous analysis is based on first-order or short-term memory; similar analysis can be made for higher order cases.However, the problem with the memory-based control is that, at the very beginning, no previous memory was accessible, and, hence, the controller was very immature and showed lots of chattering [17]. Therefore, we turned to the immune system for solutions.


## 4.2. Inspirations from the Immune System

The human immune system is a dynamic and intelligent system. Once one kind of antigen invades our body, a general immune response is initiated by the immune system. After the invading antigen gets acquainted, a more specific immune response starts to work against the antigen. In other words, because the immune system does not have any knowledge against the antigen, it selects a sort of universal antibody to take effect, and after the antigen is better understood by the intelligent system, a much more matching and effective kind of antibody is activated to fight. Note that the matching antibody could be either already existing or generated from mutation and evolution.Analogies could be made between the immune system and our control system. The immune system is like a big controller [16, 17], and different kinds of antibodies are components and subcontrollers that work together to achieve a designated objective.Therefore, a novel approach called memory/immunology-based control was developed. The controller is constructed as follows (Figure6):
(9)u=(1-σ)uN+σuA,
where u is the overall control signal (overall immune response), u1 and u2 are two candidates (two kinds of antibodies in the pool), and σ is a function adjusting the concentration of each kind of antibody, which is called immune selection function.Figure 6
Memory/immunology-based control.


### 4.2.1. Immune Selection Function

The selection ofuN is simple, uN is a general and classic proportional subcontroller, and uA is the memory-based subcontroller. However, how to choose an appropriate immune selection function continues to be a problem. In this paper, an intelligent immune selection function was introduced as follows:
(10)σ=1-exp(-μt∥e∥+ε),μ>0,ε>0.
Obviously, this immune selection function incorporates control error. It varies as the magnitude of e fluctuates. When ∥e∥ is large, which means the antigen is not well understood, the function value approaches 0, and therefore the general response tends to function more, whereas when ∥e∥ is small, which means that learned experience has been precise and has been mature, the function value goes to 1, and thus the specific antibody takes role.


## 4.2.1. Immune Selection Function

The selection ofuN is simple, uN is a general and classic proportional subcontroller, and uA is the memory-based subcontroller. However, how to choose an appropriate immune selection function continues to be a problem. In this paper, an intelligent immune selection function was introduced as follows:
(10)σ=1-exp(-μt∥e∥+ε),μ>0,ε>0.
Obviously, this immune selection function incorporates control error. It varies as the magnitude of e fluctuates. When ∥e∥ is large, which means the antigen is not well understood, the function value approaches 0, and therefore the general response tends to function more, whereas when ∥e∥ is small, which means that learned experience has been precise and has been mature, the function value goes to 1, and thus the specific antibody takes role.


## 4.3. Control Design for MSFF System

Now we show the detailed memory/immunology-based control design procedure for the MSFF system.First, we introduce a new variables=e˙+βe, where e=q-q* is the tracking error and β is an arbitrary positive real number. It is easy to prove that if s goes to zero, e will go to zero too.The proposedmth-order memory/immunology-based controller is of the form
(11)u=(1-σ)uN+σuA,uA=ω1U+(ω2mft)S,uN=mf(-ksk+q¨*-βe˙),
with t being the sampling period, U=[uk-1,uk-2,…,uk-m]T storing the past control experiences, S=[sk,sk-1,…,sk-m]Tstoring the past feedback information, ω1∈ℝm being the behavior-memory coefficient vector, and ω2∈ℝm+1 being the feedback-memory coefficient vector. The mth-order here means that the retroactive effect of the controller is to time (k-m)t.Whenm=1, we have
(12)ω1=[1]T,ω2=[-2,1]T,
and by using Euler approximation and performing a series of mathematical manipulation, it is easily shown that as T→∞, sk+1=t(Δfk-Δfk-1). Therefore, ∥sk+1∥≤t2c0, where c0=max||dΔf(·)/dt||, which denotes the variation rate of the disturbance and uncertainties that cannot be infinite, and the sampling interval t is a very small number, t square is even smaller, and so the tracking error is bound in a very narrow range.The previous analysis is based on first-order controller; similar analysis can be made for higher order case. Presumably, higher order control leads to better control precision.


## 5. Simulation Results

To verify the effectiveness of the proposed control method, we conduct numerical simulation of three-vehicle formation flight. For simplicity the first-order memory-based controller is applied to the system. The parameters are chosen as:(13)t=0.02,α=0.05,γ=0.02,β=5,k=10.The disturbance is given as(14)dis=[20sin(T)+2cos2(T)30cos(T)+6sin2(T)sin3(T)].We consider a three-spacecraft MSFF system including one leader and two followers, and our control goal is to keep the three spacecrafts in rigid formation, that is, in another word, to keep the relative position vectors between two followers and the leader constant while the leader is in any arbitrary orbital trajectory.The desired relative position for follower 1 is[4,3,1], and for follower 2 is [-10,7,-1].In Figure7, the tracking trajectories are shown, and the leader is flying in an ellipse orbit.Figure 7
Tracking trajectories.And in Figure8, we can see the control forces of follower 1, which is very smooth and bounded.Control forces of follower 1.
(a)
(b)
(c)Next, in Figure9, we show the tracking performances of follower 1 along each dimension of {x,y,z}, and at last, in Figure 10, the tracking errors of follower 1 in each dimension are given, from which we can see that under the control of the proposed method, the vehicles are driven to the destination/position quickly and precisely.Tracking performance of follower 1.
(a)
(b)
(c)Tracking errors of follower 1.
(a)
(b)
(c)


## 6. Conclusion

A memory/immunology-based concept is used to design a highly robust and adaptive control scheme for close formation flying of multiple spacecrafts. The developed control algorithms are implemented for 3-vehicle system and achieved good performance.The immunology/memory-based control brings in the inspiration from the immune system. It learns how the immune system generates an appropriate immune response by choosing different antibodies and applying this process into the system controls. This approach is especially efficient and superior when it is applied to a system with lots of disturbances and uncertainties, because of the fact that it requires little information about the disturbances and uncertainties and does not rely on precise model dynamics.



---

*Source: 456410-2013-05-02.xml*