# Optimal Control through Leadership of the Cucker and Smale Flocking Model with Time Delays

Authors: Adsadang Himakalasa; Suttida Wongkaew
Journal: Complexity
 (2021)

---

## Abstract

The Cucker and Smale model is a well-known flocking model that describes the emergence of flocks based on alignment. The first part focuses on investigating this model, including the effect of time delay and the presence of a leader. Furthermore, the control function is inserted into the dynamics of a leader to drive a group of agents to target. In the second part of this work, leadership-based optimal control is investigated. Moreover, the existence of the first-order optimality conditions for a delayed optimal control problem is discussed. Furthermore, the Runge–Kutta discretization method and the nonlinear conjugate gradient method are employed to solve the discrete optimality system. Finally, the capacity of the proposed control approach to drive a group of agents to reach the desired places or track the trajectory is demonstrated by numerical experiment results.


---

## Body

## 1. Introduction

Individual behaviors influence the dynamics of social systems. Studying the behavior of interacting individuals within a group of animals or a community of people is a new mathematical research field. The emergence of a collective level is one interesting issue raised by this type of study, with many natural systems showing collective phenomena, for instance, a flock of birds or a group of fish having coordinated movements, cells, chemical compounds, organisms or crystals, vehicular traffic, crowd dynamics, market economies, opinion formations, distributions of wealth, networks, and artificial intelligence (see, e.g., [1–6]). These collective behaviors are described as pattern configurations. In particular, collective action results from a superimposition of interactions between every possible pair of all agents. In general, the strength of such interaction forces depends on the amount of collaborative distance between agents. For example, birds orient with their nearest one in the group, and people more often agree with others who already conform to their beliefs. Nowadays, the application of these collective features is enormous and has a wide range. It appears in several engineering applications such as those using collective properties to perform complex tasks or in swarm robotics [7]. In addition, crowd models are applied in civil engineering to plan evacuation strategies in buildings (see, e.g., [8, 9]).In recent years, researchers have been interested in studying collective phenomena, starting with using a few rules of interaction among agents to predict and explain unexpected phenomena or describe the emergence of pattern structure. Despite many works studying the emergence of consensus (see, e.g., [4, 10, 11]), investigating the enforcement and stabilization of pattern formation including the presence of noise or communication with time delays is especially interesting. Communication delays result from traffic congestion or finite speeds of transmission and spread. For example, the flocking model has a processing delay when analyzing information about the location and velocity of neighboring agents. This fact has motivated an increased interest in studying the consensus problem of a flocking system, including the effect of time delays. Pioneering work on this issue was done in [12–16]. Additionally, when the system’s behavior does not realize the desired result, control is essential to force the system to attain given objectives. This problem is related to the control of a self-organized system. Most results focus on the controllability of systems in which the topology of a group communication network is fully connected. In this case, all members of the group are regulated by the same distributed control law. These model settings, however, are limited when dealing with large networks. In particular, efficient control should apply only to a few group members, and this control strategy is known as “sparsity” in the mathematical literature. From the perspective of controllability, the hierarchical leadership concept provides the aggregation states of the system and some forms of group patterns for longtime behavior. The literature on control problems in these systems has recently been documented in [17–23].The main contribution is divided into two parts. The first part discusses the Cucker and Smale (CS) flocking model, where the position and velocity of agents include the effect of time delay. Furthermore, primarily based on this model, a leader is added into the system to implement control functions. This control action can be applied to other agents through the mechanism of interaction force. The objective of the control strategy in this construction is to drive the evolution of flocking to track the desired trajectory. This control through leadership approach is constructed in the framework of optimal control problem mainly discussed in the second part of this work. In the second part, firstly, the formulation of optimal control problems with time delay is provided. The objective function includes three tracking terms and the cost of control function. The first and second terms account for the tracking error of the position of the leader and the desired target position at the final time and follow the desired path, respectively. The last tracking term is a measure of the distance between the leader and the other agents of the flock. Further, the theoretical and numerical results of optimization problems constrained with delayed CS systems are investigated. The resulting optimal control is obtained by solving an optimality system composed of the CS model’s delayed dynamical system, associated adjoint equations, and an optimality condition. Additionally, the aspect of numerical solution and implementation is discussed. An efficient conjugate gradient optimization procedure evaluates optimal control [24, 25]. The gradient of the reduced cost functional is computed using the adjoint framework by solving the forward and backward delayed CS flocking equations that appear in the optimality system. These equations are discretized using an accurate high-order Runge–Kutta scheme (see more details in [26]).The subsequent sections of our work are organized as follows. In Section2, the description of the flocking model is provided. This model is based on the Cucker and Smale model, which takes time delays and the presence of a leader into account. Additionally, the existence of solutions and stability for these systems are discussed. Furthermore, numerical simulations are demonstrated to examine the characteristic properties of our constructed model. Section 3 presents the formulation of optimal control of the Cucker and Smale system with time delays. The theoretical issues regarding the existence of first-order necessary optimality systems of delayed optimal control problems are discussed. Section 4 is comprised of two parts. In the first part, the control problem with time delays is discretized by using the Runge–Kutta scheme. Then, using the first-discretize-then-optimize strategy, a discrete gradient formula for the optimization problem is obtained. The second part of Section 4 contains numerical experiments demonstrating the validity and effectiveness of the proposed control strategy. The conclusion is provided in the last section to complete this work.


## 2. Cucker and Smale Flocking Model with Time Delays and Leadership

In this section, we investigate the flocking model, including the effect of time delays and the presence of a leader. In this work, we focus on the Cucker and Smale (CS) model. This model is one of the well-known flocking models that involve the alignment of the agent; that is, each agent changes its speed in response to an averaging weight of its relative speed to the other agents, which can be seen in [5]. Moreover, we investigate the effect of time delays, which have influenced to converge to state transition and pattern structure. In our case, we consider constant delay function as proposed in [27]. In this context, delays are defined as a lag in the processing of information. To illustrate this phenomenon, consider a situation involving two agents in which the second agent attempts to follow the first but receives information from the first agent after some time. As a result, it requires time to elaborate on its response. We define τ as a positive constant time delay. Additionally, a leader is incorporated into the CS system to facilitate developing a control strategy for guiding the flock dynamical system toward a target (described in the following section). The CS flocking system with one leader and N followers including time delay is presented as follows:(1)x˙0t=v0t,v˙0t=1N+1∑j=1Na0jt−τvjt−τ−v0t,x˙bt=vbt,v˙bt=1N+1∑j=1Nabjt−τvjt−τ−vbt+Fb0,for b=1,…,N, with given initial conditions(2)xit=xi0t,vit=vi0t,fori=0,…,Nandt∈−τ,0,where xit,vit∈ℝd×ℝd,d≥1, are position and velocity of the ith agent at time t. The initial conditions xi0t and vi0t are given continuous functions. The connectivity function aij measures the interaction strength between agents depending on the distance between ith and jth agents. The connectivity function is assumed to be positive, continuous, bounded, and nonincreasing function.(3)0<aijt≤1,∀t∈−τ,+∞,fori=0,…,Nandj=1,…,N.In this work, we consider the connectivity function with the following form:(4)abjt≔φxit−xjt=K1+xbt−xjt2σ,where K>0, σ≥0, and the notation ⋅ denotes Euclidean norm in ℝn. The force Fb0 governs the action of the leader with time delay. The leader force can take many forms, such as using short-range repulsion and long-range attraction described as Morse potential function in [18] or considering hierarchical leadership structure in [16]. In this work, the leader’s action takes the following form:(5)Fb0=γv0t−τ−vbt,where the parameter γ>0 corresponds to the strength of the interaction force taken by the leader.


### 2.1. Stability of the Cucker and Smale Model with Time Delays

The solution of the CS flocking system is influenced by the parametersK and σ. In particular, when σ<1/2, the CS model without time delays exhibits unconditional flocking for all initial configurations, in which the velocities vi approach asymptotically to the common limit velocity v∗. On the other hand, the flocking is conditional with σ≥1/2, i.e., the asymptotic behavior of the system is dependent on the value of K and the initial configuration, as detailed in references [5, 14, 27]. To analyze the solution and stability of system (1), we consider the dispersion and disagreement as defined in [16, 19].Definition 1.
(dispersion and disagreement). Given a solutionxit,vit∈ℝd×ℝd of system (1), for i=0,…,N, we define dispersion and disagreement as(6)Xt=12N+12∑i,j=1Nxit−xjt,Vt=12N+12∑i,j=1Nvit−vjt.
Then, we say that the solution tends toconsensus if(7)supt>0Xt<+∞,limt⟶+∞Vt=0.
Next, the numerical simulation results of the CS system with time delays, including the presence of a leader, are demonstrated. Figure1 depicts the numerical results of system (1) in one dimension, while Figures 2 and 3 illustrate results in two dimensions. For all experiments, we consider one leader and N=9 followers. The corresponding parameters in connectivity function are chosen as K=2 and σ=1/4. The leader interaction strength parameter is given as γ=10. Consider 1D case, and Figure 1(a) shows the trajectories of agents with no time delay, whereas the motion of flocking, including the effect of time delay (τ=5), is presented in Figures 1(b)–1(d). It is observed that due to time delay, agents require more time to adjust their alignments and follow the group.
We also consider one leader andN=9 followers in the experiments for 2D cases. The parameters for the 2D experiments are the same as those for the 1D studies. Agents begin with an initial configuration that places them all in random positions with random velocities, with a leader outside the group. After t=2, each agent begins to adapt and organize its movement to follow the leader. Figure 3 shows that for the longtime behavior t⟶∞, the value Vt tends to be constant at zero and the dispersion Xt is becoming constant. It explains that the consensus is reached after some time. The pattern configuration is formed, and each agent’s velocity is synchronized and moves together at the same speed.Figure 1
Simulation of the CS model with one leader and nine followers. (a) shows the trajectories of agents where the motion has no delay. (b)–(d) present the trajectories of agents where the movement of flocks have an effect of time delay (τ=5) in time t=0,20, t=0,100, and t=0,200, respectively. Note that the dashed line represents the trajectory of the leader, while the blue line denotes the trajectory of the follower. (a) Without time delay for t∈0,20. (b) With delay (τ=5) for t∈0,20. (c) With delay (τ=5) for t∈0,100. (d) With delay (τ=5) for t∈0,200.
(a)(b)(c)(d)Figure 2
The simulation of the leadership of CS model with time delayτ=2 for one leader and nine followers in two dimensions. Parameters in the connectivity function are chosen as K=2,σ=1/4 and leader interaction force is given as γ=10. (a)–(c) present the positions of the agents and their velocities at time t=0, t=20, t=50, respectively. The circles represent the positions of each agent and the shaded circle denotes the leader. (d) shows trajectories of agents in t∈0,50. (a) Initial configuration. (b) t=20. (c) t=50. (d) Trajectories of agents.
(a)(b)(c)(d)Figure 3
The dispersion and disagreement of the leadership of CS model with time delayτ=2 for t∈0,50. (a) Xt. (b) Vt.
(a)(b)


## 2.1. Stability of the Cucker and Smale Model with Time Delays

The solution of the CS flocking system is influenced by the parametersK and σ. In particular, when σ<1/2, the CS model without time delays exhibits unconditional flocking for all initial configurations, in which the velocities vi approach asymptotically to the common limit velocity v∗. On the other hand, the flocking is conditional with σ≥1/2, i.e., the asymptotic behavior of the system is dependent on the value of K and the initial configuration, as detailed in references [5, 14, 27]. To analyze the solution and stability of system (1), we consider the dispersion and disagreement as defined in [16, 19].Definition 1.
(dispersion and disagreement). Given a solutionxit,vit∈ℝd×ℝd of system (1), for i=0,…,N, we define dispersion and disagreement as(6)Xt=12N+12∑i,j=1Nxit−xjt,Vt=12N+12∑i,j=1Nvit−vjt.
Then, we say that the solution tends toconsensus if(7)supt>0Xt<+∞,limt⟶+∞Vt=0.
Next, the numerical simulation results of the CS system with time delays, including the presence of a leader, are demonstrated. Figure1 depicts the numerical results of system (1) in one dimension, while Figures 2 and 3 illustrate results in two dimensions. For all experiments, we consider one leader and N=9 followers. The corresponding parameters in connectivity function are chosen as K=2 and σ=1/4. The leader interaction strength parameter is given as γ=10. Consider 1D case, and Figure 1(a) shows the trajectories of agents with no time delay, whereas the motion of flocking, including the effect of time delay (τ=5), is presented in Figures 1(b)–1(d). It is observed that due to time delay, agents require more time to adjust their alignments and follow the group.
We also consider one leader andN=9 followers in the experiments for 2D cases. The parameters for the 2D experiments are the same as those for the 1D studies. Agents begin with an initial configuration that places them all in random positions with random velocities, with a leader outside the group. After t=2, each agent begins to adapt and organize its movement to follow the leader. Figure 3 shows that for the longtime behavior t⟶∞, the value Vt tends to be constant at zero and the dispersion Xt is becoming constant. It explains that the consensus is reached after some time. The pattern configuration is formed, and each agent’s velocity is synchronized and moves together at the same speed.Figure 1
Simulation of the CS model with one leader and nine followers. (a) shows the trajectories of agents where the motion has no delay. (b)–(d) present the trajectories of agents where the movement of flocks have an effect of time delay (τ=5) in time t=0,20, t=0,100, and t=0,200, respectively. Note that the dashed line represents the trajectory of the leader, while the blue line denotes the trajectory of the follower. (a) Without time delay for t∈0,20. (b) With delay (τ=5) for t∈0,20. (c) With delay (τ=5) for t∈0,100. (d) With delay (τ=5) for t∈0,200.
(a)(b)(c)(d)Figure 2
The simulation of the leadership of CS model with time delayτ=2 for one leader and nine followers in two dimensions. Parameters in the connectivity function are chosen as K=2,σ=1/4 and leader interaction force is given as γ=10. (a)–(c) present the positions of the agents and their velocities at time t=0, t=20, t=50, respectively. The circles represent the positions of each agent and the shaded circle denotes the leader. (d) shows trajectories of agents in t∈0,50. (a) Initial configuration. (b) t=20. (c) t=50. (d) Trajectories of agents.
(a)(b)(c)(d)Figure 3
The dispersion and disagreement of the leadership of CS model with time delayτ=2 for t∈0,50. (a) Xt. (b) Vt.
(a)(b)


## 3. Optimal Control of Cucker and Smale Model with Time Delays

This section discusses the optimal control problem constrained with the delayed dynamical system of the CS model, including a leader. We note that this control problem is the leadership-based control strategy which is discussed in [18] for the refined flocking model. In addition, the theoretical results corresponding to optimal control problems with delays are investigated. In particular, the existence and characterization properties of solutions and optimality conditions are provided. In the following, we consider the optimal control problem governed by the delayed CS system and the presence of a leader presented as(8)minx,uJx,u=12x0T−xdesT2+η2∫0Tx0t−xdest2dt+μ2∑b=1N∫0Tx0t−xbt4dt+ν2∫0Tut2dt,together with the delayed differential constraints given by(9)x˙0t=v0t,v˙0t=1N+1∑j=1Na0jt−τvjt−τ−v0t+ut,x˙bt=vbt,v˙bt=1N+1∑j=1Nabjt−τvjt−τ−vbt+γv0t−τ−vbt,for b=1,…,N, along with their corresponding initial conditions. The state of the system is represented by the notation(10)xt=x0t,…,xNt,v0t,…,vNt∈ℝ2N+1d.The control function is denoted byu∈L20,T;ℝd. The parameters η,μ, and ν in the cost functional are positive constants. The first term in objective functional aims to minimize the terminal position of the leader and the given target. The second term is related to track the desired path by the leader agent. Also, the distance between the leader agent and others is measured in the third term. The cost of control is minimized in the last term.To investigate the solution of our optimal control problems (8), we consider the optimal control problem that state and control variables have an effect of time delays, that is, state variable xt and control variable ut include time delays τx≥0 and τu≥0, respectively. The theoretical results of the delayed optimal control problems are investigated in [28, 29]. The general framework of an optimal control problem with delay in state and control is presented as follows:(11)minJx,u=ϕxT,subject tox˙t=ft,xt,xt−τxut,ut−τu,xt=x0t,t∈−τx,0,ut=u0t,t∈−τu,0.We assume thatxt∈W1,∞0,T,ℝnx and ut∈L∞0,T,ℝnc. The function ϕ:ℝnx⟶ℝ represents the objective function and is assumed to be continuously differentiable. f:0,T×ℝnx×ℝnx×ℝnc×ℝnc⟶ℝnx represents the dynamics of the model and is assumed to be continuously differentiable. The initial conditions x0:−τx,0⟶ℝnx and u0:−τu,0⟶ℝnc are continuous. We note that a solution of the dynamical system in (11) is uniquely determined for a given control u and the initial conditions denoted by x=xu; moreover, u↦xu is a differentiable map. For all admissible u, we aim to find a control u∗ such that(12)Jxu∗,u∗≤Jxu,u.Furthermore, to obtain the necessary optimality conditions for delayed optimal control (11), we introduce two additional variables y∈ℝnx and w∈ℝnc for delayed state and control variables, respectively, where the functions y and w are defined as follows:(13)yt=xt−τx,wt=ut−τu.The first-order optimality system can be obtained by considering the Hamiltonian function that is given by(14)ℋt,x,y,u,w,p≔p⊤ft,x,y,u,w,where p∈ℝnx.Theorem 1.
Letx∗,u∗ be locally optimal solution to optimal control problem (11). Assume that there exists an open set Ω⊂ℝnx×ℝnc such that the neighborhood Bɛx∗,u∗ with a small radius ϵ>0 of x∗,u∗ is a subset of Ω for every t∈0,T. We assume that ∇xf and ∇uf are Lipschitz continuous in Ω. Moreover ∇xJ is Lipschitz continuous in the neighborhood Bɛx∗T. Then, there exists an adjoint function p∗∈W1,∞0,T,ℝnx that satisfies the first-order optimality conditions at x∗,p∗,u∗, for a.e. t∈0,T such that(15)x˙t=ft,xt,y,ut,wt,xt=x0t,fort∈−τx,0,p˙t=−∇xℋt−χ0,T−τxt∇yℋt+τx,pT=∇xϕxT,∇uℋt+χ0,T−τxt∇wℋt+τx=0,where ∇xℋt, ∇yℋt, ∇uℋt, and ∇wℋt refer to the evaluation of the partial derivatives ℋ with respect to x,y,u, and w, respectively. The function χ is defined as(16)χa,bt=1,ift∈a,b,0,otherwise.To investigate the solution of our problem (10), we formulate our optimal control in form (11) by introducing the following equation:(17)x∧˙t=η2x0t−xdest2+μ2∑b=1Nx0t−xbt4+ν2ut2,x∧0=0.Then, the problem of optimal control (10) can be transformed into the following problem:(18)minJx,u=12x0T−xdesT2+x∧T,subject tox˙0t=v0t,v˙0t=1N+1∑j=1Na0jt−τvjt−τ−v0t+ut,x˙bt=vbt,v˙bt=1N+1∑j=1Nabjt−τvjt−τ−vbt+γv0t−τ−vbt,for b=1,…,N and(19)x∧˙t=η2x0t−xdest2+μ2∑b=1Nx0t−xbt4+ν2ut2,together with corresponding initial conditions, and the compact form can be rewritten as follows:(20)minJx˜,u=12x0T−xdesT22+x∧Tsubject tox˜.t=Ft,x˜,u.Now, the state variable is(21)x˜t=x0t,x1t,…,xNt,v0t,v1t,…,vNt,x∧tT,and Fx˜,u represents the dynamics of the transformed delayed CS systems.Further, we discuss the first-order optimality conditions for optimal control problem (20). For this purpose, the delayed system is transformed to nondelayed system by using variables(22)sit=xit−τ,zit=vit−τ,fori=0,1,…,N,and we denote yt=s0t,s1t,…,sNt,z0t,z1t,…,zNt⊤.The Hamiltonian function for system (20) is presented as(23)ℋt,x˜,y,u,p=p⊤Ft,x˜,y,u,where pt=px0t,pxbt,pv0t,pvbt,px∧t⊤∈ℝnx,nx=2N+1d+1.Assumption 1.
There exist Lagrange multipliersp∈W1,∞0,T,ℝnx corresponding to the optimization constraint equation of problem (20). Moreover, the adjoint p satisfies the following equation:(24)p˙t=−∇xFt⊤pt−χ0,T−τt∇yFt+τ⊤pt+τ,pT=∇xϕxT.
Therefore, a solution of (20) can be characterized by the first-order optimality system as presented in the following:(25)x˜˙t=Ft,x˜,u,x˜t=x˜0tfort∈−τ,0,p˙t=−∇xFt⊤pt−χ0,T−τt∇yFt+τ⊤pt+τ,pT=∇xϕxT.
In addition, the corresponding gradient can be expressed as(26)∇uJu=−∇uFt⊤pt.
In particular, the explicit formulations of adjoint equations (20) can be expressed as follows:(27)p˙x0t=−ηpx∧t,x0t−xdest−2μpx∧t,∑b=1Nx0t−xbt3+χt0,T−τt1N+1pv0t+τ,∑j=1Na0j′tvjt−v0t+τ,p˙v0t=−px0t+1N+1∑j=1Na0jt−τpv0t−γχt0,T−τt∑b=1Npvbt+τ,p˙xbt=2μpx∧t,x0t−xbt3−χt0,T−τt1N+1pv0t+τ,a0b′tvbt−v0t+τ+χt0,T−τt1N+1pv0t+τ,∑j=1Nabj′tvjt−vbt+τ,p˙vbt=−pxbt+1N+1∑j≠bNabjt−τpvbt−γpvbt−χt0,T−τt1N+1a0btpvbt+τ,p˙x∧t=0,with transversality condition(28)pxbT=x0T−xdesT,pxbT=pvbT=pv0T=0,px∧T=1.
The numerical solution of problems (20) representing as the considered optimality systems (25) will be shown and discussed in the next section.


## 4. Numerical and Implementation Aspects of Delayed Control Problems

### 4.1. First-Discretize-Then-Optimize Strategy and Discrete Optimality Conditions

The discretization of the reduced gradient is a critical step in the numerical method for solving optimal control problems. To obtain an accurate discretization for the reduced gradient, we consider the first-discretize-then-optimize strategy. To carry out this strategy, one can follow the procedures outlined below. The cost functional and corresponding differential constraints representing the optimal control problem are discretized in the first step using the Runge–Kutta method, discussed in detail in [26]. Second, the discrete Lagrangian function corresponding to the discrete optimal control problem must be constructed. The final step is to obtain the first-order optimality system for the discretized problem.For the discretization of optimal control problem (20), the Runge–Kutta scheme is employed on a uniform mesh in the time intervals 0,T such that the step-sized h is defined as(29)h=Tn,with the total number of discrete time intervals, n. It is important to match the uniform step size h>0 with positive delay τ; that is, we choose any integer fraction of h to refine the discretization grid. For this task, for any step size h>0, there exist positive integers n,mτ∈ℕ satisfying(30)τ=mτh,T=nh.By this setting, we denote the value ofx˜t at the discrete time tk by(31)x˜k=x˜tk,tk=kh,fork=0,…,n−1.Therefore, the discretization of thes-stage Runge–Kutta scheme setting for optimal control problem (20) becomes(32)minx˜,uJx˜k,uk=12x0T−xdesT22+x∧T,subject tox˜k+1=x˜k+h∑i=1sbiFtk,x˜k,x˜k−mτ,ψki,uki,x˜−k=x˜0−kh,k=0,…,mτ,ψki=x˜k+h∑j=1saijFtk,x˜k,x˜k−mτ,ψkj,ukj,for k=0,…,n−1 and 1≤i,j≤s. Note that the control uk∈Rnc×s denotes the s stages of the RK discrete control function at the stage k, which can be written in the following form:(33)uk=uk1,uk2,…,uks.We remark that the order of a Runge–Kutta discretization for optimal control problem depends on coefficientsaij and bi (see more details in [26]).The discrete optimality system that corresponds to (32) is expressed as the following equations:(34)x˜k+1=x˜k+h∑i=1sbiFtk,x˜k,x˜k−mτ,ψki,uki,x˜−k=x˜0−kh,ψki=x˜k+h∑j=1saijFtk,x˜k,x˜k−mτ,ψkj,ukj,Pk=Pk+1+∑i=1sbiζki,Pn=−∇xϕx˜n,ζki=∇xFtk,x˜k,x˜k−mτ,ψkj,ukj⊤ψk+1+∑j=1sbjaijbiζkj+χ0,T−mτk∇yFtk+mτ,x˜k+mτ,x˜k,ψkj+mτ,ukj+mτ⊤ψk+1+mτ+χ0,T−mτk∇yFtk+mτ,x˜k+mτ,x˜k,ψkj+mτ,ukj+mτ⊤∑j=1sbjaijbiζkj+mτ.Further, the gradient is in the following form:(35)∇ukiJu=−∇uFψki,uki⊤Pk+1+∑j=1sbjaijbiζkj,for 1≤i,j≤s, and 0≤k≤n−1.To solve optimization problem (32), the nonlinear conjugate gradient (NCG) strategy with the Hager and Zhang scheme [24, 25] is implemented. First, we use the following algorithm to compute the gradient specified in First, we use the following algorithm to compute the gradient specified in (35).Following that, we apply the gradient derived from Algorithm1 to the NCG scheme defined in Algorithm 2.Algorithm 1:Evaluation of the gradient at u.
Inputu;(1)
The initial conditions are provided and then solve the forward the discrete CS model;(2)
Compute the terminal condition and solve the discrete adjoint equation in (34);(3)
Evaluate the gradient∇uJu using (35);Algorithm 2:NCG with Hager and Zhang scheme.
Input: u0 index k=0, maximum kmax, tolerance tol>0.While (k<kmax and ∇Juk>tol) do(1)
Obtainx˜k from step 1 in Algorithm 1 with uk;(2)
GetPk from step 1 in Algorithm 1 with x˜k,uk;(3)
Evaluate the gradient∇uJuk from step 1 in Algorithm 1;(4)
Computeαk by using backtracking line search scheme with Armijo’s condition [30];(5)
Computeβk by using Hager and Zhang search direction formula [24];(6)
Updateuk+1=uk+αkβk;(7)
k=k+1End while


### 4.2. Numerical Results

This section presents the numerical results of our flocking models. These results demonstrate the control performance of our leader-based control strategy. The test is divided into two parts as follows:(i)
Test I: reach the desired target point.(ii)
Test II: follow the desired path.In our experiments, we investigate problem (20) in two dimensions by considering a system composed of nine agents and one leader. As a result, the state variables’ total nx=20, while the control variables’ total nc=1. In numerical test I, we focus on controlling the leader agent to reach the goal position at the final time. The purpose of numerical test II is to force the flocks to follow the desired trajectories. We provide two examples of the desired trajectory in these tests: the linear and circular paths. We use the same initial conditions in all three examples, where agents’ positions are distributed, and their initial velocities are random. Additionally, the initial position of the leader is set to x0=0,0.In numerical test I, the goal position is given asxdes=5,5 and the end time is set to T=10. The interaction force parameters are K=2,σ=0.25 and the strength of leader interaction force is γ=10. The objective of this test is to force the system to reach the desired target at the final time; as a consequence, the corresponding parameters in objective functional are μ=0.1,ν=1, and η=0. As illustrated in Figure 4, the leader is capable of reaching the desired location and leading the flock there.Figure 4
The movement of agents reaching final target position governed by leadership of CS model with time delayτ=2 for t∈0,10. (a) Initial configuration. (b) t=1. (c) t=2. (d) t=5. (e) t=10. (f) t∈0,10.
(a)(b)(c)(d)(e)(f)In numerical test II, the corresponding parameters and initial data are chosen similarly to those in the numerical test I, except thatη=10 is used to keep the leader agent tracking the desired trajectories. We illustrate two scenarios using two distinct tracking trajectories. The results of the first example are depicted in Figure 5. The plot of snapshots for ten agents traveling along the specified linear path is shown in Figure 5. Figures 5(a)–5(e) illustrate the solutions to optimal control problems at various time points. As illustrated in Figure 5(f), the leader attempts to move in the direction of a given trajectory while the group of agents follows the leader. In the last example, the desired path is given as a circle. As shown in Figure 6, the leader agent tracks the path, and the other agents follow the leader. It can be observed from three examples that agents keep some distance from the leader because time delay creates some absence of leader position. It would take some time for the group to respond to this emotion and attempt to follow the leader; however, the group eventually achieves it.Figure 5
The movement of agents following desired linear path. Time delay is given byτ=2 for t∈0,10. (a) Initial configuration. (b) t=1. (c) t=2. (d) t=5. (e) t=10. (f) t∈0,10.
(a)(b)(c)(d)(e)(f)Figure 6
The movement of agents tracking the desired circular trajectory governed by leadership of delayed CS mode. Time delay is given byτ=2 for t∈0,10. (a) Initial configuration. (b) t=1. (c) t=2. (d) t=5. (e) t=10. (f) t∈0,10.
(a)(b)(c)(d)(e)(f)


## 4.1. First-Discretize-Then-Optimize Strategy and Discrete Optimality Conditions

The discretization of the reduced gradient is a critical step in the numerical method for solving optimal control problems. To obtain an accurate discretization for the reduced gradient, we consider the first-discretize-then-optimize strategy. To carry out this strategy, one can follow the procedures outlined below. The cost functional and corresponding differential constraints representing the optimal control problem are discretized in the first step using the Runge–Kutta method, discussed in detail in [26]. Second, the discrete Lagrangian function corresponding to the discrete optimal control problem must be constructed. The final step is to obtain the first-order optimality system for the discretized problem.For the discretization of optimal control problem (20), the Runge–Kutta scheme is employed on a uniform mesh in the time intervals 0,T such that the step-sized h is defined as(29)h=Tn,with the total number of discrete time intervals, n. It is important to match the uniform step size h>0 with positive delay τ; that is, we choose any integer fraction of h to refine the discretization grid. For this task, for any step size h>0, there exist positive integers n,mτ∈ℕ satisfying(30)τ=mτh,T=nh.By this setting, we denote the value ofx˜t at the discrete time tk by(31)x˜k=x˜tk,tk=kh,fork=0,…,n−1.Therefore, the discretization of thes-stage Runge–Kutta scheme setting for optimal control problem (20) becomes(32)minx˜,uJx˜k,uk=12x0T−xdesT22+x∧T,subject tox˜k+1=x˜k+h∑i=1sbiFtk,x˜k,x˜k−mτ,ψki,uki,x˜−k=x˜0−kh,k=0,…,mτ,ψki=x˜k+h∑j=1saijFtk,x˜k,x˜k−mτ,ψkj,ukj,for k=0,…,n−1 and 1≤i,j≤s. Note that the control uk∈Rnc×s denotes the s stages of the RK discrete control function at the stage k, which can be written in the following form:(33)uk=uk1,uk2,…,uks.We remark that the order of a Runge–Kutta discretization for optimal control problem depends on coefficientsaij and bi (see more details in [26]).The discrete optimality system that corresponds to (32) is expressed as the following equations:(34)x˜k+1=x˜k+h∑i=1sbiFtk,x˜k,x˜k−mτ,ψki,uki,x˜−k=x˜0−kh,ψki=x˜k+h∑j=1saijFtk,x˜k,x˜k−mτ,ψkj,ukj,Pk=Pk+1+∑i=1sbiζki,Pn=−∇xϕx˜n,ζki=∇xFtk,x˜k,x˜k−mτ,ψkj,ukj⊤ψk+1+∑j=1sbjaijbiζkj+χ0,T−mτk∇yFtk+mτ,x˜k+mτ,x˜k,ψkj+mτ,ukj+mτ⊤ψk+1+mτ+χ0,T−mτk∇yFtk+mτ,x˜k+mτ,x˜k,ψkj+mτ,ukj+mτ⊤∑j=1sbjaijbiζkj+mτ.Further, the gradient is in the following form:(35)∇ukiJu=−∇uFψki,uki⊤Pk+1+∑j=1sbjaijbiζkj,for 1≤i,j≤s, and 0≤k≤n−1.To solve optimization problem (32), the nonlinear conjugate gradient (NCG) strategy with the Hager and Zhang scheme [24, 25] is implemented. First, we use the following algorithm to compute the gradient specified in First, we use the following algorithm to compute the gradient specified in (35).Following that, we apply the gradient derived from Algorithm1 to the NCG scheme defined in Algorithm 2.Algorithm 1:Evaluation of the gradient at u.
Inputu;(1)
The initial conditions are provided and then solve the forward the discrete CS model;(2)
Compute the terminal condition and solve the discrete adjoint equation in (34);(3)
Evaluate the gradient∇uJu using (35);Algorithm 2:NCG with Hager and Zhang scheme.
Input: u0 index k=0, maximum kmax, tolerance tol>0.While (k<kmax and ∇Juk>tol) do(1)
Obtainx˜k from step 1 in Algorithm 1 with uk;(2)
GetPk from step 1 in Algorithm 1 with x˜k,uk;(3)
Evaluate the gradient∇uJuk from step 1 in Algorithm 1;(4)
Computeαk by using backtracking line search scheme with Armijo’s condition [30];(5)
Computeβk by using Hager and Zhang search direction formula [24];(6)
Updateuk+1=uk+αkβk;(7)
k=k+1End while


## 4.2. Numerical Results

This section presents the numerical results of our flocking models. These results demonstrate the control performance of our leader-based control strategy. The test is divided into two parts as follows:(i)
Test I: reach the desired target point.(ii)
Test II: follow the desired path.In our experiments, we investigate problem (20) in two dimensions by considering a system composed of nine agents and one leader. As a result, the state variables’ total nx=20, while the control variables’ total nc=1. In numerical test I, we focus on controlling the leader agent to reach the goal position at the final time. The purpose of numerical test II is to force the flocks to follow the desired trajectories. We provide two examples of the desired trajectory in these tests: the linear and circular paths. We use the same initial conditions in all three examples, where agents’ positions are distributed, and their initial velocities are random. Additionally, the initial position of the leader is set to x0=0,0.In numerical test I, the goal position is given asxdes=5,5 and the end time is set to T=10. The interaction force parameters are K=2,σ=0.25 and the strength of leader interaction force is γ=10. The objective of this test is to force the system to reach the desired target at the final time; as a consequence, the corresponding parameters in objective functional are μ=0.1,ν=1, and η=0. As illustrated in Figure 4, the leader is capable of reaching the desired location and leading the flock there.Figure 4
The movement of agents reaching final target position governed by leadership of CS model with time delayτ=2 for t∈0,10. (a) Initial configuration. (b) t=1. (c) t=2. (d) t=5. (e) t=10. (f) t∈0,10.
(a)(b)(c)(d)(e)(f)In numerical test II, the corresponding parameters and initial data are chosen similarly to those in the numerical test I, except thatη=10 is used to keep the leader agent tracking the desired trajectories. We illustrate two scenarios using two distinct tracking trajectories. The results of the first example are depicted in Figure 5. The plot of snapshots for ten agents traveling along the specified linear path is shown in Figure 5. Figures 5(a)–5(e) illustrate the solutions to optimal control problems at various time points. As illustrated in Figure 5(f), the leader attempts to move in the direction of a given trajectory while the group of agents follows the leader. In the last example, the desired path is given as a circle. As shown in Figure 6, the leader agent tracks the path, and the other agents follow the leader. It can be observed from three examples that agents keep some distance from the leader because time delay creates some absence of leader position. It would take some time for the group to respond to this emotion and attempt to follow the leader; however, the group eventually achieves it.Figure 5
The movement of agents following desired linear path. Time delay is given byτ=2 for t∈0,10. (a) Initial configuration. (b) t=1. (c) t=2. (d) t=5. (e) t=10. (f) t∈0,10.
(a)(b)(c)(d)(e)(f)Figure 6
The movement of agents tracking the desired circular trajectory governed by leadership of delayed CS mode. Time delay is given byτ=2 for t∈0,10. (a) Initial configuration. (b) t=1. (c) t=2. (d) t=5. (e) t=10. (f) t∈0,10.
(a)(b)(c)(d)(e)(f)


## 5. Conclusion

The Cucker and Smale flocking model, including the effect of time delays and the presence of a leader, was studied. A control-based leadership technique was investigated for this model. An optimal control problem was formulated, and the corresponding discretization of an optimal problem was solved using an accurate RK method such that accurate gradients of the reduced objectives can be guaranteed. The nonlinear conjugate gradient scheme was implemented to compute the gradients. The results of the numerical experiment show that the proposed control approach is practical. From a modeling point of view, this flocking system could be enhanced by including the effects of attraction and repulsion and alignment with the vision cone and delay. In addition, the delay in control function would be considered a part of the optimal control problem.



---

*Source: 5545551-2021-08-31.xml*