Adaptive Critic-Based Solution to an Orbital Rendezvous Problem

T HE optimal continuous thrust rendezvous maneuver is investigated in this Note. The objective is for a rigid spacecraft to perform a maneuver into a destination orbit within a given final time. Recently, a method was proposed in [1] that could determine a global optimal solution using local numerical optimization steps. They developed a technique for generalizing the solution to different initial conditions. The authors of [2] proposed a hybrid continuousimpulsive control to solve the linearized problem. The shooting method for numerically solving transfer, intercept, and rendezvous problems is detailed in [3]. The generalized Hamilton–Jacobi– Bellman (generalized HJB) equation method was used in [4] to solve an orbital maneuver problem. A series-based solution to the resulting HJB equation, known as generating functions, was developed in [5] with a canonical transformation. In this study, the rendezvous problem is formulated as a fixedfinal-time optimal control problem with terminal state constraints. In general, many control, guidance, and path planning problems fall within this category [6]. One approach to solving such problems is to formulate the problem in an optimal control framework. The HJB equation is very difficult to solve for the time-dependent solution in a closed form for this class of problems. An open-loop solution is dependent on both the selected initial condition (IC) and the time-togo [7]. A method developed in [8] for problems with terminal soft constraints gives a closed-form solution to the problem, but only for a prespecified initial condition and time-to-go. The finite horizon statedependent Riccati equation (Finite-SDRE) developed in [9] gives a suboptimal closed-form solution to the problem for different initial conditions and final times in real time. Finite-SDRE shows a lot of potential but for applications with soft constraints. Series-based solutions to the optimal control problem with terminal hard constraints were investigated in [5,10,11]. The series-based methods are suitable for systems whose dynamics are given in a polynomial form and comprise weak nonlinearity, because the series can diverge in case of large nonlinearity [11]. In case of divergence, using higher order terms in the expansion for accuracy of the result may not help. The use of intelligent control for solving finite horizon optimal control problems was considered in [12–14], among which [13] solves the problems with soft terminal constraint. The controllers developed in [12,14] admit hard terminal constraints. However, the ideas in these papers cannot be used in the rendezvous problem. In [12], an intelligent controller is developed for an agile missile maneuver. It is, however, a scalar problem, wherein the final state and the control have a direct relationship. Hence, in a discrete formulation, the final state can be achieved from any state at the previous step. The rendezvous problem, on the other hand, is a vector problem, wherein the dynamics of the state elements with hard constraints do not contain control. The neurocontroller developed in [14] is synthesized for one specified set of initial conditions and, once the initial state is changed, the network needs to be retrained to give the optimal solution for the new initial condition. Hence, the method developed in [14] is not a general solution as developed in this Note. An adaptive critic-based neurocontroller is developed in this study. The technique presented in this Note is motivated by two different developments, namely, the series-based solution presented in [10] and the approximate dynamic programming (ADP) framework [15]. Novel neural network (NN) structures are selected and an algorithm is developed for learning the weights. Convergence of the NN weights is proved through a new idea by showing that the learning is a contraction mapping [16]. The trained network provides an online solution to the optimal control problem for 1) different initial conditions, as long as the resulting state trajectory lies within the domain for which the network is trained; 2) different final times not greater than the final time for which the network is trained, based on the Bellman principle of optimality [7]; and 3) shifted terminal points/curves/surfaces. This Note is organized as follows. The rendezvous problem is formulated in Sec. II. The fixed-final-time optimal control problem with a terminal state constraint is characterized in Sec. III. The proposed solution is described in Sec. IV, numerical analyses are given in Sec. V, and the work is summarized in Sec. VI.