# Non-Mypopic Beam Scheduling for Multiple Smart-Target Tracking in Phased Array Radar Networks
## 1 Introduction  
State-of-the-art radar scheduling approaches have enabled higher flexibility for tracking smart targets through phased array radar networks, where real-time beam direction control is handled electronically [1; 2; 3; 4]. Dynamic tracking of smart targets, which have the ability to be aware that they are being tracked and adapt their dynamics accordingly to hinder tracking accuracy, has drawn recent attention [5; 6; 7]. It is recognized that beam scheduling in radar networks with a trade-off between frequency of tracking and probability of maneuvering is complicated and of importance in smart-target tracking. Hence, myopic radar scheduling approaches, where the objective function considers only the current or next step [8], may lead to important tracking decisions. In contrast, non-myopic approaches take into account the accumulative function values of subsequent steps [9; 10]. This paper addresses a new model to minimize the expected total discounted error for tracking smart targets through the non-myopic dynamic beam scheduling. For smart targets, non-myopic radar scheduling problems become substantially more complex, as such targets react to radar sensing by switching their dynamics between different modes, e.g., a constant velocity (CV) mode, a constant acceleration (CA) mode, a constant turn (CT) mode, etc. [5; 6; 7]. In such scenarios, it might be beneficial to refrain from sensing too often those targets that are likely to hide or escape when tracked, as this would make them harder to track in the future. In [5], the tracking problem was formulatedas a Markov decision process (MDP) with an infinite time horizon, which was addressed through a two-stage reinforcement-learning approach with separated optimizations for detection and tracking. In [6], a modified algebraic Riccati equation (MARE) was applied in a Kalman-based target tracking system with smart targets. This work considered game-theoretic methods that aimed to optimize waveform parameters and radar modes under imperfect measurement information. In [7], a partially observable Markov decision process (POMDP) model [11] with multiple dynamic models was considered to minimize the interception risk. A roll-out method based on unscented sampling (US) was introduced to approximate the long-term reward and to select sensors based on the closest distance policy. In [12], based on a POMDP model, the posterior Cramer-Rao lower bound (PCRLB) was used as system state and multi-step cost prediction over the US. An improved decision tree search algorithm leveraging branch and bound was used to achieve non-myopic scheduling optimization for maneuvering target tracking. The POMDP-based branch-and-bound algorithm was applied to sensor allocation problems for enhanced long-term performance in [13; 14; 15]. Unfortunately, for multi-target tracking, a POMDP model is established with joint states and combinatorial actions. In general, the action space exhibits worst-case exponential complexity. As a result, computing optimal scheduling policies becomes intractable. A body of work has instead aimed to design suboptimal heuristic index policies with low computational complexity. The optimal resource management problems and index policies are addressed through the restless multi-armed bandit problem (RMABP) model [16], which is an extension to the multi-armed bandit problem (MABP) [17; 18; 19]. The RMABP focuses on selecting a maximum of \(K\) from a pool of \(N\geq K\)_projects_ (or bandits), which are versatile entities capable of being either active (i.e., chosen) or passive at each time. While optimally solving RMABP is generally intractable [20], [PERSON] proposed in [16] a heuristic index policy, which has since been widely applied [21]. The Whittle index is implicitly defined as the state-dependent critical subsidy for passivity that makes both actions optimal in a single-project subproblem. [PERSON] computed the Whittle index through an explicitly defined marginal productivity (MP) index [22].