# Non-Mypopic Beam Scheduling for Multiple Smart-Target Tracking in Phased Array Radar Networks
## 2 Model Description and Problem Formulation
### RMABP Model Formulation  
The optimal beam scheduling problem for \(K\) radars and \(N\) smart targets can be formulated as an infinite-horizon discrete-time RMABP, aiming to achieve the optimal dynamic selection of \(K\) out of \(N\) binary-action projects. For such a purpose, a target is identified with a project, which can be operated under the active action (being tracked) and the passive action (being not tracked). The corresponding target's TEC \(\mathbf{\mathrm{P}}_{n,t}\) and dynamic state estimation \(\hat{\mathbf{x}}_{n,t}\) are taken as the state of each project \(n=1,2,\ldots,N\). \(\mathbf{\mathrm{P}}_{n,t}\) moves over the state space \(\mathbb{S}_{++}^{L}\) of symmetric positive definite \(L\times L\) matrices, according to the non-linear filter as shown in Section 2.3. \(L\) is the dimension of dynamic states. The corresponding \(\hat{\mathbf{x}}_{n,t}\in\mathbb{R}^{4}\) can be obtained through the non-linear filter.  
For target \(n\) at time \(t\), the immediate cost caused by imposing action \(a_{n,t}\) is defined as  
\[C_{n}(\mathbf{\mathrm{P}}_{n,t},a_{n,t})\triangleq d_{n}\frac{\mathrm{tr}(\mathbf{ \mathrm{P}}_{n,t})}{L}+h_{n}a_{n,t}, \tag{11}\]  
where \(\mathrm{tr}(\cdot)\) denotes the trace of a matrix, \(d_{n}>0\) is the target's weight, which is used to model the importance of targets, and \(h_{n}\geq 0\) is a measurement cost.  
The radar network selects at most \(K\) out of \(N\) smart targets to track in each time slot \(t\). Each radar can only steer one beam to track one target at each time, which is formulated as the constraint  
\[\sum_{n=1}^{N}a_{n,t}\leq K,\ t=0,1,2,\ldots. \tag{12}\]  
Given the joint model probabilities \(\mathbb{U}=(\mathbf{\mathrm{U}}_{n})_{n=1}^{N}\), joint TEC state at the beginning \(\mathbb{P}_{0}=(\mathbf{\mathrm{P}}_{n,0})_{n=1}^{N}\), and joint dynamic state estimation at the beginning \(\mathbb{X}_{0}=(\hat{\mathbf{x}}_{n,0})_{n=1}^{N}\), the dynamic optimization problem with discounted cost over an infinite time horizon is formulated as follows:  
\[\min_{\mathbf{\pi}\in\Pi(K)}\mathbb{E}_{\mathbb{P}_{0},\mathbb{X}_{0}}^{\mathbf{\pi}} \left[\sum_{t=0}^{\infty}\sum_{n=1}^{N}\beta^{t}C_{n}(\mathbf{\mathrm{P}}_{n,t},a_ {n,t})  
Based on the Sigma points \(\mathbf{e}_{n,t-1}^{i}\), the dynamic states and corresponding measurements of diverse dynamic models can be sampled according to  
\[\check{\mathbf{x}}_{n,t}^{i,m}=\mathbf{F}_{n}^{m}\mathbf{e}_{n,t-1}^{i}+\mathbf{w}_{n,t-1}^{m}, \tag{21}\]  
\[\mathbf{z}_{n,t}^{i,m}=\mathrm{H}_{n}(\check{\mathbf{x}}_{n,t}^{i,m})+\mathbf{v}_{n,t}. \tag{22}\]  
Through the non-linear filter, the Sigma point \(\mathbf{e}_{n,t-1}^{i}\) can be updated with the measurement \(\mathbf{z}_{n,t}^{i,m}\). For details, the reader is referred to [38]. Then, the state estimation \(\hat{\mathbf{x}}_{n,t}^{i,m}\) and TEC estimation \(\hat{\mathbf{p}}_{n,t}^{i,m}\) are obtained.  
The dynamic state and TEC are updated by  
\[\hat{\mathbf{x}}_{n,t}=\sum_{m=1}^{M}u_{n}^{1,m}\sum_{i=0}^{2L}\alpha_{n}^{i}\hat{ \mathbf{x}}_{n,t}^{i,m}, \tag{23}\]  
\[\mathbf{P}_{n,t}=\sum_{m=1}^{M}u_{n}^{1,m}\sum_{i=0}^{2L}\alpha_{n}^{i}\hat{\mathbf{p} }_{n,t}^{i,m}. \tag{24}\]  
In the subsequent Whittle index policy, when the tracking action is performed and the target is successfully detected, the future TEC update is derived as  
\[\mathbf{P}_{n,t}=\psi_{n}^{1}\Big{(}\mathbf{P}_{n,t-1},\hat{\mathbf{x}}_{n,t-1},\mathbf{u}_{n} ^{1}\Big{)}=\sum_{m=1}^{M}u_{n}^{1,m}\sum_{i=0}^{2L}\alpha_{n}^{i}\hat{\mathbf{P} }_{n,t}^{i,m}. \tag{25}\]  
Note that (25) is only used in potential future multi-step action-making, such as the calculation of the Whittle index. Additionally, (7) is used in TEC transitions when performing online scheduling.