## 5 Mean Field Power Control Game
### Finite Difference-Based MFG Solution
#### 5.2.1 Solving HJB and FPK Equations  
The discretized form of the FPK equation is solved after application of the Lax-Friedrichs scheme and converts to (31). Here, \(M(i,j,k)\), \(P(i,j,k)\), \(\omega(i,j,k)\) represent the values of the mean field, transmit power, and interference energy at any time instant \(i\) with the energy level \(j\) and interference state \(k\) within the discretized state space:  
\[\begin{array}{l}M(i+1,j,k)\\ =\frac{[M(i,j-1,k)+M(i,j+1,k)+M(i,j,k-1)+M(i,j,k+1)]}{2}\\ +\frac{\phi_{1}[M(i,j+1,k)P(i,j+1,k)-M(i,j-1,k)P(i,j-1,k)]}{2\delta_{P}}\\ +\frac{\phi_{1}[M(i,j+1,k)P(i,j+1)\omega(i,j,k+1)-M(i,j,k-1)P(i,j,k-1)\omega(i, j,k-1)-M(i,j,k-1)P(i,j,k-1)\omega(i,j,k-1)]}{2\delta_{\beta}}.\end{array} \tag{31}\]  
The Hamiltonian in the HJB equation limits the applicability of the finite difference method in its conventional form to solve the HJB equation. Hence, the HJB equation is converted to an optimal control problem itself with the FPK equation now acting as a constraint. The newly formulated control problem is defined as:  
\[\underbrace{min}_{P_{k}(t)}\mathbb{E}\bigg{[}\int_{o}^{T_{f}}{\partial_{k}}^{ m}(t)dt+\ {\partial_{k}}^{m}\Big{(}T_{f}\Big{)}\bigg{]} \tag{32}\]  
Figure 3: Mean Field Game, (**a**) Interaction model, (**b**) Optimal power control solution.  
algorithm to obtain an optimal power control policy by evaluating against different available policies and finding the power level for each policy.  
```
Initialization:M(0, 0, 0), P(X + 1, 0, 0), \(\lambda\)(X + 1, 0, 0), Iter = 1 repeat Step 1: Mean field evaluation for all\(i=1\): 1: X do for all\(j=1\): 1: Y do for all\(j=1\): 1: Y do for all\(k=1\): 1: 1: K do  Calculate mean field \(M\)(\(i+1,j\), \(k\)) using Equation (31) ifP(\(i,j+1,k\)) = 0 \(M\)(\(i+1,j+1,k+1\)) = \(M\)(\(i,j,k\)) else: \(M\)(\(i+1,j+1,k+1\)) = 0  end if  end for  end for
```  
**Algorithm 1** Finite difference method for optimal mean field, Lagrangian and transmission power evaluation  
Each MTC user evaluates the mean field, Langrangian multipliers and eventually an optimal value for the transmit power until convergence is achieved. The concept of MFE is distinct from the notion of Nash equilibrium (NE) for conventional games (or HJB based control i.e., the SDG design) as the later considers the actual state value before taking any action leading to an exponential increase with the increasing number of players. MFG based control relies on the approximated distribution of states with MFE being achieved only asymptotically when K \(\rightarrow\infty\). The conventional SDG design sought the solution through evaluation of K PDEs which has now been reduced to exactly two (HJB and FPK) following the approximation of state space through mean field interactions instead of individual player interactions. By solving the set of MFG PDEs, a stable solution scheme can be devised even for very dense environments with larger K values. MFE is guaranteed for the proposed solution approach by considering the properties of Hessian as proven in [24]. For any discrete point (\(i\),\(j\),\(k\)), the Hessian is positive for the values of \((i,j,k)\), \(P(i,j,k)\), and \(l(i,j,k)\) and the objective function as defined in the newly formulated control problem of the HJB Equation (32). The positivity of the Hessian ensures that the formulated HJB optimization problem is convex and an optimal solution for this convex problem can be found only if the KKT conditions are met. Another important observation from this result is that the convergence of the optimization problem in (32) is the MFE of the proposed mean field power control game. In the proposed algorithm, the convergence is obtained by observing the condition that the difference between mean field values in consecutive iterations is greater than or equal to \(10^{-5}\) or the maximum iteration value \(Iter_{max}\).