## 6 Modified [PERSON]'s Method of Scoring Using Jacobi or Gauss-Seidel Over-Relaxations  
In this section we elaborate on another non-negatively constrained method for tomographic imaging, which is a modification to the standard [PERSON]'s method of scoring (FS) algorithm. This method is developed based on the following steps. Firstly, the objective function \(\Psi(\mathbf{x})\) is approximated by a quadratic function in each iteration, where the Fisher information matrix (e.g., [46]) is used to define the quadratic term; secondly, an over-relaxation method, either the Jacobi over-relaxation (JOR) or the Gauss-Seidel over-relaxation (also called the successive over-relaxation (SOR)), is employed to solve approximately the linear system derived from zeroing the derivative of this quadratic function. The resulting algorithms are called FS-JOR and FS-SOR and their detailed descriptions can be found in [47, 48]. Descriptions of the JOR and SOR methods are available, for example, in [49]. FS is a general optimization algorithm for computing maximum likelihood estimates. Its advantages over the traditional [PERSON]'s method have been documented in [50]. Briefly, FS iterations are well defined due to the non-negativeness of the Fisher information matrix, but for the [PERSON]'s method, the negative Hessian matrix may not even be non-negative definite, making it unnecessarily proceed in the uphill direction in some applications. Transmission tomography is an example where this problem for the [PERSON]'s method indeed occurs; see Example 6.2. We assume the objective function \(\Psi(\mathbf{x})\) in Equation (2) is twice differentiable and let \(F(\mathbf{x})\) be the Fisher information matrix, namely \(F(\mathbf{x})=E(-\
abla^{2}\Psi(\mathbf{x}))\). At iteration \((k+1)\) of the [PERSON] scoring algorithm, \(\Psi(\mathbf{x})\) is approximated by the following quadratic function:  
\[\Psi(\mathbf{x})\approx\Psi(\mathbf{x}^{(k)})+(\mathbf{x}-\mathbf{x}^{(k)})^{T}\
abla\Psi( \mathbf{x}^{(k)})-\frac{1}{2}(\mathbf{x}-\mathbf{x}^{(k)})^{T}F^{(k)}(\mathbf{x}-\mathbf{x}^{(k) })\equiv\Psi^{(k)}(\mathbf{x}) \tag{68}\]  
where \(F^{(k)}\) denotes the Fisher information matrix at \(\mathbf{x}^{(k)}\). Then the \(\mathbf{x}\) estimate is updated by constrained maximization of \(\Psi^{(k)}(\mathbf{x})\), namely  
\[\mathbf{x}^{(k+1)}=\arg\max_{\mathbf{x}\geq 0}\Psi^{(k)}(\mathbf{x}) \tag{69}\]  
The KKT conditions for this optimization are  
\[\
abla_{j}\Psi^{(k)}(\mathbf{x})=0\text{ if }x_{j}>0\text{ and } \tag{70}\] \[\
abla_{j}\Psi^{(k)}(\mathbf{x})\leq 0\text{ if }x_{j}=0 \tag{71}\]where  
\[\
abla_{j}\Psi^{(k)}(\mathbf{x})=\
abla_{j}\Psi(\mathbf{x}^{(k)})-F_{j}^{(k)}(\mathbf{x}-\mathbf{ x}^{(k)}) \tag{72}\]  
Here \(F_{j}^{(k)}\) denotes the \(j\)-th row of matrix \(F^{(k)}\). The JOR and SOR methods solve, for \(j=1,\ldots,p\),  
\[\
abla_{j}\Psi^{(k)}(\mathbf{x})=0 \tag{73}\]  
in different manners: JOR solves it by fixing all the \(\mathbf{x}\) elements, except \(x_{j}\), at their estimates from the last iteration (_i.e._, iteration \(k\)), but SOR solves it by fixing all the \(\mathbf{x}\) elements, except \(x_{j}\), at their most current estimates. The above illustrations describe how to incorporate JOR or SOR sub-iterations into the FS algorithm. In fact, in each iteration, JOR or SOR is used to solve approximately the linear system of equations determined by the FS algorithm, and then this approximate solution is used as the starting value for the next FS iteration. These new schemes modify the standard FS method, and are feasible for large estimation problems. Usually it suffices to run one JOR or SOR sub-iteration. But running more than one sub-iterations is also attractive as it has the potential to reduce the computations for the entire optimization process. Suppose within each Fisher scoring iteration we run \(m\) sub-iterations of JOR or SOR. The resulting algorithms are called the \(m\)-step FS-JOR and \(m\)-step FS-SOR algorithms respectively.