## 1 Introduction  
Multiple scattering of photons propagating through turbid (optical opaque) medium has remained a challenge for many multi-disciplinary applications, such as biomedical tissue diagnostics and aerosol characterization, smoke and fog analysis, and spray. The reason that simplification assumptions usually do not apply well for such problems is that the multiple scattering phenomenon is intrinsically stochastic, particularly in the intermediate regime where the optical depth (OD) of the media is somewhere between two and 10. Many have attempted to interpret multiple scattering through modeling using Monte Carlo methods [1; 2; 3; 4], or other approaches grounded on the random walk theory or empirical approximations [5; 6; 7; 8]. Monte Carlo methods have been widely applied in multiple scattering numerical modeling because of the methods simple implementations and statistical outcomes. Monte Carlo algorithms provide statistical predictions for stochastic processes through repeated random sampling within the domain of interest. More specifically, for scattering simulations, the model continuously sends one photon into the scattering medium and tracks its trajectory by predicting its propagation and scattering events. If the photon experiences a scattering event, the new propagation angle will be calculated based on the scattering phase function, determined by the physics of the scattering process. This process ends up with the photons leaving the domain of interest. Albeit that the Monte Carlo simulation can be readily implemented and allows simulations with a high fidelity, it suffers from a high computational cost for practical applications. Such a computational cost is primarily due to two reasons: (1) Monte Carlo algorithms require a great number of samples to guarantee the desired accuracy. Simulation error from Monte Carlo is inversely proportional to the square root of the number of samples \(N\). Therefore, a sufficiently large number of samples, usually millions or billions of photons in this case, is required by the Monte Carlo model for multiple scattering problems to achieve meaningful predictions; (2) The total computational cost of the Monte Carlo simulation is also impacted by the cost of each evaluation, i.e., starting from when one photon enters the domain of interest and ending when the photon leaves the domain. The computational cost of the Monte Carlo algorithm is significantly higher with a larger OD because more scattering events are expected and more evaluations are required. Given a large sample size and the relatively high computational cost in evaluating each photon, Monte Carlo simulations used in multiple scattering usually take hours for one single run with a single core CPU. The computational cost is less for forward problems as long as the optical properties of the turbid medium are known, but it is prohibitively high for inverse problems where the optical properties of the turbid medium are sought. Thousands or millions of cost function evaluations (multiple scattering calculations) are required for global optimization/reconstruction algorithms, such as simulated annealing (SA) algorithms and genetic algorithms (GA) in practical applications [9; 10; 11; 12]. Consequently, alternative algorithms and models, such as the matrix inversion methods [13] or iterative methods [14] are needed to enable multiple scattering diagnostics and reconstructions with more reasonable computational costs. With such computationally efficient methods and algorithms, more applications will be made possible, such as cancer diagnosis for skin tissue. Meanwhile, the capacity of many existing technologies will be further expanded, such as landscape imaging. Over the past decades, the Markov process has been utilized for multi-disciplinary, probability-based investigations and applications, such as economic predictions [15], computer vision [16], production control [17], and gambling [18]. There are also Markov chain related studies on the topics of diffusion [19] and radiative transfer [20].