# Locally Scaled and Stochastic Volatility Metropolis-Hastings Algorithms
###### Abstract  
Markov chain Monte Carlo (MCMC) techniques are usually used to infer model parameters when closed-form inference is not feasible, with one of the simplest MCMC methods being the random walk Metropolis-Hastings (MH) algorithm. The MH algorithm suffers from random walk behaviour, which results in inefficient exploration of the target posterior distribution. This method has been improved upon, with algorithms such as Metropolis Adjusted Langevin Monte Carlo (MALA) and Hamiltonian Monte Carlo being examples of popular modifications to MH. In this work, we revisit the MH algorithm to reduce the autocorrelations in the generated samples without adding significant computational time. We present the: (1) Stochastic Volatility Metropolis-Hastings (SVMH) algorithm, which is based on using a random scaling matrix in the MH algorithm, and (2) Locally Scaled Metropolis-Hastings (LSMH) algorithm, in which the scaled matrix depends on the local geometry of the target distribution. For both these algorithms, the proposal distribution is still Gaussian centred at the current state. The empirical results show that these minor additions to the MH algorithm significantly improve the effective sample rates and predictive performance over the vanilla MH method. The SVMH algorithm produces similar effective sample sizes to the LSMH method, with SVMH outperforming LSMH on an execution time normalised effective sample size basis. The performance of the proposed methods is also compared to the MALA and the current state-of-art method being the No-U-Turn sampler (NUTS). The analysis is performed using a simulation study based on Neal's funnel and multivariate Gaussian distributions and using real world data modeled using jump diffusion processes and Bayesian logistic regression. Although both MALA and NUTS outperform the proposed algorithms on an effective sample size basis, the SVMH algorithm has similar or better predictive performance when compared to MALA and NUTS across the various targets. In addition, the SVMH algorithm outperforms the other MCMC algorithms on a normalised effective sample size basis on the jump diffusion processes datasets. These results indicate the overall usefulness of the proposed algorithms. Bayesian methods; Metropolis-Hastings; Markov chain Monte Carlo; machine learning; algorithms; jump diffusion processes +
Footnote †: journal: _Algorithms_  
0000-0001-9999-2231-11999-2  
The random walk Metropolis-Hastings (MH) method is the most straightforward MCMC algorithm, with other methods improving on the MH method via the use of first-order or higher-order gradient information to guide the exploration of the target. The Metropolis Adjusted Langevin Algorithm (MALA) improves MH using Langevin dynamics, while Hamiltonian Monte Carlo (HMC) and Shadow Hamiltonian Monte Carlo use Hamiltonian dynamics. Magnetic Hamiltonian Monte Carlo (MHMC) adds a magnetic field to Hamiltonian dynamics, leading to faster convergence and samples with lower autocorrelations compared to HMC [9; 18]. The MH algorithm forms the foundation of more complicated algorithms such as Hamiltonian Monte Carlo, and the Metropolis Adjusted Langevin Algorithm [3; 4; 12]. The MH method uses a Gaussian distribution whose mean is the current state to pose the next state[8; 12]. This proposal distribution produces random walk behavior, which generates highly correlated samples and low sample acceptance rates [3; 8]. The MALA, HMC, and MHMC methods all employ first-order gradient information of the posterior. Although this leads to a better exploration of the target, it also results in the algorithms having a higher computational time when compared to the MH algorithm. The MH algorithm is still heavily used in practice due to its robustness and simplicity [21]. In this work, we strive to address some of the deficiencies of the MH method by designing algorithms that reduce correlations in the generated samples without a significant increase in computational cost.