  The Horseshoe is a widely used and popular continuous shrinkage prior for
high-dimensional Bayesian linear regression. Recently, regularized versions of
the Horseshoe prior have also been introduced in the literature. Various Gibbs
sampling Markov chains have been developed in the literature to generate
approximate samples from the corresponding intractable posterior densities.
Establishing geometric ergodicity of these Markov chains provides crucial
technical justification for the accuracy of asymptotic standard errors for
Markov chain based estimates of posterior quantities. In this paper, we
establish geometric ergodicity for various Gibbs samplers corresponding to the
Horseshoe prior and its regularized variants in the context of linear
regression. First, we establish geometric ergodicity of a Gibbs sampler for the
original Horseshoe posterior under strictly weaker conditions than existing
analyses in the literature. Second, we consider the regularized Horseshoe prior
introduced in Piironen and Vehtari (2017), and prove geometric ergodicity for a
Gibbs sampling Markov chain to sample from the corresponding posterior without
any truncation constraint on the global and local shrinkage parameters.
Finally, we consider a variant of this regularized Horseshoe prior introduced
in Nishimura and Suchard (2020), and again establish geometric ergodicity for a
Gibbs sampling Markov chain to sample from the corresponding posterior.
