Controlled accuracy Gibbs sampling of order-constrained non-iid ordered random variates

Abstract Order statistics arising from 𝑚 independent but not identically distributed random variables are typically constructed by arranging some <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:msub> <m:mi>X</m:mi> <m:mn>1</m:mn> </m:msub> <m:mo>,</m:mo> <m:msub> <m:mi>X</m:mi> <m:mn>2</m:mn> </m:msub> <m:mo>,</m:mo> <m:mi mathvariant="normal">…</m:mi> <m:mo>,</m:mo> <m:msub> <m:mi>X</m:mi> <m:mi>m</m:mi> </m:msub> </m:mrow> </m:math> X_{1},X_{2},\ldots,X_{m} , with <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:msub> <m:mi>X</m:mi> <m:mi>i</m:mi> </m:msub> </m:math> X_{i} having distribution function <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:msub> <m:mi>F</m:mi> <m:mi>i</m:mi> </m:msub> <m:mo>⁢</m:mo> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>x</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:mrow> </m:math> F_{i}(x) , in increasing order denoted as <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mn>1</m:mn> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> <m:mo>≤</m:mo> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mn>2</m:mn> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> <m:mo>≤</m:mo> <m:mi mathvariant="normal">⋯</m:mi> <m:mo>≤</m:mo> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>m</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> </m:mrow> </m:math> X_{(1)}\leq X_{(2)}\leq\cdots\leq X_{(m)} . In this case, <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>i</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> </m:math> X_{(i)} is not necessarily associated with <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:msub> <m:mi>F</m:mi> <m:mi>i</m:mi> </m:msub> <m:mo>⁢</m:mo> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>x</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:mrow> </m:math> F_{i}(x) . Assuming one can simulate values from each distribution, one can generate such “non-iid” order statistics by simulating <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:msub> <m:mi>X</m:mi> <m:mi>i</m:mi> </m:msub> </m:math> X_{i} from <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:msub> <m:mi>F</m:mi> <m:mi>i</m:mi> </m:msub> </m:math> F_{i} , for <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mi>i</m:mi> <m:mo>=</m:mo> <m:mrow> <m:mn>1</m:mn> <m:mo>,</m:mo> <m:mn>2</m:mn> <m:mo>,</m:mo> <m:mi mathvariant="normal">…</m:mi> <m:mo>,</m:mo> <m:mi>m</m:mi> </m:mrow> </m:mrow> </m:math> i=1,2,\ldots,m , and arranging them in order. In this paper, we consider the problem of simulating ordered values <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mn>1</m:mn> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> <m:mo>,</m:mo> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mn>2</m:mn> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> <m:mo>,</m:mo> <m:mi mathvariant="normal">…</m:mi> <m:mo>,</m:mo> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>m</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> </m:mrow> </m:math> X_{(1)},X_{(2)},\ldots,X_{(m)} such that the marginal distribution of <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:msub> <m:mi>X</m:mi> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>i</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:msub> </m:math> X_{(i)} is <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:msub> <m:mi>F</m:mi> <m:mi>i</m:mi> </m:msub> <m:mo>⁢</m:mo> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>x</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:mrow> </m:math> F_{i}(x) . This problem arises in Bayesian principal components analysis (BPCA) where the <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:msub> <m:mi>X</m:mi> <m:mi>i</m:mi> </m:msub> </m:math> X_{i} are ordered eigenvalues that are a posteriori independent but not identically distributed. We propose a novel coupling-from-the-past algorithm to “perfectly” (up to computable order of accuracy) simulate such order-constrained non-iid order statistics. We demonstrate the effectiveness of our approach for several examples, including the BPCA problem.