Original: Rapidly Mixing Multiple-try Metropolis Algorithms for Model Selection Problems
Revised: Rapidly Mixing Multiple-try Metropolis Algorithms for Model Selection Problems

Original: Abstract
Revised: Abstract

Original: The multiple-try Metropolis (MTM) algorithm is an extension of the MetropolisHastings (MH) algorithm by selecting the proposed state among multiple trials according to some weight function.
Revised: The multiple-try Metropolis (MTM) algorithm is an extension of the MetropolisHastings (MH) algorithm by selecting the proposed state among multiple trials according to some weight function.

Original: Although MTM has gained great popularity owing to its faster empirical convergence and mixing than the standard MH algorithm, its theoretical mixing property is rarely studied in the literature due to its complex proposal scheme.
Revised: Although MTM has gained great popularity owing to its faster empirical convergence and mixing than the standard MH algorithm, its theoretical mixing property is rarely studied in the literature due to its complex proposal scheme.

Original: We prove that MTM can achieve a mixing time bound smaller than that of MH by a factor of the number of trials under a general setting applicable to high-dimensional model selection problems with discrete state spaces.
Revised: We prove that MTM can achieve a mixing time bound smaller than that of MH by a factor of the number of trials under a general setting applicable to high-dimensional model selection problems with discrete state spaces.

Original: Our theoretical results motivate a new class of weight functions called locally balanced weight functions and guide the choice of the number of trials, which leads to improved performance over standard MTM algorithms.
Revised: Our theoretical results motivate a new class of weight functions called locally balanced weight functions and guide the choice of the number of trials, which leads to improved performance over standard MTM algorithms.

Original: We support our theoretical results by extensive simulation studies and real data applications with several Bayesian model selection problems.
Revised: We support our theoretical results by extensive simulation studies and real data applications with several Bayesian model selection problems.

Original:  Introduction
Revised:  Introduction

Original: The Markov chain Monte Carlo (MCMC) method has become a powerful and standard tool for
Revised: The Markov chain Monte Carlo (MCMC) method has become a powerful and standard tool for

Original: Bayesian posterior computation.
Revised: Bayesian posterior computation.

Original: In particular, the Metropolis-Hastings (MH) algorithm is widely employed in many statistical and machine learning models to sample from posterior distributions with intractable normalizing constants.
Revised: In particular, the Metropolis-Hastings (MH) algorithm is widely employed in many statistical and machine learning models to sample from posterior distributions with intractable normalizing constants.

Original: A prominent usage example of MH is the vast class of
Revised: A prominent usage example of MH is the vast class of

Original: Bayesian model selection problems with discrete-valued high-dimensional parameters, whose posterior probabilities are infeasible to evaluate because the size of model space potentially grows (super-)exponentially with the number of parameters.
Revised: Bayesian model selection problems with discrete-valued high-dimensional parameters, whose posterior probabilities are infeasible to evaluate because the size of model space potentially grows (super-)exponentially with the number of parameters.

Original: Examples of Bayesian model selection problems and their applications are ubiquitous, including Bayesian variable selection (BVS) [ and references therein], stochastic block model (SBM) [20, 34], Bayesian structure learning [10, 15, 29], change-point detection model [5, 11], spatial clustering models [23, 26, 28], and many others.
Revised: Examples of Bayesian model selection problems and their applications are ubiquitous, including Bayesian variable selection (BVS) [ and references therein], stochastic block model (SBM) [20, 34], Bayesian structure learning [10, 15, 29], change-point detection model [5, 11], spatial clustering models [23, 26, 28], and many others.

Original: Despite the popularity of MCMC algorithms, their convergence rate analysis is often challenging.
Revised: Despite the popularity of MCMC algorithms, their convergence rate analysis is often challenging.

Original: Mixing time is a key concept of interest when analyzing the convergence of MCMC, which deﬁnes the number of iterations needed to converge to a stationary distribution within a small total variation distance [25].
Revised: Mixing time is a key concept of interest when analyzing the convergence of MCMC, which deﬁnes the number of iterations needed to converge to a stationary distribution within a small total variation distance [25].

Original: We say a chain is rapidly mixing if the mixing time grows at most polynomially in some complexity parameters.
Revised: We say a chain is rapidly mixing if the mixing time grows at most polynomially in some complexity parameters.

Original: Mixing time bound has been studied extensively for problems such as card shufﬂing [3], random walks on groups [2], graph colorings [21], and the Ising model [14], across the ﬁelds of probability, physics, and computer science [40].
Revised: Mixing time bound has been studied extensively for problems such as card shufﬂing [3], random walks on groups [2], graph colorings [21], and the Ising model [14], across the ﬁelds of probability, physics, and computer science [40].

Original: Although there is a rich literature on discrete-state-space mixing time analysis in many areas, the focus has been primarily on approximate counting and random generation, but only recently has research on the MCMC convergence in the context of statistical model selection begun to emerge.
Revised: Although there is a rich literature on discrete-state-space mixing time analysis in many areas, the focus has been primarily on approximate counting and random generation, but only recently has research on the MCMC convergence in the context of statistical model selection begun to emerge.

Original: [47] proved rapid mixing of an MH algorithm in a high-dimensional BVS problem, and [38] improved the bound when a “warm start” is available.
Revised: [47] proved rapid mixing of an MH algorithm in a high-dimensional BVS problem, and [38] improved the bound when a “warm start” is available.

Original: Rapid mixing results of MH algorithms are established for Bayesian community detection problem with SBM [55] and for learning equivalence class in a high-dimensional Bayesian structure learning [52].
Revised: Rapid mixing results of MH algorithms are established for Bayesian community detection problem with SBM [55] and for learning equivalence class in a high-dimensional Bayesian structure learning [52].

Original: Recently, locally informed MH algorithms have been developed to incorporate local information about the target distribution.
Revised: Recently, locally informed MH algorithms have been developed to incorporate local information about the target distribution.

Original: [48] proposed a locally balanced proposal and proved their optimality in terms of Peskun ordering [37] for discrete space models without providing mixing time rate analysis.
Revised: [48] proposed a locally balanced proposal and proved their optimality in terms of Peskun ordering [37] for discrete space models without providing mixing time rate analysis.

Original: [54] also proposed an informed MCMC and proved a dimension-free mixing time bound but only for a BVS problem.
Revised: [54] also proposed an informed MCMC and proved a dimension-free mixing time bound but only for a BVS problem.

Original: The multiple-try Metropolis (MTM) algorithm [27] is an extension of the MH algorithm, which uses a weight function to choose the proposed state among randomly sampled trials from a proposal distribution.
Revised: The multiple-try Metropolis (MTM) algorithm [27] is an extension of the MH algorithm, which uses a weight function to choose the proposed state among randomly sampled trials from a proposal distribution.

Original: The variety of weight functions and the number of trials enrich the algorithm class but result in a much more complex transition probability than standard MH algorithms, hampering the theoretical development on the mixing time of MTM.
Revised: The variety of weight functions and the number of trials enrich the algorithm class but result in a much more complex transition probability than standard MH algorithms, hampering the theoretical development on the mixing time of MTM.

Original: As a result, there is a lack of theoretical guidance for practitioners on the choices of weight functions and number of trials in MTM and also the choice between MTM and other MCMC algorithms.
Revised: As a result, there is a lack of theoretical guidance for practitioners on the choices of weight functions and number of trials in MTM and also the choice between MTM and other MCMC algorithms.

Original: Only very recently, [46] considered the
Revised: Only very recently, [46] considered the

Original: MTM independence sampler (MTM-IS), a special case of MTM where the proposal does not depend on the current state.
Revised: MTM independence sampler (MTM-IS), a special case of MTM where the proposal does not depend on the current state.

Original: They conducted the convergence rate analysis of MTM-IS, which shows it is less efﬁcient than the simpler approach of repeated Metropolised independent sampling at the same computational cost, but they did not discuss the choices of weight functions and the number of trials.
Revised: They conducted the convergence rate analysis of MTM-IS, which shows it is less efﬁcient than the simpler approach of repeated Metropolised independent sampling at the same computational cost, but they did not discuss the choices of weight functions and the number of trials.

Original: As acknowledged in [46], MTM is signiﬁcantly different from MTM-IS, and its convergence rate is more challenging to analyze.
Revised: As acknowledged in [46], MTM is signiﬁcantly different from MTM-IS, and its convergence rate is more challenging to analyze.

Original: In this paper, we establish the mixing time bound of MTM for general model selection problems.
Revised: In this paper, we establish the mixing time bound of MTM for general model selection problems.

Original: Our main contributions are the following:
Revised: Our main contributions are the following:

Original: 1) We show that MTM with existing popular choices of weight functions can have mixing issues and propose a new class of weight functions called locally balanced weight functions .
Revised: 1) We show that MTM with existing popular choices of weight functions can have mixing issues and propose a new class of weight functions called locally balanced weight functions .

Original: With locally balanced weight functions, we prove that the mixing time bound of the MTM algorithm is smaller than that of the MH algorithm by a factor of the number of trials N in a general model selection setting, under some regularity conditions including a rate condition on N . 3)
Revised: With locally balanced weight functions, we prove that the mixing time bound of the MTM algorithm is smaller than that of the MH algorithm by a factor of the number of trials N in a general model selection setting, under some regularity conditions including a rate condition on N . 3)

Original: We provide theoretical justiﬁcation for the counterintuitive phenomenon that an increase in the number of trials N may not always lead to a proportionate improvement in mixing, and suggest a theoretically guided algorithm to choose N for practical use.
Revised: We provide theoretical justiﬁcation for the counterintuitive phenomenon that an increase in the number of trials N may not always lead to a proportionate improvement in mixing, and suggest a theoretically guided algorithm to choose N for practical use.

Original: We validate our theoretical ﬁndings via extensive simulation studies and real data applications with various model selection problems: Bayesian variable selection, stochastic block models, spatial clustering models, and structure learning models.
Revised: We validate our theoretical ﬁndings via extensive simulation studies and real data applications with various model selection problems: Bayesian variable selection, stochastic block models, spatial clustering models, and structure learning models.

Original:  Preliminary 2.1 Notation
Revised:  Preliminary 2.1 Notation

Original: For a positive integer m ∈ N , let [ m ] = { 1 , . .
Revised: For a positive integer m ∈ N , let [ m ] = { 1 , . .

Original: , m } and (cid:98) m (cid:99) be the largest integer no greater than m .
Revised: , m } and (cid:98) m (cid:99) be the largest integer no greater than m .

Original: Let | · | denote the cardinality of a set.
Revised: Let | · | denote the cardinality of a set.

Original: Let X =
Revised: Let X =

Original: X p denote a ﬁnite state space where |X| grows in a complexity parameter p ∈ N .
Revised: X p denote a ﬁnite state space where |X| grows in a complexity parameter p ∈ N .

Original: We use P : X
Revised: We use P : X

Original: × X → [0 , 1] to denote the transition probability matrix of an irreducible, aperiodic and reversible Markov chain.
Revised: × X → [0 , 1] to denote the transition probability matrix of an irreducible, aperiodic and reversible Markov chain.

Original: We denote by I an identity matrix.
Revised: We denote by I an identity matrix.

Original: We call a probability distribution π on X a stationary distribution or a target distribution of a chain
Revised: We call a probability distribution π on X a stationary distribution or a target distribution of a chain

Original: P if it satisﬁes (cid:80)
Revised: P if it satisﬁes (cid:80)

Original: z ∈X π ( z ) P ( z, x ) =
Revised: z ∈X π ( z ) P ( z, x ) =

Original: π ( x ) for all x ∈ X .
Revised: π ( x ) for all x ∈ X .

Original: In Bayesian inference, π is a posterior distribution.
Revised: In Bayesian inference, π is a posterior distribution.

Original: Let N : X → 2 X be a set-valued map called a neighborhood relation, which maps a state to a set of states.
Revised: Let N : X → 2 X be a set-valued map called a neighborhood relation, which maps a state to a set of states.

Original: We say the neighborhood relation N is symmetric if x
Revised: We say the neighborhood relation N is symmetric if x

Original: (cid:48) ∈ N ( x ) iff x ∈ N
Revised: (cid:48) ∈ N ( x ) iff x ∈ N

Original: x (cid:48) ) for all x (cid:54) =
Revised: x (cid:48) ) for all x (cid:54) =

Original: x (cid:48) .
Revised: x (cid:48) .

Original: We deﬁne the random walk proposal matrix as K RW ( x, y )
Revised: We deﬁne the random walk proposal matrix as K RW ( x, y )

Original: =
Revised: =

Original: 1 N ( x ) ( y ) /
Revised: 1 N ( x ) ( y ) /

Original: |N ( x ) | where 1 N ( x ) denotes the indicator function of N ( x ) .
Revised: |N ( x ) | where 1 N ( x ) denotes the indicator function of N ( x ) .

Original: We say K is symmetric if K ( x, y )
Revised: We say K is symmetric if K ( x, y )

Original: =
Revised: =

Original: K ( y, x ) .
Revised: K ( y, x ) .

Original: We denote a graph by ( V, E ) where V is the vertex set and E ⊂ V × V is the set of edges.
Revised: We denote a graph by ( V, E ) where V is the vertex set and E ⊂ V × V is the set of edges.

Original: We say δ ( x, y )
Revised: We say δ ( x, y )

Original: = ( v 0 , . .
Revised: = ( v 0 , . .

Original: . , v m ) is an E -path (with length m ) from x to y if e i
Revised: . , v m ) is an E -path (with length m ) from x to y if e i

Original: = ( v i − 1 , v i )
Revised: = ( v i − 1 , v i )

Original: ∈ E for i ∈
Revised: ∈ E for i ∈

Original: [ m ] , v 0 = x, v m = y , and all vertices are distinct.
Revised: [ m ] , v 0 = x, v m = y , and all vertices are distinct.

Original: [Equation]
Revised: [Equation]

Original: 2.2 Multiple-try Metropolis algorithm
Revised: 2.2 Multiple-try Metropolis algorithm

Original: We present the MTM algorithm [27] on a discrete state space in Algorithm 1.
Revised: We present the MTM algorithm [27] on a discrete state space in Algorithm 1.

Original: Here N trial states y 1 , . .
Revised: Here N trial states y 1 , . .

Original: , y N are ﬁrst sampled from the proposal K RW ( x, · ) , and then the proposed state y is selected among the trials using the weight function
Revised: , y N are ﬁrst sampled from the proposal K RW ( x, · ) , and then the proposed state y is selected among the trials using the weight function

Original: [Equation]
Revised: [Equation]

Original: where λ ( y, x ) can be any symmetric function that is positive whenever K RW ( y, x ) > 0 .
Revised: where λ ( y, x ) can be any symmetric function that is positive whenever K RW ( y, x ) > 0 .

Original: Note that
Revised: Note that

Original: Algorithm 1 becomes the standard random walk MH algorithm when the number of trials N is one.
Revised: Algorithm 1 becomes the standard random walk MH algorithm when the number of trials N is one.

Original: There have been efforts to generalize the MTM algorithm [9, 12, 30].
Revised: There have been efforts to generalize the MTM algorithm [9, 12, 30].

Original: [35] extends the weight function class by proposing a general form of acceptance probability to achieve the reversibility of a chain when the weight function does not belong to (1).
Revised: [35] extends the weight function class by proposing a general form of acceptance probability to achieve the reversibility of a chain when the weight function does not belong to (1).

Original: While one can easily ﬁnd a weight function satisfying (1), there is little research on the most appropriate form of weight functions.
Revised: While one can easily ﬁnd a weight function satisfying (1), there is little research on the most appropriate form of weight functions.

Original: [27] investigated some particular class of weight functions and claimed the performance is insensitive to the choice of weight functions within this class; w ( y | x ) =
Revised: [27] investigated some particular class of weight functions and claimed the performance is insensitive to the choice of weight functions within this class; w ( y | x ) =

Original: π ( y ) (when K RW is symmetric) and w ( y | x ) =
Revised: π ( y ) (when K RW is symmetric) and w ( y | x ) =

Original: π ( y ) / K RW ( x, y ) are popular choices in this class [6, 33].
Revised: π ( y ) / K RW ( x, y ) are popular choices in this class [6, 33].

Original: Another important component of the MTM method is the number of trials N .
Revised: Another important component of the MTM method is the number of trials N .

Original: It has been reported with empirical evidence that increasing N does not necessarily result in a corresponding improvement in mixing under the random walk proposal [32].
Revised: It has been reported with empirical evidence that increasing N does not necessarily result in a corresponding improvement in mixing under the random walk proposal [32].

Original: They illustrate a counterintuitive phenomenon that the acceptance probability for a proposal move to higher probability regions can be extremely low.
Revised: They illustrate a counterintuitive phenomenon that the acceptance probability for a proposal move to higher probability regions can be extremely low.

Original: As a solution, they suggest randomly selecting N at each iteration.
Revised: As a solution, they suggest randomly selecting N at each iteration.

Original: Despite the anecdotal observation that certain choices of N make the algorithm less efﬁcient, the principled way to choose N is rarely discussed in the literature.
Revised: Despite the anecdotal observation that certain choices of N make the algorithm less efﬁcient, the principled way to choose N is rarely discussed in the literature.

Original: 2.3 Analysis on mixing time via geometric tools
Revised: 2.3 Analysis on mixing time via geometric tools

Original: A theoretical foundation for employing MCMC methods to sample from a posterior distribution on a ﬁnite state space is the well-known convergence theorem, which states that an irreducible and aperiodic Markov chain with a stationary distribution converges to the stationary distribution regardless of the initial distribution [25, Theorem 4.9].
Revised: A theoretical foundation for employing MCMC methods to sample from a posterior distribution on a ﬁnite state space is the well-known convergence theorem, which states that an irreducible and aperiodic Markov chain with a stationary distribution converges to the stationary distribution regardless of the initial distribution [25, Theorem 4.9].

Original: However, since it is impossible to simulate a chain inﬁnitely long, our main interest is to analyze how well the chain approximates the target distribution after a certain number of steps.
Revised: However, since it is impossible to simulate a chain inﬁnitely long, our main interest is to analyze how well the chain approximates the target distribution after a certain number of steps.

Original: To this end, we introduce (cid:15) -mixing time t mix ( (cid:15) )
Revised: To this end, we introduce (cid:15) -mixing time t mix ( (cid:15) )

Original: = max x ∈X min (cid:110) t ∈ N : (cid:107) P t lazy ( x, · ) − π ( · ) (cid:107) TV ≤ (cid:15) (cid:111) , where (cid:107)·(cid:107) TV is the total variation (TV) distance, (cid:15) is a small positive constant, and P lazy =
Revised: = max x ∈X min (cid:110) t ∈ N : (cid:107) P t lazy ( x, · ) − π ( · ) (cid:107) TV ≤ (cid:15) (cid:111) , where (cid:107)·(cid:107) TV is the total variation (TV) distance, (cid:15) is a small positive constant, and P lazy =

Original: ( P + I ) / 2 denotes the lazy version of P , which is introduced merely for theoretical convenience since all eigenvalues of P lazy are non-negative (the use of lazy Markov chains is standard in mixing time analysis).
Revised: ( P + I ) / 2 denotes the lazy version of P , which is introduced merely for theoretical convenience since all eigenvalues of P lazy are non-negative (the use of lazy Markov chains is standard in mixing time analysis).

Original: In words, t mix quantiﬁes the necessary number of steps for the chain to have a small TV distance from the stationary distribution.
Revised: In words, t mix quantiﬁes the necessary number of steps for the chain to have a small TV distance from the stationary distribution.

Original: Among several techniques for characterizing the mixing property [8, 18, 25, 42], a path method is employed to derive our main result [13, 43].
Revised: Among several techniques for characterizing the mixing property [8, 18, 25, 42], a path method is employed to derive our main result [13, 43].

Original: We also refer readers to [25, Section 13.4] and [42,
Revised: We also refer readers to [25, Section 13.4] and [42,

Original: Section 3] for more details about the path method.
Revised: Section 3] for more details about the path method.

Original: Intuitively, the mixing time depends on how well the states “communicate”, especially among those which have high values of π .
Revised: Intuitively, the mixing time depends on how well the states “communicate”, especially among those which have high values of π .

Original: In contrast to samplers deﬁned on a continuous state space with a Euclidean topology, posterior landscape and modality on a ﬁnite state space are more difﬁcult to envision since it highly depends on how one deﬁnes the neighborhood relation.
Revised: In contrast to samplers deﬁned on a continuous state space with a Euclidean topology, posterior landscape and modality on a ﬁnite state space are more difﬁcult to envision since it highly depends on how one deﬁnes the neighborhood relation.

Original: Given a transition probability matrix P , we deﬁne the neighborhood of a state x by N ( x ) = { x
Revised: Given a transition probability matrix P , we deﬁne the neighborhood of a state x by N ( x ) = { x

Original: (cid:48) ∈
Revised: (cid:48) ∈

Original: X \ { x } : P ( x, x (cid:48) ) > 0 } , the set of reachable states by one-step transition from x , and we say x is a mode if π ( x ) > max x (cid:48) ∈N ( x ) π ( x (cid:48) ) .
Revised: X \ { x } : P ( x, x (cid:48) ) > 0 } , the set of reachable states by one-step transition from x , and we say x is a mode if π ( x ) > max x (cid:48) ∈N ( x ) π ( x (cid:48) ) .

Original: This speciﬁcation of the neighborhood enables us to deﬁne a graph ( X , E ) where E = {
Revised: This speciﬁcation of the neighborhood enables us to deﬁne a graph ( X , E ) where E = {

Original: ( x, x (cid:48) ) ∈
Revised: ( x, x (cid:48) ) ∈

Original: X ×
Revised: X ×

Original: X : x
Revised: X : x

Original: (cid:48) ∈ N ( x ) } is an (undirected since we have a symmetric neighborhood relation by the reversibility of P ) edge set.
Revised: (cid:48) ∈ N ( x ) } is an (undirected since we have a symmetric neighborhood relation by the reversibility of P ) edge set.

Original: For each ( x, y ) ∈
Revised: For each ( x, y ) ∈

Original: X × X with x (cid:54) =
Revised: X × X with x (cid:54) =

Original: y , we choose only one E -path from x to y and denote it as δ ( x, y ) .
Revised: y , we choose only one E -path from x to y and denote it as δ ( x, y ) .

Original: A path ensemble is deﬁned as a collection of the paths ∆ = { δ ( x, y ) :
Revised: A path ensemble is deﬁned as a collection of the paths ∆ = { δ ( x, y ) :

Original: x, y ∈ X , x (cid:54) =
Revised: x, y ∈ X , x (cid:54) =

Original: y } .
Revised: y } .

Original: A path ensemble is a “conﬁguration of path network” that helps quantify the quality of the communication within the state space X .
Revised: A path ensemble is a “conﬁguration of path network” that helps quantify the quality of the communication within the state space X .

Original: The following proposition, which serves as a base of our main result, gives a mixing time bound by means of a path ensemble.
Revised: The following proposition, which serves as a base of our main result, gives a mixing time bound by means of a path ensemble.

Original: Proposition 1 ([43]) .
Revised: Proposition 1 ([43]) .

Original: For any path ensemble ∆ ,
Revised: For any path ensemble ∆ ,

Original: [Equation]
Revised: [Equation]

Original: , where (cid:96) (∆)
Revised: , where (cid:96) (∆)

Original: =
Revised: =

Original: max x,y | δ ( x, y ) | and ρ (∆)
Revised: max x,y | δ ( x, y ) | and ρ (∆)

Original: =
Revised: =

Original: max ( u,v ) ∈
Revised: max ( u,v ) ∈

Original: E π ( u
Revised: E π ( u

Original: [Equation]
Revised: [Equation]

Original: For completeness, we provide a proof of Proposition 1 with a detailed discussion in Appendix A.1.
Revised: For completeness, we provide a proof of Proposition 1 with a detailed discussion in Appendix A.1.

Original: The quantity ρ (∆) is called the congestion parameter , and (cid:96) (∆) is the length of the longest path in ∆ .
Revised: The quantity ρ (∆) is called the congestion parameter , and (cid:96) (∆) is the length of the longest path in ∆ .

Original: To interpret ρ (∆) and (cid:96) (∆) , some terminologies from transportation networks in graph theory are often useful.
Revised: To interpret ρ (∆) and (cid:96) (∆) , some terminologies from transportation networks in graph theory are often useful.

Original: Consider π ( u ) P ( u, v ) as the capacity of an edge ( u, v ) , and π ( x ) π ( y ) as the unit ﬂow of a path δ ( x, y ) .
Revised: Consider π ( u ) P ( u, v ) as the capacity of an edge ( u, v ) , and π ( x ) π ( y ) as the unit ﬂow of a path δ ( x, y ) .

Original: Then ρ (∆) quantiﬁes the maximum loading of an edge under the conﬁguration ∆ .
Revised: Then ρ (∆) quantiﬁes the maximum loading of an edge under the conﬁguration ∆ .

Original: If one edge with a small capacity is traversed by a huge number of paths with a high unit ﬂow, it results in a constriction.
Revised: If one edge with a small capacity is traversed by a huge number of paths with a high unit ﬂow, it results in a constriction.

Original: Consider (cid:96) (∆) as the diameter of the conﬁguration ∆ .
Revised: Consider (cid:96) (∆) as the diameter of the conﬁguration ∆ .

Original: If the diameter is extremely large, it causes inefﬁcient communication, especially among the states lying on the margin of the conﬁguration.
Revised: If the diameter is extremely large, it causes inefﬁcient communication, especially among the states lying on the margin of the conﬁguration.

Original: If it is possible to design the path system in a way that no edge is overloaded by paths and no two states are too far apart, then the chain can have a good mixing behavior.
Revised: If it is possible to design the path system in a way that no edge is overloaded by paths and no two states are too far apart, then the chain can have a good mixing behavior.

Original: One of the key steps in the proof of the main result is to identify a good path ensemble ∆ ∗ (see Appendix A.3).
Revised: One of the key steps in the proof of the main result is to identify a good path ensemble ∆ ∗ (see Appendix A.3).

Original:  Main result
Revised:  Main result

Original: In this section, we prove the rapid mixing of the MTM algorithm for general model selection problems on a ﬁnite model space X , where we assume that there is only one state with the highest posterior mass, denoted by x ∗ .
Revised: In this section, we prove the rapid mixing of the MTM algorithm for general model selection problems on a ﬁnite model space X , where we assume that there is only one state with the highest posterior mass, denoted by x ∗ .

Original: Throughout the section, we may assume that |X| grows with the complexity parameter p
Revised: Throughout the section, we may assume that |X| grows with the complexity parameter p

Original: (e.g., the number of variables in BVS), and the posterior distribution π and other related objects are implicitly parameterized by p .
Revised: (e.g., the number of variables in BVS), and the posterior distribution π and other related objects are implicitly parameterized by p .

Original: 3.1 A mixing time bound with locally balanced weight functions
Revised: 3.1 A mixing time bound with locally balanced weight functions

Original: While extending the possible form of weight functions has been the main focus in the literature [33, 35], the choice of weight function is rarely discussed.
Revised: While extending the possible form of weight functions has been the main focus in the literature [33, 35], the choice of weight function is rarely discussed.

Original: We introduce a class of weight functions by using a function h : R > 

Revised: We introduce a class of weight functions by using a function h : R >

Original: → R > 0 that satisﬁes h ( u )
Revised: → R > 0 that satisﬁes h ( u )

Original: = uh (1 /u )
Revised: = uh (1 /u )

Original: for all u > 0 ; such a function h is called a balancing function [31, 48].
Revised: for all u > 0 ; such a function h is called a balancing function [31, 48].

Original: The class of balancing functions is very broad: we can choose an arbitrary non-negative function h on (0 , 1] , and then h ( u ) on u ∈ (1 , ∞ ) is deﬁned by the “balancing rule”, h ( u )
Revised: The class of balancing functions is very broad: we can choose an arbitrary non-negative function h on (0 , 1] , and then h ( u ) on u ∈ (1 , ∞ ) is deﬁned by the “balancing rule”, h ( u )

Original: = uh (1 /u ) .
Revised: = uh (1 /u ) .

Original: Typical choices of balancing functions include √ u , min { 1 , u } , max { 1 , u } , u/ ( u + 1) , and u + 1 .
Revised: Typical choices of balancing functions include √ u , min { 1 , u } , max { 1 , u } , u/ ( u + 1) , and u + 1 .

Original: The following proposition states that Algorithm 1 with the proposed weight function induces a reversible Markov chain with π as its stationary distribution.
Revised: The following proposition states that Algorithm 1 with the proposed weight function induces a reversible Markov chain with π as its stationary distribution.

Original: Proposition 2 (Locally balanced weight functions) .
Revised: Proposition 2 (Locally balanced weight functions) .

Original: Suppose that a weight function is deﬁned as
Revised: Suppose that a weight function is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where h is a balancing function.
Revised: where h is a balancing function.

Original: Then, a Markov chain P MTM induced by Algorithm 1 has a stationary distribution π .
Revised: Then, a Markov chain P MTM induced by Algorithm 1 has a stationary distribution π .

Original: The proof is deferred to Appendix A.2.
Revised: The proof is deferred to Appendix A.2.

Original: We call weight functions of the form (4) locally balanced weight functions ; though we only consider random walk proposals in this paper, K RW in Propositioncan be replaced by any other proposal matrix.
Revised: We call weight functions of the form (4) locally balanced weight functions ; though we only consider random walk proposals in this paper, K RW in Propositioncan be replaced by any other proposal matrix.

Original: Compared with the popular choice of weight function in the existing MTM literature such as w ( y | x ) =
Revised: Compared with the popular choice of weight function in the existing MTM literature such as w ( y | x ) =

Original: π ( y ) , the locally balanced weight function can be viewed as a transformed and scaled version with respect to the posterior probability at current state x .
Revised: π ( y ) , the locally balanced weight function can be viewed as a transformed and scaled version with respect to the posterior probability at current state x .

Original: For example, if h ( u ) = √ u and K RW is symmetric, then w ( y | x ) =
Revised: For example, if h ( u ) = √ u and K RW is symmetric, then w ( y | x ) =

Original: (cid:112) π
Revised: (cid:112) π

Original: ( y ) /π
Revised: ( y ) /π

Original: ( x ) .
Revised: ( x ) .

Original: Before providing a rigorous mixing time bound of MTM under our new choice of w ( y | x ) , we give an intuitive explanation for why the existing unscaled weight function may have poor mixing properties.
Revised: Before providing a rigorous mixing time bound of MTM under our new choice of w ( y | x ) , we give an intuitive explanation for why the existing unscaled weight function may have poor mixing properties.

Original: First, it is reasonable to assume that the true data-generating model, x ∗ , will have the largest posterior probability when the posterior contraction or the model selection consistency holds asymptotically.
Revised: First, it is reasonable to assume that the true data-generating model, x ∗ , will have the largest posterior probability when the posterior contraction or the model selection consistency holds asymptotically.

Original: Indeed, such results have been established for a variety of model selection problems [47, 52, 55].
Revised: Indeed, such results have been established for a variety of model selection problems [47, 52, 55].

Original: On the other hand, a model space X is often endowed with a natural neighborhood relation such that the degree of “model ﬁt” (measured by goodness of ﬁt and penalization on model size) between neighboring models tends to have a small difference.
Revised: On the other hand, a model space X is often endowed with a natural neighborhood relation such that the degree of “model ﬁt” (measured by goodness of ﬁt and penalization on model size) between neighboring models tends to have a small difference.

Original: This implies that a small modiﬁcation of a model may lead to a marginal change in the posterior probability.
Revised: This implies that a small modiﬁcation of a model may lead to a marginal change in the posterior probability.

Original: For example, in a Bayesian variable selection problem with the number of variables p , the model space can be represented as a set of binary vectors γ ∈ { 0 , 1 } p where γ j = 1 (resp.
Revised: For example, in a Bayesian variable selection problem with the number of variables p , the model space can be represented as a set of binary vectors γ ∈ { 0 , 1 } p where γ j = 1 (resp.

Original: γ j = 0 ) indicates j -th variable is included (resp.
Revised: γ j = 0 ) indicates j -th variable is included (resp.

Original: excluded) in the model.
Revised: excluded) in the model.

Original: Let us assume that γ ∗ is the true data generating model and has the highest posterior probability, and deﬁne a Hamming distance d H ( γ, γ (cid:48) )
Revised: Let us assume that γ ∗ is the true data generating model and has the highest posterior probability, and deﬁne a Hamming distance d H ( γ, γ (cid:48) )

Original: = (cid:80) p
Revised: = (cid:80) p

Original: j =1 1 { γ j (cid:54) =
Revised: j =1 1 { γ j (cid:54) =

Original: γ (cid:48) j } for γ, γ (cid:48) ∈ { 0 , 1 } p .
Revised: γ (cid:48) j } for γ, γ (cid:48) ∈ { 0 , 1 } p .

Original: reason that the weight function w ( y | x ) =
Revised: reason that the weight function w ( y | x ) =

Original: π ( y ) may fail.
Revised: π ( y ) may fail.

Original: Each vertical bar represents the posterior probability on the corresponding state, and thin lines connect neighboring states.
Revised: Each vertical bar represents the posterior probability on the corresponding state, and thin lines connect neighboring states.

Original: The expected value of the posterior probability of the neighborhood of γ (cid:48) (red) is typically greater than that of γ (blue).
Revised: The expected value of the posterior probability of the neighborhood of γ (cid:48) (red) is typically greater than that of γ (blue).

Original: One often deﬁnes the neighborhood N ( γ ) as the set of states with a small Hamming distance to γ [19, 47, 54], and models in the neighborhood tend to have similar posterior probabilities.
Revised: One often deﬁnes the neighborhood N ( γ ) as the set of states with a small Hamming distance to γ [19, 47, 54], and models in the neighborhood tend to have similar posterior probabilities.

Original: A state with a large Hamming distance from γ ∗ usually has a small posterior probability because its structure is largely different from the true data-generating model.
Revised: A state with a large Hamming distance from γ ∗ usually has a small posterior probability because its structure is largely different from the true data-generating model.

Original: Figure 1 depicts such a relationship between posterior probabilities and structural similarities among the models.
Revised: Figure 1 depicts such a relationship between posterior probabilities and structural similarities among the models.

Original: Given that γ is the current state, Step 2 of Algorithmtends to propose a neighboring state γ (cid:48) with a higher posterior probability, so γ (cid:48) tends to be closer to γ ∗ in Hamming distance, i.e. d H ( γ (cid:48) , γ ∗ ) <
Revised: Given that γ is the current state, Step 2 of Algorithmtends to propose a neighboring state γ (cid:48) with a higher posterior probability, so γ (cid:48) tends to be closer to γ ∗ in Hamming distance, i.e. d H ( γ (cid:48) , γ ∗ ) <

Original: d H ( γ, γ ∗ ) .
Revised: d H ( γ, γ ∗ ) .

Original: On average, the Hamming distance between a neighboring state of γ (cid:48) and γ ∗ will be smaller than that between a neighboring state of γ with γ ∗ .
Revised: On average, the Hamming distance between a neighboring state of γ (cid:48) and γ ∗ will be smaller than that between a neighboring state of γ with γ ∗ .

Original: Therefore, the expected value of posterior probability of a neighboring state of γ (cid:48) (in the denominator part in (2), shown in red in Figure 1) will be typically larger than that of a neighboring state of γ (in the numerator part in (2), shown in blue in Figure 1).
Revised: Therefore, the expected value of posterior probability of a neighboring state of γ (cid:48) (in the denominator part in (2), shown in red in Figure 1) will be typically larger than that of a neighboring state of γ (in the numerator part in (2), shown in blue in Figure 1).

Original: If we resort to the unscaled weight function w ( γ (cid:48) | γ )
Revised: If we resort to the unscaled weight function w ( γ (cid:48) | γ )

Original: =
Revised: =

Original: π ( γ (cid:48) ) , this can cause a signiﬁcantly low acceptance probability, especially when the number of trials
Revised: π ( γ (cid:48) ) , this can cause a signiﬁcantly low acceptance probability, especially when the number of trials

Original: N is large (due to the law of large numbers, the acceptance ratio converges to the ratio of average posterior probabilities of the two neighborhoods); see also Appendix A.4 for a concrete example.
Revised: N is large (due to the law of large numbers, the acceptance ratio converges to the ratio of average posterior probabilities of the two neighborhoods); see also Appendix A.4 for a concrete example.

Original: A locally balanced weight function (4), however, is able to mitigate this counterintuitive phenomenon by “scaling" the weight function with respect to the current state.
Revised: A locally balanced weight function (4), however, is able to mitigate this counterintuitive phenomenon by “scaling" the weight function with respect to the current state.

Original: We shall demonstrate the effectiveness of locally balanced weight functions in the simulation studies.
Revised: We shall demonstrate the effectiveness of locally balanced weight functions in the simulation studies.

Original: Although the scope of our study is limited to ﬁnite state spaces, employing (4) also improves the algorithm on continuous state spaces.
Revised: Although the scope of our study is limited to ﬁnite state spaces, employing (4) also improves the algorithm on continuous state spaces.

Original: We provide the relevant discussion in Appendix D.2.
Revised: We provide the relevant discussion in Appendix D.2.

Original: In order for the MTM procedure to efﬁciently sample states from a high posterior region, it is sensible to assign a larger weight to those states with a higher posterior probability in the proposal, i.e. Stepof Algorithm 1.
Revised: In order for the MTM procedure to efﬁciently sample states from a high posterior region, it is sensible to assign a larger weight to those states with a higher posterior probability in the proposal, i.e. Stepof Algorithm 1.

Original: To this end, we only require h to be non-decreasing.
Revised: To this end, we only require h to be non-decreasing.

Original: Now we introduce the main result, which states that the mixing time bound can be improved linearly in the number of trials N for our locally balanced weighting scheme.
Revised: Now we introduce the main result, which states that the mixing time bound can be improved linearly in the number of trials N for our locally balanced weighting scheme.

Original: Theorem 1.
Revised: Theorem 1.

Original: Let P MTM denote a Markov chain induced by Algorithm 1 with a locally balanced weight function (4) with some non-decreasing balancing function h .
Revised: Let P MTM denote a Markov chain induced by Algorithm 1 with a locally balanced weight function (4) with some non-decreasing balancing function h .

Original: Deﬁne
Revised: Deﬁne

Original: [Equation]
Revised: [Equation]

Original: for some constant t 1 and s 0 = max x ∈X |S ( x ) | .
Revised: for some constant t 1 and s 0 = max x ∈X |S ( x ) | .

Original: Suppose that the following conditions hold with p ≥ 2 and t 2 , t 3 , t 4 ≥ 0 being some constants that satisfy t 1 < t 2 , t 3 ≤ t 4 < t 2 .
Revised: Suppose that the following conditions hold with p ≥ 2 and t 2 , t 3 , t 4 ≥ 0 being some constants that satisfy t 1 < t 2 , t 3 ≤ t 4 < t 2 .

Original: (i)
Revised: (i)

Original: There exists x (cid:48) ∈
Revised: There exists x (cid:48) ∈

Original: N ( x ) such that π ( x (cid:48) ) /π
Revised: N ( x ) such that π ( x (cid:48) ) /π

Original: ( x ) ≥ p t 2 for every x (cid:54) =
Revised: ( x ) ≥ p t 2 for every x (cid:54) =

Original: x ∗ .
Revised: x ∗ .

Original: (ii) For any x ∈ X , p t 3 ≤ |N ( x ) | ≤ p t 4 .
Revised: (ii) For any x ∈ X , p t 3 ≤ |N ( x ) | ≤ p t 4 .

Original: (iii) N =
Revised: (iii) N =

Original: o (cid:16) min (cid:110) h
Revised: o (cid:16) min (cid:110) h

Original: ( p t 2+ t
Revised: ( p t 2+ t

Original: [Equation]
Revised: [Equation]

Original: 4 ) h ( p t 1+ t 4 − t 3 ) , p t 3 s 0 (cid:111)(cid:17) .
Revised: 4 ) h ( p t 1+ t 4 − t 3 ) , p t 3 s 0 (cid:111)(cid:17) .

Original: Then, we have the (cid:15) -mixing time of P MTM
Revised: Then, we have the (cid:15) -mixing time of P MTM

Original: [Equation]
Revised: [Equation]

Original: where ∆ ∗ is a path ensemble deﬁned in Appendix A.3.
Revised: where ∆ ∗ is a path ensemble deﬁned in Appendix A.3.

Original: Remark 1 .
Revised: Remark 1 .

Original: By the recent result of [53, Lemma 3] based on a reﬁned path argument, we can remove the term (cid:96) (∆ ∗ ) from (6) without changing the order of the bound.
Revised: By the recent result of [53, Lemma 3] based on a reﬁned path argument, we can remove the term (cid:96) (∆ ∗ ) from (6) without changing the order of the bound.

Original: Sketch of the proof.
Revised: Sketch of the proof.

Original: The complete proof is deferred to Appendix A.3.
Revised: The complete proof is deferred to Appendix A.3.

Original: The ﬁrst key step of our proof is to identify a suitable path ensemble ∆ ∗ by deﬁning a function g : X → X such that g ( x ) has the highest posterior probability in the neighborhood of the state x .
Revised: The ﬁrst key step of our proof is to identify a suitable path ensemble ∆ ∗ by deﬁning a function g : X → X such that g ( x ) has the highest posterior probability in the neighborhood of the state x .

Original: With π being unimodal and X being ﬁnite, repeated composition of g for any x will always lead to the mode x ∗ , that is g m ( x ) = x ∗ for some m ∈ N , which we utilize to construct δ ∗ ( x, y ) for any x
Revised: With π being unimodal and X being ﬁnite, repeated composition of g for any x will always lead to the mode x ∗ , that is g m ( x ) = x ∗ for some m ∈ N , which we utilize to construct δ ∗ ( x, y ) for any x

Original: , y ∈ X , x (cid:54) =
Revised: , y ∈ X , x (cid:54) =

Original: y .
Revised: y .

Original: See Appendix A.5 for an toy example.
Revised: See Appendix A.5 for an toy example.

Original: The other key step is to bound the congestion parameter ρ (∆ ∗ ) deﬁned in (3).
Revised: The other key step is to bound the congestion parameter ρ (∆ ∗ ) deﬁned in (3).

Original: To this end, we prove that the lower bound of the transition probability P ( x, g ( x )) gets closer to Np − tasymptotically, for any state x ∈ X\{ x ∗ } .
Revised: To this end, we prove that the lower bound of the transition probability P ( x, g ( x )) gets closer to Np − tasymptotically, for any state x ∈ X\{ x ∗ } .

Original: Condition (i) implies that the posterior distribution is unimodal with the peak at the model x ∗ .
Revised: Condition (i) implies that the posterior distribution is unimodal with the peak at the model x ∗ .

Original: We emphasize that the unimodality is with respect to the neighborhood relation N and is usually satisﬁed with some appropriate choice of a large enough neighborhood.
Revised: We emphasize that the unimodality is with respect to the neighborhood relation N and is usually satisﬁed with some appropriate choice of a large enough neighborhood.

Original: For example, for Bayesian variable selection, it is shown in [47] that their proposed posterior may not be unimodal if a neighborhood is deﬁned as a set of models within 1-Hamming distance, but expanding to the 2-Hamming distance neighborhood satisﬁes Condition (i) with t 2 = 2 .
Revised: For example, for Bayesian variable selection, it is shown in [47] that their proposed posterior may not be unimodal if a neighborhood is deﬁned as a set of models within 1-Hamming distance, but expanding to the 2-Hamming distance neighborhood satisﬁes Condition (i) with t 2 = 2 .

Original: Similarly, the community detection problem considered in [55] and the structure learning problem in [52] satisfy (i) with some positive constant tby choosing an appropriate N .
Revised: Similarly, the community detection problem considered in [55] and the structure learning problem in [52] satisfy (i) with some positive constant tby choosing an appropriate N .

Original: Next, Condition (ii) states that the neighborhood size |N
Revised: Next, Condition (ii) states that the neighborhood size |N

Original: ( x ) | should neither be too huge nor vary much from state to state.
Revised: ( x ) | should neither be too huge nor vary much from state to state.

Original: The purpose of this condition is to control the ratio K RW ( y, x ) / K RW ( x, y ) in (4) so that the posterior ratio π ( y ) /π
Revised: The purpose of this condition is to control the ratio K RW ( y, x ) / K RW ( x, y ) in (4) so that the posterior ratio π ( y ) /π

Original: ( x ) dominates the term.
Revised: ( x ) dominates the term.

Original: Last but not least, Condition (iii) implies that the number of trials N should not be arbitrarily large.
Revised: Last but not least, Condition (iii) implies that the number of trials N should not be arbitrarily large.

Original: This is consistent with the empirical observation of [32] that additional trials do not always result in better mixing.
Revised: This is consistent with the empirical observation of [32] that additional trials do not always result in better mixing.

Original: Condition (iii) also implies that s 

Revised: Condition (iii) also implies that s

Original: = o
Revised: = o

Original: ( p t 3 ) since N ≥ 1 .
Revised: ( p t 3 ) since N ≥ 1 .

Original: This seems to be a strong condition, since the number of neighboring states with a relatively large posterior probability with respect to the current state can be very large if we consider a state with the smallest posterior that is surrounded by states with greater posterior.
Revised: This seems to be a strong condition, since the number of neighboring states with a relatively large posterior probability with respect to the current state can be very large if we consider a state with the smallest posterior that is surrounded by states with greater posterior.

Original: Yet, it is a common practice to restrict the support of the prior to rule out unrealistic models by imposing some high-dimensional regularity assumptions, such as sparsity.
Revised: Yet, it is a common practice to restrict the support of the prior to rule out unrealistic models by imposing some high-dimensional regularity assumptions, such as sparsity.

Original: For example, [47] introduced the parameter of the maximum number of important covariates in variable selection, and [52] considered the maximum in-degree and out-degree in structure learning to restrict the model space.
Revised: For example, [47] introduced the parameter of the maximum number of important covariates in variable selection, and [52] considered the maximum in-degree and out-degree in structure learning to restrict the model space.

Original: In the same context, [55] suggested the use of a feasible set for the initial partition in the community detection problem.
Revised: In the same context, [55] suggested the use of a feasible set for the initial partition in the community detection problem.

Original: In this regard, we may consider X as the restricted space so that s 0 can be controlled.
Revised: In this regard, we may consider X as the restricted space so that s 0 can be controlled.

Original: Most importantly, Condition (iii) provides a key idea on how to select the number of trials N , which will be discussed in Section 3.2.
Revised: Most importantly, Condition (iii) provides a key idea on how to select the number of trials N , which will be discussed in Section 3.2.

Original: Remark 2 (Problem-speciﬁc applications) .
Revised: Remark 2 (Problem-speciﬁc applications) .

Original: Theorem 1 can be used to prove rapid mixing of the MTM algorithm for Bayesian variable selection [47] and structure learning [52] with order
Revised: Theorem 1 can be used to prove rapid mixing of the MTM algorithm for Bayesian variable selection [47] and structure learning [52] with order

Original: N improvement of the mixing bound, given that the choice of N satisﬁes the condition (iii).
Revised: N improvement of the mixing bound, given that the choice of N satisﬁes the condition (iii).

Original: Remark 3 (Computational beneﬁts) .
Revised: Remark 3 (Computational beneﬁts) .

Original: Since MTM with N trials requires calculating 2 N − 1 weight functions at each iteration, improving the mixing time bound by the factor of N leads to the same overall computational complexity as the single-try MH algorithm until the convergence of the Markov chain.
Revised: Since MTM with N trials requires calculating 2 N − 1 weight functions at each iteration, improving the mixing time bound by the factor of N leads to the same overall computational complexity as the single-try MH algorithm until the convergence of the Markov chain.

Original: However, compared to the single-try MH with N times longer sequential iterations, the calculation of N weight functions can be efﬁciently done in parallel.
Revised: However, compared to the single-try MH with N times longer sequential iterations, the calculation of N weight functions can be efﬁciently done in parallel.

Original: For example, calculating multiple weight functions can often be converted into a series of matrix multiplication problems, and optimized linear algebra libraries such as BLAS [7] can be used to exploit multiple execution units and pipe-lining.
Revised: For example, calculating multiple weight functions can often be converted into a series of matrix multiplication problems, and optimized linear algebra libraries such as BLAS [7] can be used to exploit multiple execution units and pipe-lining.

Original: As examples, we outline parallelization strategies for the Bayesian variable selection (Section 4.1) and stochastic block model (Section 4.2) in Appendix D.1.
Revised: As examples, we outline parallelization strategies for the Bayesian variable selection (Section 4.1) and stochastic block model (Section 4.2) in Appendix D.1.

Original: Indeed, Table 1 in our simulation study results shows that MTM with moderate choice of N signiﬁcantly reduces the wall-clock time until the convergence of the Markov chain.
Revised: Indeed, Table 1 in our simulation study results shows that MTM with moderate choice of N signiﬁcantly reduces the wall-clock time until the convergence of the Markov chain.

Original: 3.2 Choice of the number of trials
Revised: 3.2 Choice of the number of trials

Original: Motivated by Condition (iii) in Theorem 1, we propose an algorithm to choose the number of trials N under a general setting applicable to high-dimensional model selection problems.
Revised: Motivated by Condition (iii) in Theorem 1, we propose an algorithm to choose the number of trials N under a general setting applicable to high-dimensional model selection problems.

Original: For conciseness, we shall only focus on illustrating the case when we have balancing function h ( u ) = √ u and symmetric proposals (known t 3 = t 4 ) so that Condition (iii) becomes N =
Revised: For conciseness, we shall only focus on illustrating the case when we have balancing function h ( u ) = √ u and symmetric proposals (known t 3 = t 4 ) so that Condition (iii) becomes N =

Original: o (cid:0) min { p ( t 2 − t 1 ) / 2 , p t 3 /s 0 } (cid:1) , although the algorithm can be easily generalized to other choices of h and non-symmetric proposals.
Revised: o (cid:0) min { p ( t 2 − t 1 ) / 2 , p t 3 /s 0 } (cid:1) , although the algorithm can be easily generalized to other choices of h and non-symmetric proposals.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Algorithm 2 is depicted in Figure 2.
Revised: Algorithm 2 is depicted in Figure 2.

Original: In Step 1, the log probability ratio is calculated for all neighboring states of the initial state x 0 .
Revised: In Step 1, the log probability ratio is calculated for all neighboring states of the initial state x 0 .

Original: Step 2 identiﬁes two subsets Cand C 2 , corresponding to “bad moves” and “good moves” respectively.
Revised: Step 2 identiﬁes two subsets Cand C 2 , corresponding to “bad moves” and “good moves” respectively.

Original: In Step 3, motivated by Condition (i), we set ˆ t 2 =
Revised: In Step 3, motivated by Condition (i), we set ˆ t 2 =

Original: min C 2 , a lower bound of log probability ratios among good moves.
Revised: min C 2 , a lower bound of log probability ratios among good moves.

Original: After t 2 is ﬁxed, we ﬁrst set ˆ t 1 = max C 1 , ˆ s 0 =
Revised: After t 2 is ﬁxed, we ﬁrst set ˆ t 1 = max C 1 , ˆ s 0 =

Original: |C 2 | and gradually decrease ˆ t 1 (so that ˆ s 0 increases) until the “crossing” p (ˆ t 2 − ˆ t 1 ) / 2 ≥ p t 3 / ˆ s 0 happens.
Revised: |C 2 | and gradually decrease ˆ t 1 (so that ˆ s 0 increases) until the “crossing” p (ˆ t 2 − ˆ t 1 ) / 2 ≥ p t 3 / ˆ s 0 happens.

Original: The choice of N = (cid:98) ( p t 3 / ˆ s 0 ) ψ (cid:99) immediately after the crossing point approximates the worst-case scenario of min { p ( t 2 − t 1 ) / 2 , p t 3 /s 0 } , hence providing a conservative estimate of N .
Revised: The choice of N = (cid:98) ( p t 3 / ˆ s 0 ) ψ (cid:99) immediately after the crossing point approximates the worst-case scenario of min { p ( t 2 − t 1 ) / 2 , p t 3 /s 0 } , hence providing a conservative estimate of N .

Original: We set ψ = 0 .
Revised: We set ψ = 0 .

Original: 9 to match with the asymptotic dominance condition (iii).
Revised: 9 to match with the asymptotic dominance condition (iii).

Original: We run Algorithm 2 only once for an initial state x 0 .
Revised: We run Algorithm 2 only once for an initial state x 0 .

Original: It is based on the insight that for model selection problems, log probability ratios can often be partitioned into “bad moves” and “good moves”.
Revised: It is based on the insight that for model selection problems, log probability ratios can often be partitioned into “bad moves” and “good moves”.

Original: Although we considered any x ∈ X\{ x ∗ } in Condition (i), analyzing the initial state xis not only computationally simple but also gives a good guess of t 2 since as chain proceeds to the highest posterior state x ∗ , the number of good moves and the magnitude of ratios generally decreases, for example in the Bayesian variable selection problem [54].
Revised: Although we considered any x ∈ X\{ x ∗ } in Condition (i), analyzing the initial state xis not only computationally simple but also gives a good guess of t 2 since as chain proceeds to the highest posterior state x ∗ , the number of good moves and the magnitude of ratios generally decreases, for example in the Bayesian variable selection problem [54].

Original: Another possible way of using Algorithm 2 is to re-evaluate N after a certain number of MCMC iterations.
Revised: Another possible way of using Algorithm 2 is to re-evaluate N after a certain number of MCMC iterations.

Original: We note that another clustering algorithm can be substituted for the k-means algorithm in Step 2.
Revised: We note that another clustering algorithm can be substituted for the k-means algorithm in Step 2.

Original:  Simulation studies
Revised:  Simulation studies

Original: In this section, we present the results for two Bayesian model selection problems, BVS and SBM.
Revised: In this section, we present the results for two Bayesian model selection problems, BVS and SBM.

Original: We defer the result for spatial clustering models to Appendix B.3 due to the page limit.
Revised: We defer the result for spatial clustering models to Appendix B.3 due to the page limit.

Original: Throughout the simulation studies, we use symmetric proposal schemes.
Revised: Throughout the simulation studies, we use symmetric proposal schemes.

Original: We run single-try MH and MTM with different choices of N ∈ { 5 , 10 , 50 , 100 , 500 , 1000 , 2000 , 5000 } and four different choices of weight functions, one with (unscaled) ordinary weight function w ord
Revised: We run single-try MH and MTM with different choices of N ∈ { 5 , 10 , 50 , 100 , 500 , 1000 , 2000 , 5000 } and four different choices of weight functions, one with (unscaled) ordinary weight function w ord

Original: ( y | x ) =
Revised: ( y | x ) =

Original: π ( y ) , and three with the proposed locally balanced weight functions w sqrt
Revised: π ( y ) , and three with the proposed locally balanced weight functions w sqrt

Original: ( y | x ) =
Revised: ( y | x ) =

Original: (cid:112) π
Revised: (cid:112) π

Original: ( y ) /π
Revised: ( y ) /π

Original: ( x ) , w min ( y | x ) = min { 1 , π ( y ) /π
Revised: ( x ) , w min ( y | x ) = min { 1 , π ( y ) /π

Original: ( x ) } , and w max ( y | x ) = max { 1 , π ( y ) /π
Revised: ( x ) } , and w max ( y | x ) = max { 1 , π ( y ) /π

Original: ( x ) } .
Revised: ( x ) } .

Original: We consider the scenarios where the data-generating model x ∗ receives the highest posterior probability to verify our main theoretical results.
Revised: We consider the scenarios where the data-generating model x ∗ receives the highest posterior probability to verify our main theoretical results.

Original: Similarly to [54], two performance measures are considered: 1) H = min { t : x t = x ∗ } (hitting iteration): the number of MCMC iterations until the Markov chain ( x 0 , x 1 , . . . )
Revised: Similarly to [54], two performance measures are considered: 1) H = min { t : x t = x ∗ } (hitting iteration): the number of MCMC iterations until the Markov chain ( x 0 , x 1 , . . . )

Original: hits x ∗ , 2)
Revised: hits x ∗ , 2)

Original: T H (wall-clock hitting time): the wall-clock time taken until the chain hits x ∗ .
Revised: T H (wall-clock hitting time): the wall-clock time taken until the chain hits x ∗ .

Original: 4.1 Bayesian variable selection (BVS)
Revised: 4.1 Bayesian variable selection (BVS)

Original: Consider a high-dimensional linear model with response vector y ∈ R n and design matrix X ∈ R n × p , where the number of predictors p is much larger than the sample size n .
Revised: Consider a high-dimensional linear model with response vector y ∈ R n and design matrix X ∈ R n × p , where the number of predictors p is much larger than the sample size n .

Original: BVS seeks to ﬁnd the best subset of predictors, denoted as a binary vector γ ∈ { 0 , 1 } p described in Section 3.1, using the sparsity-inducing prior.
Revised: BVS seeks to ﬁnd the best subset of predictors, denoted as a binary vector γ ∈ { 0 , 1 } p described in Section 3.1, using the sparsity-inducing prior.

Original: We adopt the prior introduced by [47]:
Revised: We adopt the prior introduced by [47]:

Original: Linear model:
Revised: Linear model:

Original: [Equation]
Revised: [Equation]

Original: Prior on β γ and φ :
Revised: Prior on β γ and φ :

Original: [Equation]
Revised: [Equation]

Original: Sparsity prior:
Revised: Sparsity prior:

Original: [Equation]
Revised: [Equation]

Original: where X γ is a submatrix of X consisting of all j -th columns with γ j = 1 , β γ ∈ R | γ | is the subvector of β ∈ R p with nonzero coefﬁcients, where | γ | = (cid:80) pj =1 γ j .
Revised: where X γ is a submatrix of X consisting of all j -th columns with γ j = 1 , β γ ∈ R | γ | is the subvector of β ∈ R p with nonzero coefﬁcients, where | γ | = (cid:80) pj =1 γ j .

Original: The hyperparameters G , κ and s max control the weight of prior effect [50], sparsity strength and the maximum model size, respectively.
Revised: The hyperparameters G , κ and s max control the weight of prior effect [50], sparsity strength and the maximum model size, respectively.

Original: We sample γ from the posterior distribution π ( γ | y ) where other parameters β γ and φ are marginalized out; see Appendix B.1 for details.
Revised: We sample γ from the posterior distribution π ( γ | y ) where other parameters β γ and φ are marginalized out; see Appendix B.1 for details.

Original: We consider the following random walk proposal:
Revised: We consider the following random walk proposal:

Original: [Equation]
Revised: [Equation]

Original: if | γ
Revised: if | γ

Original: [Equation]
Revised: [Equation]

Original: if |
Revised: if |

Original: [Equation]
Revised: [Equation]

Original: where N 1 ( γ ) = { γ (cid:48) :
Revised: where N 1 ( γ ) = { γ (cid:48) :

Original: d H ( γ, γ (cid:48) )
Revised: d H ( γ, γ (cid:48) )

Original: = 1 } and N 2 ( γ ) =
Revised: = 1 } and N 2 ( γ ) =

Original: { γ (cid:48)
Revised: { γ (cid:48)

Original: = ( γ ∪ { j } ) \{ (cid:96) } : j (cid:54)∈ γ, (cid:96) ∈ γ } .
Revised: = ( γ ∪ { j } ) \{ (cid:96) } : j (cid:54)∈ γ, (cid:96) ∈ γ } .

Original: With n = 1000 and p = 5000 , we consider different settings of design matrix where each row x
Revised: With n = 1000 and p = 5000 , we consider different settings of design matrix where each row x

Original: i is i.i.d.
Revised: i is i.i.d.

Original: sampled from x i ∼ N ( 0 , I p ) (independent) or x i ∼ N ( 0 , Σ ) , Σ jj (cid:48)
Revised: sampled from x i ∼ N ( 0 , I p ) (independent) or x i ∼ N ( 0 , Σ ) , Σ jj (cid:48)

Original: =
Revised: =

Original: exp( −| j − j (cid:48) | ) (dependent).
Revised: exp( −| j − j (cid:48) | ) (dependent).

Original: The response vector y is generated from y ∼ N ( Xβ , I n ) , with β having only 10 nonzero entries at the ﬁrst 10 coordinates: β = SNR (cid:112) (log p ) /n
Revised: The response vector y is generated from y ∼ N ( Xβ , I n ) , with β having only 10 nonzero entries at the ﬁrst 10 coordinates: β = SNR (cid:112) (log p ) /n

Original: (2 , − 3 , 2 , 2 , − 3 , 3 , − 2 , 3 , − 2 , 3) where SNR ∈ { 2 , 4 } is a signal-to-noise ratio.
Revised: (2 , − 3 , 2 , 2 , − 3 , 3 , − 2 , 3 , − 2 , 3) where SNR ∈ { 2 , 4 } is a signal-to-noise ratio.

Original: For each of the four settings, we simulate 50 replicate datasets.
Revised: For each of the four settings, we simulate 50 replicate datasets.

Original: Per each replicate, algorithms are randomly initialized with state γ 0 such that γ 0 ∩ γ ∗ = ∅ and d H ( γ 0 , γ ∗ )
Revised: Per each replicate, algorithms are randomly initialized with state γ 0 such that γ 0 ∩ γ ∗ = ∅ and d H ( γ 0 , γ ∗ )

Original: =which implies H = 20 is the minimum required hitting iteration.
Revised: =which implies H = 20 is the minimum required hitting iteration.

Original: All hyperparameter setups, the number of MCMC iterations, and other details are deferred to Appendix B.1.
Revised: All hyperparameter setups, the number of MCMC iterations, and other details are deferred to Appendix B.1.

Original: [Figure]
Revised: [Figure]

Original: Results under independent design with SNR = 4 are summarized in Figure 3.
Revised: Results under independent design with SNR = 4 are summarized in Figure 3.

Original: The trace plot shows that single-try MH reaches the true state γ ∗ at around 20,000 iterations, whereas the MTM with
Revised: The trace plot shows that single-try MH reaches the true state γ ∗ at around 20,000 iterations, whereas the MTM with

Original: N = 10 reaches the true state γ ∗ at around 2,000 iterations, smaller by a factor of 10.
Revised: N = 10 reaches the true state γ ∗ at around 2,000 iterations, smaller by a factor of 10.

Original: The boxplot conﬁrms our ﬁndings that a larger number of trials N is not always desirable, especially when using the ordinary weight function w ord .
Revised: The boxplot conﬁrms our ﬁndings that a larger number of trials N is not always desirable, especially when using the ordinary weight function w ord .

Original: The proposed locally balanced weight functions using three different choices of h perform much better than w ord for large N .
Revised: The proposed locally balanced weight functions using three different choices of h perform much better than w ord for large N .

Original: Especially, the performance of w min seems to be robust to the choice of large N in the high SNR case.
Revised: Especially, the performance of w min seems to be robust to the choice of large N in the high SNR case.

Original: This can be explained by the fact that the denominator in (2) is often larger in high SNR data, and the choice of min { 1 , π ( y ) /π
Revised: This can be explained by the fact that the denominator in (2) is often larger in high SNR data, and the choice of min { 1 , π ( y ) /π

Original: ( x ) } limits its growth; see also Section 3.1.
Revised: ( x ) } limits its growth; see also Section 3.1.

Original: The choice of N suggested from Algorithm 2 with ψ = 0 .
Revised: The choice of N suggested from Algorithm 2 with ψ = 0 .

Original: 9 is overall reasonable, which has a median of 349 over 50 datasets.
Revised: 9 is overall reasonable, which has a median of 349 over 50 datasets.

Original: The detailed results for other settings are available in Appendix B.1.
Revised: The detailed results for other settings are available in Appendix B.1.

Original: 4.2 Stochastic block model (SBM)
Revised: 4.2 Stochastic block model (SBM)

Original: SBM [20] is a popular generative model of an undirected graph which assumes a block structure of edge connection probabilities Q to describe community structure.
Revised: SBM [20] is a popular generative model of an undirected graph which assumes a block structure of edge connection probabilities Q to describe community structure.

Original: We consider Bayesian SBM [34] by assigning prior on Q and partition z
Revised: We consider Bayesian SBM [34] by assigning prior on Q and partition z

Original: = ( z 1 , . . .
Revised: = ( z 1 , . . .

Original: , z p ) ∈ { 1 , . .
Revised: , z p ) ∈ { 1 , . .

Original: . , K } p with ﬁxed number of blocks
Revised: . , K } p with ﬁxed number of blocks

Original: K , where z i indicates membership label of i -th node.
Revised: K , where z i indicates membership label of i -th node.

Original: The main goal is to ﬁnd partition z (up to a label permutation) that best describes the graph, denoted as an adjacency matrix A ∈ { 0 , 1 } p × p .
Revised: The main goal is to ﬁnd partition z (up to a label permutation) that best describes the graph, denoted as an adjacency matrix A ∈ { 0 , 1 } p × p .

Original: We follow the prior of [55]:
Revised: We follow the prior of [55]:

Original: Edge appearance:
Revised: Edge appearance:

Original: [Equation]
Revised: [Equation]

Original: Prior on blockwise probabilities:
Revised: Prior on blockwise probabilities:

Original: [Equation]
Revised: [Equation]

Original: partition:
Revised: partition:

Original: [Equation]
Revised: [Equation]

Original: with restricted partition space S
Revised: with restricted partition space S

Original: α = { z : (cid:80) pi
Revised: α = { z : (cid:80) pi

Original: =1 1 ( z i = u ) ∈ (cid:2) pαK , αpK (cid:3) for all u = 1 , . . .
Revised: =1 1 ( z i = u ) ∈ (cid:2) pαK , αpK (cid:3) for all u = 1 , . . .

Original: , K } for some α > 0 , which excludes partitions whose block sizes differ too much.
Revised: , K } for some α > 0 , which excludes partitions whose block sizes differ too much.

Original: We sample z from the posterior π ( z | A ) where Q is marginalized out; see Appendix B.2 for details.
Revised: We sample z from the posterior π ( z | A ) where Q is marginalized out; see Appendix B.2 for details.

Original: We consider the following proposal where ˜ d H is a permutation-invariant Hamming distance [51]:
Revised: We consider the following proposal where ˜ d H is a permutation-invariant Hamming distance [51]:

Original: [Equation]
Revised: [Equation]

Original: With the number of nodes p = 1000 , we consider two different settings of the blocks K ∈ { 2 , 5 } with true partition z ∗ = (1 , . .
Revised: With the number of nodes p = 1000 , we consider two different settings of the blocks K ∈ { 2 , 5 } with true partition z ∗ = (1 , . .

Original: , 1 , · · · , K, . . .
Revised: , 1 , · · · , K, . . .

Original: , K ) being balanced ( p/K times each) which ensures z ∗ ∈ S α .
Revised: , K ) being balanced ( p/K times each) which ensures z ∗ ∈ S α .

Original: In each setting of K , we generate a graph from the homogeneous SBM [16, 55], where
Revised: In each setting of K , we generate a graph from the homogeneous SBM [16, 55], where

Original: Q uv = a if u = v and Q uv = b otherwise.
Revised: Q uv = a if u = v and Q uv = b otherwise.

Original: We choose probability pairs ( a, b ) based on the ChernoffHellinger divergence ( CH ) [1], deﬁned as CH :
Revised: We choose probability pairs ( a, b ) based on the ChernoffHellinger divergence ( CH ) [1], deﬁned as CH :

Original: = p ( √ a −√ b ) 2 / ( K log p ) for a, b (cid:16) (log p ) /p , which can be interpreted as a signal-to-noise ratio.
Revised: = p ( √ a −√ b ) 2 / ( K log p ) for a, b (cid:16) (log p ) /p , which can be interpreted as a signal-to-noise ratio.

Original: Our choices are CH ≈ 2 (weak signal) and CH ≈(high signal), both being greater than 1 which is necessary to satisfy posterior consistency [1, 51, 55].
Revised: Our choices are CH ≈ 2 (weak signal) and CH ≈(high signal), both being greater than 1 which is necessary to satisfy posterior consistency [1, 51, 55].

Original: For each of the the four combinations of ( K, CH) , we simulate 50 replicate datasets.
Revised: For each of the the four combinations of ( K, CH) , we simulate 50 replicate datasets.

Original: For each replicate, algorithms are randomly initialized with state z 0 such that ˜ d H ( z 0 , z ∗ ) = 400 which implies
Revised: For each replicate, algorithms are randomly initialized with state z 0 such that ˜ d H ( z 0 , z ∗ ) = 400 which implies

Original: H = 400 is the minimum required hitting iteration.
Revised: H = 400 is the minimum required hitting iteration.

Original: All hyperparameter setups, the number of
Revised: All hyperparameter setups, the number of

Original: MCMC iterations, and other details are deferred to Appendix B.2.
Revised: MCMC iterations, and other details are deferred to Appendix B.2.

Original: [Figure]
Revised: [Figure]

Original: Results for K = 2 and CH ≈ 2 are summarized in Figure 4.
Revised: Results for K = 2 and CH ≈ 2 are summarized in Figure 4.

Original: The boxplot of H is similar to that of
Revised: The boxplot of H is similar to that of

Original: Figure 3: w ord quickly deteriorates as N increases, but the locally balanced weight functions mostly converge with a larger choice of N , although some Markov chains fail to converge due to the low
Revised: Figure 3: w ord quickly deteriorates as N increases, but the locally balanced weight functions mostly converge with a larger choice of N , although some Markov chains fail to converge due to the low

Original: CH .
Revised: CH .

Original: Especially, now the performance of w max or w sqrt is slightly better than w min when N is large.
Revised: Especially, now the performance of w max or w sqrt is slightly better than w min when N is large.

Original: This can be explained by the fact that in low signal case ( CH ≈ 2 ), there are few neighboring states that have high probability ratios.
Revised: This can be explained by the fact that in low signal case ( CH ≈ 2 ), there are few neighboring states that have high probability ratios.

Original: It is important to catch such states to increase the acceptance ratio (2), and the w min cannot do this efﬁciently as it is upper bounded by 1 .
Revised: It is important to catch such states to increase the acceptance ratio (2), and the w min cannot do this efﬁciently as it is upper bounded by 1 .

Original: The choice of N suggested from Algorithm 2 with ψ = 0 .
Revised: The choice of N suggested from Algorithm 2 with ψ = 0 .

Original: 9 has a median of 15 over 50 datasets.
Revised: 9 has a median of 15 over 50 datasets.

Original: The detailed simulation results for other settings are available at Appendix B.2.
Revised: The detailed simulation results for other settings are available at Appendix B.2.

Original: [Table]
Revised: [Table]

Original: Finally, Table 1 provides the summary of wall-clock hitting time T H for the previous two examples, which shows the clear computational beneﬁt of MTM over the single-try MH thanks to the parallelism.
Revised: Finally, Table 1 provides the summary of wall-clock hitting time T H for the previous two examples, which shows the clear computational beneﬁt of MTM over the single-try MH thanks to the parallelism.

Original: As N increases, computation becomes more demanding compared to the decreasing rate of H , so the optimal choice of N with respect to T H is often less than the minimizer of H .
Revised: As N increases, computation becomes more demanding compared to the decreasing rate of H , so the optimal choice of N with respect to T H is often less than the minimizer of H .

Original: Spatial clustering models and additional information.
Revised: Spatial clustering models and additional information.

Original: In Appendix B.3, we present simulation study results with spatial clustering models where the key ﬁndings align with those from BVS and
Revised: In Appendix B.3, we present simulation study results with spatial clustering models where the key ﬁndings align with those from BVS and

Original: SBM.
Revised: SBM.

Original: In Appendix B.4, we also analyze the behavior of MTM algorithms on multimodal target distributions.
Revised: In Appendix B.4, we also analyze the behavior of MTM algorithms on multimodal target distributions.

Original: The code is available at https://github.com/changwoo-lee/rapidMTM .
Revised: The code is available at https://github.com/changwoo-lee/rapidMTM .

Original:  Real data applications and discussion
Revised:  Real data applications and discussion

Original: Real data applications.
Revised: Real data applications.

Original: We carry out two real data application analyses to corroborate our ﬁndings beyond the scope of the simulation settings.
Revised: We carry out two real data application analyses to corroborate our ﬁndings beyond the scope of the simulation settings.

Original: The performance measures suggested in Section 4, the hitting iteration H and the corresponding wall-clock hitting time T H , are no longer available for real data analysis since we cannot identify the true data generating state x ∗ .
Revised: The performance measures suggested in Section 4, the hitting iteration H and the corresponding wall-clock hitting time T H , are no longer available for real data analysis since we cannot identify the true data generating state x ∗ .

Original: Instead, we evaluate the performance based on the acceptance rate and the number of unique states visited.
Revised: Instead, we evaluate the performance based on the acceptance rate and the number of unique states visited.

Original: Both real data analyses conﬁrm our theory: the performance deteriorates signiﬁcantly as the number of trials N grows when using w ord , but it does not when using w sqrt , w min , or w max .
Revised: Both real data analyses conﬁrm our theory: the performance deteriorates signiﬁcantly as the number of trials N grows when using w ord , but it does not when using w sqrt , w min , or w max .

Original: The ﬁrst real data application is to ﬁnd a subset of genetic variants from a genome-wide association study (GWAS) dataset that best explains the cup-to-disk ratio averaged over two eyes, which is used to assess the risk of glaucoma.
Revised: The ﬁrst real data application is to ﬁnd a subset of genetic variants from a genome-wide association study (GWAS) dataset that best explains the cup-to-disk ratio averaged over two eyes, which is used to assess the risk of glaucoma.

Original: We employ BVS model described in Section 4.1, with the number of samples n = 5418 and the number of gene variants p = 7255 which are selected after the preliminary screening procedure described in [54, Section 6].
Revised: We employ BVS model described in Section 4.1, with the number of samples n = 5418 and the number of gene variants p = 7255 which are selected after the preliminary screening procedure described in [54, Section 6].

Original: We tabulate the acceptance rate and the number of unique states visited along the different number of trials N , and also report the posterior inclusion probabilities of the top 10 genetic variants in Appendix C.1 and E with discussion.
Revised: We tabulate the acceptance rate and the number of unique states visited along the different number of trials N , and also report the posterior inclusion probabilities of the top 10 genetic variants in Appendix C.1 and E with discussion.

Original: The other application is to learn the underlying directed acyclic graph (DAG) model for the single-cell
Revised: The other application is to learn the underlying directed acyclic graph (DAG) model for the single-cell

Original: RNA dataset on Alzheimer’s disease [22].
Revised: RNA dataset on Alzheimer’s disease [22].

Original: We preprocess the dataset as [10, Section 6], which yields the sample size n = 1666 and 73 genes, and we utilize the DAG model described in [10, Section 2.2], whose model size is equal to 73!
Revised: We preprocess the dataset as [10, Section 6], which yields the sample size n = 1666 and 73 genes, and we utilize the DAG model described in [10, Section 2.2], whose model size is equal to 73!

Original: ≈ 4 .
Revised: ≈ 4 .

Original: 5 × 10 105 .
Revised: 5 × 10 105 .

Original: We defer the performance result and other details (including the log-posterior trace plots for 4 different weight functions) to Appendix C.2.
Revised: We defer the performance result and other details (including the log-posterior trace plots for 4 different weight functions) to Appendix C.2.

Original: Discussion and future work.
Revised: Discussion and future work.

Original: We prove that, under some assumptions, the mixing time bound of the MTM algorithm is smaller than that of the MH algorithm by a factor of the number of trials.
Revised: We prove that, under some assumptions, the mixing time bound of the MTM algorithm is smaller than that of the MH algorithm by a factor of the number of trials.

Original: Motivated by the observation that popular choices of weight function can cause mixing problems, we propose locally balanced weight functions for which a mixing time bound is proven.
Revised: Motivated by the observation that popular choices of weight function can cause mixing problems, we propose locally balanced weight functions for which a mixing time bound is proven.

Original: We also suggest a theoretically guided choice of N for practical use.
Revised: We also suggest a theoretically guided choice of N for practical use.

Original: Future research may investigate the mixing time of other generalizations of MTM such as the case with correlated weight functions.
Revised: Future research may investigate the mixing time of other generalizations of MTM such as the case with correlated weight functions.

Original: It will also be of interest to extend MTM and analyze its mixing time to better handle the potential multimodality in the target distribution, by combining it with techniques such as annealing or tempering [9].
Revised: It will also be of interest to extend MTM and analyze its mixing time to better handle the potential multimodality in the target distribution, by combining it with techniques such as annealing or tempering [9].

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: We thank James Cai for providing us with the processed single-cell data set on Alzheimer’s disease.
Revised: We thank James Cai for providing us with the processed single-cell data set on Alzheimer’s disease.

Original: We thank the submitters and participants of the two dbGaP studies (phs000308.v1.p1 and phs000238.v1.p1), which were funded by NIH.
Revised: We thank the submitters and participants of the two dbGaP studies (phs000308.v1.p1 and phs000238.v1.p1), which were funded by NIH.

Original: We thank Florian Maire, Giacomo Zanella, and
Revised: We thank Florian Maire, Giacomo Zanella, and

Original: Philippe Gagnon for the helpful discussion at ISBA 2022.
Revised: Philippe Gagnon for the helpful discussion at ISBA 2022.

Original: The research of Changwoo Lee, Zhao
Revised: The research of Changwoo Lee, Zhao

Original: Tang Luo, and Huiyan Sang was partially supported by NSF DMS-1854655 and DMS-2210456.
Revised: Tang Luo, and Huiyan Sang was partially supported by NSF DMS-1854655 and DMS-2210456.

Original: Appendices
Revised: Appendices

Original: In Appendix A, we provide proofs of Proposition 1, Proposition 2, and Theorem 1 in the main text.
Revised: In Appendix A, we provide proofs of Proposition 1, Proposition 2, and Theorem 1 in the main text.

Original: In Appendix B, we provide more details of the Bayesian variable selection (BVS) and stochastic block model (SBM) in Section 4 as well as a detailed simulation study on the spatial clustering model (SCM).
Revised: In Appendix B, we provide more details of the Bayesian variable selection (BVS) and stochastic block model (SBM) in Section 4 as well as a detailed simulation study on the spatial clustering model (SCM).

Original: In addition, we study the performance of multiple-try Metropolis for the case with multimodal target distributions, following the BVS simulation setting of [54].
Revised: In addition, we study the performance of multiple-try Metropolis for the case with multimodal target distributions, following the BVS simulation setting of [54].

Original: In Appendix C, we present details of two real data application analyses.
Revised: In Appendix C, we present details of two real data application analyses.

Original: In Appendix D, we add a more detailed discussion on parallelization, state space of interest, and the behavior of MTM on continuous state space.
Revised: In Appendix D, we add a more detailed discussion on parallelization, state space of interest, and the behavior of MTM on continuous state space.

Original: Finally, we provide additional tables on the real data analysis results in Appendix E.
Revised: Finally, we provide additional tables on the real data analysis results in Appendix E.

Original: A A path method for proving the mixing time bound for multiple-try Metropolis algorithm A.1 Proof of Proposition
Revised: A A path method for proving the mixing time bound for multiple-try Metropolis algorithm A.1 Proof of Proposition

Original: This section aims to provide a summary of the existing results on proving mixing time bound via path methods.
Revised: This section aims to provide a summary of the existing results on proving mixing time bound via path methods.

Original: We refer readers to [13, 25, 42, 43] for more details.
Revised: We refer readers to [13, 25, 42, 43] for more details.

Original: Let P ( x, y ) denote the transition probability for an irreducible, aperiodic chain on the ﬁnite state space X .
Revised: Let P ( x, y ) denote the transition probability for an irreducible, aperiodic chain on the ﬁnite state space X .

Original: Assume P satisﬁes the detailed balance condition with respect to the probability distribution π , that is, π ( x ) P ( x, y ) =
Revised: Assume P satisﬁes the detailed balance condition with respect to the probability distribution π , that is, π ( x ) P ( x, y ) =

Original: π ( y ) P ( y, x ) for x, y ∈ X , which leads to π being stationary for P [25, Proposition 1.20].
Revised: π ( y ) P ( y, x ) for x, y ∈ X , which leads to π being stationary for P [25, Proposition 1.20].

Original: P may be thought as a |X| × |X| stochastic matrix, which means P ( x, y ) ≥ 

Revised: P may be thought as a |X| × |X| stochastic matrix, which means P ( x, y ) ≥

Original: and (cid:80) z ∈X P ( x, z )
Revised: and (cid:80) z ∈X P ( x, z )

Original: = 1 for all x, y ∈ X , and π can be regarded as a |X| -dimensional stochastic vector since (cid:80)
Revised: = 1 for all x, y ∈ X , and π can be regarded as a |X| -dimensional stochastic vector since (cid:80)

Original: x ∈X π ( x ) = 1 .
Revised: x ∈X π ( x ) = 1 .

Original: By the spectral decomposition, we can sort the eigenvalues of P as
Revised: By the spectral decomposition, we can sort the eigenvalues of P as

Original: [Equation]
Revised: [Equation]

Original: due to P being stochastic, irreducible and aperiodic [25, Lemma 12.1].
Revised: due to P being stochastic, irreducible and aperiodic [25, Lemma 12.1].

Original: Let λ max = max { λ 1 , | λ |X|− 1 |} .
Revised: Let λ max = max { λ 1 , | λ |X|− 1 |} .

Original: We say Gap( P ) = 1 − λ max is the spectral gap of the chain P .
Revised: We say Gap( P ) = 1 − λ max is the spectral gap of the chain P .

Original: Intuitively, if the spectral gap is close to zero, the chain requires a large number of steps to be close to the stationary distribution in total variation distance.
Revised: Intuitively, if the spectral gap is close to zero, the chain requires a large number of steps to be close to the stationary distribution in total variation distance.

Original: The following lemma draws the connection between the spectral gap and mixing time deﬁned in Section 2.3.
Revised: The following lemma draws the connection between the spectral gap and mixing time deﬁned in Section 2.3.

Original: Lemma 1.
Revised: Lemma 1.

Original: Let t mix ( (cid:15) ) denote (cid:15) -mixing time deﬁned in Section 2.3, then
Revised: Let t mix ( (cid:15) ) denote (cid:15) -mixing time deﬁned in Section 2.3, then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We consider P lazy =
Revised: We consider P lazy =

Original: ( I + P ) / 2 so that all eigenvalues of P lazy are positive and the spectral gap becomes Gap( P lazy )
Revised: ( I + P ) / 2 so that all eigenvalues of P lazy are positive and the spectral gap becomes Gap( P lazy )

Original: =
Revised: =

Original: 1 − λ 1 .
Revised: 1 − λ 1 .

Original: By (1.10) of [13, Proposition 3],
Revised: By (1.10) of [13, Proposition 3],

Original: [Equation]
Revised: [Equation]

Original: for t ∈ N .
Revised: for t ∈ N .

Original: We set max x ∈X { (1 − π ( x )) /π
Revised: We set max x ∈X { (1 − π ( x )) /π

Original: ( x ) } exp[ − 2 t Gap( P lazy )] ≤ 4 (cid:15) 2 , and solving the inequality with t gives
Revised: ( x ) } exp[ − 2 t Gap( P lazy )] ≤ 4 (cid:15) 2 , and solving the inequality with t gives

Original: [Equation]
Revised: [Equation]

Original: Since
Revised: Since

Original: [Equation]
Revised: [Equation]

Original: min x ∈X π ( x ) (cid:27) − 1 (cid:35) ≤ 2 { Gap( P ) } −(cid:34) log(1 /(cid:15) )
Revised: min x ∈X π ( x ) (cid:27) − 1 (cid:35) ≤ 2 { Gap( P ) } −(cid:34) log(1 /(cid:15) )

Original: + log (cid:26)
Revised: + log (cid:26)

Original: min x ∈X π ( x ) (cid:27) − 1 (cid:35) , it follows that to achieve “ (cid:15) -mixing”, it sufﬁces to choose
Revised: min x ∈X π ( x ) (cid:27) − 1 (cid:35) , it follows that to achieve “ (cid:15) -mixing”, it sufﬁces to choose

Original: [Equation]
Revised: [Equation]

Original: This concludes the proof.
Revised: This concludes the proof.

Original: Our next interest is to ﬁnd the lower bound of the spectral gap, which leads to the upper bound of the mixing time.
Revised: Our next interest is to ﬁnd the lower bound of the spectral gap, which leads to the upper bound of the mixing time.

Original: It is often the case that the functional analysis tool is useful to establish such bound.
Revised: It is often the case that the functional analysis tool is useful to establish such bound.

Original: Let R X
Revised: Let R X

Original: = { f : X → R } and (cid:96) 2 ( π ) ⊂
Revised: = { f : X → R } and (cid:96) 2 ( π ) ⊂

Original: R X be the vector space equipped with an inner product (cid:104)· , ·(cid:105) π , which is deﬁned by (cid:104) f 1 , f 2 (cid:105) π
Revised: R X be the vector space equipped with an inner product (cid:104)· , ·(cid:105) π , which is deﬁned by (cid:104) f 1 , f 2 (cid:105) π

Original: = (cid:80) x ∈X f 1 ( x ) f 2 ( x ) π ( x ) for all f 1 , f 2 ∈ (cid:96) 2 ( π ) .
Revised: = (cid:80) x ∈X f 1 ( x ) f 2 ( x ) π ( x ) for all f 1 , f 2 ∈ (cid:96) 2 ( π ) .

Original: We can regard the transition probability P as a function operator in (cid:96) 2 ( π ) , which can be deﬁned as
Revised: We can regard the transition probability P as a function operator in (cid:96) 2 ( π ) , which can be deﬁned as

Original: P f ( x ) =
Revised: P f ( x ) =

Original: (cid:80) y ∈X P ( x, y ) f ( y ) .
Revised: (cid:80) y ∈X P ( x, y ) f ( y ) .

Original: We deﬁne the Dirichlet form associated to the pair ( P , π ) by
Revised: We deﬁne the Dirichlet form associated to the pair ( P , π ) by

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: π for f 1 , f 2 ∈ (cid:96) 2 ( π ) .
Revised: π for f 1 , f 2 ∈ (cid:96) 2 ( π ) .

Original: By the reversibility of P , we can easily check that E ( f ) =
Revised: By the reversibility of P , we can easily check that E ( f ) =

Original: E ( f, f ) = 12 (cid:80) x,y ∈X [ f ( x ) − f ( y )] 2 π ( x ) P ( x, y ) .
Revised: E ( f, f ) = 12 (cid:80) x,y ∈X [ f ( x ) − f ( y )] 2 π ( x ) P ( x, y ) .

Original: The spectral gap can be deﬁned using the Dirichlet form as follows [25, Remark 13.8]:
Revised: The spectral gap can be deﬁned using the Dirichlet form as follows [25, Remark 13.8]:

Original: [Equation]
Revised: [Equation]

Original: where Var π ( f ) = (cid:80) x ∈X ( f ( x ) − E π f ) 2 π ( x ) .
Revised: where Var π ( f ) = (cid:80) x ∈X ( f ( x ) − E π f ) 2 π ( x ) .

Original: This deﬁnition also has a link to the famous Poincaré inequality [4], that is, Var π ( f ) ≤ C E ( f ) for all f ∈ (cid:96) 2 ( π ) , because the smallest constant C is equal to { Gap( P ) } − 1 .
Revised: This deﬁnition also has a link to the famous Poincaré inequality [4], that is, Var π ( f ) ≤ C E ( f ) for all f ∈ (cid:96) 2 ( π ) , because the smallest constant C is equal to { Gap( P ) } − 1 .

Original: The next lemma uses Poincaré inequality and an arbitrary path ensemble ∆ deﬁned in the main text.
Revised: The next lemma uses Poincaré inequality and an arbitrary path ensemble ∆ deﬁned in the main text.

Original: Lemma 2 (Corollory 6, [43]) .
Revised: Lemma 2 (Corollory 6, [43]) .

Original: For an arbitrary path ensemble ∆ ,
Revised: For an arbitrary path ensemble ∆ ,

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) (∆)
Revised: where (cid:96) (∆)

Original: =
Revised: =

Original: max x,y | δ ( x, y ) | and
Revised: max x,y | δ ( x, y ) | and

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We follow the proof given in [42, Theorem 3.2.1].
Revised: We follow the proof given in [42, Theorem 3.2.1].

Original: For each ( x, y ) ∈
Revised: For each ( x, y ) ∈

Original: X × X and for any function f ∈ R X , we can write f ( y ) − f ( x ) =
Revised: X × X and for any function f ∈ R X , we can write f ( y ) − f ( x ) =

Original: (cid:80) ( u,v ) ∈
Revised: (cid:80) ( u,v ) ∈

Original: δ ( x,y ) f
Revised: δ ( x,y ) f

Original: ( v ) − f ( u ) .
Revised: ( v ) − f ( u ) .

Original: By using Cauchy-Schwarz, multiplying π ( x ) π ( y ) / 2 , and summing over x and y ,
Revised: By using Cauchy-Schwarz, multiplying π ( x ) π ( y ) / 2 , and summing over x and y ,

Original: [Equation]
Revised: [Equation]

Original: = ⇒  (cid:88) x,y | f ( y )
Revised: = ⇒  (cid:88) x,y | f ( y )

Original: [Equation]
Revised: [Equation]

Original: f ( x ) |
Revised: f ( x ) |

Original: [Equation]
Revised: [Equation]

Original: ( x )
Revised: ( x )

Original: [Equation]
Revised: [Equation]

Original: ( y ) (cid:124) (cid:123)(cid:122) (cid:125) =
Revised: ( y ) (cid:124) (cid:123)(cid:122) (cid:125) =

Original: Var π ( f ) ≤ (cid:88) x
Revised: Var π ( f ) ≤ (cid:88) x

Original: [Equation]
Revised: [Equation]

Original: y | δ ( x, y ) | (cid:88) ( u,v ) ∈
Revised: y | δ ( x, y ) | (cid:88) ( u,v ) ∈

Original: [Equation]
Revised: [Equation]

Original: , where the right-hand side becomes
Revised: , where the right-hand side becomes

Original: [Equation]
Revised: [Equation]

Original: This satisﬁes the Poincaré inequality, which yields the conclusion.
Revised: This satisﬁes the Poincaré inequality, which yields the conclusion.

Original: By combining the results of Lemma 1 and Lemma 2, we get the conclusion of Proposition 1.
Revised: By combining the results of Lemma 1 and Lemma 2, we get the conclusion of Proposition 1.

Original: A.2 Proof of Proposition
Revised: A.2 Proof of Proposition

Original: Proof.
Revised: Proof.

Original: Recall that the form of the weight function suggested in [27] is given by
Revised: Recall that the form of the weight function suggested in [27] is given by

Original: [Equation]
Revised: [Equation]

Original: where λ ( x, y )
Revised: where λ ( x, y )

Original: =
Revised: =

Original: λ ( y, x ) is a non-negative symmetric function in x and y , and satisﬁes λ ( x, y ) >
Revised: λ ( y, x ) is a non-negative symmetric function in x and y , and satisﬁes λ ( x, y ) >

Original: whenever K RW ( x, y ) > 0 .
Revised: whenever K RW ( x, y ) > 0 .

Original: If we put λ ( x, y )
Revised: If we put λ ( x, y )

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: it is easy to check that the conditions are met.
Revised: it is easy to check that the conditions are met.

Original: A.3 Proof of Theorem
Revised: A.3 Proof of Theorem

Original: In this section, we prove our main result by using Proposition 1.
Revised: In this section, we prove our main result by using Proposition 1.

Original: The main step of the proof is identifying the path ensemble ∆ ∗ that makes the mixing time bound tight.
Revised: The main step of the proof is identifying the path ensemble ∆ ∗ that makes the mixing time bound tight.

Original: To this end, we need to choose exactly one path δ ∗ ( x, y ) for each tuple ( x, y ) ∈
Revised: To this end, we need to choose exactly one path δ ∗ ( x, y ) for each tuple ( x, y ) ∈

Original: X × X .
Revised: X × X .

Original: (Note that x
Revised: (Note that x

Original: and y cannot be identical by the deﬁnition of path).
Revised: and y cannot be identical by the deﬁnition of path).

Original: From the intuition described in Section 2.3, an edge ( u, v ) has a large capacity if π ( u ) and π ( v ) are large.
Revised: From the intuition described in Section 2.3, an edge ( u, v ) has a large capacity if π ( u ) and π ( v ) are large.

Original: For example, if an edge contains the highest posterior state x ∗ , we can let the edge be traversed by a large number of paths.
Revised: For example, if an edge contains the highest posterior state x ∗ , we can let the edge be traversed by a large number of paths.

Original: Given an edge with a small capacity, however, we need to ensure that the edge overlies with a small number of paths.
Revised: Given an edge with a small capacity, however, we need to ensure that the edge overlies with a small number of paths.

Original: Importantly, we do not let a path δ ∗ ( x, y ) pass through an edge with a small capacity if both π ( x ) and π ( y ) are large, so that the edge can maintain a small unit ﬂow size.
Revised: Importantly, we do not let a path δ ∗ ( x, y ) pass through an edge with a small capacity if both π ( x ) and π ( y ) are large, so that the edge can maintain a small unit ﬂow size.

Original: We may envision the topography of the path ensemble; x ∗ becomes the hub, while states with low posterior probability are located on the outskirt.
Revised: We may envision the topography of the path ensemble; x ∗ becomes the hub, while states with low posterior probability are located on the outskirt.

Original: We construct the path ensemble ∆ ∗ according to the description above.
Revised: We construct the path ensemble ∆ ∗ according to the description above.

Original: The construction of ∆ ∗ is similar to that of [47] and [52].
Revised: The construction of ∆ ∗ is similar to that of [47] and [52].

Original: With a neighborhood relation N that satisﬁes the conditions in
Revised: With a neighborhood relation N that satisﬁes the conditions in

Original: Theorem 1, g : X → X ,
Revised: Theorem 1, g : X → X ,

Original: [Equation]
Revised: [Equation]

Original: By Condition (i) in Theorem 1, there exists m ∈ N such that g m ( x ) =
Revised: By Condition (i) in Theorem 1, there exists m ∈ N such that g m ( x ) =

Original: ( m
Revised: ( m

Original: [Equation]
Revised: [Equation]

Original: g ◦ · · · ◦ g )( x ) =
Revised: g ◦ · · · ◦ g )( x ) =

Original: x ∗ for any x ∈ X and x ∗ is the only ﬁxed point of g .
Revised: x ∗ for any x ∈ X and x ∗ is the only ﬁxed point of g .

Original: ( x ∗ can be thought as an attractor in dynamic systems.)
Revised: ( x ∗ can be thought as an attractor in dynamic systems.)

Original: For all x, y ∈ X with x (cid:54)
Revised: For all x, y ∈ X with x (cid:54)

Original: =
Revised: =

Original: y , we have three cases; (i) g m ( x ) =
Revised: y , we have three cases; (i) g m ( x ) =

Original: y for some m ∈ N \ { 0 } , (ii) g m ( y ) =
Revised: y for some m ∈ N \ { 0 } , (ii) g m ( y ) =

Original: x for some m ∈ N \ { 0 } , or (iii) neither (i) nor (ii).
Revised: x for some m ∈ N \ { 0 } , or (iii) neither (i) nor (ii).

Original: If ( x, y ) ∈ X × X belongs to (i), we deﬁne δ ∗ ( x, y ) =
Revised: If ( x, y ) ∈ X × X belongs to (i), we deﬁne δ ∗ ( x, y ) =

Original: ( x, g ( x ) , . . .
Revised: ( x, g ( x ) , . . .

Original: , g m ( x ) = y ) .
Revised: , g m ( x ) = y ) .

Original: Similarly, if ( x, y ) ∈
Revised: Similarly, if ( x, y ) ∈

Original: X × X belongs to (ii), let δ ∗ ( x, y ) =
Revised: X × X belongs to (ii), let δ ∗ ( x, y ) =

Original: ( x = g m ( y ) , . .
Revised: ( x = g m ( y ) , . .

Original: , g ( y ) , y ) .
Revised: , g ( y ) , y ) .

Original: For the case (iii), if m 1 , m 2 ∈ N \ { 0 } are the minimum numbers that satisfy g m 1 ( x ) = x ∗ , g m 2 ( y ) =
Revised: For the case (iii), if m 1 , m 2 ∈ N \ { 0 } are the minimum numbers that satisfy g m 1 ( x ) = x ∗ , g m 2 ( y ) =

Original: x ∗ , respectively, we let δ ∗ ( x, y ) =
Revised: x ∗ , respectively, we let δ ∗ ( x, y ) =

Original: ( x, g ( x ) , . . .
Revised: ( x, g ( x ) , . . .

Original: , g m 1 ( x ) = x ∗ =
Revised: , g m 1 ( x ) = x ∗ =

Original: g m 2 ( y ) , . .
Revised: g m 2 ( y ) , . .

Original: , g ( y ) , y ) .
Revised: , g ( y ) , y ) .

Original: This yields the path ensemble ∆ ∗ .
Revised: This yields the path ensemble ∆ ∗ .

Original: We provide a toy example on how to construct g and any path δ ∗ ( x, y ) for each tuple ( x, y ) ∈
Revised: We provide a toy example on how to construct g and any path δ ∗ ( x, y ) for each tuple ( x, y ) ∈

Original: X × X associated with g in Appendix A.5.
Revised: X × X associated with g in Appendix A.5.

Original: Next, we make a bound for the congestion parameter ρ (∆ ∗ ) .
Revised: Next, we make a bound for the congestion parameter ρ (∆ ∗ ) .

Original: We let Λ( u ) = { x ∈ X : u = g k
Revised: We let Λ( u ) = { x ∈ X : u = g k

Original: x ) , k ∈ N } denote the ancestor set of u with respect to g .
Revised: x ) , k ∈ N } denote the ancestor set of u with respect to g .

Original: If ( u, v ) ∈ δ ∗ ( x, y ) for some x, y ∈ X , we can easily verify that x ∈ Λ( u ) by the construction of δ ∗ .
Revised: If ( u, v ) ∈ δ ∗ ( x, y ) for some x, y ∈ X , we can easily verify that x ∈ Λ( u ) by the construction of δ ∗ .

Original: This implies { ( x, y ) ∈
Revised: This implies { ( x, y ) ∈

Original: X ×
Revised: X ×

Original: X : ( u, v ) ∈
Revised: X : ( u, v ) ∈

Original: δ ∗ ( x, y ) } ⊆ Λ( u ) × X .
Revised: δ ∗ ( x, y ) } ⊆ Λ( u ) × X .

Original: It follows that
Revised: It follows that

Original: [Equation]
Revised: [Equation]

Original: The last inequality holds by the following.
Revised: The last inequality holds by the following.

Original: Let g − k ( u ) =
Revised: Let g − k ( u ) =

Original: { x ∈ X : g k ( x )
Revised: { x ∈ X : g k ( x )

Original: =
Revised: =

Original: u, g k − 1 ( x ) (cid:54) =
Revised: u, g k − 1 ( x ) (cid:54) =

Original: u } for k ∈ N , then Λ( u ) =
Revised: u } for k ∈ N , then Λ( u ) =

Original: (cid:93) k ∈ N g − k
Revised: (cid:93) k ∈ N g − k

Original: ( u ) .
Revised: ( u ) .

Original: By Condition (ii) in Theorem 1, we have | g − k
Revised: By Condition (ii) in Theorem 1, we have | g − k

Original: ( u ) |
Revised: ( u ) |

Original: ≤ p kt 4 which yields,
Revised: ≤ p kt 4 which yields,

Original: [Equation]
Revised: [Equation]

Original: where we use Condition (i) and the deﬁnition of g , and t 2 > t 4 in Theorem 1.
Revised: where we use Condition (i) and the deﬁnition of g , and t 2 > t 4 in Theorem 1.

Original: Finally, we show that P ( x, g ( x )) ≥ C (cid:48)
Revised: Finally, we show that P ( x, g ( x )) ≥ C (cid:48)

Original: Np t 4 for x (cid:54) =
Revised: Np t 4 for x (cid:54) =

Original: x ∗ and for some universal constant C (cid:48) > 0 .
Revised: x ∗ and for some universal constant C (cid:48) > 0 .

Original: Recall that (see also [27, Theorem 1])
Revised: Recall that (see also [27, Theorem 1])

Original: [Equation]
Revised: [Equation]

Original: and we denote the ﬁrst and the second terms of the right-hand side by
Revised: and we denote the ﬁrst and the second terms of the right-hand side by

Original: K ( x, g ( x ))
Revised: K ( x, g ( x ))

Original: =
Revised: =

Original: N
Revised: N

Original: [Equation]
Revised: [Equation]

Original: Hence, we have P ( x, g ( x )) ≥ K ( x, g ( x ))
Revised: Hence, we have P ( x, g ( x )) ≥ K ( x, g ( x ))

Original: η ( x, g ( x )) .
Revised: η ( x, g ( x )) .

Original: We remark that the formulation of K ( x, g ( x )) in (8) is based on the exchangeability, where the N -th trial state y N = g ( x ) is assumed to be selected as a proposal state and its probability is multiplied by N .
Revised: We remark that the formulation of K ( x, g ( x )) in (8) is based on the exchangeability, where the N -th trial state y N = g ( x ) is assumed to be selected as a proposal state and its probability is multiplied by N .

Original: However, we will not utilize such exchangeability to calculate a lower bound of K ( x, g ( x )) .
Revised: However, we will not utilize such exchangeability to calculate a lower bound of K ( x, g ( x )) .

Original: Instead, we deﬁne an event A that selects g ( x ) as the proposal from Step 1 and Step 2 in Algorithm 1, so that P ( A ) =
Revised: Instead, we deﬁne an event A that selects g ( x ) as the proposal from Step 1 and Step 2 in Algorithm 1, so that P ( A ) =

Original: K ( x, g ( x )) .
Revised: K ( x, g ( x )) .

Original: We aim to lower bound the P ( A ) by using the law of total probability.
Revised: We aim to lower bound the P ( A ) by using the law of total probability.

Original: To this end, we introduce the event F for
Revised: To this end, we introduce the event F for

Original: Step 1 in Algorithm 1 that we include the state g ( x ) at least once among the N trials while we don’t sample any “high” posterior states of the neighborhood of x for the rest of trials, i.e. they do not belong to the set S ( x ) .
Revised: Step 1 in Algorithm 1 that we include the state g ( x ) at least once among the N trials while we don’t sample any “high” posterior states of the neighborhood of x for the rest of trials, i.e. they do not belong to the set S ( x ) .

Original: Using conditional probability rule, the probability of the event F is equal to
Revised: Using conditional probability rule, the probability of the event F is equal to

Original: [Equation]
Revised: [Equation]

Original: where we take the probability over the uniform samples in Step 1 in Algorithm 1.
Revised: where we take the probability over the uniform samples in Step 1 in Algorithm 1.

Original: Using the inequality a n − b n = ( a − b )( a n − 1 + a n − 2 b + · · · + b n − 1 ) ≥ ( a − b ) nb n − 1 for any a ≥ b ≥ 0 , we ﬁnd that the lower bound of P ( F ) can be obtained by
Revised: Using the inequality a n − b n = ( a − b )( a n − 1 + a n − 2 b + · · · + b n − 1 ) ≥ ( a − b ) nb n − 1 for any a ≥ b ≥ 0 , we ﬁnd that the lower bound of P ( F ) can be obtained by

Original: [Equation]
Revised: [Equation]

Original: x ) | (cid:19) N − (cid:18) 1 −
Revised: x ) | (cid:19) N − (cid:18) 1 −

Original: [Equation]
Revised: [Equation]

Original: S ( x ) | |N
Revised: x ) [Equation]

Original:  ( x ) | (cid:19) N (cid:33) ≥
Revised: |N ( x ) | (cid:19) [Equation] ≥

Original: N |N ( x ) | (cid:18) 1 −|
Revised: N |N ( x ) | (cid:18) 1 −

Original: [Equation]
Revised: [Equation]

Original: ( x ) | |N ( x ) | (cid:19)
Revised: x ) | |N ( x ) | (cid:19)

Original: [Equation]
Revised: [Equation]

Original: −≥ N p
Revised: ≥ N p

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: (cid:19) ≥ N p
Revised: (cid:19) ≥ N p

Original: [Equation]
Revised: [Equation]

Original: where Condition (ii) and Bernoulli’s inequality are used in the second inequality, and Condition (iii) is used in the last inequality.
Revised: where Condition (ii) and Bernoulli’s inequality are used in the second inequality, and Condition (iii) is used in the last inequality.

Original: We further deﬁne F k as the event F with k number of g ( x ) among N trials.
Revised: We further deﬁne F k as the event F with k number of g ( x ) among N trials.

Original: (Note that since we sample trials with replacement in Step 1 of Algorithm 1, we may sample g ( x ) multiple times.)
Revised: (Note that since we sample trials with replacement in Step 1 of Algorithm 1, we may sample g ( x ) multiple times.)

Original: Observe that F = (cid:93) Nk
Revised: Observe that F = (cid:93) Nk

Original: =1 F k .
Revised: =1 F k .

Original: Given the event F k , the probability to select g ( x ) in (2) of Algorithm 1 is upper bounded by
Revised: Given the event F k , the probability to select g ( x ) in (2) of Algorithm 1 is upper bounded by

Original: [Equation]
Revised: [Equation]

Original: To see ( (cid:63) ) , we have used the fact that h is a non-decreasing function, Condition (iii), and we have
Revised: To see ( (cid:63) ) , we have used the fact that h is a non-decreasing function, Condition (iii), and we have

Original: [Equation]
Revised: [Equation]

Original: from Condition (i) and (ii).
Revised: from Condition (i) and (ii).

Original: Note that the right-hand side of the inequality ( (cid:63) ) does not depend on k .
Revised: Note that the right-hand side of the inequality ( (cid:63) ) does not depend on k .

Original: Using the law of total probability, we yield the lower bound of P ( A )
Revised: Using the law of total probability, we yield the lower bound of P ( A )

Original: =
Revised: =

Original: K ( x, g ( x )) by combining the previous results:
Revised: K ( x, g ( x )) by combining the previous results:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Similarly
Revised: Similarly

Original: [Equation]
Revised: [Equation]

Original: we can calculate the lower bound of η ( x, g ( x )) .
Revised: we can calculate the lower bound of η ( x, g ( x )) .

Original: We consider the event G that we don’t select any “high” posterior states of the neighborhood of g ( x ) for N − 1 trials, that is, any of them are not in the set S ( g ( x )) .
Revised: We consider the event G that we don’t select any “high” posterior states of the neighborhood of g ( x ) for N − 1 trials, that is, any of them are not in the set S ( g ( x )) .

Original: A simple calculation yields
Revised: A simple calculation yields

Original: [Equation]
Revised: [Equation]

Original: Under the event G , on the other hand,
Revised: Under the event G , on the other hand,

Original: [Equation]
Revised: [Equation]

Original: where the second equality is due to the property of the balancing function h ( u )
Revised: where the second equality is due to the property of the balancing function h ( u )

Original: = uh (1 /u )
Revised: = uh (1 /u )

Original: and the last inequality follows from a similar argument as before using Conditions (i), (ii) and
Revised: and the last inequality follows from a similar argument as before using Conditions (i), (ii) and

Original: [Equation]
Revised: [Equation]

Original: by non-decreasing h .
Revised: by non-decreasing h .

Original: Then for x (cid:54) = x ∗ , by Conditions (ii), (iii) and we have
Revised: Then for x (cid:54) = x ∗ , by Conditions (ii), (iii) and we have

Original: [Equation]
Revised: [Equation]

Original: g
Revised: g

Original: [Equation]
Revised: [Equation]

Original: Combining the lower bounds (11) and (12) leads to P ( x, g ( x )) ≥ C (cid:48)
Revised: Combining the lower bounds (11) and (12) leads to P ( x, g ( x )) ≥ C (cid:48)

Original: Np t 4 , which concludes the proof of the theorem.
Revised: Np t 4 , which concludes the proof of the theorem.

Original: A.4 An example on a different weight function.
Revised: A.4 An example on a different weight function.

Original: The example below shows an undesirable behavior of the MTM algorithm if we use a weight function which is not in the class of (4).
Revised: The example below shows an undesirable behavior of the MTM algorithm if we use a weight function which is not in the class of (4).

Original: Here an undesirable behavior means that the acceptance probability is close to zero even when π ( x prop ) (cid:29) π ( x curr ) , where we denote x curr , x prop as the current state and the proposed state from (2) in Algorithm 1, respectively.
Revised: Here an undesirable behavior means that the acceptance probability is close to zero even when π ( x prop ) (cid:29) π ( x curr ) , where we denote x curr , x prop as the current state and the proposed state from (2) in Algorithm 1, respectively.

Original: Example 1.
Revised: Example 1.

Original: We consider the weight function w ( y | x ) =
Revised: We consider the weight function w ( y | x ) =

Original: π ( y ) .
Revised: π ( y ) .

Original: For the sake of simplicity, let |N ( x ) |
Revised: For the sake of simplicity, let |N ( x ) |

Original: =
Revised: =

Original: p t 3 and for x (cid:48) ∈
Revised: p t 3 and for x (cid:48) ∈

Original: N ( x ) , π ( x (cid:48) )
Revised: N ( x ) , π ( x (cid:48) )

Original: π ( x ) =
Revised: π ( x ) =

Original: (cid:26) p t 2 , for x (cid:48) ∈ S ( x ) , p t 1 / 2 , for x (cid:48) (cid:54)∈ S ( x ) , for all x (cid:54) = x ∗ .
Revised: (cid:26) p t 2 , for x (cid:48) ∈ S ( x ) , p t 1 / 2 , for x (cid:48) (cid:54)∈ S ( x ) , for all x (cid:54) = x ∗ .

Original: Let x j , x (cid:63)j be uniform samples from N ( x curr ) and N ( x prop ) , respectively for j ∈ [ N − 1] .
Revised: Let x j , x (cid:63)j be uniform samples from N ( x curr ) and N ( x prop ) , respectively for j ∈ [ N − 1] .

Original: Assume x prop ∈ S ( x curr ) to reﬂect π ( x prop ) (cid:29) π ( x curr ) .
Revised: Assume x prop ∈ S ( x curr ) to reﬂect π ( x prop ) (cid:29) π ( x curr ) .

Original: Although the conditions on
Revised: Although the conditions on

Original: Theorem 1 are met (except for those related to a balancing function h ), the acceptance probability from x curr to x prop is upper bounded as
Revised: Theorem 1 are met (except for those related to a balancing function h ), the acceptance probability from x curr to x prop is upper bounded as

Original: [Equation]
Revised: [Equation]

Original: Notice that the acceptance probability α ( x curr , x prop ) = 1 in the MH algorithm.
Revised: Notice that the acceptance probability α ( x curr , x prop ) = 1 in the MH algorithm.

Original: A.5 A toy example on path construction.
Revised: A.5 A toy example on path construction.

Original: We provide a simple example on how to construct the canonical path ensemble ∆ ∗ described in A.3.
Revised: We provide a simple example on how to construct the canonical path ensemble ∆ ∗ described in A.3.

Original: Deﬁne X
Revised: Deﬁne X

Original: = { 0 , 1 } 3 as our state space and let x ∗ = (1 , 1 , 0) be the mode of the distribution on X .
Revised: = { 0 , 1 } 3 as our state space and let x ∗ = (1 , 1 , 0) be the mode of the distribution on X .

Original: Let the target distribution π ( x ) ∝ exp( − d H ( x, x ∗ )) , where d H is a Hamming distance.
Revised: Let the target distribution π ( x ) ∝ exp( − d H ( x, x ∗ )) , where d H is a Hamming distance.

Original: We specify a neighborhood of any x ∈ X as N ( x ) =
Revised: We specify a neighborhood of any x ∈ X as N ( x ) =

Original: { y ∈ X : d H ( y, x ) = 1 } , where d H is a Hamming distance.
Revised: { y ∈ X : d H ( y, x ) = 1 } , where d H is a Hamming distance.

Original: In the left panel of Figure 5, neighboring states are linked to black undirected edges.
Revised: In the left panel of Figure 5, neighboring states are linked to black undirected edges.

Original: We follow the rule described in (7) to deﬁne a transition function g : X → X and black directed edges indicate the deﬁned moves by a function g .
Revised: We follow the rule described in (7) to deﬁne a transition function g : X → X and black directed edges indicate the deﬁned moves by a function g .

Original: The height of each bar indicates π ( x ) associated with the corresponding state x .
Revised: The height of each bar indicates π ( x ) associated with the corresponding state x .

Original: In the right panel of Figure 5, we provide three examples to illustrate the three possible cases to construct a path from the deﬁned transition g , described in A.3.
Revised: In the right panel of Figure 5, we provide three examples to illustrate the three possible cases to construct a path from the deﬁned transition g , described in A.3.

Original: • The red directed edges indicate a path from (0 , 0 , 1) to (1 , 0 , 0) , which corresponds to the case (i), since (1 , 0 , 0) =
Revised: • The red directed edges indicate a path from (0 , 0 , 1) to (1 , 0 , 0) , which corresponds to the case (i), since (1 , 0 , 0) =

Original: g 2 ((0 , 0 , 1)) , • The green directed edge indicates a path from (1 , 1 , 0) to (0 , 1 , 0) , which corresponds to the case (ii), since (1 , 1 , 0) =
Revised: g 2 ((0 , 0 , 1)) , • The green directed edge indicates a path from (1 , 1 , 0) to (0 , 1 , 0) , which corresponds to the case (ii), since (1 , 1 , 0) =

Original: g ((0 , 1 , 0))
Revised: g ((0 , 1 , 0))

Original: , • The blue directed edges indicate a path from (0 , 0 , 1) to (1 , 1 , 1) , which corresponds to the case (iii).
Revised: , • The blue directed edges indicate a path from (0 , 0 , 1) to (1 , 1 , 1) , which corresponds to the case (iii).

Original: B Details of simulation studies
Revised: B Details of simulation studies

Original: The scope of this paper is to theoretically study the mixing time for the family of MTM algorithms, and hence we mainly focus on experiments to empirically verify our theoretical insights, that the
Revised: The scope of this paper is to theoretically study the mixing time for the family of MTM algorithms, and hence we mainly focus on experiments to empirically verify our theoretical insights, that the

Original: MTM mixing time is smaller by a factor of the number of trials N and that locally balanced weight functions tend to perform better under suitable assumptions.
Revised: MTM mixing time is smaller by a factor of the number of trials N and that locally balanced weight functions tend to perform better under suitable assumptions.

Original: Nevertheless, in some experiments, we compare the MTM algorithm with the locally balanced MH algorithm
Revised: Nevertheless, in some experiments, we compare the MTM algorithm with the locally balanced MH algorithm

Original: (denoted as LBMH)
Revised: (denoted as LBMH)

Original: [48], as it has been reported to outperform the other state-of-the-art methods.
Revised: [48], as it has been reported to outperform the other state-of-the-art methods.

Original: Before describing the details of simulation studies, here we brieﬂy describe the locally balanced MH algorithm.
Revised: Before describing the details of simulation studies, here we brieﬂy describe the locally balanced MH algorithm.

Original: Speciﬁcation of LBMH requires balancing function h and uninformed symmetric distribution
Revised: Speciﬁcation of LBMH requires balancing function h and uninformed symmetric distribution

Original: K sym ( x, · ) supported on N ( x ) .
Revised: K sym ( x, · ) supported on N ( x ) .

Original: LBMH chooses a proposal state y from a pointwise informed proposal distribution
Revised: LBMH chooses a proposal state y from a pointwise informed proposal distribution

Original: [Equation]
Revised: [Equation]

Original: where Z h ( x ) = (cid:80) z ∈N ( x ) h ( π ( z ) /π ( x ))
Revised: where Z h ( x ) = (cid:80) z ∈N ( x ) h ( π ( z ) /π ( x ))

Original: K sym ( x, z ) is a normalizing constant.
Revised: K sym ( x, z ) is a normalizing constant.

Original: Then, y is accepted with probability α = min { 1 , π ( y ) Q h
Revised: Then, y is accepted with probability α = min { 1 , π ( y ) Q h

Original: ( y,x ) π ( x ) Q h ( x,y ) } =
Revised: ( y,x ) π ( x ) Q h ( x,y ) } =

Original: min { 1 , Z h
Revised: min { 1 , Z h

Original: ( x ) Z h ( y ) } , by deﬁntion of balancing function h ( u )
Revised: ( x ) Z h ( y ) } , by deﬁntion of balancing function h ( u )

Original: =
Revised: =

Original: uh (1 /u ) and symmetry of K sym .
Revised: uh (1 /u ) and symmetry of K sym .

Original: Unlike MTM where a subset of N ( x ) is selected (with replacement) as a trial and choose a proposal among them, LBMH needs to evaluate h ( π ( y ) /π
Revised: Unlike MTM where a subset of N ( x ) is selected (with replacement) as a trial and choose a proposal among them, LBMH needs to evaluate h ( π ( y ) /π

Original: ( x )) for all y ∈ N ( x ) to get a proposal state y which can be viewed as an exhaustive search of N ( x ) .
Revised: ( x )) for all y ∈ N ( x ) to get a proposal state y which can be viewed as an exhaustive search of N ( x ) .

Original: In terms of computation, MTM requires calculating 2 N − 1 weight functions at each iteration where N can be chosen at one’s disposal, LBMH requires calculating |N
Revised: In terms of computation, MTM requires calculating 2 N − 1 weight functions at each iteration where N can be chosen at one’s disposal, LBMH requires calculating |N

Original: ( y )
Revised: ( y )

Original: | number of ratios to calculate Z h ( y ) at each iteration, where proposal probabilities { Q h ( x, y ) : y ∈ N ( x ) } and normalizing
Revised: | number of ratios to calculate Z h ( y ) at each iteration, where proposal probabilities { Q h ( x, y ) : y ∈ N ( x ) } and normalizing

Original: constant
Revised: constant

Original: Z h ( x ) can be saved and reused from the previous iteration.
Revised: Z h ( x ) can be saved and reused from the previous iteration.

Original: Since random walk proposals in BVS and SBM examples are both symmetric, we compare MTM with LBMH by letting K sym
Revised: Since random walk proposals in BVS and SBM examples are both symmetric, we compare MTM with LBMH by letting K sym

Original: = K RW with three different balancing functions: h ( u ) = √ u , h ( u ) =
Revised: = K RW with three different balancing functions: h ( u ) = √ u , h ( u ) =

Original: min { 1 , u } and h ( u ) = max { 1 , u } (corresponding to w sqrt , w min , w max respectively).
Revised: min { 1 , u } and h ( u ) = max { 1 , u } (corresponding to w sqrt , w min , w max respectively).

Original: B.1 Details of Bayesian variable selection (BVS)
Revised: B.1 Details of Bayesian variable selection (BVS)

Original: After marginalizing out β and φ , the posterior distribution π ( γ | y ) is written as [47, §A.1]
Revised: After marginalizing out β and φ , the posterior distribution π ( γ | y ) is written as [47, §A.1]

Original: [Equation]
Revised: [Equation]

Original: (14) where SSR( γ ) =
Revised: (14) where SSR( γ ) =

Original: y (cid:62) (cid:16) I n − GG +1 X γ (cid:0) X (cid:62) γ X γ (cid:1) − 1 X (cid:62) γ (cid:17)
Revised: y (cid:62) (cid:16) I n − GG +1 X γ (cid:0) X (cid:62) γ X γ (cid:1) − 1 X (cid:62) γ (cid:17)

Original: y is a term having a similar role as a sum of squared residuals and C is a normalizing constant.
Revised: y is a term having a similar role as a sum of squared residuals and C is a normalizing constant.

Original: MCMC setup.
Revised: MCMC setup.

Original: Hyperparameters are speciﬁed as G
Revised: Hyperparameters are speciﬁed as G

Original: = p 3 = 5000 3 , κ = 2 , and s max = 100 .
Revised: = p 3 = 5000 3 , κ = 2 , and s max = 100 .

Original: For each dataset, we run a chain of 10 5 iteration for single-try MH, 2 × 10 4 iteration for MTM with N = 5 , and 10 4 iteration for MTM with N = 5 , 10 , 50 , 100 , 500 , 1000 , 2000 , 5000 using four different weight functions.
Revised: For each dataset, we run a chain of 10 5 iteration for single-try MH, 2 × 10 4 iteration for MTM with N = 5 , and 10 4 iteration for MTM with N = 5 , 10 , 50 , 100 , 500 , 1000 , 2000 , 5000 using four different weight functions.

Original: Algorithms are randomly initialized with state γ 0 such that γ 0 ∩ γ ∗ = ∅ and d H ( γ 0 , γ ∗ )
Revised: Algorithms are randomly initialized with state γ 0 such that γ 0 ∩ γ ∗ = ∅ and d H ( γ 0 , γ ∗ )

Original: = 20 which implies H
Revised: = 20 which implies H

Original: = 20 is the minimum required hitting iteration.
Revised: = 20 is the minimum required hitting iteration.

Original: For each simulated dataset, the true data generated model achieves the highest posterior probability ( γ ∗ = x ∗ ).
Revised: For each simulated dataset, the true data generated model achieves the highest posterior probability ( γ ∗ = x ∗ ).

Original: All simulation studies are performed on a Linux cluster with Intel(R)
Revised: All simulation studies are performed on a Linux cluster with Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Gold 6132 CPU @ 2.60GHz and 96GB memory.
Revised: Gold 6132 CPU @ 2.60GHz and 96GB memory.

Original: Results from Table 2 show that H decreases roughly by a factor of N until N = 100 , which conﬁrms our theoretical ﬁndings, given that the model setting satisﬁes that the mixing time is equivalent to the hitting iteration up to constant factors [36].
Revised: Results from Table 2 show that H decreases roughly by a factor of N until N = 100 , which conﬁrms our theoretical ﬁndings, given that the model setting satisﬁes that the mixing time is equivalent to the hitting iteration up to constant factors [36].

Original: When N becomes larger, the performance of unscaled weight function w ord deteriorates and never converges when N = 5000 .
Revised: When N becomes larger, the performance of unscaled weight function w ord deteriorates and never converges when N = 5000 .

Original: In contrast, locally balanced weight functions generally perform well even when N is large.
Revised: In contrast, locally balanced weight functions generally perform well even when N is large.

Original: Table 3 suggests that choosing moderate N is beneﬁcial in terms of computational savings.
Revised: Table 3 suggests that choosing moderate N is beneﬁcial in terms of computational savings.

Original: When the design matrix is correlated and
Revised: When the design matrix is correlated and

Original: SNR = 2 , the result suggests that the chain often stuck when we choose the weight function as w max .
Revised: SNR = 2 , the result suggests that the chain often stuck when we choose the weight function as w max .

Original: Since the shape of the posterior distribution becomes irregular when the SNR is intermediate [47] and design matrix is correlated, to get a more clear insight we further perform additional simulation study when the posterior distribution exhibits multimodality; see Appendix B.4.
Revised: Since the shape of the posterior distribution becomes irregular when the SNR is intermediate [47] and design matrix is correlated, to get a more clear insight we further perform additional simulation study when the posterior distribution exhibits multimodality; see Appendix B.4.

Original: Finally, under different settings of the design and SNR, the median N estimated from Algorithm 2 using ψ = 0 .
Revised: Finally, under different settings of the design and SNR, the median N estimated from Algorithm 2 using ψ = 0 .

Original: 9 overreplicate datasets is ˆ N = 349 (indep, SNR=4), 501 (indep, SNR=2), 328 (dep, SNR=4) and 158 (dep,
Revised: 9 overreplicate datasets is ˆ N = 349 (indep, SNR=4), 501 (indep, SNR=2), 328 (dep, SNR=4) and 158 (dep,

Original: SNR=2).
Revised: SNR=2).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: In contrast to MTM, LBMH fails to converge to γ ∗ when h ( u ) = √ u or h ( u ) = max { 1 , u } .
Revised: In contrast to MTM, LBMH fails to converge to γ ∗ when h ( u ) = √ u or h ( u ) = max { 1 , u } .

Original: It is easier for MTM to escape from such local modes by randomly searching part of its neighborhood to select the proposal.
Revised: It is easier for MTM to escape from such local modes by randomly searching part of its neighborhood to select the proposal.

Original: The exhaustive search nature of LBMH makes it difﬁcult to escape from the local mode since some high values of π ( y ∗ ) /π
Revised: The exhaustive search nature of LBMH makes it difﬁcult to escape from the local mode since some high values of π ( y ∗ ) /π

Original: ( y ) , y ∗ ∈ N ( y ) involved in the denominator makes the acceptance ratio small.
Revised: ( y ) , y ∗ ∈ N ( y ) involved in the denominator makes the acceptance ratio small.

Original: This phenomenon disappears when h ( u ) =
Revised: This phenomenon disappears when h ( u ) =

Original: min { 1 , u } is used.
Revised: min { 1 , u } is used.

Original: In terms of wall-clock hitting time T H , LBMH is not as efﬁcient as MTM with a smaller choice of N .
Revised: In terms of wall-clock hitting time T H , LBMH is not as efﬁcient as MTM with a smaller choice of N .

Original: In addition, we also consider the case when SNR = 0 .
Revised: In addition, we also consider the case when SNR = 0 .

Original: 5 (very weak SNR) so that the null model γ ∗ = 0 receives the highest probability across all simulated datasets.
Revised: 5 (very weak SNR) so that the null model γ ∗ = 0 receives the highest probability across all simulated datasets.

Original: For each replicated dataset, algorithms are randomly initialized with state γ 0 such that d H ( γ 0 , γ ∗ ) = 10 which implies H =is the minimum required hitting iteration.
Revised: For each replicated dataset, algorithms are randomly initialized with state γ 0 such that d H ( γ 0 , γ ∗ ) = 10 which implies H =is the minimum required hitting iteration.

Original: Table 4 provides the result similar to Tables 2 and 3, since the posterior distribution is unimodal with the peak at the null model γ ∗ = 

Revised: Table 4 provides the result similar to Tables 2 and 3, since the posterior distribution is unimodal with the peak at the null model γ ∗ =

Original: due to the sparsity prior.
Revised: due to the sparsity prior.

Original: The median N estimated from Algorithm 2 using ψ = 0 .
Revised: The median N estimated from Algorithm 2 using ψ = 0 .

Original: 9 over 50 replicate datasets is ˆ N = 171 for independent design and is ˆ N = 212 for dependent design.
Revised: 9 over 50 replicate datasets is ˆ N = 171 for independent design and is ˆ N = 212 for dependent design.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: B.2 Details of stochastic block model (SBM)
Revised: B.2 Details of stochastic block model (SBM)

Original: After marginalizing out { Q uv } 1 ≤ u ≤ v ≤ K , the posterior distribution π ( z | A ) is written as (see [24, §2.1] and [55, §2.2])
Revised: After marginalizing out { Q uv } 1 ≤ u ≤ v ≤ K , the posterior distribution π ( z | A ) is written as (see [24, §2.1] and [55, §2.2])

Original: [Equation]
Revised: [Equation]

Original: where B ( κ 1 , κ 2 )
Revised: where B ( κ 1 , κ 2 )

Original: =
Revised: =

Original: Γ( κ 1 )Γ( κ 2 ) /
Revised: Γ( κ 1 )Γ( κ 2 ) /

Original: Γ( κ 1 + κ 2 ) is a beta function, C is a normalizing constant,
Revised: Γ( κ 1 + κ 2 ) is a beta function, C is a normalizing constant,

Original: [Equation]
Revised: [Equation]

Original: is the number of edges between blocks u and v , and using the notation n u ( z )
Revised: is the number of edges between blocks u and v , and using the notation n u ( z )

Original: =
Revised: =

Original: (cid:80) i 

Revised: (cid:80) i

Original: ( z i = u ) ,
Revised: ( z i = u ) ,

Original: [Equation]
Revised: [Equation]

Original: if
Revised: if

Original: [Equation]
Revised: [Equation]

Original: if u = v, is the number of non-edges between blocks u and v .
Revised: if u = v, is the number of non-edges between blocks u and v .

Original: We note that π ( z | A ) is invariant of a label permutation.
Revised: We note that π ( z | A ) is invariant of a label permutation.

Original: Data generation.
Revised: Data generation.

Original: When K = 2 , there are two true clusters (blocks) of nodes each with 500 nodes.
Revised: When K = 2 , there are two true clusters (blocks) of nodes each with 500 nodes.

Original: When K = 5 , there are ﬁve true clusters of nodes each with 200 nodes.
Revised: When K = 5 , there are ﬁve true clusters of nodes each with 200 nodes.

Original: We generated a graph from the homogeneous SBM and where within- and cross-community edge connection probabilities are a and b respectively.
Revised: We generated a graph from the homogeneous SBM and where within- and cross-community edge connection probabilities are a and b respectively.

Original: Speciﬁcally, for K = 2 we set ( a, b ) =
Revised: Speciﬁcally, for K = 2 we set ( a, b ) =

Original: (0 . 222 , 0 . 01) and ( a, b ) =
Revised: (0 . 222 , 0 . 01) and ( a, b ) =

Original: so that CH ≈ 10 and CH ≈ 2 , and for K = 5 we set ( a, b ) = (0 . 473 , 0 . 01) and ( a, b ) = (0 . 13 , 0 . 01)
Revised: so that CH ≈ 10 and CH ≈ 2 , and for K = 5 we set ( a, b ) = (0 . 473 , 0 . 01) and ( a, b ) = (0 . 13 , 0 . 01)

Original: so that CH ≈ 10 and CH ≈ 2 .
Revised: so that CH ≈ 10 and CH ≈ 2 .

Original: For each setting, we simulate 50 datasets.
Revised: For each setting, we simulate 50 datasets.

Original: MCMC setup.
Revised: MCMC setup.

Original: Hyperparameters are speciﬁed as κ 1 = κ 2 = 1 , and α = 1000 so that the size of the feasible set S α is maximized.
Revised: Hyperparameters are speciﬁed as κ 1 = κ 2 = 1 , and α = 1000 so that the size of the feasible set S α is maximized.

Original: For each dataset, we run a chain of 10 5 iteration for singletry MH, 5 × 10 4 iteration for MTM with N = 5 , and 2 × 10 4 iteration for MTM with N = 5 , 10 , 50 , 100 , 500 , 1000 , 2000 , 5000 using four different weight functions.
Revised: For each dataset, we run a chain of 10 5 iteration for singletry MH, 5 × 10 4 iteration for MTM with N = 5 , and 2 × 10 4 iteration for MTM with N = 5 , 10 , 50 , 100 , 500 , 1000 , 2000 , 5000 using four different weight functions.

Original: Algorithms are randomly initialized with state z 0 such that ˜ d H ( z 0 , z ∗ ) = 400 which implies H = 400 is the minimum required hitting iteration.
Revised: Algorithms are randomly initialized with state z 0 such that ˜ d H ( z 0 , z ∗ ) = 400 which implies H = 400 is the minimum required hitting iteration.

Original: For each simulated dataset, the true data generated model achieves the highest posterior probability ( z ∗ = x ∗ ).
Revised: For each simulated dataset, the true data generated model achieves the highest posterior probability ( z ∗ = x ∗ ).

Original: All simulations are performed on a Linux cluster with Intel(R)
Revised: All simulations are performed on a Linux cluster with Intel(R)

Original: Xeon(R) Gold 6132 CPU @ 2.60GHz and 96GB memory.
Revised: Xeon(R) Gold 6132 CPU @ 2.60GHz and 96GB memory.

Original: Results from Table 5 show that H decreases roughly by a factor of N until N = 10 for locally balanced weight functions, but not for unscaled weight function w ord .
Revised: Results from Table 5 show that H decreases roughly by a factor of N until N = 10 for locally balanced weight functions, but not for unscaled weight function w ord .

Original: Even when N ≥ 50 , MTM with w ord never converges to the highest probability model, highlighting the necessity of the use of locally balanced weight function for the general model selection problems.
Revised: Even when N ≥ 50 , MTM with w ord never converges to the highest probability model, highlighting the necessity of the use of locally balanced weight function for the general model selection problems.

Original: When N is very large, the performance of locally balanced weight functions generally deteriorates, which matches with our theoretical ﬁndings regarding the rate condition on N .
Revised: When N is very large, the performance of locally balanced weight functions generally deteriorates, which matches with our theoretical ﬁndings regarding the rate condition on N .

Original: Table 6 also suggests that a moderate choice of N (in SBM case, around 10) is beneﬁcial in terms of computation savings.
Revised: Table 6 also suggests that a moderate choice of N (in SBM case, around 10) is beneﬁcial in terms of computation savings.

Original: Finally, under different settings of ( K , CH), the median N estimated from Algorithm 2 using ψ = 0 .
Revised: Finally, under different settings of ( K , CH), the median N estimated from Algorithm 2 using ψ = 0 .

Original: 9 over 50 replicate datasets is ˆ N = 15 for ( K , CH) =
Revised: 9 over 50 replicate datasets is ˆ N = 15 for ( K , CH) =

Original: (2, 2), ˆ N = 8 for (2, 10), ˆ N = 5 for (5, 2) and ˆ N = 4 for (5, 10).
Revised: (2, 2), ˆ N = 8 for (2, 10), ˆ N = 5 for (5, 2) and ˆ N = 4 for (5, 10).

Original: Comparison of hitting iteration H with LBMH gives an insight similar to the BVS example.
Revised: Comparison of hitting iteration H with LBMH gives an insight similar to the BVS example.

Original: When
Revised: When

Original: K = 5 , LBMH often gets stuck at a local mode and never converges to z ∗ .
Revised: K = 5 , LBMH often gets stuck at a local mode and never converges to z ∗ .

Original: However when K = 2 ,
Revised: However when K = 2 ,

Original: LBMH performs similarly to MTM with larger choices of N .
Revised: LBMH performs similarly to MTM with larger choices of N .

Original: We note that when K = 2 , the shape of posterior distribution can be signiﬁcantly different from that of K = 5 , as the minimax rate and posterior contraction rate analysis are often treated separately when K = 2 and K ≥ 3 [51, 55].
Revised: We note that when K = 2 , the shape of posterior distribution can be signiﬁcantly different from that of K = 5 , as the minimax rate and posterior contraction rate analysis are often treated separately when K = 2 and K ≥ 3 [51, 55].

Original: The comparison of wall-clock hitting time T H also suggests MTM with moderate choice of N is much more efﬁcient.
Revised: The comparison of wall-clock hitting time T H also suggests MTM with moderate choice of N is much more efﬁcient.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Fail w max 2.
Revised: Fail w max 2.

Original: Fail
Revised: Fail

Original: [Table]
Revised: [Table]

Original: Fail Fail
Revised: Fail Fail

Original: Fail Fail
Revised: Fail Fail

Original: Fail Fail
Revised: Fail Fail

Original: N/A
Revised: N/A

Original: [Table]
Revised: [Table]

Original: Fail
Revised: Fail

Original: [Table]
Revised: [Table]

Original: min 1.
Revised: min 1.

Original: 6.52 w max 1.
Revised: 6.52 w max 1.

Original: Fail
Revised: Fail

Original: B.3 Spatial clustering model (SCM)
Revised: B.3 Spatial clustering model (SCM)

Original: We consider a spatial clustering problem for a given set of spatial locations S
Revised: We consider a spatial clustering problem for a given set of spatial locations S

Original: = { s 1 , . . .
Revised: = { s 1 , . . .

Original: , s p } ⊂
Revised: , s p } ⊂

Original: Rwhere the responses ζ ( s i ) ’s are observed.
Revised: Rwhere the responses ζ ( s i ) ’s are observed.

Original: The goal of an SCM is to identify a spatially contiguous partition on S , denoted by P = {S 1 , . . .
Revised: The goal of an SCM is to identify a spatially contiguous partition on S , denoted by P = {S 1 , . . .

Original: , S K } , where S j ’s are disjoint subsets of S whose union is S , such that the responses within a cluster { ζ ( s ) : s ∈ S j } are identically distributed and have different means across clusters.
Revised: , S K } , where S j ’s are disjoint subsets of S whose union is S , such that the responses within a cluster { ζ ( s ) : s ∈ S j } are identically distributed and have different means across clusters.

Original: We follow [28] to adopt a probabilistic model for P that utilizes a spanning tree graph T on S (with p vertices and p − 1 edges) as a “spatial order” of S .
Revised: We follow [28] to adopt a probabilistic model for P that utilizes a spanning tree graph T on S (with p vertices and p − 1 edges) as a “spatial order” of S .

Original: The spanning tree T is chosen in a way that two locations connected by an edge are spatially proximate to each other.
Revised: The spanning tree T is chosen in a way that two locations connected by an edge are spatially proximate to each other.

Original: A partition P with K clusters can be deﬁned by removing K − 1 edges from T .
Revised: A partition P with K clusters can be deﬁned by removing K − 1 edges from T .

Original: Speciﬁcally, the SCM we consider can be written as
Revised: Speciﬁcally, the SCM we consider can be written as

Original: [Equation]
Revised: [Equation]

Original: with µ ( s i ) =
Revised: with µ ( s i ) =

Original: µ j if s i ∈ S j , for i ∈
Revised: µ j if s i ∈ S j , for i ∈

Original: [ p ] , (16)
Revised: [ p ] , (16)

Original: [Equation]
Revised: [Equation]

Original: for j ∈ [ K ] ,
Revised: for j ∈ [ K ] ,

Original: [Equation]
Revised: [Equation]

Original: InvGamma
Revised: InvGamma

Original: [Equation]
Revised: [Equation]

Original: π
Revised: π

Original: [Equation]
Revised: [Equation]

Original: P|
Revised: P|

Original: [Equation]
Revised: [Equation]

Original: ∝ 1 {P can be obtained by removing K − 1 edges from T } ,
Revised: ∝ 1 {P can be obtained by removing K − 1 edges from T } ,

Original: [Equation]
Revised: [Equation]

Original: where a 0 > 0 , b 0 > 0 , 0 ≤ c 0 < 1 , and λ > 0 are hyperparameters.
Revised: where a 0 > 0 , b 0 > 0 , 0 ≤ c 0 < 1 , and λ > 0 are hyperparameters.

Original: See Figure 6a for an example of partition P obtained by cutting edges from the spanning tree T .
Revised: See Figure 6a for an example of partition P obtained by cutting edges from the spanning tree T .

Original: Thanks to the conjugate priors, µ j ’s and σ 2 can be analytically marginalized out, and hence the inference problem boils down to drawing samples from the (discrete) posterior distribution π ( P | data) .
Revised: Thanks to the conjugate priors, µ j ’s and σ 2 can be analytically marginalized out, and hence the inference problem boils down to drawing samples from the (discrete) posterior distribution π ( P | data) .

Original: Although [28] considered random spanning trees by assigning a prior distribution on T , we stress that the main focus of this paper is the mixing time analysis on the posterior distribution of P .
Revised: Although [28] considered random spanning trees by assigning a prior distribution on T , we stress that the main focus of this paper is the mixing time analysis on the posterior distribution of P .

Original: Thus, following [26], we ﬁx T as the Euclidean minimum spanning tree in a Delaunay triangulation graph on S , otherwise we can only sample from the conditional distribution π
Revised: Thus, following [26], we ﬁx T as the Euclidean minimum spanning tree in a Delaunay triangulation graph on S , otherwise we can only sample from the conditional distribution π

Original: ( P | T , data) which complicates the mixing time analysis of our target distribution π ( P | data) .
Revised: ( P | T , data) which complicates the mixing time analysis of our target distribution π ( P | data) .

Original: We consider the following proposal:
Revised: We consider the following proposal:

Original: [Equation]
Revised: [Equation]

Original: where N b ( P ) is the set of all possible partitions obtained by splitting a cluster in P into two clusters by selecting a cut-edge of T and N d ( P ) is the set of all possible partitions obtained by merging two neighboring (with respect to T ) clusters in P .
Revised: where N b ( P ) is the set of all possible partitions obtained by splitting a cluster in P into two clusters by selecting a cut-edge of T and N d ( P ) is the set of all possible partitions obtained by merging two neighboring (with respect to T ) clusters in P .

Original: See [23, 28] for detailed discussion on how to perform an appropriate split or merge on P given a spanning tree T .
Revised: See [23, 28] for detailed discussion on how to perform an appropriate split or merge on P given a spanning tree T .

Original: Data generation .
Revised: Data generation .

Original: We generate p = 1000 uniform locations s i iid ∼ Unif ([0 , 1] 2 ) , i ∈
Revised: We generate p = 1000 uniform locations s i iid ∼ Unif ([0 , 1] 2 ) , i ∈

Original: [ p ] and specify the true means { µ ( s i ) } as in Figure 6a.
Revised: [ p ] and specify the true means { µ ( s i ) } as in Figure 6a.

Original: Responses are generated according to (16) with σ = (cid:112)
Revised: Responses are generated according to (16) with σ = (cid:112)

Original: Var ( µ ( s )) /
Revised: Var ( µ ( s )) /

Original: SNR and we simulate 50 replicate datasets under SNR ∈ { 3 , 10 } respectively.
Revised: SNR and we simulate 50 replicate datasets under SNR ∈ { 3 , 10 } respectively.

Original: MCMC setup .
Revised: MCMC setup .

Original: Following [28], we initialize the chain using the estimates from the spatially clustered coefﬁcient model of [26].
Revised: Following [28], we initialize the chain using the estimates from the spatially clustered coefﬁcient model of [26].

Original: However, this initialization does not guarantee the same minimum number of iterations required to hit the true partition P true for different replicate datasets.
Revised: However, this initialization does not guarantee the same minimum number of iterations required to hit the true partition P true for different replicate datasets.

Original: For a fair comparison, throughout this subsection, we redeﬁne H as the number of extra iterations until hit, which is the iterations until hit minus the minimum number of iterations required to reach P true .
Revised: For a fair comparison, throughout this subsection, we redeﬁne H as the number of extra iterations until hit, which is the iterations until hit minus the minimum number of iterations required to reach P true .

Original: Hyperparameters are speciﬁed as a 0 = b 0 = 1 , c 0 = 0 .
Revised: Hyperparameters are speciﬁed as a 0 = b 0 = 1 , c 0 = 0 .

Original: 5 , and λ = 0 .
Revised: 5 , and λ = 0 .

Original: We consider the number of trials N ∈ { 5 , 10 , 100 , 500 , 1000 } .
Revised: We consider the number of trials N ∈ { 5 , 10 , 100 , 500 , 1000 } .

Original: For each replicate dataset, we run a chain of 10 , 000 iterations for each MTM speciﬁcation and a chain of 30 , 000 for standard single-try MH.
Revised: For each replicate dataset, we run a chain of 10 , 000 iterations for each MTM speciﬁcation and a chain of 30 , 000 for standard single-try MH.

Original: All simulation studies are performed on a Linux cluster with Intel(R)
Revised: All simulation studies are performed on a Linux cluster with Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Gold 6132 CPU @ 2.60GHz and 96GB memory.
Revised: Gold 6132 CPU @ 2.60GHz and 96GB memory.

Original: Table 7 summarizes H and T H of various weight functions and numbers of trials N .
Revised: Table 7 summarizes H and T H of various weight functions and numbers of trials N .

Original: The distributions of H for the setting of SNR = 10 are provided in Figure 6b.
Revised: The distributions of H for the setting of SNR = 10 are provided in Figure 6b.

Original: The results from single-try MH are also included as a baseline.
Revised: The results from single-try MH are also included as a baseline.

Original: When SNR = 10, the proposed locally balanced weight functions, especially w sqrt and w min , considerably outperform the ordinary weight function w ord and the single-try MH, in the sense that the proposed ones can reach P true by much fewer iterations when
Revised: When SNR = 10, the proposed locally balanced weight functions, especially w sqrt and w min , considerably outperform the ordinary weight function w ord and the single-try MH, in the sense that the proposed ones can reach P true by much fewer iterations when

Original: N ∈ { 100 , 500 , 1000 } .
Revised: N ∈ { 100 , 500 , 1000 } .

Original: In contrast, the performance of w ord deteriorates when N ≥ 

Revised: In contrast, the performance of w ord deteriorates when N ≥

Original: and it fails to reach P true when N = 500 or 1000 .
Revised: and it fails to reach P true when N = 500 or 1000 .

Original: For the proposed weight functions, the wall-clock time until hit T H is minimized when N = 100 , since the beneﬁt of having fewer iterations until hit is offset by the computational cost of extra trials when N is large.
Revised: For the proposed weight functions, the wall-clock time until hit T H is minimized when N = 100 , since the beneﬁt of having fewer iterations until hit is offset by the computational cost of extra trials when N is large.

Original: When SNR = 3, the chains never visit the true partition, possibly because P true does not lead to the highest posterior probability.
Revised: When SNR = 3, the chains never visit the true partition, possibly because P true does not lead to the highest posterior probability.

Original: In this case, we redeﬁne H and T H to be the number of extra iterations and the wall-clock time, respectively, to reach the 0.99 Rand index neighborhood of P true , deﬁned as
Revised: In this case, we redeﬁne H and T H to be the number of extra iterations and the wall-clock time, respectively, to reach the 0.99 Rand index neighborhood of P true , deﬁned as

Original: N Rand ( P true ) :
Revised: N Rand ( P true ) :

Original: = {P : Rand( P , P true ) ≥ 0 . 99 } , where Rand( · , · ) is the Rand index [39] measuring the proportion of agreements between two partitions.
Revised: = {P : Rand( P , P true ) ≥ 0 . 99 } , where Rand( · , · ) is the Rand index [39] measuring the proportion of agreements between two partitions.

Original: The ﬁndings on H and T H for reaching N Rand ( P true ) are similar to the ones when SNR = 10.
Revised: The ﬁndings on H and T H for reaching N Rand ( P true ) are similar to the ones when SNR = 10.

Original: Finally, the median N estimated from Algorithm 2 using ψ = 0 .
Revised: Finally, the median N estimated from Algorithm 2 using ψ = 0 .

Original: 9 over 50 replicate datasets is ˆ
Revised: 9 over 50 replicate datasets is ˆ

Original: N = 13 when SNR = 10 and ˆ N = 21 when SNR = 3.
Revised: N = 13 when SNR = 10 and ˆ N = 21 when SNR = 3.

Original: (a) l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l
Revised: (a) l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: s v −1.
Revised: s v −1.

Original: [Figure]
Revised: [Figure]

Original: lllllll llllll lllll ll l l ll l l lll ll l ll ll l l ll l ll l l l l l ll l l ll llll lll l
Revised: lllllll llllll lllll ll l l ll l l lll ll l ll ll l l ll l ll l l l l l ll l l ll llll lll l

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Fail Fail
Revised: Fail Fail

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Fail
Revised: Fail

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: B.4 MTM algorithm on multimodal target distributions
Revised: B.4 MTM algorithm on multimodal target distributions

Original: In this section, we analyze the performance of the MTM algorithm with different choices of weight functions and N on the multimodal target distribution.
Revised: In this section, we analyze the performance of the MTM algorithm with different choices of weight functions and N on the multimodal target distribution.

Original: Following [54], we generate a multimodal dataset in the context of a Bayesian variable selection problem.
Revised: Following [54], we generate a multimodal dataset in the context of a Bayesian variable selection problem.

Original: Data generation .
Revised: Data generation .

Original: We let sample size n = 1000 and number of variables p
Revised: We let sample size n = 1000 and number of variables p

Original: =
Revised: =

Original: Each row of design matrix is independently sampled from x i iid ∼ N (0 , Σ) for i = 1 , . . .
Revised: Each row of design matrix is independently sampled from x i iid ∼ N (0 , Σ) for i = 1 , . . .

Original: , n where Σ = diag (Σ 20 , . .
Revised: , n where Σ = diag (Σ 20 , . .

Original: . , Σ 20 ) is block-diagonal.
Revised: . , Σ 20 ) is block-diagonal.

Original: Each block Σ 20 has dimension 20 × 20 , and (Σ 20 ) jk =
Revised: Each block Σ 20 has dimension 20 × 20 , and (Σ 20 ) jk =

Original: exp( −| j − k | / 3) .
Revised: exp( −| j − k | / 3) .

Original: We generate true coefﬁcient β true by ﬁrst sampling 100 indices j 1 , . .
Revised: We generate true coefﬁcient β true by ﬁrst sampling 100 indices j 1 , . .

Original: , juniformly at random (without replacement) from [ p ] and let β j (cid:96) iid ∼ N (0 , σ 2 β ) for (cid:96) = 1 , . .
Revised: , juniformly at random (without replacement) from [ p ] and let β j (cid:96) iid ∼ N (0 , σ 2 β ) for (cid:96) = 1 , . .

Original: 100 , and β k = 0 if k (cid:54)∈ { j 1 , . .
Revised: 100 , and β k = 0 if k (cid:54)∈ { j 1 , . .

Original: . , j 100 } .
Revised: . , j 100 } .

Original: Then the response vector y is generated from y ∼ N
Revised: Then the response vector y is generated from y ∼ N

Original: ( Xβ true , I n ) .
Revised: ( Xβ true , I n ) .

Original: We consider three settings of σ β = 0 .
Revised: We consider three settings of σ β = 0 .

Original: 5 to simulate the coefﬁcients and data.
Revised: 5 to simulate the coefﬁcients and data.

Original: For each setting, we simulate 20 datasets.
Revised: For each setting, we simulate 20 datasets.

Original: MCMC setup .
Revised: MCMC setup .

Original: We use the same BVS model described in Section 4.
Revised: We use the same BVS model described in Section 4.

Original: Hyperparameters are speciﬁed as G
Revised: Hyperparameters are speciﬁed as G

Original: = p = 5000 , κ = 1 , and s
Revised: = p = 5000 , κ = 1 , and s

Original: max = 100 .
Revised: max = 100 .

Original: For each dataset, we run a chain of 10,000 iteration for
Revised: For each dataset, we run a chain of 10,000 iteration for

Original: MTM with N = 50 , 100 , 500 , 1000 , 2000 , 5000 using four different weight functions.
Revised: MTM with N = 50 , 100 , 500 , 1000 , 2000 , 5000 using four different weight functions.

Original: The ﬁrstiterations are discarded since the behavior of the chain (e.g. acceptance ratio) during the burn-in stage may be different from the behavior of the chain which entered stationarity; see also Figure 7. for trace plots.
Revised: The ﬁrstiterations are discarded since the behavior of the chain (e.g. acceptance ratio) during the burn-in stage may be different from the behavior of the chain which entered stationarity; see also Figure 7. for trace plots.

Original: Algorithms are all initialized with null model γ 0 = 0 .
Revised: Algorithms are all initialized with null model γ 0 = 0 .

Original: All simulation studies are performed on a Linux cluster with Intel(R)
Revised: All simulation studies are performed on a Linux cluster with Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Gold 6132 CPU @ 2.60GHz and 96GB memory.
Revised: Gold 6132 CPU @ 2.60GHz and 96GB memory.

Original: Since the target distribution is no longer unimodal, hitting iteration H and wall-clock hitting time
Revised: Since the target distribution is no longer unimodal, hitting iteration H and wall-clock hitting time

Original: T H are not appropriate metrics to compare mixing performance.
Revised: T H are not appropriate metrics to compare mixing performance.

Original: Instead, we use three different metrics to evaluate the quality of the mixing: 1) acceptance ratio, 2) the number of unique states visited by the chain, denoted by #(unique γ ), and 3) ESS/Time, where ESS is the effective sample size calculated from the hamming distances d H (ˆ γ max , γ t ) , t = 2001 , . . .
Revised: Instead, we use three different metrics to evaluate the quality of the mixing: 1) acceptance ratio, 2) the number of unique states visited by the chain, denoted by #(unique γ ), and 3) ESS/Time, where ESS is the effective sample size calculated from the hamming distances d H (ˆ γ max , γ t ) , t = 2001 , . . .

Original: , 10000 from the maximum posterior state ˆ γ max found in a chain, Time is wall-clock time usage, measured in seconds.
Revised: , 10000 from the maximum posterior state ˆ γ max found in a chain, Time is wall-clock time usage, measured in seconds.

Original: The results are summarized in Table 8.
Revised: The results are summarized in Table 8.

Original: Figure 7 shows that as N increases, chains generally move faster towards the high posterior states which suggests that discarding the ﬁrst 2000 samples is reasonable.
Revised: Figure 7 shows that as N increases, chains generally move faster towards the high posterior states which suggests that discarding the ﬁrst 2000 samples is reasonable.

Original: One exception is when N = 5000 ,
Revised: One exception is when N = 5000 ,

Original: [Table]
Revised: [Table]

Original: the chain using ordinary weight function barely moves to another state, whereas the chain using locally balanced weight functions exhibits better mixing properties than N = 50 and N = 500 .
Revised: the chain using ordinary weight function barely moves to another state, whereas the chain using locally balanced weight functions exhibits better mixing properties than N = 50 and N = 500 .

Original: Finally, Table 8 summarizes the multimodal simulation results.
Revised: Finally, Table 8 summarizes the multimodal simulation results.

Original: Similar to the previous unimodal results, the performance of w ord deteriorates as N being large, especially when σ β is large.
Revised: Similar to the previous unimodal results, the performance of w ord deteriorates as N being large, especially when σ β is large.

Original: When N is moderate, the weighting functions w ord and w min has a better mixing property than w sqrt , while w max being the worst.
Revised: When N is moderate, the weighting functions w ord and w min has a better mixing property than w sqrt , while w max being the worst.

Original: The inferior performance of w max is also observed at Table 2.
Revised: The inferior performance of w max is also observed at Table 2.

Original: If the current state x is one of the local modes, we may have max { 1 , π ( x (cid:63) ) /π
Revised: If the current state x is one of the local modes, we may have max { 1 , π ( x (cid:63) ) /π

Original: ( y ) } >> max { 1 , π ( y ) /π
Revised: ( y ) } >> max { 1 , π ( y ) /π

Original: ( x ) } where y is the proposed state, and x (cid:63) is one of the trials of y in Step 3 of Algorithm 1, which makes the acceptance probability very small so that the chain is stuck at x when using w max .
Revised: ( x ) } where y is the proposed state, and x (cid:63) is one of the trials of y in Step 3 of Algorithm 1, which makes the acceptance probability very small so that the chain is stuck at x when using w max .

Original: On the other hand, the magnitude of the difference between h ( π ( x (cid:63) ) /π
Revised: On the other hand, the magnitude of the difference between h ( π ( x (cid:63) ) /π

Original: ( y )) and h ( π ( y ) /π
Revised: ( y )) and h ( π ( y ) /π

Original: ( x )) will be reduced if we use w sqrt , w min so that the chain can traverse among local modes.
Revised: ( x )) will be reduced if we use w sqrt , w min so that the chain can traverse among local modes.

Original: To summarize, the simulation results suggest that w min would be the best choice of the weight function if the multimodality exists, since it not only traverses the multimodal posterior efﬁciently, but also is robust to large N .
Revised: To summarize, the simulation results suggest that w min would be the best choice of the weight function if the multimodality exists, since it not only traverses the multimodal posterior efﬁciently, but also is robust to large N .

Original: We envision that there are a number of ways to improve the mixing under the multimodal posterior by combining it with techniques such as annealing or tempering [9].
Revised: We envision that there are a number of ways to improve the mixing under the multimodal posterior by combining it with techniques such as annealing or tempering [9].

Original: C Real data applications C.1 GWAS dataset for Bayesian variable selection
Revised: C Real data applications C.1 GWAS dataset for Bayesian variable selection

Original: We consider a genome-wide association study (GWAS) dataset on glaucoma studied in [54] with sample size n = 5418 and number of genetic variants p = 7255 .
Revised: We consider a genome-wide association study (GWAS) dataset on glaucoma studied in [54] with sample size n = 5418 and number of genetic variants p = 7255 .

Original: The response variable y ∈
Revised: The response variable y ∈

Original: Ris the standardized cut-to-disk ratio measurements averaged over two eyes.
Revised: Ris the standardized cut-to-disk ratio measurements averaged over two eyes.

Original: We use the BVS model described in Section 4.1, with hyperparameters G = 100 and κ = 0 . 8 .
Revised: We use the BVS model described in Section 4.1, with hyperparameters G = 100 and κ = 0 . 8 .

Original: Since the “true" state is not available, we compare the acceptance rate and the number of unique states visited, averaging overchains.
Revised: Since the “true" state is not available, we compare the acceptance rate and the number of unique states visited, averaging overchains.

Original: From Table
Revised: From Table

Original: [Table]
Revised: [Table]

Original: it is clear that the performance of w ord deteriorates signiﬁcantly as N grows whereas w sqrt , w min , w max does not.
Revised: it is clear that the performance of w ord deteriorates signiﬁcantly as N grows whereas w sqrt , w min , w max does not.

Original: We also report the posterior inclusion probabilities of the top 10 genetic variants in Tables 12, 13, 14, 15, 16, and 17 in Appendix E. All results generally agree with the result of [54], except that when we use w ord with N = 5000 , the chain is stuck at local modes and fails to ﬁnd the signiﬁcant genetic variants.
Revised: We also report the posterior inclusion probabilities of the top 10 genetic variants in Tables 12, 13, 14, 15, 16, and 17 in Appendix E. All results generally agree with the result of [54], except that when we use w ord with N = 5000 , the chain is stuck at local modes and fails to ﬁnd the signiﬁcant genetic variants.

Original: [Table]
Revised: [Table]

Original: C.2 Single-cell RNA dataset for structure learning
Revised: C.2 Single-cell RNA dataset for structure learning

Original: We consider a gene expression dataset on Alzheimer’s disease used in [10] with the sample size n = 1666 and the number of genes p = 73 .
Revised: We consider a gene expression dataset on Alzheimer’s disease used in [10] with the sample size n = 1666 and the number of genes p = 73 .

Original: The goal is to learn the underlying directed acyclic graph (DAG) model among the p genes.
Revised: The goal is to learn the underlying directed acyclic graph (DAG) model among the p genes.

Original: Due to acyclicity, each DAG has at least one ordering of the nodes.
Revised: Due to acyclicity, each DAG has at least one ordering of the nodes.

Original: For example, the ordering for the DAG a → b ← c can be either ( a, c, b ) or ( c, a, b ) .
Revised: For example, the ordering for the DAG a → b ← c can be either ( a, c, b ) or ( c, a, b ) .

Original: A popular
Revised: A popular

Original: Bayesian structure learning strategy is to use MCMC sampling to ﬁrst learn the marginal posterior distribution on the order space and then ﬁnd one or multiple best DAGs for each sampled ordering.
Revised: Bayesian structure learning strategy is to use MCMC sampling to ﬁrst learn the marginal posterior distribution on the order space and then ﬁnd one or multiple best DAGs for each sampled ordering.

Original: We use an MTM implementation of the order MCMC sampler proposed in [10], which aims to learn the posterior distribution on the order space S p , the permutation group on { 1 , . . .
Revised: We use an MTM implementation of the order MCMC sampler proposed in [10], which aims to learn the posterior distribution on the order space S p , the permutation group on { 1 , . . .

Original: , p } .
Revised: , p } .

Original: The size of our model space S p is equal to 73!
Revised: The size of our model space S p is equal to 73!

Original: ≈ 4 .
Revised: ≈ 4 .

Original: 5 × 10 105 .
Revised: 5 × 10 105 .

Original: For each weight function and each setting, we simulate 30 chains, initialized at (1 , . . . , 73) .
Revised: For each weight function and each setting, we simulate 30 chains, initialized at (1 , . . . , 73) .

Original: It is clear from Table 10 that the acceptance probability with ordinary weight function w ord signiﬁcantly deteriorates, which is consistent with our theory.
Revised: It is clear from Table 10 that the acceptance probability with ordinary weight function w ord signiﬁcantly deteriorates, which is consistent with our theory.

Original: We can see this tendency more clearly in the log-posterior trace plots for all weight functions in Figure 8.
Revised: We can see this tendency more clearly in the log-posterior trace plots for all weight functions in Figure 8.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: D Additional discussion D.1 On parallelization (vectorization)
Revised: D Additional discussion D.1 On parallelization (vectorization)

Original: As discussed in Remark 3 in Section 3.1, the overall theoretical computational complexity of the
Revised: As discussed in Remark 3 in Section 3.1, the overall theoretical computational complexity of the

Original: MTM algorithm until the convergence remains the same as a usual MH algorithm.
Revised: MTM algorithm until the convergence remains the same as a usual MH algorithm.

Original: However, MTM enables parallel computations when evaluating N weight functions and hence leads to a signiﬁcant practical computational gain as evidenced by the reduced wall-clock hitting time reported in Table 1.
Revised: However, MTM enables parallel computations when evaluating N weight functions and hence leads to a signiﬁcant practical computational gain as evidenced by the reduced wall-clock hitting time reported in Table 1.

Original: Under the random walk proposal K RW , the evaluation of weight functions is equivalent to the evaluation of target distribution at N states y 1 , . . .
Revised: Under the random walk proposal K RW , the evaluation of weight functions is equivalent to the evaluation of target distribution at N states y 1 , . . .

Original: , y N .
Revised: , y N .

Original: Here we clarify that the scope of parallelism we consider is the instruction-level parallelism [41], also called vectorization.
Revised: Here we clarify that the scope of parallelism we consider is the instruction-level parallelism [41], also called vectorization.

Original: Thanks to modern
Revised: Thanks to modern

Original: When task-level parallelism, assigning a set of independent tasks in parallel across several processors, is employed to the MTM algorithm within each MCMC iteration, it suffers from communication overhead unless the evaluation of target distributions takes extremely long.
Revised: When task-level parallelism, assigning a set of independent tasks in parallel across several processors, is employed to the MTM algorithm within each MCMC iteration, it suffers from communication overhead unless the evaluation of target distributions takes extremely long.

Original: Thanks to the optimized linear algebra libraries such as BLAS [7], the easiest way to achieve instruction-level parallelism is to convert the problem of evaluating target distribution at multiple states π ( y 1 ) , . .
Revised: Thanks to the optimized linear algebra libraries such as BLAS [7], the easiest way to achieve instruction-level parallelism is to convert the problem of evaluating target distribution at multiple states π ( y 1 ) , . .

Original: , π ( y N ) to a series of matrix multiplication problems.
Revised: , π ( y N ) to a series of matrix multiplication problems.

Original: Here we outline the computational strategy to simultaneously calculate π ( y 1 ) , . . .
Revised: Here we outline the computational strategy to simultaneously calculate π ( y 1 ) , . . .

Original: , π ( y N ) for BVS and SBM.
Revised: , π ( y N ) for BVS and SBM.

Original: For BVS, since only one variable is added or deleted in the proposal, the Cholesky rankupdate [17, 44] is utilized to get π ( y 1 ) , . . .
Revised: For BVS, since only one variable is added or deleted in the proposal, the Cholesky rankupdate [17, 44] is utilized to get π ( y 1 ) , . . .

Original: , π ( y N ) from π ( x ) .
Revised: , π ( y N ) from π ( x ) .

Original: To be speciﬁc, assume γ (cid:48) 1 . .
Revised: To be speciﬁc, assume γ (cid:48) 1 . .

Original: , γ (cid:48) N are obtained by adding a variable from γ .
Revised: , γ (cid:48) N are obtained by adding a variable from γ .

Original: By (14), the evaluation of π ( γ j | y ) , j = 1 , . . .
Revised: By (14), the evaluation of π ( γ j | y ) , j = 1 , . . .

Original: , N corresponding to evaluating SSR( γ j ) , j = 1 , . .
Revised: , N corresponding to evaluating SSR( γ j ) , j = 1 , . .

Original: , N from SSR( γ ) saved from the previous iteration.
Revised: , N from SSR( γ ) saved from the previous iteration.

Original: We refer [49, Appendix B] for details of vectorization procedure with Cholesky rank-1 update.
Revised: We refer [49, Appendix B] for details of vectorization procedure with Cholesky rank-1 update.

Original: For
Revised: For

Original: SBM, let A i ∈ { 0 , 1 } p be the i th column of adjacency matrix and Z ∈ { 0 , 1 } p × K be one-hot encoded partition matrix such that Z i,k
Revised: SBM, let A i ∈ { 0 , 1 } p be the i th column of adjacency matrix and Z ∈ { 0 , 1 } p × K be one-hot encoded partition matrix such that Z i,k

Original: = 1 if z i
Revised: = 1 if z i

Original: = k and 0 otherwise.
Revised: = k and 0 otherwise.

Original: Since a node is assigned to another block one at a time, the calculation of π ( y 1 ) . .
Revised: Since a node is assigned to another block one at a time, the calculation of π ( y 1 ) . .

Original: , π ( y N ) given current state π ( x ) can be done by counting the change of the number of edges between blocks; see (15).
Revised: , π ( y N ) given current state π ( x ) can be done by counting the change of the number of edges between blocks; see (15).

Original: Letting A J ∈ { 0 , 1 } p × N where column A j corresponds to the j th proposal, the matrix-matrix multiplication Z (cid:62)
Revised: Letting A J ∈ { 0 , 1 } p × N where column A j corresponds to the j th proposal, the matrix-matrix multiplication Z (cid:62)

Original: A J allows to calculate π ( y 1 ) . .
Revised: A J allows to calculate π ( y 1 ) . .

Original: , π ( y N ) simultaneously from π ( x ) .
Revised: , π ( y N ) simultaneously from π ( x ) .

Original: In addition, if the graph is sparse, then sparse matrix multiplication algorithms can be utilized for further speedup.
Revised: In addition, if the graph is sparse, then sparse matrix multiplication algorithms can be utilized for further speedup.

Original: D.2 On state space
Revised: D.2 On state space

Original: Our state space of interest is ﬁnite (so discrete), but the proposed locally balanced MTM algorithm is also applicable to continuous state spaces which will be shown shortly.
Revised: Our state space of interest is ﬁnite (so discrete), but the proposed locally balanced MTM algorithm is also applicable to continuous state spaces which will be shown shortly.

Original: We choose to focus on the discrete case since the theory on continuous state spaces is usually developed under very different frameworks (and likewise, the theory on continuous spaces often cannot be readily applied to discrete ones).
Revised: We choose to focus on the discrete case since the theory on continuous state spaces is usually developed under very different frameworks (and likewise, the theory on continuous spaces often cannot be readily applied to discrete ones).

Original: Indeed, developing MCMC theory or methodology on discrete spaces is often regarded as more challenging than on continuous ones [48, Section 1], due to the lack of gradient information and a widely accepted theoretical framework supported by statistical theory (for comparison, on continuous spaces, one often assumes log-concavity or asymptotic normality of the target posterior distribution).
Revised: Indeed, developing MCMC theory or methodology on discrete spaces is often regarded as more challenging than on continuous ones [48, Section 1], due to the lack of gradient information and a widely accepted theoretical framework supported by statistical theory (for comparison, on continuous spaces, one often assumes log-concavity or asymptotic normality of the target posterior distribution).

Original: To some extent, the proposed MTM method is conceptually similar to MALA (Metropolis adjusted
Revised: To some extent, the proposed MTM method is conceptually similar to MALA (Metropolis adjusted

Original: Langevin algorithm) or HMC (Hamiltonian Monte Carlo) on continuous spaces in that MTM evaluates the “gradient” by a random search of neighboring states.
Revised: Langevin algorithm) or HMC (Hamiltonian Monte Carlo) on continuous spaces in that MTM evaluates the “gradient” by a random search of neighboring states.

Original: This suggests that for continuous-state-space problems where the gradient of log-posterior cannot be easily evaluated (e.g. Bayesian inverse problems and Gaussian process regression models), the proposed MTM method can be quite useful.
Revised: This suggests that for continuous-state-space problems where the gradient of log-posterior cannot be easily evaluated (e.g. Bayesian inverse problems and Gaussian process regression models), the proposed MTM method can be quite useful.

Original: We conclude this section with a simulation study that shows the weight function proposed in Proposition 2 can lead to an improved MTM algorithm on continuous spaces.
Revised: We conclude this section with a simulation study that shows the weight function proposed in Proposition 2 can lead to an improved MTM algorithm on continuous spaces.

Original: Suppose our target distribution is the 10-dimensional Gaussian distribution N (0 , I 10 ) .
Revised: Suppose our target distribution is the 10-dimensional Gaussian distribution N (0 , I 10 ) .

Original: We set our proposal distribution q ( ·| x ) =
Revised: We set our proposal distribution q ( ·| x ) =

Original: N 10 ( x, 10 − 2 I 10 ) , initialize the chains at x 0 = (10 , 10 , . .
Revised: N 10 ( x, 10 − 2 I 10 ) , initialize the chains at x 0 = (10 , 10 , . .

Original: . , 10) , and run 10,000 iterations for each chain.
Revised: . , 10) , and run 10,000 iterations for each chain.

Original: The result is summarized in Table 11, where for each setting we repeat the simulation 30 times.
Revised: The result is summarized in Table 11, where for each setting we repeat the simulation 30 times.

Original: The advantage of the weight functions considered in Proposition 2 over w ord is substantial.
Revised: The advantage of the weight functions considered in Proposition 2 over w ord is substantial.

Original: We present the log-posterior traceplots in Figure 9 and the MCMC sample trajectories in Figure 10.
Revised: We present the log-posterior traceplots in Figure 9 and the MCMC sample trajectories in Figure 10.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ordinary weight function w ord gets stuck at its early stage, whereas the chains with the other weight functions move to the region with a high posterior, which supports our claim.
Revised: ordinary weight function w ord gets stuck at its early stage, whereas the chains with the other weight functions move to the region with a high posterior, which supports our claim.

Original: E Additional tables
Revised: E Additional tables

Original: Here we report genetic variants with the top 10 highest posterior inclusion probabilities found from
Revised: Here we report genetic variants with the top 10 highest posterior inclusion probabilities found from

Original: BVS model with MTM algorithm, under the different choices of N = 50, 100, 500, 1000, 2000, 5000 and weight functions w ord , w sqrt , w min and w max .
Revised: BVS model with MTM algorithm, under the different choices of N = 50, 100, 500, 1000, 2000, 5000 and weight functions w ord , w sqrt , w min and w max .

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]