Talk:Metropolis–Hastings algorithm

message: The article says that the Gibbs-sampler is a special case of the MH-algorithm. As far as I know it is a complementary MCMC-method which works quite differently. --Smeyen 01:07, 24 February 2007 (UTC)
replies: text: In Gibbs sampler, its proposal distribution is the full conditional distribution of some part of parameter space conditional on the rest, which always resultes in a acceptance probability of 1. See [http:://www.stat.purdue.edu/~lebanon/notes/metropolis.pdf] (Postdoc 05:46, 30 July 2007 (UTC)
thread_title:  [[Gibbs sampling]] 

message: :The Markov chain is started from a random initial value \displaystyle x^0 and the algorithm is run for many iterations until this initial state is "forgotten"
no bounds are known on the number of needed iterations, e.g. the mixing time of the Markov chain? Can we really claim in the introduction that this algorithm allow to compute something if we don't known how long it should be iterate? Levochik (talk) 07:43, 26 August 2009 (UTC)
thread_title: Complexity

message: First of all, thank you for this article. It's the best I could find in the web about Monte Carlo simualtion. But unfortunately, I still don't get it. How can I compute a=a_1a_2 if I don't know P(x)? Perhaps additionally, you could include an example. Something where one can see the step-by-step instruction applied to a simple example. Just like a record of a view steps from a real simulation of some simple case.  —Preceding unsigned comment added by Askamaster (talk • contribs) 10:30, 25 March 2010 (UTC)
replies: text: You don't need P(x), you only need a function proportional to P(x), as the normalizations cancel in the ratio. It is quite often the case that you can easily determine a function proportional to P(x), but the integral to normalize it is very difficult to perform. This comes up quite often in Bayesian computations of the posterior distribution.  Rjljr2 (talk) 15:49, 11 April 2010 (UTC)
replies: text:  article is NOT good, is NOT clear, you want improvement?
best part is who created it 
then explain why, what, when, are no answered I will give you it tells how
does it give algorithm in detail so one can code it? nope, does it provide example? noJuror1 (talk) 10:28, 30 November 2017 (UTC)
thread_title: Example

message: Clearly the Markov chain doesn't converge for all proposal densities Q.  Certainly Q > 0 is sufficient, but I don't believe it is necessary.  What condition is required for the proposal density?  — Preceding unsigned comment added by WuTheFWasThat (talk • contribs) 19:24, 16 May 2012 (UTC)
thread_title: Proposal density

message: The link which is labeled "in practice" and leads to Bayesian statistics should be rephrased so as to not surprise the reader, as per WP:EASTEREGG. There should probably be some brief remark explaining what Bayesian statistics are and how they relate, but I don't know how to write such a remark.  — Preceding unsigned comment added by 132.65.153.93 (talk) 18:03, 12 November 2013 (UTC)
thread_title:  Easter egg link 

message: The paragraph on thinning is very poor. In particular the statement "this means that if we want a set of independent samples, we have to throw away the majority of samples and only take every nth sample" is plain wrong. 
There is no such guarantee of independence, regardless of the interval chosen. In addition, most authors advise against the practice, since while thinning reduces correlation between subsequent steps, it also greatly reduces the number of samples finally obtained. There are some exceptions, as Art Owen discusses here http://statweb.stanford.edu/~owen/reports/bestthinning.pdf.
Patrick1894 (talk) 14:42, 17 April 2019 (UTC)
thread_title:  Thinning 

message: In my experience, supported for example by the frequencies of the terms "Metropolis", "Metropolis-Hastings" and "Monte Carlo" recorded between 1953 and 2012 in Google's Ngram viewer , this algorithm is far more often called "Metropolis" than "Metropolis-Hastings."  Therefore I suggest the article be renamed (moved to) "Metropolis algorithm" with Metropolis-Hastings redirecting to it, and with a subsection, now lacking, that specifically explains and compares Hastings' 1970 improvement and generalization to the original 1953 algorithm.CharlesHBennett (talk) 22:06, 22 January 2020 (UTC)
replies: text: Neutral. In my experience, "Metropolis" is more common than "Metropolis-Hastings" as well. However, I don't think we get a lot by moving this. Metropolis algorithm already redirects to this article. BernardoSulzbach (talk) 10:59, 23 January 2020 (UTC)
thread_title:  Rename?   (In Wikipedia parlance suggested Move) 

message: It is stated that the transition probability P(x'\mid x) can be separated into a proposal step and accept/reject step. Mathematically, 
: P(x'\mid x) = g(x' \mid x) A(x', x).
where g(x' \mid x) is the proposal distribution and A(x', x) the acceptance distribution.
This transition distribution does not take into account rejection however. The transition probability is complete with an additional rejection term
: P(x'\mid x) = g(x' \mid x) A(x', x) + \delta(x' - x) (1 - A(x', x))
where \delta(x) is understood as the Dirac delta function. It is only later in the derivation with the choice for A(x', x) that the last term vanishes, since a transition where x' = x will always be accepted (A(x, x) = 1).  — Preceding unsigned comment added by 213.125.242.82 (talk) 19:46, 8 April 2020 (UTC)
thread_title:  Formal derivation of transition probability incomplete 

message: Calculate the acceptance ratio \alpha = f(x')/f(x_t), which will be used to decide whether to accept or reject the candidate. Because f is proportional to the density of P, we have that \alpha = f(x')/f(x_t) = P(x')/P(x_t).
* Accept or reject: 
** Generate a uniform random number u \in [0, 1].
** If u \le \alpha, then accept the candidate by setting x_{t+1} = x',
** If u > \alpha, then reject the candidate and set x_{t+1} = x_t instead.
May I suggest a rewrite as follows which shows the Hastings step more clearly? The above is correct, but:
* It forces a generation of u unnecessarily when \alpha > 1, and
* It requires thinking to see that it works, specifically in the case \alpha = 1.
Suggested revision:
* Accept or reject: 
** If \alpha > 1, then accept the candidate by setting x_{t+1} = x'.
** If \alpha \le 1, then
*** Generate a uniform random number u \in [0, 1].
*** If u \le \alpha, then accept the candidate with probability \alpha by setting x_{t+1} = x'.
*** If u > \alpha, then reject the candidate and set set x_{t+1} = x_t instead.
bayesianlogic.1@gmail.com 
 19:28, 14 December 2020 (UTC)
thread_title:  Oughtn't the core procedure be made a tad clearer?  Suggest revision. 

message: The rosenbrock function is typically used as an example for minimization, not maximization. The Metropolis Hastings can be seen as a maximization algorithm though, and indeed the caption is written as if we were maximizing something. Perhaps the function being used is 1/rosenbrock, in such a way that the minimum becomes a maximum? 
Lnz.Rossi (talk) 16:59, 22 May 2021 (UTC)
replies: text: Maximization of some function f is the same as minimization of -f. Here, instead of minimizing Rosenbrock, the Metropolis-Hastings algorithm is maximizing -1×Rosenbrock. RFZYNSPY talk 02:47, 21 March 2024 (UTC)
thread_title:  The plot with the three chains needs clarification 

message: Under 'Intuition', I see the following parsing error instead of the acceptance ratio formula:
"Calculate the acceptance ratio Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "/mathoid/local/v1/":): {\displaystyle \alpha = f(x')/f(x_t)} , which will be used to decide whether to accept or reject the candidate. Because f is proportional to the density of P, we have that..."
When I try to edit, this error doesn't show up in the preview, so not sure how to fix this. 128.12.123.150 (talk) 03:22, 17 February 2023 (UTC)
replies: text: It looks fine from my end. Perhaps some sort of browser incompatibility? Qflib, aka KeeYou Flib (talk) 04:43, 18 February 2023 (UTC)
thread_title:  Acceptance ratio formula failing to parse