Talk:Conjugate prior

message: You may want to see some of the pages on Empirical Bayes, the Beta-Binomial Model, Bayesian Linear RegressionCharlesmartin14 23:43, 19 October 2006 (UTC)
replies: text: This was my comment last night, sorry I didn't sign it, I've just corrected this on the page. Bazugb07 (talk) 14:32, 12 May 2009 (UTC)

text: Wasn't the "dependence" referring to dependence among the parameters, not the data? --128.187.80.2 (talk) 23:00, 30 March 2009 (UTC)
thread_title: Untitled

message: Has anyone else noticed the posterior parameters are wrong? At least according (Degroot, 1970), the multivariate normal distribution posterior in terms of precision is listed incorrectly: it should be what the multivariate normal distribution in terms of the covariance matrix is listed as on the table. I don't really have the time to make these changes right now or check any of the other posterior parameters for accuracy, but someone needs to double check these tables. Maybe I'll do it when I'm not so busy. Also the, Fink (1995) article disagrees with DeGroot on a number of points, so I question it's legitimacy, given that the latter is published work and former is an ongoing report. Maybe it should be removed as a source?
DeverLite (talk) 23:22, 8 January 2010 (UTC)
thread_title:  Incorrect posterior parameters 

message: I think it would be useful to augment the tables with the "marginal distribution" as well. The drawback here is the tables will widen, and they are already pretty dense. Thoughts? --128.187.80.2 (talk) 23:00, 30 March 2009 (UTC)
replies: text: I am not clear what you mean by marginal distribution here ... if it is what I first thought (marginal dist of the observations) then these marginal distributions might find a better and useful place under an article named like compound distributions. Or is it the marginal distribution of new observations conditional on the existing observations marginalised over the parameters (ie predictive distributions)? Melcombe (talk) 08:59, 31 March 2009 (UTC)
replies: text: I was referring to the marginal distribution of the observations (not the predictive distribution). I often use this page as a reference guide (much simpler than pulling out my copy of Gelman et al.) and at times I have wanted to know the marginal distribution of the data. Granted, many books don't include this information, but it would be useful. As an example, in the Poisson-Gamma model \mathbf{x} \sim NegBin \left( \alpha, \frac{\beta}{1+\beta} \right) (when the gamma is parameterized by rate). This information is largely contained in Negative binomial#Gamma-Poisson_mixture but that article does not specifically mention that it is the marginal distribution of the data in the Bayesian setting. Plus, it would be more convenient to have the information in one place.  Your proposal to put it on a dedicated page may be a reasonable compromise since the tables are already large and this information is used much less frequently. --128.187.80.2 (talk) 17:27, 1 April 2009 (UTC)
replies: text: I thought that giving such marginal distributions would be unusual in a Bayesian context, but I see that Bernardo & Smith do include them in the table in their book .. but they do this by a having a separate list of results for each distribution/model, which would be a drastic rearrangement of what is here. An article on compound distributions does seem to be needed for its own sake. Melcombe (talk) 13:21, 2 April 2009 (UTC)
thread_title: Marginal distributions

message: It keeps getting changed back and forth, but I have the hyperparameters as: 
\alpha + n,\ \beta + \sum_{i=1}^n x_i\!
-- There is certainly a problem as it currently stands. The wikipedia page on the gamma explicitly gives both the two forms. Particularly k=alpha, beta=1/theta. Hence the update rules must be consistent with this notation. I have corrected this for now.  —Preceding unsigned comment added by 129.215.197.80 (talk) 15:23, 20 January 2010 (UTC)
thread_title:  Poisson-Gamma 

message: Just a cheeky comment to say that this is the hardest article to understand of all those I've read so far. It assumes a lot of background knowledge of statistics. Maybe a real-world analogy or example would help clarify what a conjugate prior is. Abstractions are valuable but people need concrete examples if they want to jump in half-way through the course. I'm really keen to understand the relationship between the beta distribution and the binomial distribution, but this article (and the ones it links to) just leave me befuddled. 111.69.251.147 (talk) 00:39, 21 June 2010 (UTC)
replies: text: You haven't read enough of Wikipedia if you think this is it's most unintelligible !  However, I agree that it's baffling.  I came here with no prior knowledge of what a conjugate prior is, following a link from a page that mentioned the beta distribution being the conjugate of various other distributions.  I find myself reading a page that tells me a conjugate prior is (in effect) a likelihood function that changes metaparameters but not form when given new data; this does not tell me how this prior's form is conjugate *to* any other distribution, which was what I was trying to glean.  Lurking in the back is the fact that the variate being modelled has a distribution, let's call it X; when the prior for its parameter has distribution Y, then data about the primary refines our knowledge of the parameter to a posterior likelihood of the same form as the prior Y; in such a case, I'm guessing "the form of Y" is what's being described as "conjugate to" (possibly the form of) X; but I don't actually see the text **saying that** so I'm left wondering whether I've guessed wrong.  An early remark about the gaussian seemed to be saying that, but it was hard to be sure because it was being described as self-conjugate and similar phrasing was used to describe the prior and posterior as conjugate, so I was left in doubt as to whether X=gauss has Y=gauss work.  I lost hope of finding any confirmation or correction for my guess as the subsequent page descended into unintelligible gibberish. (It might not seem like that to its author, but that's the problem of knowing what you're talking about and only being used to talking about it to others who already understand it: as you talk about it, you say the things you think when you think about it and can't see that, although it all fits nicely together within any mind that understands it already, it *conveys nothing* to anyone who doesn't already understand it.  Such writing will satisfy examiners or your peers that you understand the subject matter, but won't teach a student anything.)  -- Eddy 84.215.30.244 (talk) 06:14, 30 July 2015 (UTC)
thread_title:  Most unintelligible article on Wikipedia 

message: Paragraphs 1 through to contents - Great.
The rest - incomprehensible. I have no doubt that if you already know the content, it is probably superb, but I saw a long trail of introduced jargon going seemingly in no particular direction. I was looking for a what & some "WHY do this", but I did not find it here. Many thanks for the opening paragraphs. Yes, I may be asking for you to be the first ever to actually explain Bayesian (conjugate) priors in an intuitive way. [not logged in]  — Preceding unsigned comment added by 131.203.13.81 (talk) 20:13, 1 August 2011 (UTC)
thread_title: Another less cheeky comment.

message: wat
Simple English sans maths in the intro would be great.  —Preceding unsigned comment added by 78.101.145.17 (talk) 14:48, 24 March 2011 (UTC)
thread_title:  This article 

message: The external link is broken. Should I remove it?  — Preceding unsigned comment added by 163.1.211.163 (talk) 17:38, 12 December 2011 (UTC)
thread_title: Broken link

message: Some of the posterior are wrong. I just discovered one: 
Normal with known precision τ	μ (mean)
The posterior variance is (τ0+nτ)^-1.  — Preceding unsigned comment added by 173.19.34.157 (talk) 04:09, 15 May 2012 (UTC)
thread_title:  Wrong posterior 

message: Yeah...  That table, while informative, is not formatted very well.  It wasn't clear at first what the Posterior Hyperparameters column represented, or what any of the variables meant in the Posterior Predictive column.  — Preceding unsigned comment added by 129.93.5.131 (talk) 05:00, 10 December 2013 (UTC)
thread_title:  That Table 

message: Substituted at 19:53, 1 May 2016 (UTC)
thread_title: Assessment comment

message: I've recently used bayesian conjugate priors for computing the probability that there will be at least 1 rental car available in my area at any given day. Would there be any appetite for a section showing how one can use the table to compute something like this? If so i would write that up in the next few days. I figure it might help with making the page a little more understandable.  — Preceding unsigned comment added by Rasmusbergpalm (talk • contribs) 08:59, 11 February 2020 (UTC)
replies: text: Rasmusbergpalm I've found the practical example very helpful, thank you. However, I was wondering about the correctness of the gamma distribution hyperparameters picked; perhaps I'm missing something. Because in the gamma distribution mean=alfa/beta and variance=alfa/beta^2, we can calculate beta=mean/variance and alfa=beta*mean. From the example we have mean=8/3 and sample variance=7/3, from which we can calculate beta=8/7 and alfa=64/21. Does this make sense?--Gciriani (talk) 16:18, 2 December 2020 (UTC)
replies: text: Gciriani Thanks! Glad you liked it :) Remember, the gamma distribution is a distribution over the rate of the poisson distribution (from which the samples are drawn), not over the samples directly. The information from the data enters through the likelihood term, not the prior term. Also, the prior hyperparameters are inherently subjective. There's no "right" answer. They represents your prior belief. Ideally you should set them before you observe any data. One way to check if your prior belief is reasonable is what is called prior predictive samples; You sample parameters from your prior and then sample data from your likelihood model given those parameters and see if they're reasonable. I made a small notebook where I do this that you can check out. link to notebook. I hope this clears things up. If you wish to learn more here some excellent resources: http://www.stat.columbia.edu/~gelman/book/ and https://arxiv.org/pdf/2011.01808.pdf. Rasmusbergpalm (talk)  — Preceding undated comment added 20:03, 3 December 2020 (UTC)
thread_title:  Any appetite for a "practical application" section? 

message: I am not persuaded that saying the interpretation of a Beta prior/posterior distribution with hyperparameters α and β is \alpha - 1 successes and \beta - 1 failures, with the note 
The exact interpretation of the parameters of a beta distribution in terms of number of successes and failures depends on what function is used to extract a point estimate from the distribution.  The mode of a beta distribution is \frac{\alpha - 1}{\alpha + \beta - 2}, which corresponds to \alpha - 1 successes and \beta - 1 failures; but the mean is \frac{\alpha}{\alpha + \beta}, which corresponds to \alpha successes and \beta failures.  The use of \alpha - 1 and \beta - 1 has the advantage that a uniform {\rm Beta}(1,1) prior corresponds to 0 successes and 0 failures, but the use of \alpha and \beta is somewhat more convenient mathematically and also corresponds well with the fact that Bayesians generally prefer to use the posterior mean rather than the posterior mode as a point estimate.  The same issues apply to the Dirichlet distribution.
My problem is that this becomes a nonsense when used with the Jeffreys prior \alpha=\beta=\tfrac12: the fractional values might be explainable away, but the negative values really cannot. I would much prefer saying the interpretation of α and β is \alpha successes and \beta failures with a note like the following - 11:03, 23 June 2020 (UTC)
thread_title:  Beta note 

message: For the a poisson likelihood and gamma prior, the predictive prior is given as \operatorname{NB}\left(\tilde{x}\mid k', {\theta'}\right) when specifying the Gamma Distribution using scale (\theta); and \operatorname{NB}\left(\tilde{x}\mid\alpha', \frac{1}{1 + \beta'}\right) when specifying the Gamma Distribution using rate \beta. Since  \theta'=1/\beta', I do not see how both equations can be correct.
Furthermore, there are a number of conventions for the Negative binomial (NB) and it does not specify which is used.
The root of all these problems is that the equations are unsourced. Every single equation should be sourced, or deleted. Adpete (talk) 01:38, 26 August 2020 (UTC)
thread_title:  looks like an error in predictive priors 

message: These https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf notes present a chained prior for the Gaussian when neither mean or variance are fixed - is it a good idea to put this in the table too? Thank you,  — Preceding unsigned comment added by Reim (talk • contribs) 08:16, 6 November 2020 (UTC)
thread_title:  mean + variance gaussian chain