Talk:Rule of succession

message: Why does Laplace use the expected value (mean) of the posterior?  This is not obvious to me, but it's crucial. Could someone who knows add something to the entry about the justification for this?
Alex Holcombe 11:13, 14 October 2007 (UTC)
replies: text: When the parameter whose posterior probability distribution we're finding is itself a probability, then its expected value is the marginal probability, conditional on current data, of the event in question---in this case, tomorrow's sunrise.  That is the content of the law of total probability. Michael Hardy 16:06, 14 October 2007 (UTC)

text: I think I'm interested in a different kind of answer. As I understand it, the expected value minimizes the *average* difference between the estimate and the actual unknown parameter value. But in other contexts, Bayesians may minimize the average sum of squared difference, and in maximum likelihood the mode of the posterior distribution is used. Did Laplace have any particular reason for choosing the mean? Is this somehow embedded in the law of total probability? Sorry if this is a naive question. Alex Holcombe 10:23, 15 October 2007 (UTC)
replies: text:  No, it does not.  The expected value does not minimize that average distance.  The median does.  The expeected values minimizes the average square of the distance.  Besides, it's not clear that estimation is what's being done here. Michael Hardy (talk) 19:07, 7 January 2008 (UTC)
thread_title:  Why does Laplace use the expected value (mean) of the posterior? 

message: The logic behind this generalisation is not satisfactory, and there are no appropriate references where a more formal argument is made.  I can provide a reference to a more formal generalisation given in PROBABILITY THEORY: THE LOGIC OF SCIENCE.  page 563-568 discusses the original rule of succession, and page 568-571 discuss the generalisation to m categories.
One simple way to check the logical consistency of the results, is that if we observe nothing, the rule of succession reverts back to the principle of indifference (i.e. initial probability = 1/2 before any event is observed).  Therefore, this principle should be maintained when generalising in the way described, but it clearly does not (unless m=2) for we get a prior probability for success of
:P(s=0, n=0, m)={0+{2 \over m} \over 0+2}={1 \over m} 
But this is clearly not expressing prior ignorance about a success unless m=2, for if m=4 say, this means that, prior to observing any data, we assume that it is 3 times as likely for failure as for success (Pr(success)=1/4).  This easily becomes absurdly informative if the number of categories is huge (say m=10500 categories then this method assumes that the probability of an arbitrary half of these categories occurring is 10-500~0 which is hardly "...assuming absolutely no prior knowledge about them, except the number of them, and that any possibility being considered is from that set...." stated at the beginning of the section).
The way to proceed from here is very carefully, and to re-derive the results from first principles (this is what Jaynes and Bretthorst do in their book) rather than to introduce an intuitively sensible generalisation.
Setting a uniform prior over the initial m categories, and using the multinomial distribution (the multivariate generalisation of the binomial).
Letting Ai denote the event that the next observation is in category i (i=1,...,m), and let n_{i} denote the number of times category i (i=1,...,m) was observed and let
n=n1+...+nm be the total number of observations made.
The result, in terms of the original m categories is:
:P(A_{i} | n_{1},...,n_{m}, m)={n_{i} + 1 \over n + m} 
Because the propositions or events Ai are mutually exclusive, to collapse to two categories you simply add up the i values which correspond to "success" to get the probability of success.  Supposing that you do this, and that you aggregate c categories as "success" and m-c categories as "failure".  Let s denote the sum of the relevant n_{i} values which have been termed "success". The probability of "success" at the next trial is then:
:P(success| n_{1},...,n_{m}, m, c)={s + c \over n + m} 
Which is different from the intuitive rule 2/m unless 2/m=c and m=2 which just gives the binary rule of succession.
This indicates that mere knowledge of more than two outcomes which we are sure are possible (i.e. we are sure they would occur if we sampled indefinitely) is relevant information when collapsing these categories down into just two.  In "shrinkage" terminology, the more categories that are enumerated (i.e. the larger m is), the more the probability of success is "shrunk" towards the value c/m (which is 1/2 in the example you give).
I believe something analogous to a "law of large numbers" style process occurring here, in that as you aggregate a larger and larger number of categories, the variability of the aggregation tends to decrease, and thus a priori one is more confident that the true proportion will actually be c/m.  —Preceding unsigned comment added by 124.168.210.217 (talk) 16:20, 22 January 2011 (UTC)
thread_title:  Generalization to any number of possibilities 

message: "This gives a slightly different probability in the binary case of s + 0.5 n + 1 {\displaystyle {\frac {s+0.5}{n+1}}} {\frac {s+0.5}{n+1}}. "
You may add this is the Perks's indifference rule.
See for example
https://www.jstor.org/stable/41138962  — Preceding unsigned comment added by MUrqui (talk • contribs) 08:41, 3 June 2022 (UTC)
thread_title:  Perks's rule