Talk:Principle of maximum entropy

message: I think the minus sign on the equations to find lambda values is wrong. I'll remove it. --163.117.155.37 18:01, 12 January 2007 (UTC)
thread_title: equation minus sign

message: (what a confusing title...)
I was intrigued by the comment about the n&s condition for the existence of a sufficient statistic, so I read 
Pitman-Koopman_theorem
and it says:
"...only in exponential families is there a sufficient statistic whose dimension remains bounded as sample size increases."
I rephrased the comment slightly but I know very little about the subject, so if any of you knows a little about the subject please consider fixing it...
Amitushtush 11:30, 22 September 2007 (UTC)
thread_title:  neccessary and sufficient condition for a sufficient statistic 

message: Jaynes had a lifelong controversy with authors who claimed that his maxent principle would allow paradoxes, most famously the Dawid-Stone-Zideck paradox. After his death, the paradox was acknowledged by Kavin van Horn. I do not see any mention of paradoxes at all. -- Zz (talk) 13:30, 18 January 2008 (UTC)
replies: text: Well, this is only a year and a half late, but what the heck. The DSZ paradox is an argument against improper priors, not the maxent principle. Cyan (talk) 01:48, 8 August 2009 (UTC)
replies: text: The paradox is also not a paradox, as Jaynes argues in his 2003 book "Probability Theory": DSZ simply made a mistake in the derivation; probabliy due to uncareful notation they assume that two cases that are different are the same; if this mistake is resolved, then it appears that in most of their examples, the paradox goes away for one improper prior which, if the parameter in question is a scaling parameter, turns out to be Jeffrey's prior.  So instead of treating the DSZ thing as a paradox, it could be seen (and has been seen as such by Jaynes) as a new and elegant mathematical way to find an uninformative improper prior.147.86.212.162 (talk) 06:14, 28 August 2009 (UTC)
replies: text: I think most people now seem to agree that Jaynes was wrong on the marginalization paradox, such as here. There is also a reply from the original authors. It is somewhat relevant to the article, as many maxent priors are improper, but perhaps it deserves an article of its own. —3mta3 (talk) 21:39, 27 September 2009 (UTC)

text: I just ran across this three-year-old comment, and I have to correct it, as it misrepresents what I wrote (I'm Kevin Van Horn).  Yes, Jaynes made a mistake in attributing the "paradox" to two distinct states of information.  However, the Marginalization Paradox is not a paradox -- it is a failure to strictly follow the rules of probability theory.  The problem is that DSZ go directly from "p(zeta | y, z) does not depend on y" to "p(zeta | z) = p(zeta | y, z)".  This step requires integrating out y: p(zeta | z) = (INTEGRAL y: p(y | z) * p(zeta | y,z)).  The problem is that p(y | z) is also improper, so you're taking a divergent integral, making this step invalid.   Ksvanhorn (talk) 01:08, 20 March 2011 (UTC)
thread_title:  Paradoxes 

message: The term "least biased" in the 2nd sentence seems likely to be incorrect, at least if "bias" has its usual statistical meaning  Least incorrect? Least additional information? Melcombe (talk) 10:09, 30 April 2008 (UTC)
thread_title: least biased?

message: That relativity bit in the lead seems pretty creative to me, and I'm not sure I buy the metaphor. Is this attributable to anyone outside of WP editor(s)? Cretog8 (talk) 03:11, 12 July 2008 (UTC)
replies: text: OK, I'll cut it. CRETOG8(t/c) 07:42, 4 September 2008 (UTC)
thread_title:  Relativity 

message: The first sentence in the Overview section of the article reads:
In most practical cases, the testable information is given by a set of conserved quantities (average values of some moment functions)...
I do not understand why conserved quantities serves as testable information. Consider a distribution over energy, if I say energy is conserved, then the testable information becomes the expectation of energy. I can not see any relation between these two. Why the testable information related to energy conservation can not be the standard deviation of energy or some thing else? The word "conserved" here is also confusing. The probability distribution implies that the energy is fluctuating, then how can a fluctuating quantity being considered as conserved? Everett (talk) 13:12, 26 August 2009 (UTC)
thread_title:  Conserved Quantities 

message: :Are you sure you posted this to the right talk page?
:This article doesn't even mention Swenson, whoever he is.  It's mostly about E.T. Jaynes and his followers; and is recognised as an important and mainstream in probability theory and in philosophical discussions of the foundations of statistical thermodynamics.  Jheald (talk) 00:43, 8 January 2010 (UTC)
thread_title:  sorry. deleted section posted in error

message: Good edit! Chjoaygame (talk) 19:38, 27 January 2013 (UTC)
thread_title:  Good edit ! 

message: * Maximum entropy estimation — make proposed page title track back to this comment via "what links here" 
Maximum likelihood estimation has its own page.  I'm not sure whether MEE should be covered here (this is already a large page), or on its own page.  MEE is not the most common term in academic paper titles, but it's not a ghost town, either. — MaxEnt 17:47, 7 May 2017 (UTC)
replies: text: I should add that the problem for me is that many papers in machine learning embed the phrase "maximum entropy" in the title, and it implicitly references either a form of maximum entropy estimation, or a class of objective functions which take you to much the same place.  I'm just going to lump all these papers under MEE in my own notes.  I'm certainly not using Principle of maximum entropy as the topknot for this collection of papers. If the current carving of the ME cake doesn't work for me locally, does it server the readers here any better?  Definitely scratching my own itch, but it raises a broader question. — MaxEnt 17:57, 7 May 2017 (UTC)
thread_title:  Maximum entropy estimation 

message: Hello fellow Wikipedians,
I have just modified one external link on Principle of maximum entropy. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20060603144738/http://www.phys.uu.nl/~wwwgrnsl/jos/mepabst/mep.pdf to http://www.phys.uu.nl/~wwwgrnsl/jos/mepabst/mep.pdf
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
Cheers.—InternetArchiveBot (Report bug) 01:41, 30 December 2017 (UTC)
thread_title:  External links modified 

message: I think in the "Discrete case" paragraph, it should first be taken into consideration the MEP, exhibiting the ME expression, ie what it has to be maximized, just as it has been done in the continuous case.  Filipsnew (talk) 19:49, 19 December 2018 (UTC)
thread_title:  Lacking of what it must be maximized in the discrete case 

message: If we talk purely about decimals, what numbers contain maximum entropy in the distribution of numbers, (ea what would be the worst number to compress by an ideal compression formula)
From the possible number, how do the physic constants and mathematical constants compare to it?, I've not studied this area, but its easy to imagine that it has some impact on various current theories about our universe.
If indeed our universe was is pure information based. Also closely related is how much information can a blackhole surface contain, or any theoretical surface size in our universe.  — Preceding unsigned comment added by 2001:1C04:3806:7E00:2500:37FD:FDF7:46BB (talk) 13:29, 10 August 2022 (UTC)
thread_title:  interesting topic but missing some content 

message: Support the 2019 proposal to merge Entropy maximization to here, specifically in the Discrete section. The contents are duplicated, so the only thing really to move is the 'reference'. Klbrain (talk) 09:08, 3 August 2020 (UTC)
thread_title:  Entropy maximization merge