Talk:Minimum description length

message: The following comment pertains to the previous version of this page, of which only traces remain in the current version. Any new comments are welcome! I'll check this discussion page now and then for the next couple of weeks (september 2004).
----
see the disccussion Minimum message length which may be relevant to stress the difference between MML and MDL (perhaps the easiest is in the historical sense / pratical issues) - Meduz 12:15, 4 August 2006 (UTC)
thread_title: Comments

message: This text in the article: "Central to MDL theory is the 1-1 correspondence between code length functions and probability distributions. (The lemma involved is the Kraft-McMillan inequality.)" linked to the function disambiguation page.  I altered the link to point to function (mathematics).  I'm fairly certain this is correct, but since the article does reference Turing machines, I'm hoping the text does not refer to subroutines by the word "function."  Correct me if I am wrong. Volfy 02:07, 23 November 2005 (UTC)
thread_title:  fix link to function to be [[function (mathematics)]] 

message: I think this section needs a citation, clarification, or confirmation by an expert (preferably by citation).  It looks like original research as it stands, and I tend to think the conclusion is incorrect.
The simplest counter-example is that of a loss-less graphic compression algorithm.  These map pixel data into a computably-feasible model space, and using techniques such as run-length encoding often effectively exploit the fact that not all images appear with equal likelihood, and that color bias and spacial consistency often allow a reduction.  This amounts to a model selection.  Even a random image of 1000 pixels which are selected according to a severe bias (95% black and 5% white) will be compressible to less than 1000 bits, because at those odds run-length encoding works well, as does an "all are black but these" model.  With those languages, should you encounter a picture that is 50/50 random, you may need give up a constant number of bits to say "no better compression possible", but making these practical trade-offs is one of the advantages of MDL versus Kolomorov computation:  as the article says, in MDL the actual constants matter.
12.232.236.2 (talk) 16:36, 5 July 2011 (UTC)
thread_title:  Coin example, never selecting a biased coin model? 

message: Hello fellow Wikipedians,
I have just added archive links to  on Minimum description length. Please take a moment to review my edit. If necessary, add  after the link to keep me from modifying it. Alternatively, you can add  to keep me off the page altogether. I made the following changes:
*Added archive http://web.archive.org/web/20100218134039/http://www.mdl-research.org:80/ to http://www.mdl-research.org/
When you have finished reviewing my changes, please set the checked parameter below to true or failed to let others know (documentation at ).
Cheers.—cyberbot IITalk to my owner:Online 00:18, 19 March 2016 (UTC)
thread_title:  External links modified 

message: Hello fellow Wikipedians,
I have just modified one external link on Minimum description length. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20100616234207/http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=11155 to http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=11155
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
Cheers.—InternetArchiveBot (Report bug) 02:14, 1 February 2018 (UTC)
thread_title:  External links modified (February 2018) 

message: * Specific text to be added or removed: In practice, when finding a machine learning model that best fits the data according to the MDL principle, one selects the model that minimizes the length of a  two-part code (program). As an example, in the supervised case of learning the best fitting rule list (the probabilistic form of a decision list) from a dataset, one chooses the rule list that minimizes the two-part code composed of: 1) the code that uniquely defines a rule list given all possible rule lists that could be generated from that dataset, i.e., taking into account the number of variables and their possible combinations; and 2) the code that returns the dataset given the specific rule list as defined in the first code. The first code is specifically made to avoid overfitting through the penalization of more complex rule lists that would naturally be more prone to overfitting. Note that using a two-part code is essential when one needs to distinguish the different models, such as in the case of finding the model and its parameters that best fit the data. This contrasts with the task of merely wanting to know the size of the model (e.g., the number of rules in a rule list) necessary for the best performance.
* Reason for the change: The MDL in Machine Learning section is quite poor and misses the most common problem for beginners in the use of MDL for ML, i.e., how to pass from its abstract form to a concrete problem. I decided to add an example based on a recent work that I know well and that is accessible to the machine learning practitioner and found some level of success. Given COI, I cannot be a partial judge but believe this to add some significant clarification regarding what it means in practice to use MDL in machine learning and what does it consist of. As a last note, even if this edit is not approved I would suggest for someone to add an example of this in practice. A significant extension would be to add concrete equations and formulas to help. Afternoonfriend (talk) 17:11, 1 January 2022 (UTC)
replies: text: Not done: please provide reliable sources that support the change you want to be made.   Thinker78 (talk) 00:23, 3 December 2022 (UTC)
thread_title:  Adding an example to "MDL in Machine Learning"