Talk:Kolmogorov complexity

message: This article does not say whether the description language is required to be Turing complete.  But it links to minimum message length, which makes the claim that Kolmogorov complexity allows only Turing complete description languages.  I don't know whether that claim is true or not.  If that claim is true, then this requirement should be described in the first paragraph that mentions description languages (the existing link to description language is useless, and links to a page listing languages like XML which are not Turing complete).  If that claim is not true, then the first theorem here should be fixed, by adding a phrase requiring that L1 and L2 be Turing complete.  Clearly, there are cases where the theorem is false, if L1 and L2 don't have to be Turing complete.
:It is understood that "the" Kolmogorov complexity is with respect to a universal description system -- i.e. the description language is Turing complete.  In the literature there are also definitions of Kolmogorov complexity relative to a particlar (not necessarily universal) description system, but that is not what is generally intended by "the" Kolmogorov complexity. The latter expression is still a slight abuse of terminology, because the KC will still depend on which universal system is used.--r.e.s. (talk) 19:19, 28 July 2008 (UTC)
thread_title: Must description languages be Turing complete?

message: My bot semiautomatically fixed a mistake in this page, see  doing the replacement  
:parameterisation ->
:parametrisation
I got a an anonimous message today that the former is also valid spelling. Is that true? I myself would think that it is not in this context. Oleg Alexandrov 6 July 2005 15:02 (UTC)
:I think your bot spells correctly, although I am an American, and would use a 'z' instead of an 's'.   The former spelling looks incorrect to me in any case. -- 130.94.162.61 23:59, 21 February 2006 (UTC)
replies: text: See these pages   in Merriam-Webster Online Dictionary. :-) CiaPan 01:17, 22 February 2006 (UTC)
thread_title: Spelling

message: The first sentence reads as follows:
In computer science, algorithmic information theory is a field of study which attempts to capture the concept of complexity by using tools from theoretical computer science.
Typically the first sentence of an article is some definition or a thumbnail sketch of one.  The above statement is not.  Additionally, it could have equally applied to the better known field of Computational complexity theory.
I plan to fix this as time permits. Vonkje 22:42, 23 July 2005 (UTC)
thread_title: Opening definition

message: I propose that this page be moved to Kolmogorov complexity.  This is the main subject of the article and would simplify the article's introduction.--CSTAR 01:56, 3 October 2005 (UTC)
thread_title:  Suggestion to move 

message: Why does Talk:Algorithmic information theory forward to Talk:Kolmogorov complexity?
:: Fixed. The following  comment copied over to appropriate talk page.--CSTAR 17:31, 17 June 2006 (UTC)
thread_title:  Some issues 

message: "The first string admits a short English language description namely "50 repetitions of '01'", which consists of 20 characters. The second one has no obvious simple description other than writing down the string itself, which has 100 characters." <----- This is not true. One could reference this string as eg. "the uncompressable 100 character string in http://en.wikipedia.org/wiki/Kolmogorov_complexity" which would have 95 characters.
What do you wiki-authors do with this now? I didn't dare to put it directly in the main page and wanted to spur some discussion. As far as I remember Martin Garder is talking about this paradox, but I cannot find the source right now. (Markus Gaelli)
:This is just a red herring.  The paradox is just because Kolmogorov complexity is only defined relative to a particular prefix free universal machine.  There are plenty of machines that make the repeated 01 string have higher Kolmogorov complexity than the other string, by carefully choosing indices.  So the correct reading of the intro paragraph is to view it as an informal motivating example, which is fine for an intro paragraph.  Once the formal definition is given, it becomes clear that the intro paragraph is speaking of some intuitive idea of description which is made formal via universal machines. CMummert 23:57, 13 August 2006 (UTC)
thread_title: Example strings

message: Here it says, if I understand correctly, that there are two different definitions of Kolmogorov complexity (plain and prefix). It would be nice to be able to read about that in the article. --Tgr 12:21, 18 September 2007 (UTC)
replies: text: I don't believe, as claimed, that the leading texts distinguish between these two definitions, which I've never seen before reading this blog post.  In the classroom, generally one definition is given, so that should suffice here, barring a reliable source to the contrary.  I suspect that "prefix complexity" means that the program includes information about how long it is, whereas "plain complexity" would not include this information.  This is not generally an issue.  I believe "plain complexity" is usual definition.  However, although the program description is generally not assumed to include a termination indicator, the program contains information about when to terminate during its runtime. Calbaer 00:34, 19 September 2007 (UTC)
replies: text: Yes, there is plain Kolmogorov complexity and prefix-free complexity. The latter is crucial for things like Chaitin's omega. There is some description at Chaitin's constant. This article only covers the regular Kolmogorov complexity. — Carl (CBM · talk) 02:48, 19 September 2007 (UTC)
thread_title:  Two kinds of Kolmogorov complexity? 

message: The English phrase "The binary representation of 14,439,066,989,718,820,758" has less than 64 characters. Army1987 (talk) 00:19, 21 June 2008 (UTC)
replies: text: Fixed. The new string was made of the lowest 7 bits of bytes from /dev/random, discarding those < 32 or = 127. So, it's very unlikely to be described in less than 64 characters. --Army1987 (talk) 11:05, 21 June 2008 (UTC)

text: I think it would be preferable to rephrase the introductory example in terms of bits, because "character" is so poorly specified (after all, if we're talking Unicode characters, you could probably encode even your random string in half the space). I'm going to attempt this. Dcoetzee 19:22, 21 June 2008 (UTC)
replies: text: In the end I decided binary strings were infeasible because it's really hard to describe anything in 64 bits. So I went with specifying the alphabet more carefully: only lowercase letters, numbers, and spaces are allowed. This ensures that the second example is likely to be an incompressible string, while still providing enough characters for English text. Dcoetzee 19:47, 21 June 2008 (UTC)
thread_title:  Example in the intro 

message: Dear all,
I’m rather a fan of Kolmogorov complexity, than an expert. I believe however that Kolmogorov was on the right track to mathematically describe information and its impact. Unfortunately, there is so much confusion about what is information, especially from IT perspective. And this confusion is reflected in very messy Information article.
I have outlined how the article should present the concept on the related discussion page, but without much of an echo. If there are mathematicians among you who would like to get involved, they will be welcomed.
Kind regards, Damir Ibrisimovic (talk) 04:15, 18 February 2009 (UTC)
replies: text: In pursuit of exactness, mathematicians very often hijack Wikipedia articles away from being understandable or usable by those seeking to understand the topic. Ywaz (talk) 23:23, 19 January 2016 (UTC)
thread_title:  Kolmogorov Complexity & Information 

message: What happens if which divide the Kolmogorov Complexity by the length of the digit, ie : 
K(s)/|s|, where |s| is the size of s (note, it may be that an appropriate notion of entropy density would be arrived at by dividing by some function of s, or by another complexity metric aside from the Kolmogorov complexity).  Do we find that, in some sense, there is a theoretical limitation to the entropy density which we can obtain?  Does this not provide a theoretical limitation upon the degree of complexity which can be attained by both formal systems and also applied systems?  Is the actual maximum entropy density attainable related to the Chaitin constant?  
ConcernedScientist (talk) 18:19, 28 February 2009 (UTC)
thread_title:  Complexity Density or Entropy Density Comment (tentative) 

message: I added a link to my recently created page about Binary Lambda Calculus,
which provides a relatively simple and easy to work with concrete definition
of both plain and prefix complexity.
--Tromp (talk) 14:50, 13 March 2009 (UTC)
thread_title:  Binary Lambda Calculus 

message: the article suggests that the 64 characters
"4c1j5b2p0cv4w1 8rx2y39umgw5q85s7ur qbjfdppa0q7nieieqe9noc4cvafzf"
is a description of itself, which is highly misleading. There absolutely needs
to be an indication that the string is to be taken literally.
How else are we going to describe the string "ab 32 times" ?
Here we also get into issues with quoting. How do we describe (in English) the
string "\"ab 32 times\"" ?
Probably the best thing is to avoid character escapes and use a description like
"the 13 character string "ab 32 times""
Thus, I suggest the example in the article should have description:
"the 64 character string 4c1j5b2p0cv4w1 8rx2y39umgw5q85s7ur qbjfdppa0q7nieieqe9noc4cvafzf"
--216.223.242.211 (talk) 17:28, 13 March 2009 (UTC)
replies: text: I see your point, but I don't think it that much of a problem.  While the example can be made more accurate with a more formal "description language" than English, the example works to get the point across to people who don't really understand this subject. Wrs1864 (talk) 10:59, 16 March 2009 (UTC)
thread_title: Problem with main example

message: Hello, the function GenerateComplexString(int n) will only terminate for any n if K has no upper bound. Of course, there is no upper bound for K, because if there was, every string (out of about aleph-0 strings) could be generated by a program with a length less than or equal to that bound (of which only a finite number exist).  While the lack of proof for the termination of the function is thus just a technicality, it nevertheless requires the reader to think a bit on his own (or, more likely, ignore it). As I am no expert on the topic and would rather not complicate the proof due to a misunderstanding, I will ignore WP:BB and just wait for an expert to verify or refute my concerns. X127 (talk) 07:09, 16 March 2009 (UTC)
replies: text: I think the Pigeonhole principle is pretty obvious, but I guess in the top of the "basic results" section where it discusses the maximum length of K(s), something could be added about this. Wrs1864 (talk) 11:18, 16 March 2009 (UTC)

text: First off, as per WP:NOTFORUM, talk pages are supposed to be for discussing the article itself, not for general discussions on the subject matter.  As far as what can be computed/proved with atypical systems, I think they would be best discussed on those article pages.  E.g., something on Kolmogorov complexity on an Oracle machine, would be best discussed on the Oracle machine page.  (The same goes for all other proofs that may change when you can assume you have an oracle.)  As far as approximations, as discussed in the article, all strings have a Kolmogorov complexity that is at most the length of the string plus a (small) constant, so yes, there is an obvious polynomial approximation. Wrs1864 (talk) 19:26, 4 April 2009 (UTC)

text: Yes, it is true. Actually, Ω wtt-computes the halting problem . — Carl (CBM · talk) 04:11, 1 June 2010 (UTC)
thread_title:  Incomputability of Kolmogorov complexity: termination of GenerateComplexString 

message: This may sound a silly point.  And I may be well off the mark in my understanding of what the 'Incomputability of Kolmogorov complexity' entails.  BUT, HOW incomputable is Kolmogorov complexity? According to the article Oracle machine, even undecidable problems can be solved by an oracle based equipped Turing machine.  So what is the minimal complexity class of decision problems which an oracle attached to a Turing machine (for the purpose of computing the Kolmogorov complexity of numbers) must be capable of solving?  
As a separate point, do there exist Polynomial-time approximation schemes which can determine approximations to the Kolmogorov complexity to within an arbitrary constant value of the actual Kolmogorov complexity (it would seem clear that the answer is no in the sense that having a computed Kolmogorov complexity value within 1% of the actual value would enable the Kolmogorov complexity to be calculated - however, some type of probabilistic polynomial-time randomized approximation scheme might enable Kolmogorov complexity evaluation to some probabilistic bound of the 'answer').  
Are there any fundamental problems with any of this?
ConcernedScientist (talk) 18:19, 4 April 2009 (UTC)
replies: text: First off, as per WP:NOTFORUM, talk pages are supposed to be for discussing the article itself, not for general discussions on the subject matter.  As far as what can be computed/proved with atypical systems, I think they would be best discussed on those article pages.  E.g., something on Kolmogorov complexity on an Oracle machine, would be best discussed on the Oracle machine page.  (The same goes for all other proofs that may change when you can assume you have an oracle.)  As far as approximations, as discussed in the article, all strings have a Kolmogorov complexity that is at most the length of the string plus a (small) constant, so yes, there is an obvious polynomial approximation. Wrs1864 (talk) 19:26, 4 April 2009 (UTC)

text: Yes, it is true. Actually, Ω wtt-computes the halting problem . — Carl (CBM · talk) 04:11, 1 June 2010 (UTC)
thread_title: HOW Incomputable is Kolmogorov complexity/HOW undecidable is the issue of determining the Kolmogorov complexity?

message: Do we need a more formal definition/specification of what qualifies as a description language in this article?  Right now, the article is pretty vague, just using a link to the vague description language article.  Say we have a description language that consists of only Hollerith constants.  We can generate any string, but we also do such things as calculate the exact Kolmogorov complexity for each string (the length string, plus one, plus the log base 10 of the string).  I thought about throwing in a "turning complete" requirement, but realized that maybe the proofs don't require that powerful of a language, for example, a Linear bounded automaton might still work.  Thoughts? Wrs1864 (talk) 02:38, 5 April 2009 (UTC)
replies: text: I think one normally defines Kolmogorov complexity relative to a fixed universal Turing machine U.  That is, first choose U, and then for a given string s, KU(s), the complexity of s, is the shortest program for U that generates s.  Yes, this should be explained more precisely in the article. 69.228.170.24 (talk) 20:36, 31 May 2010 (UTC)
thread_title:  what is a "description language"? 

message: I don't see why MML is refered to in this section. The notation of MML there is so vague it shows nothing about the relation with CIT, I almost erased it, but I'd rather put it in another section for now. --210.231.57.36 (talk) 12:11, 19 July 2009 (UTC)
replies: text: The Wallace/Dowe paper explains it.  69.228.170.24 (talk) 16:42, 29 May 2010 (UTC)
thread_title: Chaitin's incompleteness theorem and "minimum message length" ?

message: The Invariance theorem article is a stub with little promise of expansion. The material could easily be merged here. Also, the term "Invariance theorem" pretty generic and could refer to several theorems in a variety of area. It seem to me that once the merge is complete then the existing Invariance theorem article should be changed to either a redirect to Invariant (mathematics) or a dab page.--RDBury (talk) 22:14, 12 July 2010 (UTC)
thread_title:  Merge with [[Invariance theorem]] 

message: The pseudocode:
   function GenerateComplexString(int n)
      for i = 1 to infinity:
         for each string s of length exactly i
            if KolmogorovComplexity(s) >= n
               return s
               quit
and:
   function GenerateParadoxicalString()
       return GenerateComplexString(n0)
Seem to use two different languages. In the first case "return" means something like "save this value for the calling function but continue executing like normal", which necessitates the "quit" statement directly after. The second example does not have a "quit" statement and instead works like normal programming languages like C, Pascal, Python, etc. 
Suggestion: remove the "quit" statements in all the examples to make the pseudocode function more in line with how "return" is normally (always?) interpreted.
--Boxed (talk) 12:28, 20 January 2011 (UTC)
replies: text: Okay by me. Personally I don't like pseudocode, especially "structured" pseudocode, because it doesn't lend itself to a rapid translation into a flowchart. What I've done is create a "sandbox page" in my namespace, here: User:Wvbailey/Kolmogorov complexity. There you can experiment to your heart's content before inserting it in the article, plus others can go there too if they want and comment.  If you find the need for a more formal algorithm I'd recommend unstructured Basic with line numbers coupled with Knuth's presentation style (see the Euclid's algorithm example at Algorithm). Bill Wvbailey (talk) 16:06, 11 February 2011 (UTC)
thread_title:  Inconsistencies in the pseudocode? 

message: An anonymous editor deleted all mentions of Gregory Chaitin from this article, alleging it to be self-promotion.  Is there some evidence of self-promotion in editing this article by Gregory Chaitin?  Since long before Wikipedia existed, I've heard the name Chaitin associated with this concept.  It seems bizarre to exclude Chaitin from an account of this topic. Michael Hardy (talk) 16:36, 4 February 2011 (UTC)
replies: text: I'm in total agreement with you. I've read his popularization MetaMath! The quest for Omega and learned a bunch from it (and pp. 204-206 it has an interesting Bibliography). Also I have an old source ca 1980's written by Martin Davis that mentions Chaitin prominently -- that was where I first encountered his name (if we need the source I'll produce it + I think it's in a talk discussion a few up from this one. But I don't think we will. We can discount this edit as grafitti). Bill Wvbailey (talk) 18:48, 4 February 2011 (UTC)
replies: text: Chaitin is given to self-promotion (read any of his popular writings to see this) and the article may have gotten into a state that featured his work too prominently compared to that of others.  But eliminating all mention of him is going overboard. 71.141.88.54 (talk) 08:15, 10 February 2011 (UTC)

text: I'd like to shorten the stuff about Chaitin's incompleteness theorem and AIT and add a summary/citation of some of Panu Raatikainen's exposition about those topics.  I'd also like to get rid of the stuff about Mark Burgin and super-recursiveness, which seems irrelevant to the subject of this article.  71.141.88.54 (talk) 08:29, 10 February 2011 (UTC)

text: I have no strong feelings one way or the other, but I worry about self-promotion; we'd have to look at your edits. What I'd suggest is leave the Chaitin material alone but edit out Burgin and add material that you think would benefit the article. But rather than do a major edit as "anonymous", why not reveal yourself first by signing up as an editor? You may notice that Michael Hardey and I aren't hiding behind an IP number. BillWvbailey (talk) 15:21, 10 February 2011 (UTC)
thread_title:  Chaitin 

message: To explain my edit from:
"It can be shown that the Kolmogorov complexity of any string cannot be too much larger than the length of the string itself."
to
"It can be shown that the Kolmogorov complexity of any string cannot be more than a few bytes larger than the length of the string itself."
The former phrase was vague, and might mean a multiplicative factor rather than an additive factor. Any incompressible string can be wrapped in a print function. The  extra length is just the length of the print function, which is a few bytes in any reasonable description language. A description language can be specified that automatically prints any string that starts without a recognized language keyword, so nearly all incompressible strings will be printed without any overhead. The remainder require wrapping with a print function, but on average the overhead will be negligible. Any string that has a significant proportion of description language keywords will not be incompressible. At any rate, a string that is designed to require maximum overhead beyond its own length in a given description language can be represented in another description language with no more than a few bytes to specify the choice of that description language.Enon (talk) 02:49, 26 July 2010 (UTC)
replies: text: Hmmm.  Depending on quoting characters required, it might actually require multiples.  Depending on the description language, it may be possible that the simple description might end up being a multiple.  You may be right that in "a natural" programming language, the shortest description would be a bounded quantity longer than the string, but it isn't obvious, and would require a source.  — Arthur Rubin  (talk) 07:34, 23 September 2011 (UTC)
replies: text: It's true in any universal code because it's true in the trivial code $f$ such that $f(s) =s$ for every string $s$. By the definition of universality, for any effective code there is a constant such that the universal system is no worse than that fixed code plus a constant. So for any universal system used to define $K$ there is a constant $C$ such that $K(s) \leq |s| + C$.  — Carl (CBM · talk) 11:52, 23 September 2011 (UTC)
thread_title: Maximum minimum description length

message: ""namely "ab 32 times", which consists of 11 characters"" according to w.org measuring unit rules should write as
""namely "ab 32 time"s, which consists of 10 characters.
...just an observation
86.121.67.182 (talk) 19:23, 22 September 2011 (UTC)
replies: text: Actually, not, if you're rational.  And, besides, it should be "'ab' 32 times", making it 13 characters, because of the required quotes.  — Arthur Rubin  (talk) 07:34, 23 September 2011 (UTC)
replies: text:  Why not just say "ab×32", which is 5 characters? - dcljr (talk) 20:03, 6 December 2013 (UTC)
replies: text: That would be abbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.  One might accept "'ab'x32" which is 7 characters.  Specific representation schemes do not have any place in this article, though.  — Arthur Rubin  (talk) 02:38, 7 December 2013 (UTC)
thread_title: minimum desciption length

message: After some study, it appears that whether a (rational) complex number z is in the Mandelbrot set may be undecidable, so it may not be a good example.  Perhaps the Koch snowflake?  — Arthur Rubin  (talk) 06:48, 11 August 2014 (UTC)
replies: text: Huh? The image in question was in fact generated by a (presumably small) computer program, so it can necessarily be encoded by a small program. Probably that program has a threshold on the number of iterations it uses before giving up and assuming that the point in question is in the Mandelbrot set. If you have some more precise mathematical definition of the image in mind than "whatever that Mandelbrot program happened to generate" then you need to start worrying about things like anti-aliasing because just going by whether a mathematical point at the center of a pixel is in or out doesn't produce good images. But if you really did want the point at the center of the pixel, you could encode the image precisely by storing a version of the program that hardcoded a good-enough threshold. (Different crops and zooms would require different thresholds, and the threshold itself might not be easy to compute, but we're only talking about existence of a short code so that's not a problem.) —David Eppstein (talk) 07:00, 11 August 2014 (UTC)
replies: text: The image looks complicated, and is computable; however, its relationship to the Mandelbrot set is that of being "close". The formal definition of the Mandelbrot set provides a short description, but it does not necessarily represent something computable.  However, if none of the points is on the boundary, then \left| z_N(c) \right | < 2 represents the set for sufficiently large N; again, we have no algorithm to determine N; log N could be larger than the number of points.  As far as I know, the question of whether a square in the complex plane is contained within the Mandelbrot set is believed undecidable.  If we make it clear that we don't know whether the image is exactly the Mandelbrot set, it's true that it's likely that the complexity of the description of the program is less than that of the raw image, but we cannot be sure.  I still like the Koch snowflake better.  — Arthur Rubin  (talk) 07:25, 11 August 2014 (UTC)
thread_title:  Mandelbrot set 

message: A few attempts have been made to define the Kolmogorov complexity of numbers. https://oeis.org/A168650 defines the complexity of an integer as the length of the shortest expression in C/C++/Java/Python that evaluates to that integer. Also there is a paper about the complexity of real numbers: L. Staiger, "The Kolmogorov complexity of real numbers", Theoretical Computer Science, pp. 455-466, 2002. Should these be added to the article? 118.210.239.185 (talk) 02:00, 29 August 2015 (UTC)
thread_title:  Kolmogorov complexity of Numbers 

message: Hello fellow Wikipedians,
I have just added archive links to  on Kolmogorov complexity. Please take a moment to review my edit. If necessary, add  after the link to keep me from modifying it. Alternatively, you can add  to keep me off the page altogether. I made the following changes:
*Added archive http://web.archive.org/web/20110928032821/http://www.ece.umd.edu/~abarg/ppi/contents/1-65-abstracts.html to http://www.ece.umd.edu/~abarg/ppi/contents/1-65-abstracts.html#1-65.2
*Added archive http://web.archive.org/web/20160131130238/http://www.cs.umaine.edu/~chaitin/ to http://www.cs.umaine.edu/~chaitin/
*Added archive http://web.archive.org/web/20060619060230/http://mitpress.mit.edu:80/catalog/item/default.asp?sid=4C100C6F-2255-40FF-A2ED-02FC49FEBE7C&ttype=2&tid=10478 to http://mitpress.mit.edu/catalog/item/default.asp?sid=4C100C6F-2255-40FF-A2ED-02FC49FEBE7C&ttype=2&tid=10478
When you have finished reviewing my changes, please set the checked parameter below to true to let others know.
Cheers.—cyberbot IITalk to my owner:Online 07:07, 27 February 2016 (UTC)
thread_title:  External links modified 

message: Hello fellow Wikipedians,
I have just modified one external link on Kolmogorov complexity. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20050115091525/http://kolmogorov.com/ to http://www.kolmogorov.com/
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
Cheers.—InternetArchiveBot (Report bug) 01:37, 12 December 2017 (UTC)
thread_title:  External links modified 

message: KC is essentially a formalization of Occam's Razor -- given that, it seems quite odd that Occam's Razor is not even mentioned in the article until the External Links section all the way at the bottom. It seems like the kind of thing that should be in the opening paragraph, or at least somewhere in the text. Or am I missing sth? --2A02:8071:7141:6400:9DC6:E5A6:8DEB:FF7E (talk) 09:21, 26 May 2024 (UTC)
thread_title:  Occam's Razor?