Talk:Algorithmic information theory

message: Here are slightly edited version extending Pexatus's proposed sentences.  Are they OK with everyone?
 
 Unlike classical information theory, algorithmic information theory gives a formal, 
 rigorous definition of a random string. (The set of random strings depends on the choice
 of the universal Turing machine used to define Kolmogorov complexity, but any choice
 gives identical asymptotic results because the Kolmogorov complexity of a string is
 invariant up to an additive constant depending only on the choice of universal Turing  
 machine.)
 Ω is a real number which expresses the probability that a self-delimiting
 universal Turing machine will halt when its input is supplied by flips
 of a fair coin.  Athough Ω is easily defined, it is impossible to determine 
 infinitely many of its binary digits in any consistent axiom system; this 
 result is similar to Gödel's Incompleteness Theorem in establishing the
 unprovability of certain statements in formal theories.  Although the digits of Ω 
 cannot be determined, many properties of Ω are known; for example, it is an 
 algorithmically random sequence and thus its binary digits are equidistributed.
 From the point of view of computability theory, Omega is of the same Turing degree
 as the halting problem.
Pexatus:  thanks for pointing out algorithmically random sequence.  I would never have found it with that name.
They are called 1-random reals in computability theory. CMummert 12:05, 20 June 2006 (UTC)
replies: text: Looks good. I personally think that it is easier to see that Ω is uncomputable because access to it allows one to solve the halting problem, but since I'm not a logic guy, I guess I have a different notion of what is easy to see. I would remove the part about its digits being equidistributed (i.e., it is simply normal), as that might give the false impression that that is the definition of a random sequence. At some point, self-delimiting Turing machine should be given its own entry and linked from here. (a Turing machine with a read-only input tape whose read head may only move to the right, and which is defined such that its computation is invalid if the read head is not on the final character of the input sequence when the machine halts).  Pexatus 18:35, 20 June 2006 (UTC)
replies: text: Thanks for the feedback. I'll cut the equidistributed part (the whole clause starting with and thus).  The Turing degree is the easiest way for me to see that Omega is uncomputable, as well, although I don't know any really easy proof.  CMummert 22:49, 20 June 2006 (UTC)
replies: text: I edited the page using these suggestions as guidelines, since it's been two years and no action had been taken. - skeptical scientist (talk) 16:29, 9 July 2008 (UTC)
thread_title:  OK? 

message: The article now states "This use of the term "information" might be a bit misleading, as it depends upon the concept of compressibility."
Is this fair? I'm not sure if it does depend on compressibility. In the case of infinite strings, there are equivalent definitions using betting strategies and constructive null sets which make no mention of compressibility, and even in the finite case there is an equivalent definition using termgales. Furthermore, I think there are good reasons to use compressibility and information interchangeably, as different representations of the same piece of data should have the same information content, and no piece of data can contain more information than its most compact representation. The use of the term "information" may be misleading, but I think this has more to do with the distinction between "information" and "useful information" which is described elsewhere. skeptical scientist (talk) 14:38, 21 July 2008 (UTC)
thread_title:  "Information" misleading, means compressibility? 

message: IMHO, useful information refer to an economical concept, not a computer science one. First, there is a cost associated to processing information, and there is a trade off to make with space / time / energy. Kolmogorov Complexity is the minimum size, given you are willing to spend whatever amount of your time computing, it's an extrema of a space/time function. A random sequence could be seen as a fully (lossless) compressed one, therefore it cost time and energy to uncompressed it before using it somehow. The operation may be far from the optimal cost, so useless. Second, the information could have no utility because it can't be processed, like a text written in Chinese is useless to me alone. All of this utility consideration are relative to a context unrelated to the theory.77.192.56.161 (talk) 10:33, 19 October 2010 (UTC)
thread_title:   Utility is an economical concept 

message: The last section, in particular the last paragraph needs references : 
"AC, AP, and AR are the core sub-disciplines of AIT, but AIT spawns into many other areas. It serves as the foundation of the Minimum Description Length (MDL) principle, can simplify proofs in computational complexity theory, has been used to define a universal similarity metric between objects, solves the Maxwell daemon problem, and many others."  — Preceding unsigned comment added by 2A02:2788:925:E21B:91A9:A15:BAF:C904 (talk) 23:37, 17 February 2021 (UTC)
thread_title:  [[Lack of references]]

message: The Scholarpedia article on this topic was reviewed by Paul Vitanyi, which is sufficient to make it a notable and (at least when Vitanyi reviewed it) reliable resource. Thus, I have restored it as an external link.  Kiefer.Wolfowitz  (talk) 12:37, 10 February 2011 (UTC)
thread_title:  [[Scholarpedia]] 

message: There appear to be at least two outstanding errors in this article
http://en.wikipedia.org/wiki/Algorithmic_information_theory
First, 
'From this point of view, a 3000 page encyclopedia actually contains less information than 3000 pages of completely random letters'
is not completely true.  In point of fact one can plot words or the sounds of a word made up of letters, to determine if said sounds represent actual language, versus pure jibberish.   Therefore, written language cannot be said to contain random information, such as random letters.   Even if you jumbled the letters of the encycopedia, then plotted their numbers, specific letters would occur periodically more often than others based on use in language, therefore they are not random.   
Secondly, 
'1100100001100001110111101110110011111010010000100101011110010110   presumably has no simple description other than writing down the string itself'
This above statement is also not entirely true.   A program or algorithm can convert the binary numbers to octal, hexadecimal, base-10, or any other base numerical system, including expressing the above string of 1's and 0's in base 2^n groupings, for example.   Such alternate representation can also be algorithmic expression of the binary string.  — Preceding unsigned comment added by Contributions/ (talk) 14:14, 2 August 2013 (UTC)
thread_title:  [[Other errors]]