Talk:Arithmetic coding

message: The sentence that reads...
Instead, we represent the sequence as a rational number between 0 and 1 in base 3
Should read...
Instead, we represent the sequence as a rational number between 0 and 2 in base 3
I didn't want to edit the main page, maybe one of you who is more involved can  —Preceding unsigned comment added by 66.244.163.200 (talk) 18:17, 10 April 2008 (UTC)
thread_title: Older stuff not in a section

message: These parts are entitled with “Encoding and decoding: overview” and “Encoding and decoding: example” but the overview part describes the encoding process only while the example part describes the decoding process only. This is very inconsistent and makes it hard for a reader to understand both processes as the encoding remains theory whilst the decoding example uses an input value coming out-of-nothing.  — Preceding unsigned comment added by 77.12.61.89 (talk) 17:39, 16 April 2013 (UTC)
thread_title:  Encoding and decoding: overview and example 

message: I think the entropy estimate in the example is wrong.  Each character seems to contain (-log2(.6) * .6) + (-log2(.2) * .2) + (-log2(.1) * .1) + (-log2(.1) * .1) = 1.57 bits of entropy.  Ignoring the fact that an end of message only occurs at the end (which would reduce entropy of the entire message), a three character message like the example would have an entropy of 3*1.57=4.71. The article says 7.381.
As a check; you could actually encode any three character message of a four symbol alphabet in only 6 bits by simply assigning a 2 bit code to each of the inputs. Our example must have less entropy because the probabilities are skewed away from .25/.25/.25/.25.
Mfinn68916 15:53, 27 March 2007 (UTC)
thread_title:  Entropy number wrong 

message: I think that the 111 is wrong. The binary 0.00110 is the shortest sufficient stream ((5/27 + 6/27)/2 in binary). 
(Of course you cut off the leading 0 and you get 001100 later). I guess the 111 suggests 0.00111 but this would be wrong, too.
Le me explain why 0.00110:
Let's say 1 = NEUTRAL, 2 = NEGATIVE, 3 = END OF STREAM and the string to compress is "123" (as in the article)
No input (Input 0):
You have no clue about any numbers after the dot. In worst case it is 0.1111111... which goes asymptotically towards 1. Therefore
0 => [0,1)
 Input 0.0:
You have no clue about any numbers that follow. In worst case it is 0.0111111... which goes asymptotically towards dec(0.1) = 0.5. Therefore
0.0 => [0, 0.5)
Now we can say that the first word will be 1 or 2 but not 3 which had the interval [0.66,1)
 Input 0.00:
You have no clue about any numbers that follow. In worst case it is 0.00111111... which goes asymptotically towards dec(0.01) = 0.25. Therefore
0.00 => [0, 0.25)
Now we can say that the first word will be 1 for sure. the second word could still be 1,2 or 3, because [0, 0.25) contains [0,1/9), [1/9,2/9) and a part of [2/9,3/9)
 Input 0.001:
You have no clue about any numbers that follow. In worst case it is as before 0.00111111... which goes asymptotically towards dec(0.01) = 0.25. Therefore
0.001 => [dec(0,001), 0.25) = [0.125, 0.25)
Now the second must be 2 or 3 because the interval doesn't contain [0,1/9)
 Input 0.0011:
You have no clue about any numbers that follow. In worst case it is as before 0.00111111... which goes asymptotically towards dec(0.01) = 0.25. Therefore
0.0011 => [dec(0,0011), 0.25) = [0.1875, 0.25)
This is still not sufficient to know the second word because it still contains [1/9,2/9) and a part of [2/9,3/9)
 Input 0.00110:
You have no clue about any numbers that follow. In worst case it is 0.00110111... which goes asymptotically towards dec(0.00111) = 0.21875. Therefore
0.00110 => [dec(0,00110), 0.25) = [0.1875, 0.21875)
Now it is in [1/9,2/9) and the second word is 2
Now if we go for the third word then [0.1875, 0.21875) is also exclusively in [5/27,6/27) = [0.1851...,0.2222...). Therefore we don't need more bits to know that the third word is 3
0.00111 would require a lot more bits by the way because then you would need to shrink [0.2185, 0.25) to [0.2185, 0.2222) to include the word 3
Now we cut the leading 0. because it is always 0. and we get 00111 which is 5 bits!  — Preceding unsigned comment added by 78.34.236.143 (talk) 20:49, 8 April 2012 (UTC)
thread_title:  111 in sources of inefficiency wrong 

message: The article currently mentions range coding (which is a good thing for it to do), but it does seem to support what I understand to be some common misconceptions about range coding in relation to arithmetic coding.
I've already expounded on this in Talk:Range encoding#Range encoding and arithmetic encoding, but the key points are:-
#Range coding seems to be a different interpretation of the same thing, rather than being a different thing itself.
##As I understand it, all range coders (decoders/encoders) are the corresponding arithmetic coders, and vice-versa.  (It's just a matter of what you interpret the actual range/arithmetic code as being.  You could interpret it as being an arithmetic code in the usual way, or you could interpret it as being the common prefix of a range of natural numbers (non-negative integers), in the usual range coding way.)
##Differences in renormalisation are implementational, and there's no reason why byte-based renormalisation can't be applied in implementations of arithmetic coding.
#Although I am not a lawyer, it does seem that the 'myth' that arithmetic-coding-applicable patents don't apply to range coding relies on range coding being different to arithmetic coding.  I strongly suspect it's a myth, a rumour that keeps getting (unwisely) propagated.  But, as I said, I am not a lawyer.  More on this below.
--Simon G Best 17:24, 20 July 2006 (UTC)
thread_title:  Range coding and arithmetic coding 

message: The article says:-
Range encoding, unlike arithmetic coding, is generally believed not to be covered by any company's patents.
Source?  It's a rumour I've come across quite a number of times, now, but I strongly suspect it's a myth (as range coding and arithmetic coding do seem to be the same thing (just interpreted in two, different ways)).  I'm certainly doubtful that it's "generally believed", as there seems to be no shortage of people who look at range coding, and conclude that it's really just arithmetic coding interpreted a different way.  (In my experience, this kind of observation seems to be made, by someone or other, whenever range coding comes up on the net.  (And no, not just by me :o) ).)  "Rumoured" might be better than "generally believed".
I should emphasise, of course, that I am not a lawyer.
--Simon G Best 17:24, 20 July 2006 (UTC)
thread_title: Range coding and arithmetic coding patents

message: (I've added a subsection heading for this, so that it doesn't seem to be part of the preceding subsection on patents.  --Simon G Best 21:38, 20 July 2006 (UTC)
thread_title: Changes to this subsection of the article

message: What common programs/file formats/standards use Arithmetic coding? --Apoc2400 08:18, 10 October 2006 (UTC)
replies: text: bzip (ok, not exactly common, sadly), PAQ, most PPM implementations, DjVu, JBIG/JBIG2, JPEG 2000, H.264/MPEG-4 AVC...  --Piet Delport 01:21, 11 October 2006 (UTC)
thread_title:  Applications 

message: The history of AC would be quite dandy. When it started development, first implementations, how it has developed over the years to improve, who were the major contributors (such as Jorma Rissanen and Glen G. Langdon Jr.), etc. Spodi (talk) 11:41, 30 December 2007 (UTC)
thread_title:  History 

message: I found myself often confuzed when permutations and variations and combinations are all called combinations. (Lets say you have apple, banana and pear. There are 27 variations, 6 permutations and 1 combination of 3 of those 3 fruits.) Is there any particular reason for doing so? 84.16.123.194 (talk) 10:59, 27 September 2008 (UTC)
thread_title:  combinatorics 

message: This article needs to be brought into agreement with the range encoding article.   Arithmetic coding and range coding are mathematically equivalent and range coding was introduced in an article in the year 1979.  Therefore the article should not suggest that the patent situation precludes the implementation of arithmetic coding without licensing.  Only specific implementations of arithmetic coding necessary to implement certain standards are covered by patents.  The general arithmetic coding algorithm however was prior art by the time these patents were issued.  Agalmic (talk) 20:02, 1 November 2008 (UTC)
replies: text: This was mentioned a few times. Without getting an IP lawyer involved, I'm uncertain of just what the scope of the arithmetic coding patent is - I think it might be more accurate to say that its scope hasn't been tested in court. Dcoetzee 22:46, 2 November 2008 (UTC)
thread_title:  Patent Section Incorrect 

message: I sincerely hope that I have not altered this article in any adverse way as pertaining to the technical information contained herein. I am unfamiliar with this subject and would ask someone else more knowledgeable to read my edit, and check that I have done some good. If you judge otherwise, feel free to revert me with no hard feelings.--Paraballo (talk) 03:42, 16 May 2009 (UTC)
thread_title: minor copyediting

message: Some of the links on this page are obsolete or with restricted access, e.g.
an article "An Introduction to Arithmetic Coding, IBM J. RES. DEVELOP. VOL. 28, No 2, March 1984" in "Range encoding" section which requires a valid login 
or
an article "Arithmetic coding" in "References" section which does not seem to be right.
Radekg1000 (talk) 23:48, 25 November 2010 (UTC)
thread_title: Invalid of obsolete links

message: This article is better than the last time I looked, but it could still be a lot better. A couple of things that jumped out at me were:
1) In the introductory paragraph it states "Normally, a string of characters such as the words "hello there" is represented using a fixed number of bits per character, as in the ASCII code" - This page is about a specific, quite advanced form of data compression and does not need to spell out this simple concept. The pages for entropy, lossless compress etc... are linked right beforehand and are enough for uninformed users to follow.
2) In the explanatory section the article states "A more efficient solution is to represent the sequence as a rational number between 0 and 1 in base 3, where each digit represents a symbol. For example, the sequence "ABBCAB" could become 0.0112013" - without any explanation of how this is performed. An example and some detail come much later in the article but I think this idea, which is fundamental to the principle of arithmetic coding, should be explained in much more detail here.  — Preceding unsigned comment added by 91.143.178.131 (talk) 17:09, 19 October 2011 (UTC)
thread_title: Suggestions

message: Could someone look at this equation please, as it doesn't balance. It's in the theoretical limit section. The LHS appears to be correct.
:n \log_2(n) - \sum_{i=1}^A f_i \log_2(f_i) = -\left[\sum_{i=1}^A p_i \log_2(p_i)\right] * n
UKWikiGuy (talk) 23:31, 30 May 2013 (UTC)
thread_title:  Theoretical Limit...  

message: There is lots of going on about this alternative to arithmetic coding, which have similar complexity to Huffman coding, but handles fractional bits of information - have accuracy like arithmetic coding.
Some implementations: https://github.com/Cyan4973/FiniteStateEntropy https://github.com/rygorous/ryg_rans
Blogs: http://fastcompression.blogspot.fr/ http://fgiesen.wordpress.com/2014/02/02/rans-notes/ http://cbloomrants.blogspot.com/
Maybe it should be mentioned somewhere in the article?  — Preceding unsigned comment added by 128.211.178.20 (talk) 21:56, 4 February 2014 (UTC)
replies: text: No.  This is blatant self-promotion; see Talk:Huffman coding#Asymmetric numeral systems (ANS) - combining speed of Huffman with accuracy of arithmetic?.  Additions about this to other articles violated WP:SOAP, WP:FORUM, WP:NOR, WP:RS, and WP:notability. Calbaer (talk) 00:14, 7 June 2014 (UTC)
thread_title:  Asymmetric numeral systems (ANS) - combining speed of Huffman with accuracy of arithmetic? 

message: "The cumulative frequency is the total of all frequencies below it in a frequency distribution (a running total of frequencies)."
What does that even mean? it's already listed under total frequency, yet the values are different...
Can we get someone who isn't trying to jerk off their ego by using so much jargon to rewrite the article?  — Preceding unsigned comment added by 2601:405:4202:C7F0:E0FA:1C06:932F:9C6D (talk) 20:12, 4 September 2016 (UTC)
replies: text: Where is the Jargon?  A Shortfall Of Gravitas (talk) 18:15, 4 September 2018 (UTC)
thread_title:  What does this even mean?! 

message: Hello fellow Wikipedians,
I have just modified  on Arithmetic coding. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20030228094950/http://planetmath.org:80/encyclopedia/ArithmeticEncoding.html to http://planetmath.org/encyclopedia/ArithmeticEncoding.html
When you have finished reviewing my changes, please set the checked parameter below to true or failed to let others know (documentation at ).
Cheers.—InternetArchiveBot (Report bug) 21:58, 17 October 2016 (UTC)
thread_title:  External links modified 

message: Hello fellow Wikipedians,
I have just modified 2 external links on Arithmetic coding. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20071222053346/http://www.inference.phy.cam.ac.uk/mackay/itila/book.html to http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
*Corrected formatting/usage for http://planetmath.org/encyclopedia/ArithmeticEncoding.html
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
Cheers.—InternetArchiveBot (Report bug) 02:16, 9 July 2017 (UTC)
thread_title:  External links modified 

message: Hi,
after working on radix based stuff for a while, I just noticed this example and that it is actually wrong. The radix wouldn't be chosen based on the size of the string (just check, that any value from 4 to 5 in the thusly implied 6 symbol alphabet would never be used. The radix should actually be 4 (or exactly 2 bits). So the radix encoded value of the string "DABDDB" would be 6*2 = 12 bits.
Best regards,
Dresdenboy (talk) 09:30, 28 August 2020 (UTC)
thread_title:  Error in radix example in "Arithmetic coding as a generalized change of radix" 

message: I'm surprised to not find Rich Pasco's name in the article at all.  His 1976 dissertation laid out the algorithms.  Most (or at least many) works on arithmetic coding attribute it him, along with Rissanen who published part of it earlier in 1976.  Pasco referenced Rissanen saying "One algorithm of the family was developed independently by Rissanen
[1976]. It shifts the code element to the most significant end of the accumulator, using a pointer obtained by addition and exponentiation. We shall now compare the alternatives in the three choices, and see that it is preferable to shift the code element rather than the accumulator, and to add code elements to the least significant end of the accumulator." I'll work on adding something about that. Dicklyon (talk) 22:47, 11 September 2020 (UTC)
thread_title:  Pasco