Talk:Huffman coding

message: hi! i think the example in the basic example, the one in the captation of the image is wrong. it shoud be
a3	110
a4	111
and not:
a3	111
a4	110
both the image is wrong and the table. i think one can't mix the way the 0/1 is set. PedroCarvalho (talk) 06:23, 28 November 2007 (UTC)
replies: text: well, you can mix the way 0/1 is set, the output code will be good anyway, you just have to remember it when you decode it! anyway, you'd better not to change the way 0/1 is set, so I have fixed both image and caption. Alessio Damato 13:59, 4 December 2007 (UTC)
replies: text:  it still mixes the way 0/1 is set. In the first split it gives 0 to the least common (a1: 0.4), and 1 to the most common ({a2,a3,a4}: 0.6), but in the two other splits is gives 0 to the most common and 1 to the least common. -Thomas Nygreen (talk) 09:59, 9 May 2008 (UTC)
replies: text: Huffman code generators can select 0/1 in any manner they choose (including randomly) and still generate a valid Huffman code. Nevertheless the diagram ought to be consistent with the pseudocode. Dcoetzee 16:30, 9 May 2008 (UTC)
replies: text:  Exactly like it says in the text below it, the hoffman coding should be done from left to right, taking the lower probability each time. This is NOT what the picture shows, so it is clearly wrong. Yomach (talk) 06:27, 26 June 2010 (UTC)
replies: text:  Isn't the frequency for the 'f' character also wrong? Should be 2 not 3? Moulsonp (talk) 10:54, 11 May 2012 (UTC)
thread_title:  basic example image is wrong? 

message: The algorithm in "Basic technique" is ambiguous. It's not clear how the two queues are used. It seems that the second queue is never read from, only added to.
[BREAK INTO DISCUSSION: 30 July 2007 - Simply does not compute - I have been trying to make sense of this 'pseudo code' for a couple of hours now. I am no slouch, but I just can't get the algorithm from this description. Somebody - please post something more intelligible, then remove this comment. - TropicalCoder (Google me) EXIT DISCUSSION]
~
I believe I may have better solved the example "Sample-1".
I may be wrong, but I found the algorithm ill explained in my notes so I visited wikipedia to better understand it... I tried to solve it in the most logical way to me... building a complete binary tree with apropriate depth and using the percentage costs as guides to choose which boundaries to choose.
What I got was this:
There is 1 character with weight 30; 1 with weight 20; 3 * 15; 1 * 10 and 3 * 5 for a total sum of 120.
Dividing all by 120 and using the percentage to define how much of the tree to take away you get that 30/120 = 25%... and you go on like that... using this technic I got:
Character	Weight	Coding
h        	30    	00
e        	20    	010
b        	15    	011
d        	15    	100
g        	15    	101
a        	10    	1100
c        	5     	1101
f        	5     	1110
i        	5     	1111
Total cost is exactly the same, but the coding varies in size between 2 and 4 bits, and in #Basic_technique it says that "It is generally beneficial to minimize the variance of codeword length."
I believe that makes my solution better, so if anyone with more than a mere day's knowledge on the matter agrees, then by all means make the correction :) --nunocordeiro 02:02, 11 July 2006 (UTC)
replies: text: You got the right answer for the wrong reasons.  You used Shannon-Fano coding, which in this case happens yield a result identical (in terms of each symbol's weight) to the bottom-merge Huffman code, the Huffman code with the lowest variance and lowest maximum length (among optimal codes).  In general, these are not the same, and Shannon-Fano is suboptimal.  The example should be replaced by one that either yields only one Huffman code (again, in terms of symbol weights, so {0,1} is the same code as {1,0}) or explain bottom-merge Huffman coding.  I'll put this on my to do list, but if someone wants to fix this, go ahead. Calbaer 02:10, 27 July 2006 (UTC)

text: Oh no!  This is wrong, wrong, wrong (in the context of the article).  For most applications, costs are probabilities, pure and simple.  But they need not be, because they are just multiplicative factors (or, as the Basic technique section has it, weights) for the expression to be minimized.  Adding to the confusion, the Huffman coding with unequal letter costs subsection uses cost in the way you two and the linked paper are discussing, as output costs!  It seems as though an information theorist wrote most the article but a computer scientist wrote the Problem definition section.  Anyway, I think most people who read the entire article would come away very confused.  Looks like it's time for a rewrite (or at least a find/replace of the word 'cost').... Calbaer 02:22, 27 July 2006 (UTC)
thread_title:  older 

message: The Huffman coding is not one-to-one for finite messages. Suppose the message consists completely of the same character (e.g. a file full of 'aaaaaaaaaaaaaaaaaaaa...'); then that character would be assigned the empty bitstring; and the message would encode to the empty string. But such messages of different lengths would all encode to the empty string, hence the information about the length is lost. If you think about decoding such a message, you would also see the problem: you have an empty encoded message, but you also have a symbol whose code is the empty bitstring, so you don't know how many characters to write out.
This problem does not occur for infinite messages, of course; since there is only one infinite message consisting solely of the same character, for any given character.
Perhaps this should be mentioned in the article? --Spoon! (talk) 04:18, 12 January 2008 (UTC)
replies: text: This is not entirely correct.  The weights in the Huffman algorithm represent probabilities, not observed frequencies: so you can only assign the empty bitstring to a character if you know that such character will occur with certainty.  Hence this is a degenerate case, where the only information being transmitted is the length of the string.  In practice, if all you have is finite sample data, then you need to use something alike to Laplace's rule of succession to assign probabilities.  —Preceding unsigned comment added by Classicalecon (talk • contribs) 11:50, 11 February 2008 (UTC)
replies: text: Another thing to keep in mind is that the empty string is not part of a valid Huffman code.  A valid Huffman code has no codeword of length less than one.  If the most likely outcome is more than 40% likely, its codeword will only have one bit, but it cannot have less, even if it is 99.9999% likely.  If it is 100% likely — or even just much more likely than 50% — then a practical compressor wouldn't use a Huffman code per symbol, but perhaps instead a run-length encoding on the entire sequence.  If there are relatively few possible sequences, that run-length code could be a Huffman code. Calbaer (talk) 22:27, 11 February 2008 (UTC)
thread_title:  not one-to-one on finite messages 

message: The section on length-limited Huffman coding needs to be clarified, since finding the optimal length-limited code given the initial weight distribution is trivial.  In what sense is this an unsolved problem?  —Preceding unsigned comment added by Classicalecon (talk • contribs) 10:56, 11 February 2008 (UTC)
replies: text: How is it trivial?  If the Huffman code has a length longer than the length limit, we have to throw away this code and find the optimal length-limited code using a different method, e.g., the Package-Merge algorithm.  It might be trivial to find a length-limited code in general — a fixed-length code certainly limits length — but to find an optimal length-limited code is another matter.  No method is known to solve this problem in linear or logarithmic time. Calbaer (talk) 22:27, 11 February 2008 (UTC)
replies: text: The article section currently claims that it isn't linearithmic, but I don't see how that's the case. The package-merge algorithm finishes in O(nL) time. But if L is greater than log n in base phi, code length is not a limiting factor. (See Witten, Moffat, and Bell, Managing Gigabytes, , pp. 345-350, for a proof.) Otherwise, L ≤ log n, and O(nL) ≤ O(n log n). So why isn't that linearithmic? --Damian Yerrick (talk | stalk) 21:21, 24 April 2012 (UTC)
replies: text: Because Huffman coding is used where the weights/probabilities/frequencies are not all identical.  If the probability of the jth item were 2-j, then the optimal (length-unlimited) code would have two codewords of length n-1.  If you wanted to limit lengths to n/2, then the Package-Merge method would take O(n2). Calbaer (talk) 20:33, 6 June 2014 (UTC)
thread_title:  Length-limited Huffman coding 

message: In the article it describe the basic technique like so:
   1. Start with as many leaves as there are symbols.
   2. Enqueue all leaf nodes into the first queue (by probability in increasing order so that the least likely item is in the head of the queue).
   3. While there is more than one node in the queues:
         1. Dequeue the two nodes with the lowest weight.
         2. Create a new internal node, with the two just-removed nodes as children (either node can be either child) and the sum of their weights as the new weight.
         3. Enqueue the new node into the rear of the second queue.
   4. The remaining node is the root node; the tree has now been generated.
However, it never says what the second queue is used for! From what I understand shouldn't this node be inserted back into the first queue and the queue sorted? I'm having real problems working out how to generate the Huffman tree (since it differs from a normal binary tree as the highest weight should only be a single bit).  —Preceding unsigned comment added by 88.107.145.7 (talk) 19:30, 6 May 2008 (UTC)
replies: text: This description is correct but confusing. The step "Dequeue the two nodes with the lowest weight" is implicitly examining the fronts of both queues. A conceptually simpler implementation uses priority queues, but is not linear time. I will attempt to clarify the exposition. Dcoetzee 19:35, 6 May 2008 (UTC)
thread_title:  Basic technique? 

message: I think a more programming-focused version of the pseudocode should be added to the article, as the current explanation is a bit difficult for programmers to implement and somewhat ambiguous. Archer1742 (talk) 10:29, 26 March 2009 (UTC)
thread_title:  Alternate Pseudocode 

message: Please explain your reason for mass removal of links from this article. Please do not simply refer to WP:LINKFARM. Explain why each link you removed was bad.Biophys (talk) 14:56, 9 April 2009 (UTC)
thread_title: Removal of links

message: Hey there, this is not 100% clear is it? Huffman avoided the major flaw of the suboptimal Shannon-Fano coding by building the tree from the bottom up instead of from the top down. So was the flaw created "by building the tree from the bottom up", or was the "bottom up" way the good method? Heh, if you have no idea about H. C. at first, you might be more confused than you were at the beginning. -andy 217.50.52.128 (talk) 15:54, 27 October 2012 (UTC)
thread_title:  Ambiguity alert 

message: Nuf' said.  — Preceding unsigned comment added by 99.245.3.15 (talk) 10:49, 25 August 2013 (UTC)
thread_title:  Second paragraph is poorly worded 

message: My removal of the external links, none of which appear to meet our guideline on external links was reverted.  Can I ask how the remaining links meet that guideline? Thargor Orlando (talk) 17:05, 21 January 2014 (UTC)
thread_title:  External links 

message: Is it my imagination, or does the image in the Problem definition section (:File:HuffmanCodeAlg.png) not match the Example in the Problem definition section? (Yet they use the same symbols a, b, c, d, and e, inviting the reader to try to correlate them.) —Steve Summit (talk) 12:41, 19 June 2015 (UTC)
thread_title:  Examples don't match? 

message: I removed the See also section and most External links.  WP:ALSO states:
:Editors should provide a brief annotation when a link's relevance is not immediately apparent, when the meaning of the term may not be generally known, or when the term is ambiguous.
This was not done here, except for Modified Huffman coding, which I can restore if desired.  However, its article is short and not very illuminating, and that article's external link doesn't indicate how the code was created, so the link might not be useful as is. Others were to items already in the article or technologies that happened to use Huffman coding and have the syllable "Huff" in them.  For example, Huffyuv seems similar to Lossless JPEG and JPEG-LS in its use of Huffman coding methods.  That makes its inclusion arbitrary, which is true of most of the section; better to leave it out and have links inline where appropriate.
Of the three external links, one was broken and another was to an open source library.  Only the third - to readable examples of code - seemed useful.  As WP:EL states:
:Some acceptable links include those that contain further research that is accurate and on-topic, information that could not be added to the article for reasons such as copyright or amount of detail, or other meaningful, relevant content that is not suitable for inclusion in an article for reasons unrelated to its accuracy.
:Some external links are welcome (see § What can normally be linked), but it is not Wikipedia's purpose to include a lengthy or comprehensive list of external links related to each topic. No page should be linked from a Wikipedia article unless its inclusion is justifiable according to this guideline and common sense. The burden of providing this justification is on the person who wants to include an external link.
I kept the link I believe satisfied this, being instructional and (wiki) collaborative, i.e., not just a single person trying to get a page they made linked from Wikipedia.  The section should be short with only truly useful links, to keep it from devolving to the link dump it was before this change. Calbaer (talk) 14:55, 8 June 2017 (UTC)
thread_title:  Remove most External links, all See also 

message: Under the n-ary Huffman coding section, it is mentioned that "The same algorithm applies as for binary (n equals 2) codes, except that the n least probable symbols are taken together, instead of just the 2 least probable." - but is that really the case?
Let's look at a simple example: trinary alphabet (n equals 3), with four symbols (call them w, x, y and z, in increasing order of probability). The algorithm as described will group w, x and y under a single node, and then (supposedly, as there are only two nodes remaining, not three, so this isn't explained explicitly, but I think this is the reasonable way to understand it) that new node and z under one more new node, which ends up as the root:
     root
     /  \
   wxy   z
  / | \
 w  x  y
So the root only has two children (which is OK, the article does mention the possibility that "additional 0-probability place holders must be added"). But surely this can't be the optimum, as better compression would be achieved by moving one of the first three symbols (optimally y) to occupy the free position directly under the root, rather than its current, deeper position:
     root
    /  | \
   wx  y  z
  /  \
 w    x
This prompted me to look at Huffman's paper (http://compression.ru/download/articles/huff/huffman_1952_minimum-redundancy-codes.pdf as referred to from the article), where I think the issue is actually addressed, by saying (left column of the fourth - and last - page, which is page 1101) "... it will be required that all these brackets, with the possible exception of one combining the least probable messages of the original ensemble, always combine a number of messages equal to D." (emphasis mine).
I think the article should be fixed to reflect this and to explain the algorithm more clearly. And I'd rather leave this task to someone more proficient than myself (but I might do it myself eventually if I notice in some time that this hasn't been addressed). 95.90.235.121 (talk) 12:00, 29 December 2020 (UTC)
thread_title:  ''n''-ary Huffman coding 

message: What it's called Huffman tree it's called in bibliography as Kraft Tree--138.4.177.124 (talk) 15:20, 16 February 2021 (UTC)
thread_title:  ¿Huffman tree? 

message: The applications section needs some work. The first paragraph compares Huffman codes to Arithmetic coding (see the rest of the drama in this talk page), which is not an application. The second paragraph describes some compression algorithms that use prefix codes which are specifically described as NOT Huffman codes.
So... the applications section has no applications in it.
J2kun (talk) 02:00, 4 May 2021 (UTC)
replies: text: Gof for it. Prefixed codes are not huffman codes for sure (judging by papers that I've seen so far). AXONOV (talk) ⚑ 15:14, 3 December 2021 (UTC)
thread_title:  Better applications section? 

message: The definition speaks of sets, but doesn't say finite.  Does the tree construction still work out OK for infinite alphabets and trees of infinite depth?  Clearly one can't work from the leaves combining least-probably nodes if there are infinitely many.  See Golomb code, the last part of this edit where I'm not so sure about what I said. Dicklyon (talk) 16:07, 12 July 2021 (UTC)
thread_title:  Finite vs infinite alphabet 

message: Current § Problem definition subsection is overly abstract. I got a paper described in much simpler terms how to get huffman code for a given set of symbols (probability values). No hard math involved, take a look: Huffman Offline.pdf  AXONOV (talk) ⚑ 15:18, 3 December 2021 (UTC)
thread_title:  Simpler explanation 

message: I think there´s some kind of "mistake" in the description: "this code requires 135 (or 147) bits, as opposed to 288 (or 180) bits if 36 characters of 8 (or 5) bits were used.". There are only 16 sympbols listed in the table given below the tree-graphics, so for a FAIR comparison, only 4 bits are required to binary code 16 different symbols. Please don´t compare most wasteful ASCII-Coding with the Huffman Code, please don´t limit Huffman Coding to text-data! 
If it comes to the GAIN everybody is greedily looking for, it´s 135 bits used by huffman-coding (without codebook and decoder) versus 36 chars * 4 binary code bits = 144 bits (NOT 288 or 180 given in the description!).
I know it does look much less impressive, if you honestly say "only 144-135=9 bits gain in this example, and overhead for codebook and decoder is even discounted!".
I ask for a fair comparison, because everybody should face the truth how hard it is to find powerful algorithms for data compression.
Don´t try to make Huffman Coding better than it is, don´t feed the "magic function theory".
And PLEASE also show an example where beloved Huffman Coding clearly FAILS!  — Preceding unsigned comment added by 84.115.238.220 (talk) 05:47, 3 June 2023 (UTC)
thread_title:  Wrong bit quantity/comparison in the example at the beginning