Talk:Lempel–Ziv–Welch

message: the article says: "There are thus 26 symbols in the plaintext alphabet (the 26 capital letters A through Z), plus the stop code #".
seems like it's suppose to be "There are thus 27 symbols..." , isn't it?  — Preceding unsigned comment added by 109.66.119.64 (talk) 14:31, 16 December 2013 (UTC)
replies: text: No.  The stop code is not used as a symbol; the algorithm handles it differently than plaintext symbols. -- Elphion (talk) 23:31, 16 December 2013 (UTC)
thread_title:  counting mistake? 

message: The introduction to the article incorrectly states that LZW is derived from LZ77, when it's really derived from LZ78. I'll change this when I come up with a decent wording. --Kgaughan 22:12, 27 May 2005 (UTC)
thread_title:  Incorrect derivation 

message: I've just added redirects to here from both Ziv-Lempel-Welch and ZLW. --Heavyphotons 07:55, 27 July 2005 (UTC)
thread_title:  redirects 

message: It seems like this page is very, very lacking in detail about the workings of the LZW algorithm (Description of the algorithm).  Any other thoughts? Stack 16:17, 28 July 2005 (UTC)
thread_title:  Lack of detail 

message: The article claimed that the IBM US patent expired on 11 Aug 2006, which is seven months in the future.  I'm not sure when it will expire or if it has already (based on the patent I'd say it expired in 2005), so I removed the claim.  If someone can verify that, I'd appreciate it.  --DCrazy talk/contrib 18:39, 17 January 2006 (UTC)
replies: text: Just caught the comment above.  Seems there is a bit of dispute on this point, unless I'm misunderstanding it.  --DCrazy talk/contrib 18:41, 17 January 2006 (UTC)
thread_title:  IBM Patent 

message: I think the Python example should be translated to pseudocode by someone who knows how.  Not all readers understand Python.
Also, the given python code is not entirely clear.
Additionally, I believe the given pseudocode is overly simplistic; I did a direct translation (and I mean "my code looks exactly like the pseudocode" direct), and it failed on "xxx".
This is a known issue in the simplistic representation of LZW which is indicated at http://marknelson.us/1989/10/01/lzw-data-compression/.
If I get some time, I'll rework my code or clean up the given python to look more pseudocodey, then post that.
I have cleaned up the Python code a little bit:
* removed unnecessary class
* put doctsrings at correct place (after the def line)
* removed unnecessary and "unpythonic" type checks
* moved utf-8 encoding to the end of compress()
* removed that ugly __join() by using the unicode strings directly as keys
* some minor clean ups
The result can be seen here: http://paste.pocoo.org/show/4497/
Any objections against replacing the existing code in the article? Marrin 10:24, 25 September 2007 (UTC)
thread_title:  Python -> pseudocode 

message: Which pseudo code failed on "xxx"? I tested the algorithm for compression and decompression on xxx and it worked!
I think the last line of the algorithm:
done
   output the code for w;
is not correct -- I implemented the algorithm and the resulting code had a duplicate of the penultimate code and no final code.
I suggest the last line should be:
done
   output the code for wk;
158.48.32.2 12:49, 4 August 2007 (UTC)
thread_title:  failure on xxx 

message: That table that was deleted on Nov. 13th was the single most useful thing on the page for understanding how the LZW worked. Deleting REALLY hurts how easy it is to understand the LZW for a novice user. 68.207.120.151 (talk) 00:39, 15 November 2008 (UTC)
replies: text: It was removed by User:Thumperward at 13:25, 13 November 2008 in this edit with edit summary "Algorithm:  the actual algorithm, rewritten in English and not pseudocode, might be useful. this barrage of university homework is not." I think it would be useful to describe the algorithm in a general way, but it would also be helpful to have an illustrative example. Could I invite Thumperward to expand on their justification? Dcoetzee 02:16, 15 November 2008 (UTC)
replies: text: I have added examples of LZW encoding and decoding at GIF of a simple image. Cuddlyable3 (talk) 16:13, 23 November 2008 (UTC)
thread_title:  table deletion 

message: The article doesn't mention this but clear code handling is especially arcane. After a clear code some 'padding' bits are expected to follow. And it is not padding to the next byte boundary, but rather to a boundary that is a multiple of the clear code's length multiplied by eight. The sourceforge ncompress project (based on original BSD sources), and the lzw decompressor from GNU gzip both contain impenetrable bit buffer positioning logic with what appears to be the possibility of seeking backwards into the bit stream under some conditions. If anyone has more information on the padding expected after a Clear code, please ammend this article! DLeonard (talk) 21:15, 12 May 2009 (UTC)
replies: text: The clear code is mentioned as one of the special codes in the Encoder and Decoder diagrams. The article is not exclusively about the implementation of LZW coding in the GIF file format but should be consistent with it. A GIF encoder may deliver a clear code at any time. Its effect is to reset the code table to root+special codes only and to reset the packing of code into bytes. Packing varies during encoding as the table builds with longer codes so that only the essential number of bits is stored.Cuddlyable3 (talk) 06:35, 13 May 2009 (UTC)
replies: text: I know of no requirement in the algorithm itself to add padding after a clear code.  That would certainly break GIF encoding, which treats the clear code as just another code. It resets the code width back to the original value as Cuddlyable3 indicates, but inserts no padding, and there's no particular point to inserting any padding.  (Of course there are many flavors of LZW; as long as the encoder and decoder agree on the details things ought to work out.) Elphion (talk) 02:50, 6 November 2009 (UTC)
thread_title:  Clear codes 

message: I find the diagrams incomprehensible and unhelpful.  Can we remove them, or explain them better?  --Doradus (talk) 16:13, 14 July 2009 (UTC)
replies: text: The diagrams should be kept until improvements are made. Anyone who understands the article is free to do that. Cuddlyable3 (talk) 08:28, 5 August 2009 (UTC)
replies: text: I respectfully disagree.  There are times when no diagram is better than a poor one.  --Doradus (talk) 15:48, 13 August 2009 (UTC)
replies: text: Is your plan to criticize or to contribute material ? Cuddlyable3 (talk) 19:42, 13 August 2009 (UTC)
replies: text: I don't see how my plans are relevant here, but since you asked, it is my plan to improve the article.  In this case, I was trying to do so by removing a confusing diagram.  --Doradus (talk) 17:05, 25 August 2009 (UTC)

text: I didn't realize until now that you are the creator of this diagram.  I apologize if I have been brusque.  But your pronouncement that "diagrams should be kept until improvements are made" is a personal opinion; it is not Wikipedia policy, and it doesn't in any way explain how you believe my edit harmed the quality of the article, so it is not a reasonable justification for reverting my edit.  --Doradus (talk) 19:27, 25 August 2009 (UTC)

text: I agree. The diagrams are incomprehensible and add no value to the article. 81.152.169.67 (talk) 16:21, 28 August 2009 (UTC)
thread_title:  Diagrams 

message: IP 124.124.105.162 left in the article the following question, which I have moved here:
:someone plz give an explanation for "Encoded length   = 6*5 + 11*6 =  96 bits."..how did 6 and 11 come?
Answer:  in the example, 6 codes were emitted with width 5 bits, and 11 codes were emitted at 6 bits.  (Just count the codes under "Output -- bits".) -- Elphion (talk) 06:10, 30 April 2010 (UTC)
replies: text: Hmm. When I count the number of codes built in the extended dictionary, I see codes 27-31 added.  These numbers are encoded in 5 bits.  So wouldn't that be 5 5-bit codes? Likewise, the 6-bit codes are 32-41.  So wouldn't that be 10 codes?  — Preceding unsigned comment added by Netskink (talk • contribs) 12:23, 22 May 2017 (UTC)
replies: text:  No, the "encoded length" refers to the number of bits sent in the output, not to codes in the dictionary (many of which will not be transmitted).  The point is the degree of compression of the encoded text, not the details of the compression machinery. -- Elphion (talk) 13:04, 22 May 2017 (UTC)

text: This is perhaps the source of the misunderstanding:  when the encoder adds code 32 to the dictionary, it issues all following codes with 6 bits rather than 5 bits even though the codes it transmits may not require the wider width.  That is, it increases the code width immediately after generating a wider code in the dictionary, not later (when it first transmits a code requiring the wider width).  This is required so that the decoder can keep track of the width of the codes:  how is it to know when code width increases?  Answer:  when the decoder itself generates a wider code in its own dictionary, which it generates in parallel with the encoder. (Added: There's a subtle difference:  since the decoder is always one step behind the encoder, it increases the code width from p to p+1 when it generates the last code of width p, not the first code of length p+1, as the encoder does.) -- Elphion (talk) 14:15, 22 May 2017 (UTC)
thread_title:  Encoding question 

message: The second sentence of this article on 01-jan-2011 says: "The algorithm is designed to be fast to implement but is not usually optimal because it performs only limited analysis of the data". This got flagged for citation needed, but I am taking it out. The LZW implementation of LZ78 is known to  asymptotically optimal, so an examination of its performance needs to be a bit more nuanced than this.
A good discussion of performance of LZW wrt other compressors would be great, and a rewrite of that second sentence would be nice too.
--Snorkelman 12:54, 21 January 2011 (UTC)
thread_title:  Not Optimal 

message: The following paragraph was added to the article by user:One.Ouch.Zero.  It's not entirely clear how the convention is supposed to work; and more importantly I've never seen this implemented.  I think we need some documentation before including it (including formats that use it).  -- Elphion (talk) 20:27, 19 January 2012 (UTC)
thread_title:  Putative "Late Change" convention 

message: http://www.stevenpigeon.com/Publications/publications/LossyLZW.pdf talks about at least three ways of maximizing compression by allowing lzw to introduce small errors. Should we include that in the article?
--Peterpall (talk) 19:21, 22 July 2014 (UTC)
replies: text: Neither the scheme introduced in the paper (which clearly has not been through any editing process, let alone a reviewing process) nor the earlier schemes mentioned in the paper look very practical.  Do these have any real-world application?  If lossy compression is acceptable, JPG will get far better results, both in terms of compression and visual quality. -- Elphion (talk) 18:28, 24 July 2014 (UTC)
thread_title:  Lossy LZW 

message: The lede states that LZW is an improved implementation of LZ78, but the article does not describe what these improvements are.  Could someone who knows the difference clarify what they are exactly?  --Doradus (talk) 13:04, 12 December 2014 (UTC)
thread_title:  Contrast wth LZ78 

message: Re: merging Reduced Offset Lempel Ziv: If it's got to go somewhere, LZ77 and LZ78 is probably a better choice. -- Elphion (talk) 21:29, 9 December 2015 (UTC)
thread_title:  Proposed merge 

message: It says:
Buffer input characters in a sequence ω until ω + next character is not in the dictionary. Emit the code for ω, and add ω + next character to the dictionary. Start buffering again with the next character.
But this is missing some key information.
After the first sentence (which is fine as it is), the second sentence is "Emit the code for ω, and add ω + next character to the dictionary". Now while this describes what to do if no match is found, it never describes what to do if a match IS found. If a match is found, what code should be emitted? Should no code be emitted if a match is found? Please explain. I'm trying to implement LZW in my software, and have turned to Wikipedia for the info that I will need to implement this in my software. Please provide a good enough description that a programmer can use that description to actually implement it in software. Benhut1 (talk) 20:38, 22 August 2016 (UTC)
replies: text: Hi, . Depending on what language you're using, you may already find it at the Rosetta Code page about LZW, which has its implementation in a number of languages. While the article here will need updating, that should help you in the interim. Hope that helps! — Sasuke Sarutobi (talk) 13:05, 23 August 2016 (UTC)
replies: text: I disagree somewhat with OP.  The high-level description of the encoding algorithm in section 1.1 makes it pretty clear that no output is generated while the buffered string matches entries in the dictionary.  (The detailed example in section 2.1 shows this explicitly.)  And the premise that a WP article should necessarily provide enough information to implement the algorithm in solid code goes way beyond what WP aims to do.  This is an encyclopedia, not a text book. -- Elphion (talk) 14:50, 25 August 2016 (UTC)
thread_title:  Not a good enough description of the encoding process. 

message: Rule 4 says to include W and the following input into the dictionary
Rule 5 says to go to 2
Rule 2 says to find the longest string that matches current input
Wasn't the current input just added to the dictionary added to the dictionary? Rule 2 will always match what was just included in Rule 4. 
I'm not exactly certain of the algorithm, so am not confident rewriting this section or I would. Sorry.
139.102.14.207 (talk) 17:55, 25 February 2019 (UTC)
replies: text: Rule 3 removes W from the input, so when you loop back to rule 2, you will start scanning at the first character beyond W. -- Elphion (talk) 21:46, 25 February 2019 (UTC)
thread_title:  Encoding -Rule 4 in conflict with Rule 5 (indirectly with rule 2) 

message: As a *nix user it would be interesting to see if wikipedia could show some comparisons with .tar.lz to other formats,
in particular compression rate, memory usage, time to create and extract such archives etc..
A simple table should suffice, but of course it has to be correct. 2A02:8388:1641:8380:3AD5:47FF:FE18:CC7F (talk) 17:24, 10 May 2020 (UTC)
thread_title:  Comparison to other archive formats, in particular .tar.xz files 

message: I have reverted edits to the description of the encryption and decryption algorithms, which have made the process more difficult to understand.  The goal of WP is not to provide code for algorithms, but to describe them in enough detail that the reader understands how they work. -- Elphion (talk) 14:31, 2 May 2024 (UTC)
replies: text: Good job; WP:NOTHOWTO. TJRC (talk) 18:31, 2 May 2024 (UTC)
thread_title:  Over-detailed algorithms