Talk:Prediction by partial matching

message: Some PPM algorithms have the useful property of being able to interpret any collection of bytes as valid compressed input. An algorithm with this property is said to be bijective. Bijectiveness has positive implications for compression efficiency and security because there is no way to distinguish random data from valid output.
This is not terminology in general usage; it is the terminology used (misused?) by one particular person, David A. Scott, who has been for years obsessively describing the importance of "bijective compression" and his work on it on comp.compression.* and sci.crypt.*.  Bijective compression works, but it is at best mildly useful.  It is not, in the real world, so valuable for encryption that the NSA deliberately surpressed it.  When he describes it, he nearly always mentions the fact that it is "more efficient" than 'non-bijective' methods, but he does not mention that the savings is usually an amount less than a few bytes.  Nor does he acknowledge the major drawback of his method:  his "savings" comes from eliminating the internal termination of the 
data stream.  Yet the information of where the message stops, where the decoder must stop reading, obviously needs to come from somewhere.  Scott has actually argued that every file worth dealing with is located in a file system that keeps track of this information anyways, a claim which is manifestly untrue.
Obviously, the claims for bijectivity do not go so far in this article, but they do overestimate its usefulness with no mentions of its disadvantages.  -- Antaeus Feldspar 19:07, 24 Sep 2004 (UTC)
thread_title:  D. Scott infamous bijective nonsense 

message: While i don't think this article/talk page is the best place to discuss bijective compression, i must try and clear up some of the statements made above:
* Some PPM algorithms have the useful property of being able to interpret any collection of bytes as valid compressed input.  An algorithm with this property is said to be bijective.
:(I'm only noting this one for completeness.)  This statement is misleading, of course:  the bijectivity in question doesn't have anything to do with the entropy model (PPM), but is a property of the entropy coder that the model is linked to.
* Bijective compression works, but it is at best mildly useful. It is not, in the real world, so valuable for encryption [...]
:Bijectivity is probably not interesting to the average user, but i think it's more than just "mildly useful" in the context of cryptography.  The fact is that bijective compression is the only kind of compression that can be used with encryption without introducing a weakness.  (With compressed plaintext, an attacker can always eliminate all keys with ill-formed decryptions from consideration ("ill-formed" meaning "doesn't decompress and recompress back to itself").  Bijective compressors, though, will decompress and recompress anything back to itself (by definition), which means that they keep the entire keyspace as valid and well-formed as it was before; they don't introduce anything for an attacker to leverage.)
:Of course, in practice, all of that only matters to the extent that you actually want to compress your plaintext in the first place.  This is a more complicated issue, i think, but the sci.crypt FAQ has this to say (8.5 "How do I use compression with encryption?"):
::Compression aids encryption by reducing the redundancy of the plaintext.  This increases the amount of ciphertext you can send encrypted under a given number of key bits.  (See "unicity distance")
::[...]
::Compression is generally of value, however, because it removes other known plaintext in the middle of the file being encrypted.  In general, the lower the redundancy of the plaintext being fed an encryption algorithm, the more difficult the cryptanalysis of that algorithm.
::In addition, compression shortens the input file, shortening the output file and reducing the amount of CPU required to do the encryption algorithm, so even if there were no enhancement of security, compression before encryption would be worthwhile.
* Nor does he acknowledge the major drawback of his method: his "savings" comes from eliminating the internal termination of the data stream. Yet the information of where the message stops, where the decoder must stop reading, obviously needs to come from somewhere.
:I'm not familiar with "his method", but this is definitely not true of bijective encoding in general.  For an example, take a look at Matt Timmermans's bijective implementation of arithmetic coding, which is self-terminating (very efficiently so).
:--Piet Delport 06:48, 11 March 2006 (UTC)
thread_title:  In defense of bijective compression 

message: --SyP 12:39, 2 January 2006 (UTC)
thread_title:  now ZipSMS uses PPM to compress SMS on modern mobile phones 

message: This article could use a section explaining where this algorithm is implemented and used in practice, and another section with a worked-out example. -- Beland (talk) 22:38, 27 July 2008 (UTC)
thread_title:  Applications, example 

message: "PPMD increments the pseudocount of the "never-seen" symbol every time the "never-seen" symbol is used." - is it about PPMD or about PPMd method? `a5b (talk) 06:08, 20 September 2011 (UTC)
thread_title:  PPMd vs PPMD