HDTenEightyP/NetNews huggingface dataset:
From: trb@ima.isc.com (Andrew Tannenbaum)
Newsgroups: comp.compression
Subject: jpeg notes, progressive vs hierarchical, etc.
Date: 10 Apr 92 21:55:16 GMT
Article-I.D.: ima.1992Apr10.215516.20817
Organization: Interactive Systems, Cambridge, MA 02138-5302

I'm starting to get wrapped up in JPEG and compression, and some ideas
are puzzling me.  First, I'm wondering why there are different JPEG
specs for progressive and hierarchical encoding.  They seem so similar,
and I would think that the hierarchical method would do the job of
progressive presentation better than the progressive mode.  In
hierarchical mode, you can put up a low-res image and then sharpen it
up as you go, in progressive mode, you'd put out interlaced scans,
which seems lame by comparison.

Next, some people worry about how slow it will be to get index images
(thumbnails) out of (IJG-style) JPEG.  I figure that this isn't so,
since you can just grab the DC component out of each block (MCU) and
use that as the pixel value of the 8x8 block (giving you a 1:8 scale
index image).  It won't be as easy as having a raw thumbnail in an as
yet unspecified header, but at least you don't have to invert the DCT.

A comment on fear of lossy compression.  After watching captains of
industry force their wage slaves to type memos into PC word processors,
print them on paper, fax them, and have them scanned back into other
PC's, and so on, I can understand fears about the abuse of technology.
Where would USENET if it relied on faxes to send text?  Look at how
much bandwidth is wasted sending alt.binaries.* around, compared to the
value of the content, even at 1 picture == 1000 words.

One iteration of compression losing artifacts which are beyond the
perceptual threshold is not important to me.  Do you really care to
know that your pixels have square corners and flat sides?   If we
operated on reality with JPEG, we wouldn't call it image processing,
now would we?  Your source image isn't necessarily any better a
representation of reality than the decompressed JPEG image, assuming a
carefully-chosen Q (quantization quality value).

A legitimate fear of lossiness is a fear of multi-generational
degeneration.  JPEG isn't idempotent.  It could be, couldn't it?  I
mean, if I JPEG compress image I(0) with a Q=75 to get J(0) then I
decompress that to get I(1) then I compress that image with Q=75 to get
J(1), wouldn't it be nice if J(0) == J(1)?  This would be lossy, but
controllably so.  Unless I'm mistaken, you run the image through the
loop a few times and, well, you look at some kind of diff, Hosaka plots
or something, and you say "close enough, it's lossy anyway."  That
fear of the unknown, the hard to quantify, is the scary part of lossy
compression.

Someone here said that someday soon, storage will be so cheap that we
won't need lossy compression.  Well, I think photographers push the
limits of storage of image data on film, and I think that it'll be a
while before digital methods will compete with darkroom film in terms
of price/performance for storing pictures, so don't be so quick to
write off lossy compression.

	Andrew Tannenbaum   Interactive   Cambridge, MA   +1 617 661 7474
From: cornet@duteca3.et.tudelft.nl (Jan-Pieter Cornet)
Newsgroups: comp.compression
Subject: DISLITE available for anonymous ftp
Summary: ... as promised ...
Keywords: DISLITE, PKLITE, ftp
Date: 13 Apr 92 16:08:54 GMT
Organization: Delft University of Technology, Dept. of Electrical Engineering

The program I wrote to reverse any PKLITE compression, DISLITE,
is now available for anonymous ftp from at least two sites.

The filename to look for is DISLT100.LZH. You will need LHA to extract
the program and the doc file.

Sites:
donau.et.tudelft.nl (130.161.144.100)
directory pub/ (Well... it _might_ be in pub/irc/ too)

dnpap.et.tudelft.nl (130.161.144.65)
directory Msdos/

Regards,
-- Jan-Pieter
cornet@duteca.et.tudelft.nl
From: robtow@colour.Eng.Sun.COM (Rob Tow)
Newsgroups: comp.compression
Subject: Re: Human visual system scaling for jpeg
Date: 16 Apr 92 19:06:18 GMT
Organization: Sun Microsystems, Mt. View, Ca.
NNTP-Posting-Host: colour.eng.sun.com

In article <1992Apr15.171503.10372@ima.isc.com: trb@ima.isc.com (Andrew Tannenbaum) writes:
:In article <2013@marlin.NOSC.MIL: bryan@marlin.NOSC.MIL (Dale Bryan) writes:
:: I am trying to find a good reference for scaling a DCT
:: coefficient matrix (n x n) using the contrast sensitivity
:: of the human visual system.  Any help would be greatly
:: appreciated.
:
:I'm not sure what you mean by scaling here, but here are pointers to a
:couple of Kodak patents that address halftoning and the human visual
:system.  (The patent docs have further references in them.)
:
:US Pat 5,051,844
:Digital Halftoning with Error Diffusion
:James R. Sullivan
:Eastman Kodak Company
:Sep 24, 1991
:
:US Pat 5,070,413
:Color Digital Halftoning with Vector Error Diffusion
:James R. Sullivan
:Eastman Kodak Company
:Dec 3, 1991
:
:These patents describe methods for improving computer generated
:graphics using a human visual system response function.
:
:	Andrew Tannenbaum   Interactive   Cambridge, MA   +1 617 661 7474
:	(Interactive is a Kodak Company)

See also "Color Halftoning Using Error Diffusion and a Human Visual
System Model", by R. Miller and J. Sullivan. Proceedings of the 43rd Annual
SPSE Conference. May 1990. Published by SPSE - The Society for Imaging
Science and Technology, 700 Kilworth Lane, Springfield, VA 22151. 
(703)-642-9090.


-Rob-
Xref: sparky comp.compression:1577 sci.math:6065 comp.graphics:5429
From: jack@multimedia.gatech.edu (Tom Rodriguez)
Newsgroups: comp.compression,sci.math,comp.graphics
Subject: Integer Cosine Transform for JPEG
Keywords: DCT ICT image compression
Date: 15 Apr 92 02:04:48 GMT
Followup-To: comp.graphics
Organization: Software Research Center / Georgia Tech


	I've implemented the Integer Cosine Transform for use with the
PD JPEG implementation but i'm having trouble producing an equivalent
inverse transform.  I'm operating on the assumption that i can derive
the inverse transform by inverting the matrix and applying a suitable
scaling factor to the final result.  Am I off in space?  Would anyone
know of a paper on this topic?  I've looked around but can't find any
that discuss the Inverse ICT.  I'd appreciate any help.

	For those interested, the paper I'm using for the ICT is by
F.S. Wu and W.K. Cham, "A comparison of error behaviour in the
implementation of the DCT and the ICT", from IEEE TENCON '90.  It
resulted in about a 2 times speed up of the JPEG compression and a
very slight but consistent improvement in compression, on the order of
2%.  The approximate MSE as compared with the DCT code distributed
with the PD JPEG code was 1.

	tom

--
Tom Rodriguez  (jack@cc.gatech.edu)
Multimedia Computing Group - GVU Lab
Georgia Institute of Technology
Atlanta, Georgia 30332-0280
From: nevries@accucx.cc.ruu.nl (Nico de Vries.)
Newsgroups: comp.compression
Subject: Re: Strange Zip file.
Date: 14 Apr 92 18:21:24 GMT
Organization: Academic Computer Centre Utrecht

In <1992Apr14.135104.14440@cci632.cci.com> sik@cci632.cci.com (Shawn Kammerdiener) writes:

>In article <6APR199216494690@vms.huji.ac.il> gilf@vms.huji.ac.il (GIL FREUND) writes:
>>viewing the file (With pkunzip -v or the XTreeGold Archive manager) I find
>>that some of the files have UNKNOWEN written at the compression method
>>column.

>	It probably means that the version of UNZIP that you are using is
>older than the archive that you are trying to undo. Get a new version of
>PKUNZIP from an FTP site or BBS, and the problem should go away.

>sik@cci.com

Unfortunately the problem is larger. There is a large chance the 1.93 beta
version has been used. Since this version has some bugs (it is a beta
after all) many sites and BBSes don't use/have it. The "new" version of
PKZIP is stil 1.10 and this version won't solve the problem.

PKZIP 2.20 is supposed to arive about now and is supposed to be 1.93
compatible so that should solve the problem.

Nico
Newsgroups: comp.compression
From: prknoerr@immd4.informatik.uni-erlangen.de (Peter Knoerrich)
Subject: Re: PKZIP encryption contest available from garbo
Organization: CSD., University of Erlangen, Germany
Date: Thu, 9 Apr 1992 10:55:17 GMT

ts@uwasa.fi (Timo Salmi) writes:

Hi,

>Thank you Jean-loup.  Your upload is now available as
>14433 Apr 3 14:39 garbo.uwasa.fi:/pc/arcutil/pkcontst.zip

I can't get it! (Must be part of the contest :-)

*ftp> cd /pc/arcutil
*250 CWD command successful.
*ftp> dir pkcontst.zip
*200 PORT command successful.
*150 Opening ASCII mode data connection for /bin/ls.
*-rw-------  1 10016     110         14433 Apr  3 11:39 pkcontst.zip
*226 Transfer complete.

						    Bye, Peter

-------------------------------------------------------------------------
.sig? You must be kidding with all those SIGKILLs around on this machine!
Peter Knoerrich,        email: prknoerr@medusa.informatik.uni-erlangen.de
---
-------------------------------------------------------------------------
Watch the sunfall, and that flock of .sigs peacefully crossing the sky...
Peter Knoerrich,        email: prknoerr@medusa.informatik.uni-erlangen.de
From: nevries@accucx.cc.ruu.nl (Nico de Vries.)
Newsgroups: comp.compression
Subject: Re: PKUNZIP problem
Date: 11 Apr 92 07:34:36 GMT
Organization: Academic Computer Centre Utrecht

In <14318@pitt.UUCP> jblst5@icarus.lis.pitt.edu (Jan B Larsen) writes:


>I don`t know if this is the right place to post this question, but here
>we go:

>When I ftp ZIP-files from wuarchive.wustl.edu using binary mode, I allways
>get this error message when I try to PKUNIZIP it on the PC:
>Warning! Don't know how to handle: filename
I assume you are using PUNZIP 1.10 and try to decompress files made by
PKZIP 1.93a (a beta for PKZIP 2.20 which should be out in a week).

>If I ftp to my SunOS account and use UnZip (by S.H.Smith) it works just fine.
Probably you have UnZip 4.2 whoch I believe is capable of handling PKZ1.93

>Help!

>- Jan


Nico de Vries
-----------------------------------------------------------------------------
USENET: nevries@cc.ruu.nl                 FIDO      : 2:281/708.1  (very new) 
VOICE : +31-3404-24931 (+31-30-316931)    FAX       : +31-30-312033 
BBS   : +31-30-341401 (Nico De_vries)     COMPUSERVE: (comming)
MAIL  : Hortensialaan 45, 3702VE, Zeist, The Netherlands 

IMPORT StdDisclaimer, MyPersonalOpinion;          - O   Place reserved for
Complaints about my spelling to DEV/NUL please.    |    funny joke. Sugestions 
I work for BITECH, I don't SPEAK for BITECH.      \_/   apreciated.
From: coulter@malibu.janet.ucla.edu (Michael L. Coulter)
Newsgroups: comp.compression
Subject: Re: Too much lossy encoding?
Keywords: PASC, audio
Date: 9 Apr 92 09:13:25 GMT
Organization: University of California at Los Angeles, EE Dept.

In article <1992Apr8.160300.5981@tc.fluke.COM> kurt@tc.fluke.COM (Kurt Guntheroth) writes:
>Is it just me or is anyone else out there concerned that we are using lossy
>encoding "too much" as a substitute for denser storage and better transmission
>methods?
>

Mr. Guntheroth,
I'm not concerned at all.  In the video compression arena, you can't expect
companies like LSI Logic, IIT, C-Cubed, AT&T, etc. to sit back and wait until
the year 2010 when fiber makes to everyones house or business.  There is
obviously an application for video compression (JPEG, MPEG, Px64, etc.) or
these companies would not be pursuing it.  There is definitely a lot of money
to be made.  Sure, there is a sacrifice in video quality when bandwidth
requirements have to be met, but there are a number of applications where top  
quality video is not nearly as important as the compression ratio.  
You may rationalize that as soon as fiber reaches everyone, that there is
no need for compression, but you would be wrong.  Video compression is needed
in the military and space areas such as satellite systems, missiles, unmanned
aerial vehicles, etc.

mike
From: jj@alice.att.com (that old grouch jj)
Newsgroups: comp.compression
Subject: Re: Human visual system scaling for jpeg
Date: 16 Apr 92 16:21:00 GMT
Reply-To: jj@alice.UUCP (that old grouch jj)
Organization: NJ State Home for Bewildered Terminals

In article <2013@marlin.NOSC.MIL> bryan@marlin.NOSC.MIL (Dale Bryan) writes:
> I am trying to find a good reference for scaling a DCT
> coefficient matrix (n x n) using the contrast sensitivity
> of the human visual system.  Any help would be greatly
> appreciated.

Well, you could also check out a paper by Safranek and Johnston, from
AT&T-Bell Labs, in ICASSP-89, on image coding for some other ideas.

There are other papers on such issues, and a good picture of the
original porobmel in many vision books.
-- 
Vote for    *Copyright alice!jj 1992, all rights reserved, except transmission
President 1992 *by USENET and like free facilities granted. Said permission is
___________________*granted only for complete copies that include this notice.
Just Say !!!NO!!!       *Use on pay-for-read services specifically disallowed.
From: nevries@accucx.cc.ruu.nl (Nico de Vries.)
Newsgroups: comp.compression
Subject: Re: Strange Zip file.
Date: 7 Apr 92 12:25:25 GMT
Organization: Academic Computer Centre Utrecht

In <6APR199216494690@vms.huji.ac.il> gilf@vms.huji.ac.il (GIL FREUND) writes:

>I have reacently recieved a ZIP file which I cannot decompress. When
>viewing the file (With pkunzip -v or the XTreeGold Archive manager) I find
>that some of the files have UNKNOWEN written at the compression method
>column.
>What does this mean? The freind who gave me the file isn't around.

>Thanks

>Gil

There are two possibilities. The program is compressed with PKZIP 1.93 (beta
version) or with PKZIP 2.01 (cracked fake version). You are probably using
PKZIP 1.10 which is not capable of handling those.


Nico de Vries (nevries@cc.ruu.nl)              

#include <std disclaimer & my personal opinion.h++>     - O   Don't
Complaints about my spelling to DEV/NUL please.          |    worry,
I work for BITECH, I don't SPEAK for BITECH.            \_/   be happy!
From: dat91jla@ludat.lth.se (Larsson Jesper)
Newsgroups: comp.compression
Subject: Re: Stupit compressor idea (but interesting).
Date: 14 Apr 92 16:49:18 GMT
Organization: Lund Institute of Technology, Sweden

In article <2367@accucx.cc.ruu.nl> nevries@accucx.cc.ruu.nl (Nico de Vries.) writes:
>Suppose we take a compressed file (doesn't matter which compressor) and
>we remove a single byte. The first one. Now what the decompressor does
>is trying ann 256 options and takes the file which looks most likely (this
>is the weak part of cource but we could try decompression an compressio
>and if the decompressor goes wrong we just don't remove the byte).
>This seems to work to me.
>
>Perhaps could this system be enhanced to compress even better (e.g. removing
>each 4th byte and trying again and again to repair it. If decompression
>fails we dont remove it.). An extra bit needs to be inserted to determine
>if a byte has to be removed).

I think what you have described is really an extremely simple approximation
of a context sensitive Huffman-encoder. Your "Huffman tree" always has the
node that corresponds to the most likely next byte value at depth one, and
all the others at equal depth (that would be 9).

You had me fooled for a while with your talk about "trying all 256 options".
You make it sound like it would be an awful lot of work for the decompressor
to determine which byte is the most likely. It does not have to be.

Intuitively one might think that a hard-working decompressor makes an
efficient compression algorithm, but this is not necessarily true. The
decompressor must have the information about which is the next byte in some
way (otherwise it wouldn't be able to decompress), and there is no reason
why a smart algorithm would not be able to process this information fast.

............................................................................
:                       J e s p e r   L a r s s o n                        :
:                          dat91jla@ludat.lth.se                           :
:        ``Bad programming practice can lead to death'' P J Plauger        :
:..........................................................................:
From: nevries@accucx.cc.ruu.nl (Nico de Vries.)
Newsgroups: comp.compression
Subject: Re: PKZIP encryption contest available from garbo
Date: 11 Apr 92 07:41:35 GMT
Organization: Academic Computer Centre Utrecht

In <1992Apr11.045843.28777@uwasa.fi> ts@uwasa.fi (Timo Salmi) writes:

>[some talk about PKCONTST availability]
>Of the contest itself.  The price of $1000 seems frankly quite low
>to be attractive.  The challenge is ok and interesting, though. 
It sure is interesting but it is NOT realistic and it does prove nothing. It
is just a nice marketing stunt. I have the fi\ollowing problems with the contest:

- the password uses all 256 byte values. Very very very few users do this
- there is only 1 encrypted file in the PKZIP archive. One of the weaknesses
  od the scheme is that mostly multiple files are present which significantly
  lowers the number of candidate passwords. In the contest file the number
  of files which pass the header CRC16 test (which IMHO is the weakest part
  of the PKZIP method) is probably 256^5 while in a real file with e.g. 5
  files there will only be a few (if more than one at all) wrong passwords
  pass the header test.
  The main difference is that in the contest the plaintext needs to be investigated while this in practive almost never is needed for cracking the PKZIP
  scheme. I personally prefect e.g. NSEA since it does have this problem
  much less.

>...................................................................
>Prof. Timo Salmi


Nico de Vries
-----------------------------------------------------------------------------
USENET: nevries@cc.ruu.nl                 FIDO      : 2:281/708.1  (very new) 
VOICE : +31-3404-24931 (+31-30-316931)    FAX       : +31-30-312033 
BBS   : +31-30-341401 (Nico De_vries)     COMPUSERVE: (comming)
MAIL  : Hortensialaan 45, 3702VE, Zeist, The Netherlands 

IMPORT StdDisclaimer, MyPersonalOpinion;          - O   Place reserved for
Complaints about my spelling to DEV/NUL please.    |    funny joke. Sugestions 
I work for BITECH, I don't SPEAK for BITECH.      \_/   apreciated.
Newsgroups: comp.compression
From: kocherp@leland.Stanford.EDU (Paul Carl Kocher)
Subject: Re: PKZIP encryption contest
Organization: DSG, Stanford University, CA 94305, USA
Date: Wed, 15 Apr 92 02:17:03 GMT

In nevries@accucx.cc.ruu.nl (Nico de Vries.) quotes:
>>I have no reason to believe that they did not use PKZIP 1.10 to make
>>the file.  However, I also have no reason to believe that they did.
>>They haven't said whether they did or not.  If they did not, this also
>>adds greatly to the bogosity.

pkzip -vt reveals that the file was stored using PKZIP version 1.1.
It doesn't really matter, though, since the file is not compressed.

>Just another thougth BTW. Does anyone know how PK generates its random
>numbers. If it is some derivate from the timer than it should be possible
>to reconstruct it from the file time (in theory, it would at least limit
>the possibilities). If it also depends on the plaintext and the password
>it might become a lot harder to do.

I've already partially explored this route without success.  The time-of-
day is included, but so are many other things, such as the password itself
and the position where things are loaded in RAM (could be a result of any
of a number of things...)  I'd be curious if anyone has done a complete or
partial reverse-assembly to see whether this data might possibly be
as a 16- or 32-bit random number seed (unlikely, but worth exploring).

-- Paul Kocher

________kocherp@leland.stanford.edu_________kocherp@jacobs.cs.orst.edu________
I am an undergrad at Stanford seeking summer work or contract jobs programming
(IBM PC), doing technical writing, or assisting with research. Send e-mail for
resume if interested!  - Paul Kocher/Box 13554/Stanford CA 94309, 415/497-6438
From: dat91jla@ludat.lth.se (Larsson Jesper)
Newsgroups: comp.compression
Subject: Re: 16: 1 data compression
Keywords: bogus, compression
Date: 14 Apr 92 17:05:36 GMT
Organization: Lund Institute of Technology, Sweden

In article <1213@lysator.liu.se> boberg@lysator.liu.se (Stefan Boberg) writes:
>N02@csci1.csci.unt.edu (Orion) writes:
>I've invented a method to compress files to 0 bits
>Yes, that's right: ZERO BITS! It's called `delete', or `rm' depending on
>which version you have.

So that's what "too much lossy encoding" is really about!

............................................................................
:                       J e s p e r   L a r s s o n                        :
:                          dat91jla@ludat.lth.se                           :
:        ``Bad programming practice can lead to death'' P J Plauger        :
:..........................................................................:
From: ry01@ns1.cc.lehigh.edu (ROBERT YUNG)
Newsgroups: comp.compression
Subject: 16:1 compression?
Date: 10 Apr 92 16:38:55 GMT
Organization: Lehigh University

A little side-news thing in Pc-Mag says a company named WEB technology has
made a compressor that does 16 to 1 compression. The little paragraph also
says this compressor is better and different from Pkzip's. Any truth to this
or are they compressing files like this:

   00000000000000000000000000
   00000000000000000000000000
   00000000000000000000000000 :-)

BTW, doesn't that longer to compress a file than to uncompress it? All the
compression time is taken up by trying to find the right way to compress a
certain strin g of data. Once that's done, uncompression should be a breeze!
So why am I hearing people saying they can compress so and so but can't
uncompress...????
-- 
 (%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%)
  |  Off-campus address...                                          |
  |     first alley down this street, left refrigerator box. :-)    |
  |                                                                 |
   \_________Bobby Yung_______RY01@Lehigh.Edu________X-0471________/
From: ATDYW@ASUACAD.BITNET (Dan Wilson)
Newsgroups: comp.compression
Subject: PKZIP 2.0 Advert in PC MAG
Date: 12 Apr 92 04:33:50 GMT
Article-I.D.: ASUACAD.92102.213350ATDYW
Organization: Arizona State University

Well, I don't know if this even worth reporting, given all the false
alarms we've seen on this board, but here it is anyway. I just got my
new issue of PC MAGAZINE (April 28,1992) and on page 100 there is a
large advertisement for PKZIP 2.0 and PKLITE. PKZIP 2.0 is priced at
$47.00 and PKLITE at $46.00/ PKLITE PROFESSIONAL is priced at $146.00.

So, what does this mean? Are they actually shipping? Or is it still
vaporware?
Newsgroups: comp.compression
From: iyengar@Panda1.UOttawa.CA (Giri Iyengar)
Subject: Wavelets for image compression
Organization: Computing & Comm. Services, University of Ottawa
Distribution: world 
Date: Mon, 13 Apr 92 12:48:38 GMT

Hi eveybody:
I am looking for journal/conf papers/texts which talk about wavelets for
image compression. Pls email your replies to me. I'll post a summary on the
net after I collect substantial info.
Thanks
-Giri
iyengar@aix1.uottawa.ca
From: bryan@marlin.NOSC.MIL (Dale Bryan)
Newsgroups: comp.compression
Subject: Human visual system scaling for jpeg
Date: 14 Apr 92 14:56:40 GMT
Organization: Naval Ocean Systems Center, San Diego

I am trying to find a good reference for scaling a DCT
coefficient matrix (n x n) using the contrast sensitivity
of the human visual system.  Any help would be greatly
appreciated.

dale bryan
email: bryan@marlin.nosc.mil