HDTenEightyP/NetNews huggingface dataset:
From: lip@s1.gov (Loren I. Petrich)
Newsgroups: comp.compression
Subject: Re: 16-1 compression
Date: 28 Jul 92 23:30:30 GMT
Organization: LLNL

In article <1992Jul28.165529.18628@lugb.latrobe.edu.au> HCHA8904593X@LUST2.LATROBE.EDU.AU (PERRETT, ANDREW) writes:
>This has probably been FAQ'd to death but can anybody tell me if the report in
>Byte of two months ago was correct ie come mob claiming a 16-1 compression
>ratio, and being able to do it again to the same file ?? Stranger things have
>happened (hell the earths round!) but i think ill remain agnostic till i get to
>turn my HD into a couple of gigabytes...


	In fact, there is something in this newsgroup's FAQ about this
supposed compression method, called "WEB" (if I remember correctly).
It supposedly does lossless compression on all kinds of files by a
factor of 16, and it can supposedly be repeated.

	It is easy to show that _lossless_ compression (where one can
recover the original file without corruption) of any possible file is
an impossibility. This comes from counting the number of possible
files of a given length and pointing out that the number of compressed
files is less than the number of original files. Thus, there will be
at least one compressed file which will map onto more than one of the
original files.

	For _lossy_ compression (as on sound or image information),
and also for lossless compression practiced on files with certain
features (as in text files, with their internal patterns), the story
is different.
Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
From: gof@casbah.acns.nwu.edu (Gerald Fountain)
Subject: Re: Compression artifacts in NBC Olympic coverage
Organization: Northwestern University, Evanston Illinois.
Date: Mon, 27 Jul 1992 00:27:07 GMT

In article <tlm1.712195270@Ra.MsState.Edu> tlm1@Ra.MsState.Edu (Terry Matula) writes:
>tlm1@Ra.MsState.Edu (Terry Matula) writes:
>>monta@image.mit.edu (Peter Monta) writes:
>
>>>Finally, if you have a tape of the afternoon coverage, take
>>>a look and try to confirm that I'm not off my rocker.
>
>>I haven't taped anything, but I will keep an eye out for it next time I
>>watch some of the coverage.
>
>I just watched some of the floor exercises, and yes, there is certainly
>something visible around fast moving objects that may be compression 
>artifacts.  So you're not completely off your rocker. :)
>
>--
>Terry Matula
>tlm1@ra.msstate.edu
>matula@ee.msstate.edu

Hmmm, I was thinking that it seemed like they were speeding up the feed
slightly.  I was watching the diving and swimming competitions and noticed
what seemed like fast forward shots.  Is it possible they are doing a 10%
speedup or so to also reduce broadcast time?  This would either save them
bandwidth or allow them to make more 'complete' coverage.


-- 
Jerry Fountain                       
Northwestern University                        gof@stokes.chem-eng.nwu.edu
Department of Chemical Engineering             (708) 491-3555
2145 Sheridan Road, Evanston, IL  60208
Xref: sparky comp.compression:2817 rec.video:9598 rec.video.satellite:3782 rec.video.production:1141
From: syd@DSI.COM (Syd Weinstein)
Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
Subject: Re: Compression artifacts in NBC Olympic coverage
Date: 27 Jul 92 03:47:51 GMT
Reply-To: syd@DSI.COM
Organization: Datacomp Systems, Inc.  Huntingdon Valley, PA

monta@image.mit.edu (Peter Monta) writes:
>While I was watching the US-Angola basketball game and the coverage
>of the early gymnastics compulsories, I thought I saw several image
>artifacts.
Motion artifacts, yes, but I doubt due to compression.  Remember
the pool feed is in 50hz, like the lighting.  Thus it needs to be
converted to NTSC from the local TV standard.  That is known to
cause some motion artifacts.
-- 
========================================================================
Sydney S. Weinstein, CDP, CCP          Elm Coordinator - Current 2.3PL11
Datacomp Systems, Inc.                 Projected 2.4 Release: Oct 1,1992
syd@DSI.COM or dsinc!syd      Voice: (215) 947-9900, FAX: (215) 938-0235
Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
From: elball@babbage.waterloo.edu (Eric Ladd Ball)
Subject: Re: NBC Olympic coverage - reason
Organization: University of Waterloo
Date: Wed, 29 Jul 1992 13:33:40 GMT

I do believe I stand corrected.  There has been enough posts by people
"in the know" to my hypotheses out to dry.  But...I still think that there
is the possibility that non-interlaced frame-type cameras are being used.
High "shutter-speed" is great to prevent blurring on freeze-frame, but it
still does not prevent interlaced time-dependent artifacts.
If they are using some sort of compression I'd be very impressed.  That
sort of equipment is not cheap.
  eric
Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
From: tope@enea.se (Tommy Petersson)
Subject: Re: Compression artifacts in NBC Olympic coverage
Organization: Enea Data AB
Date: Fri, 31 Jul 1992 13:53:54 GMT

ojala@funet.fi (Petri Ojala) writes:
- 
- > Which HDTV standard is this- Japanese version? I would imagine that is
- > designed to augment an NTSC signal, so how does it relate to PAL? While I
- > can see HDTV 
- > worlwide as having the same aspect ratio and number of lines, is there a
- > worldwide standard for frame rate? And will the only difference in US
- > digital HDTV be in the brodcast mechanism, not in the actual frames.
- 
- (Has there been any HDTV broadcasts from Barcelona to Japan or US ?)
- 
- I have a very short paper about the Barcelona HDTV setup (in Finland) and
- if I understand it correctly, the transmission from Barcelona is
- European HDTV, HD-MAC.  These transmissions are done by European
- Vision 1250 production project and the transmissions are sent as
- European-wide satellite transmission.
- 
- Accordingly to this paper there is 1250 lines, 50 frames/sec and the
- broadcasting technology is HD-MAC in the European HDTV while in the
- Japan they have 1125 lines, 60 frames/sec and MUSE broadcasting.
- 
- Petri

People hoped that the HDTV would be a world-wide accepted standard, but it
seems there will be three standards in the beginning (and then the French
will invent a fourth, only slightly different :-)

One main concern in Europe was to be able to view old PAL broadcasts on
the new HDTV equipment, which HD-MAC allows (the old backward-compatibility
crippling).

-- 
=============================================================================
 Tommy Petersson              tope@enea.se              Enea Data AB, Sweden
=============================================================================

Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
From: edb9140@tamsun.tamu.edu (Edward Dale Boling)
Subject: Re: Compression artifacts in NBC Olympic coverage
Organization: Texas A&M University, College Station
Date: Mon, 27 Jul 1992 02:11:49 GMT

I noticed it also :)
From: An-Ly Yao <anlyyao@igc.org>
Newsgroups: comp.compression
Date: 30 Jul 92 06:28 PDT
Subject: 16:1, not again  p l e a s e !
Nf-ID: #N:cdp:1345600008:000:273
Nf-From: cdp.UUCP!anlyyao    Jul 30 06:28:00 1992


There have been lengthy discussions on WEB, 16:1 compression (where the
problem might be the 1:16 decompression) etc.

Please let's not waste time and wait, until WEB's program ist really
available for verification.

bye, Goetz Kluge, anlyyao@igc.apc.org, Seoul 1992/07/30

Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
From: maynard@esther.msc.cornell.edu (Maynard J. Handley)
Subject: Re: Compression artifacts in NBC Olympic coverage
Organization: Cornell-Materials-Science-Center
Date: Thu, 30 Jul 1992 21:59:48 GMT

In article <OJALA.92Jul30143437@dolphin.hut.fi>, ojala@funet.fi (Petri
Ojala) writes:
>
>> While HDTV will be at the Olympics, none of the cameras are on-line for
>the
>> world feed.  All of that is PAL (4:3).  The HDTV stuff will be used
>later to
>> show off the quality of HDTV.
>>
>> Though, there may be some sites where HDTV can be seen live.  It was
>done for
>> the winter Olympics.  But, none of it (as far as I know) is being
>broadcast.
>
>I believe that they are actually broadcasting the HDTV stuff over
>satellite
>to several locations, at least inside Europe.

Which HDTV standard is this- Japanese version? I would imagine that is
designed to augment an NTSC signal, so how does it relate to PAL? While I
can see HDTV 
worlwide as having the same aspect ratio and number of lines, is there a
worldwide standard for frame rate? And will the only difference in US
digital HDTV be in the brodcast mechanism, not in the actual frames.

Maynard Handley
From: jones@pyrite.cs.uiowa.edu (Douglas W. Jones,201H MLH,3193350740,3193382879)
Newsgroups: comp.compression
Subject: Re: entropy
Date: 22 Jul 92 15:46:14 GMT

From article <1992Jul22.144104.27576@usenet.ins.cwru.edu>, by daf10@po.CWRU.Edu (David A. Ferrance):
> 
> Right now I compute the entropy of a file by summing the probability of
> each character ocurring times the log (base2) of the reciprocal of that
> probability.  This gives me the theoretical minimum number of bits
> needed to represent each character, on average (from what I understand).
> What I would like to know is, does this number only work for first order
> (1-ary?) compression techniques or does it serve also as a limit for
> more complex techniques?

This is a universal way to compute entropy, but the answer you get depends
on how you compute the probability of a character occuring.  The first
order entropy is obtained by measuring probabilities in isolation -- the
probability of a letter is taken out of context.  To get the second order
entropy, you look at the probability of each letter in the context of the
immediately preceeding letter.  For the third order, you look at each letter
in the context of each pair of preceeding letters, and so on.

The absolute entropy of a bit of text can only be determined if you use an
infinite order model, taking into account every bit of context information
available.  Shannon did a very elegant experiment to measure this by having
human subjects read bits of text up to random points and then guess the next
letter.  This is how he arrived at the estimate that the entropy of English
text is a bit over one bit per character.

				Doug Jones
				jones@cs.uiowa.edu
From: ts@cup.portal.com (Tim W Smith)
Newsgroups: comp.compression
Subject: Re: PERPETUAL MOTION
Date: Thu, 23 Jul 92 21:42:17 PDT
Organization: The Portal System (TM)
  <7486@public.BTR.COM>

> I have not seen any counterexample to the primary refutation, that if
> you consider all files of size less then or equal to N bits, you have a
> certain number of different possible files, and if you compress them
> ALL using a single method, either all will end up exactly the same
> size, or at least one of them will get larger, or two different files
> will compress to the same result meaning the method isn't reversible,
> or the method simply will refuse to work on at least one file.

Most impossible compressors claim to only work on files past a certain
size.  Let that size be M bits.  Let F(X) be the total number of files
of X bits or less.  Let f(X) be the total number of files of X bits.

Assume we have some ordering of all files.  With each file, associate
a number, which is the position of the file in this ordering relative
to the first file of the same size.

Now consider the following compressor.  For files of length M+1, the
first F(M) of them are mapped to files of length M or less.  The rest
of the files of length M+1 are mapped to the files whose number is
F(M) less than theirs.

This leaves us F(M) files of length M+1 that do not have files mapped
onto them.

We can now do a similar thing for files of length M+2.  We map the
first F(M) onto the files of length M+1 that have not been mapped to,
and them map the rest to files whose number is F(M) less.

This leaves us F(M) files of length M+2 that have not been mapped
onto.

Repeat forever.  At each step, there are F(M) files that compress, and
applying this to a file of size N at most f(N)/F(M) times will compress
it.

I don't recall all the WEB claims, but I think that this satisfies them
in spirit.

Actually, the above is needlessly complex.  Arrange all possible files
by increasing size.  Call the position of a given file on this list
the index of that file.  "Compress" a file by mapping it to the file
whose index is one less.  No file gets bigger, some files get smaller,
and repeated application to a given file will make it smaller.

The above is the kind of thing I was trying to capture with the
"subtract one" compressor.  I screwed up on the leading zeros,
though.

Note that the above is completely useless as a real compressor.
The fraction of files actually compressed by a single pass goes
to zero as the file size goes up.  If you apply it repeatedly, you
will use as much space to remember how many times you applied it
as you save by using it.  And the compression ratio goes down
as the file size goes up, approaching a limit of 1.

However, I believe that it *DOES* provide a counterexample to the
usual counting argument. 

--Tim Smith
From: meyer@dbsun.uucp (Don Meyer)
Subject: Re: Compression artifacts in NBC Olympic coverage
Organization: BioMerieux-Vitek, St. Louis Mo.
Date: Mon, 27 Jul 1992 16:24:49 GMT

In article <1992Jul27.035759.153188@zeus.calpoly.edu> jemenake@zeus.calpoly.edu (Joe Emenaker) writes:
>Well, bravo. I'm happy for you that you're smart enough to notice such
>things. I'm sure you must be a very bright little boy. To quote Monty
>Python: "Gee, we're all really impressed down here, I can tell you.".
>However, I'm sad for you that knowing that NBC is compressing the signal is
>ruining your experience of the Olymics so much.
>Is it REALLY that unbearable? For heaven's sake, just sit a few feet futher
>from the screen or take your glasses off or something.

Ahhh, I see the problem.  You're looking for rec.bad.video.and.proud.of.it.

I've found this thread interesting.  If you don't care about these
things, why the heck are you reading this group?  Might I suggest you
go away?

Don



-- 
-------------------------------------------------------------------------
meyer%dbsun.uucp@wupost.wustl.edu        |||     I am at two with nature.
Opinions expressed are personal.         |||       -- Woody Allen
Newsgroups: comp.compression
From: dcarr@gandalf.ca (Dave Carr)
Subject: Re: Expanding/Unshrinking in ZIP
Organization: Gandalf Data Ltd.
Distribution: na
Date: Fri, 24 Jul 1992 12:30:11 GMT

In <36044@sdcc12.ucsd.edu> alehecka@sdcc13.ucsd.edu (Augustin Lehecka) writes:

>Can anyone tell me what 'Unshrinking' and 'Expanding' messages 
>in PKUNZIP mean?  It seems to me that the program is using
>two different compressing algorithms and utilizes each on files better
>suited for each algorithm. That is my theory. If you know, please
>respond!

You forgot IMPLODING, STORING, and the newest algorithm DEFLATING.  
Yes, ZIP picks a different algorithm based on the first N characters
of the file, and also on whether the file actually was compressed. 
-- 
Dave Carr               | dcarr@gandalf.ca       | It's what you learn, 
Principal Designer      | TEL (613) 723-6500     | after you know it all,
Gandalf Data Limited    | FAX (613) 226-1717     | that counts. 
Newsgroups: comp.compression
From: lip@s1.gov (Loren I. Petrich)
Subject: Re: more entropy
Organization: LLNL
Date: Fri, 24 Jul 1992 00:37:09 GMT

In article <1992Jul23.174740.14559@usenet.ins.cwru.edu> daf10@po.CWRU.Edu (David A. Ferrance) writes:

>If I have an unsigned int count[256][256], what is wrong with
>calculating entropy like this:

>for (i=0;i<256;i++) for (j=0;j<256;j++)  {
>    freq = count[i][j] / total;
>    ent += freq * log10(1/freq) / 0.30103;
>    }

	I presume that the code also included:

	ent = 0;
	total = 0;
	for (i=0;i<256;i++) for (j=0;j<256;j++)
	  total += count[i][j];

>Where total and ent are doubles, total is the # of bytes total, ent
>starts off as 0, and the values of the array are the # of occurances of
>each 2 letter combination?

>I get values > 8.

	The theoretical maximum value is log2(256*256) = 16.

	Yes, some versions of C do have a "log2" function (logarithm
to base two).
From: gordona@mof.govt.nz
Newsgroups: comp.compression
Subject: PKZIP password encryption?
Date: 22 Jul 92 23:24:59 GMT
Organization: New Zealand Ministry of Forestry

Hi,
   I have a registed version of PKZIP version 1.10 but the password
   encryption option does not seem to be present.
   In manual.doc and in the printed manual I have it states the following :

          -s<password>  Scramble files with password
          ------------------------------------------

   However when the try the following command

   PKZIP tmp.zip -sSecret  tmp.txt

   I just get the standard usage info as follows :

PKZIP (R)   FAST!   Create/Update Utility   Version 1.1.  03-15-90
Copr. 1989-1990 PKWARE Inc. All Rights Reserved.  PKZIP/h for help
PKZIP Reg. U.S. Pat. and Tm. Off.

Usage: PKZIP [-b[path]] [options] zipfile [@list] [files...]
Options summary - consult the PKWARE documentation for additional information
  -x<filespec|@list> = eXclude filespec(s)            -z = add zipfile comment
                        .
                        .
                   (text deleted)
                        .

  Nowhere in the options information that it prints does it mention -s
  for password enryption?


  Any suggestions much appreciated,
  Joel Gordon.
Newsgroups: comp.compression
From: bx304@cleveland.Freenet.Edu (Jeff Epler)
Subject: Re: My New Compression Algorithm
Reply-To: bx304@cleveland.Freenet.Edu (Jeff Epler)
Organization: Case Western Reserve University, Cleveland, OH (USA)
Date: Fri, 31 Jul 92 02:15:56 GMT


In a previous article, madler@cco.caltech.edu (Mark Adler) says:

>
>>> - Please mail me a -200MB file of yours - now THAT would increase my disk 
>
>While part of this file is in my CPU do I have more registers?

Or imagine the effect of loading the file in one process (Say, a text
editor) while you use something really memory-intensive.

Or having the file repeatedly transferred over networks, to increase
throughput.

At a guess, the decoding takes a negative (Perhaps imaginary?) amount of
time, and would therefore increase the speed of your computer's
processor as well.
>
>mark
Jeff
-- 
|Jeff Epler               Additions Welcome           c(-8 ;-) >{8-) |
| :) (=( =-] (-= Celebrating the variety of faces =-> :^) {-= |-) (: |
|                                                   Lincoln, Nebraska|
Newsgroups: comp.compression
From: dcarr@gandalf.ca (Dave Carr)
Subject: Re: PKZip 2.x
Organization: Gandalf Data Ltd.
Date: Fri, 24 Jul 1992 12:26:30 GMT

In <14n1vhINNrhc@agate.berkeley.edu> bho@ocf.berkeley.edu () writes:


>I was wondering if the current ZIP file format fully machine and
>platform independent; i.e., will it support extended file
>attributes in OS/2, extended attributes/filenames in Unix, etc.

>Along that vein, has anybody heard of any announcement for a
>PKZip for OS/2?

ZIP yes.  PKZIP no.  You need the INFO-ZIP 2.x compression source.
It should be released shortly after PKZIP 2.x.  They are
compatible.

-- 
Dave Carr               | dcarr@gandalf.ca       | It's what you learn, 
Principal Designer      | TEL (613) 723-6500     | after you know it all,
Gandalf Data Limited    | FAX (613) 226-1717     | that counts. 
Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
From: harry@cs.tu-berlin.de (Harald Brinkmann)
Subject: Re: Compression artifacts in NBC Olympic coverage
Organization: Techn. University of Berlin, Germany
Date: Mon, 27 Jul 1992 20:14:40 GMT

Hi There!

Well, I haven't seen those artifacts that you lot in the US rave
about. Anyway here some some thoughts of mine about that:

They could be standard conversion atrifacts, but I'm surprised that
no-one has ever noticed them elsewhere. I have heard that they use at
least for slow motion (I'm not sure about standard conversion) a new
system with motion estimation, which is intended to make movements
more smooth. Fast movements on existing standard converters looks
really messy. (For those of you who can pick up PAL signals look for
Spacenet S2 at 69 deg 3920 MHz Vertical which is supposed to be
ECO/Galavision for Europe already PAL transcoded. Look for for moving
letters like scrolling subtitles or credits.)

American TV programmers often insist on using their own cameras in
international events. They may relay the international feed and mix it
with their own output. Often their own cameras are in NTSC, so there
would be no stanard conversion then. Indoors with 50 Hz lighting this
may be impractical though, but I have seen that during the Winter
Olympics for Outside broadcasts.

There is full HDTV coverage with is aired in Europe on Olympus,
TV-SAT, TDF, Telecom 2A and Eutelsat II F3. Compatible reception in
D2MAC yields some artifacts (I wrote about them during HDTV coverage
of the French Open Tennis Championchip). However in the meantime the
coders have been modified to produce fewer artifacts and the
difference is clearly visible. The artifacts are still there, but far
less annoying. I doubt that any HDTV feeds will be used for "normal"
TV. This at least didn't happen for sure during the opening ceremony
which I watched in both D2MAC (HDTV) and the normal international
feed. Anyway if they did use the feeds *I* would use a proper
transcoder which avoids these artifacts. I went to the SFB today where
an HDTV demo, with a rpoper decoder ran and saw no artifacts on there
either.

Hope this is of some help

  Harald

Newsgroups: comp.compression,rec.video,rec.video.satellite,rec.video.production
From: dhd@exnet.co.uk (Damon)
Subject: Re: Compression artifacts in NBC Olympic coverage
Organization: ExNet Systems Ltd Public Access News, London, UK
Date: Mon, 27 Jul 1992 21:07:09 GMT

In article <1992Jul27.064849.15994@noao.edu> tody@noao.edu writes:
>In article <1992Jul27.035759.153188@zeus.calpoly.edu>, jemenake@zeus.calpoly.edu (Joe Emenaker) writes:
>> Well, bravo. I'm happy for you that you're smart enough to notice such
>> things. I'm sure you must be a very bright little boy. To quote Monty
>> Python: "Gee, we're all really impressed down here, I can tell you.".
>> However, I'm sad for you that knowing that NBC is compressing the signal is
>> ruining your experience of the Olymics so much...
>>
>> .. Is it REALLY that unbearable? For heaven's sake, just sit a few feet futher
>> from the screen or take your glasses off or something.
>>
>> Granted, it is a subtle thing and I AM truly impressed that you caught it.
>> And I acknowledge your attempt to use it to cash in on your 15 minutes of
>> fame that Andy Warhol says each of us will have. But, I think that you're
>> going overboard by expecting NBC to fork over more money to make the
>> transmission absolutely perfect when so few people are even going to
>> nitice... and fewer are even going to care.
>
>Lighten up.  I (and I think many others) also noticed it - it was very
>apparent, distracting, and mildly annoying.  Personally I would just ignore
>it, I suspect NBC is just experimenting with some new technology and they
>will probably get enough complaints so that something is done about it.
>
>Doug Tody  (NOAO)

Yes, do lighten up.  NBC *want* people to watch.  If what they do irritates
people (eg playing muzak over the action) then they'd rather people told them
so they could change it.  So yes I *would* expect NBC to spend lots of money on
getting this sort of thing right to keep the viewers happy.  That's why we use
TV rather than radio, yes?  (BTW, I'm in the UK and so don't see NBC at
all---this makes me very happy B^>)

Damon
-- 
Damon Hart-Davis | Tel/Fax: +44 81 755 0077 |1.21|| ALL MAIL FREE.
Internet: dhd@exnet.co.uk | Also: Damon@ed.ac.uk || Hotrod motor groups soon.
--------------------------+----------------------++ >12 mail&news polls / day.
Public access UNIX (Suns), news and mail for #5 per month.  FIRST MONTH FREE.
Xref: sparky comp.compression:2872 rec.video:9782 rec.video.satellite:3879 rec.video.production:1198