HDTenEightyP/NetNews huggingface dataset:
From: wdw@math2.sma.usna.navy.MIL (Wm. Douglas Withers -- math  FACULTY <wdw@sma.usna.navy.mil>)
Newsgroups: comp.compression
Subject: Re: Math Question
Date: 16 Nov 92 13:44:02 GMT
Organization: U.S. Naval Academy

In article <24005@hacgate.SCG.HAC.COM> abirenbo@rigel.cel.scg.hac.com (Aaron Birenboim) writes:
>
>  I seem to remember some terribly nasty formulas for handling the
>differentiation before evaluating any integral.
>
>  I think it involved the bounds, derivitives of the bounds, the integrand
>and the derivitave of the integrand.
>
>  any advise, help, or refrences are welcome.

I think you want the following:

       /            / g(q)        \
      |            |               |
d/dq  |   Integral |    f(x,q) dx  |  =
      |            |               |
       \          / h(q)          /


                                              / g(q)
                                             |
     f( g(q) ,q) dg/dq - f( h(q) ,q) dh/dq + |   (df/dq)(x,q) dx
                                             |
                                            / h(q)

-- 
Els Withers                   email: wdw@math2.sma.usna.navy.mil
Department of Mathematics     phone: (410)267-3192 / fax: (410)267-4883
United States Naval Academy
Annapolis, MD  21402-5002     Senri no tabi mo ippo yori...
Newsgroups: comp.compression
From: bithead@cs.montana.edu (Bob Wall)
Subject: Re: Need a compressor for sparse bit datastream
Keywords: compression technique
Organization: Dept. of Computer Science, MSU, Bozeman Mt 59717
Date: Mon, 16 Nov 1992 23:20:15 GMT

In article <1992Nov13.120505.29654@spectrum.xerox.com> landells.sbd-e@rx.xerox.com writes:
>I have an application that generates binary output.  The output is relatively random, but there are approximately twice as many off bits as on bits.  My objective is to compress this as much as possible.  
>
>I have tried several 'standard' compressors, arj 2.2, lharc, pkzip 1.1, and have only managed to achieve very minimal compression in the order of 4% at best (on a 40K file).  Now I know that a truly random binary datastream cannot be compressed, but I was kind of hoping for better than 4%.  Am I missing something fundamental, or is this really the best that can be achieved?  
>
>If there is a technique to compress this type of data, I would appreciate some pointers to some source code that implements it.
>
>
>Richard Landells  (landells.sbd-e@rx.xerox.com)
>Rank Xerox System Centre

   You might try a variation of arithmetic coding referred to as the BAC
(Binary Arithmetic Code).  It is described in the paper

   Glen G. Langdon, Jr. and Jorma Rissanenm, "A Simple General Binary Source
      Code," _IEEE Transactions on Information Theory_, Vol. IT-28, No. 5,
      Sept. 1982, pp. 800-803.

   I implemented this once, and I remember that it did OK on binary data
streams where the frequency of one bit was substantially higher than the
frequency of the other bit (and I would classify 2 to 1 as substantially
higher) but I don't remember just what compression ratio OK would be.

   It's a pretty simple algorithm, although the article is written in
math, instead of English  B^(.  A more straightforward explanation (in
near-English) is given in

   Glen G. Langdon, Jr., "An Introduction to Arithmetic Coding," _IBM
      Journal of Research and Development_, Vol. 28, No. 2, March 1984,
      pp. 135-149.

   This article also has some other references that might help, although I
haven't tracked them down.


Hope this helps,
Bob


-- 
=============================================================================
  bithead@fubar.cs.montana.edu          (Bob Wall,  CS grad student)
   "Software means never having to say you're finished."
			--J. D. Hildebrand in UNIX REVIEW
Xref: sparky comp.compression:3883 comp.compression.research:283
From: nevries@accucx.cc.ruu.nl (Nico E de Vries)
Newsgroups: comp.compression,comp.compression.research
Subject: Data compression SOURCES available NOW !!!
Date: 17 Nov 92 18:14:43 GMT
Followup-To: comp.compression
Organization: Academic Computer Centre Utrecht

Hi,

It took some time but LDS_10 finally made it to garbo! Thanks
to Timo for helping me with that.

I made version 1 of the LDS (lossless datacompression sources) kit.
I noticed lots of people want sources for datacompressors and it
can be hard to get them. Especially good ones. I have spend a fortune
on collecting compression sources (I need them for an archiver I
am working on) and decided to create a package containing the ones
I think are most wanted and interesting.

The kit includes AR002 (comparable with ARJ 2.39, LHA, ZOO 2.11,
PKZIP 1.93a, ZIP 1.9), LZW, Huffman, Splaytree's, Ashford, LZHUF,
LZSS, LZARI, ARI, higher order ARI etc.

The file is available on garbo.uwasa.fi, /pc/programming/LDS_10.ZIP


Nico E. de Vries  (nevries@cc.ruu.nl) |------------------*   AA   III  PPP
_ This text is supplied AS IS, no warranties of any kind |  A  A   I   P  P
| apply. No rights can be derived from this text. This   |  AAAA   I   PPP
| text is likely to contain spelling and grammar errors. |  A  A   I   P
*---------------------------( Donate to GreenPeace! )----*  A  A  III  P

"The IBM PC is still waiting for a version of the CP/M OS.", G.M. Vose, 1982.
Newsgroups: comp.compression
From: skow@cs.utwente.nl (Jacek Skowronek)
Subject: HQX compression format question
Organization: University of Twente, Dept. of Computer Science
Date: Wed, 18 Nov 1992 10:06:42 GMT

We are working on a project with some companies which use Macs to
write their documents. They send us those compressed in HQX (BinHex)
format.
Does anybody know how to unpack them on the Sun? How this format
can be converted to any other format such as compress/uncompress or tar?
Is it at all possible to use those formats on the Mac (then we can ask them
to compress it differently).

Jacek

-- 
-------------------------------------------------------------------
Jacek Skowronek				e-mail: <skow@cs.utwente.nl>
Department of Computer Science		tel: +31-53-893688
Information Systems Workgroup		fax: +31-53-339605
University of Twente
P.O. Box 217
7500 AE Enschede, The Netherlands
Newsgroups: comp.compression
From: a7657@mindlink.bc.ca (Stephen H. Kawamoto)
Subject: Re: PKZIP - What's the newest version
Organization: MIND LINK! - British Columbia, Canada
Date: Sun, 15 Nov 1992 19:53:55 GMT

Look for version 1.93a of PKZIP but please! whatever you do, don't archive
files with it!
--
Stephen Kawamoto <a7657@mindlink.bc.ca>
    "It is the very mind itself
     That leads the mind astray;
     Of the mind,
     Do not be mindless."
     (an old Japanese folk song)
From: mchst12@cislabs.pitt.edu (Mark    C Hahn)
Newsgroups: comp.compression
Subject: non-dictionary, non-statistical compression
Summary: any source available?
Keywords: DMC PPM compression context state
Date: 21 Nov 92 02:14:46 GMT

We seem to have lots of sources available for various versions of
LZ-derived dictionary methods, and some statistical ones using
Huffman coding, arithmetic, etc.  Does anyone know of sources 
some of the other techniques described in Cleary,Witten&Bell?

thanks, Mark.
Xref: sparky comp.compression:3910 comp.compression.research:296
From: cmcgldrk@unix2.tcd.ie (Wowbagger - the infinitely Prolonged)
Newsgroups: comp.compression
Subject: RM8 - Where to get it?
Summary: RM8-Where to get document?
Keywords: RM8 Where to find?
Date: 18 Nov 92 17:33:04 GMT
Organization: Trinity College, Dublin

Can someone please mail me details of where
to find document RM8, as referred to in CCITT
 H261 spec.
E-mail to me or else send me a site address,
if one exists.
Thanks.
Ciaran Mc Goldrick.
From: mjn@tik.vtt.fi
Newsgroups: comp.compression
Subject: CCIR Recommendation 601
Date: 19 Nov 1992 15:05:31 +0200
Organization: Technical Research Centre of Finland, Laboratory for Information Processing (VTT/TIK)
Distribution: world
NNTP-Posting-Host: tik.vtt.fi

Could anyone tell me where I can get the CCIR Recommendation 601 ?
Is there an electronic version which could be obtained by ftp ?

mjn@tik.vtt.fi

Newsgroups: comp.compression
From: ohumukin@spectra.com (Kawika Ohumukini)
Subject: Voice Mail Card and Stacker
Organization: Spectragraphics Corporation
Date: Mon, 16 Nov 92 21:45:30 GMT

Hi,

Just wanted to follow up on my findings.

I received the card and I recorded a message exactly 15 seconds long
with constant talking.

Stacker reported   1.2:1 compression ratio

Not to bad.  Luckily my card has a blanking feature where if the
volume from the line falls below a threshold and stays there for
XX seconds (stop talking, pause, etc..) then the voice file comes
out smaller by stopping recording.  It is all done in RAM so I 
should be able to set it to 1 second without problems.

Thanks to all who responded and told me not to expect much which is
what I got.  Time to buy another harddisk.

Cheers,
Kawika
From: nevries@accucx.cc.ruu.nl (Nico E de Vries)
Newsgroups: comp.compression
Subject: Re: Pkzip 2.0 in the 23'rd century?
Date: 16 Nov 92 17:24:00 GMT
Organization: Academic Computer Centre Utrecht

In <17484@mindlink.bc.ca> Robert_Salesas@mindlink.bc.ca (Robert Salesas) writes:

>Why, did version 1.1 stop compressing files?  You bought a product that
>worked, it still does, I'd say that's a good investment.  Is it a bad
>investment to buy a 1993 Accord because the 1994 will "soon be released"?

>In my opinion the only mistake PKWare did was announce 2.0.  They need not
>apologize for not releasing it if it isn't ready, for whatever reason.

Sounds reasonable. Unfortunately the REAL problem is PKWARE has competition.
To try to keep customers they released a alpha version of 2.0 and announced
2.0. If PKWARE would have been completely quiet by now much more people
would be using LHA, ZOO, ARJ etc because those compress better than PKZIP
1.10 does.

>Rob

Nico E. de Vries  (nevries@cc.ruu.nl) |------------------*   AA   III  PPP
_ This text is supplied AS IS, no warranties of any kind |  A  A   I   P  P
| apply. No rights can be derived from this text. This   |  AAAA   I   PPP
| text is likely to contain spelling and grammar errors. |  A  A   I   P
*---------------------------( Donate to GreenPeace! )----*  A  A  III  P

"The IBM PC is still waiting for a version of the CP/M OS.", G.M. Vose, 1982.
Newsgroups: comp.compression
From: Mischa_Sandberg@mindlink.bc.ca (Mischa Sandberg)
Subject: Re: Need a compressor for sparse bit datastream
Organization: MIND LINK! - British Columbia, Canada
Date: Fri, 20 Nov 1992 04:22:17 GMT

Tom Lane writes:
> richard@garfield.noname (richard_landells.sbd-e@rx.xerox.com) writes:
> > I have an application that generates binary output.  The output is
> > relatively random, but there are approximately twice as many off bits as
> on
> > bits.  My objective is to compress this as much as possible.
>
> The fact that standard compressors can do as well as 4% suggests to me that
> there is more order in the input bitstream than you've let on.
>                                 regards, tom lane

... yes; I'm trying to figure out how the subject line ("sparse"?) relates
to data that is 67% 0's instead of 50% 0's.

Every compressor has its model, even supposedly "all-purpose" compressors
like LZ or arithmetic coding. Clearly, if standard stream compressors
flunk out on your data, then there's no strong serial dependency between
one byte and the next, nor does it chunk in 8-bit symbols.

But then, perhaps there's some information you have about the source of
this data. Perhaps it chunks on a different unit size (which might be VERY
large)? Perhaps the sequence of zero-bit-counts between unit-bits has some
significant distribution, or even a significant markovian?

It would probably give people a better crack at your problem if you
could relate the kind of data you're storing. Good luck, in any case.
--
Mischa Sandberg ... Mischa_Sandberg@mindlink.bc.ca
                 or uunet!van-bc!rsoft!mindlink!Mischa_Sandberg
*-*-*-*-*-*-*-*-*-*-*
Engineers think equations are an approximation of reality.
Physicists think reality is an approximation of the equations.
Mathematicians never make the connection.
From: nevries@accucx.cc.ruu.nl (Nico E de Vries)
Newsgroups: comp.compression
Subject: Re: Pkzip 2.0 in the 23'rd century?
Date: 17 Nov 92 18:17:16 GMT
Organization: Academic Computer Centre Utrecht

(Timo Salmi) writes:

>>PKWare has stated (maybe this is *the* time) that PKZip 2.x will be
>>released at the end of November, and that it is in final beta.

>Sure, but which year?   (Sorry :-)

PKWARE announced end november (1992) on their BBS. Either they are again
making fools of theirselves or they finally made it.

>   All the best, Timo

Nico
From: ck060@cleveland.Freenet.Edu (Eric L. Hinson)
Newsgroups: comp.compression
Subject: uuencode/decode & ".Z" compress.decompress files for MS-DOS
Date: 16 Nov 1992 05:35:36 GMT
Organization: Case Western Reserve University, Cleveland, OH (USA)
Reply-To: ck060@cleveland.Freenet.Edu (Eric L. Hinson)
NNTP-Posting-Host: slc10.ins.cwru.edu


I have an MS-DOS machine, and am looking for the .exe or .com files
nessisary to uuencode/decode and compress/decompress using the ".Z" type of
compression found on unix based ftp machines.

Could someone who has these files please e-mail them to me uuencoded?  I
have access to uudecode/.ZDecompression on the system I use, but it would
be much easier (and cheaper for me too :) to be able to do that sort of
stuff offline [those pc persuit hours add up in a hurry!]

Thanks for your help.

Eric L. Hinson (ck060@cleveland.freenet.edu)
From: ujacampbe@memstvx1.memst.edu (James Campbell)
Newsgroups: comp.compression
Subject: Re: ARJ password cracker.
Date: 16 Nov 92 14:13:57 -0600
Followup-To: comp.compression
Organization: Memphis State University

In article <3327@accucx.cc.ruu.nl>, 
                    nevries@accucx.cc.ruu.nl (Nico E de Vries) writes:

>          ...Unfortunately most common ones are crackable (WP, PKZIP, ARJ).
> I know of WP and PKZIP crack programs but unfortunately not of ARJ crack
> programs. ARJ encryption is very weak but nooane seems to have made a tool 
> for it.

Why is ARJ's encryption "weak?"  Also, where did you see a crack program for
PKZIP?  All I've seen has been a couple of brute-force pass phrase checkers.


  ========================================================================;-)
  James Campbell, Math Sciences Department, MSU; ujacampbe@memstvx1.memst.edu
  ---------------------------------------------------------------------------
  The above text has not been reviewed by DIRNSA, and should not be construed
  as an official release of the National Security Agency or any of its member
  organizations. No non-public information shall be included in this message.
  ===========================================================================
From: tgl+@cs.cmu.edu (Tom Lane)
Newsgroups: comp.compression
Subject: Re: Need a compressor for sparse bit datastream
Keywords: compression technique
Date: 17 Nov 92 01:07:53 GMT
Organization: School of Computer Science, Carnegie Mellon

richard@garfield.noname (richard_landells.sbd-e@rx.xerox.com) writes:
> I have an application that generates binary output.  The output is
> relatively random, but there are approximately twice as many off bits as on
> bits.  My objective is to compress this as much as possible.  

> I have tried several 'standard' compressors, arj 2.2, lharc, pkzip 1.1, and
> have only managed to achieve very minimal compression in the order of 4% at
> best (on a 40K file).  Now I know that a truly random binary datastream
> cannot be compressed, but I was kind of hoping for better than 4%.  Am I
> missing something fundamental, or is this really the best that can be
> achieved?  

Most 'standard' compressors work by looking for repeated byte strings.
If your source is really random then this won't be very effective since
you won't have much repetition at the byte level.

However, you should be able to exploit the 2:1 ratio of 0s to 1s to get
better than 4% compression.  To do this will require arithmetic compression.
Huffman would work if the symbols were bigger than a bit, but since Huffman
codes can't be less than one bit, you can't get any compression with
single-bit input symbols.  Arithmetic coding wins because it can encode
source symbols into "fractional bits".  Arithmetic compression code is easy
to come by, see the comp.compression FAQ.  (But if you want to use this for
any commercial purpose, beware of patents.)

But wait... if all we know about your source is that 0's have probability
.67 and 1's have probability .33, then the entropy is
	-(.67*log2(.67) + .33*log2(.33))
or 0.92, which means you could theoretically achieve at most 8% compression
this way.  I'd say implementing arithmetic coding on individual bits is
probably not worth the trouble.  Are you sure the 0's and 1's are
uncorrelated?  If not, higher-order arithmetic coding might work.
The fact that standard compressors can do as well as 4% suggests to me that
there is more order in the input bitstream than you've let on.

				regards, tom lane
Newsgroups: comp.compression
From: eduard@htsa.aha.nl (Eduard Beijnes)
Subject: compressor writing uuencoded files
Date: Thu, 19 Nov 1992 14:19:20 GMT
Organization: Algemene Hogeschool Amsterdam, Polytechnic Institute


Are there compressors who write uuencoded files ?

The files are for transport via the net.

Something like:
arj a -vv60000 -uuencoded postfile program.exe

(the result is compressed files uuencoded < normal uuencoded files)
Organization: Penn State University
Date: Thu, 19 Nov 1992 14:48:50 EST
From: <ECN101@psuvm.psu.edu>
Newsgroups: comp.compression
Subject: LHZ File Type

Could anyone tell me where to get the Decompressor for this type, I just downlo
aded a file and unzipped it to find that it was of type LZH (Typo in Subject He
ading) The best thing that I can do right now is just look at the directory str
ucture with one file that I got of the CAC-PC at Penn State.  Any help would be
appreciated.
Newsgroups: comp.compression
From: ts@uwasa.fi (Timo Salmi)
Subject: Re: Pkzip 2.0 in the 23'rd century?
Organization: University of Vaasa, Finland
Date: Mon, 16 Nov 1992 09:07:03 GMT

In article <1992Nov15.162050.1683@msus1.msus.edu> cs272017@eeyore.stcloud.msus.edu (cs272 user 017) writes:
>PKWare has stated (maybe this is *the* time) that PKZip 2.x will be
>released at the end of November, and that it is in final beta.

Sure, but which year?   (Sorry :-)

   All the best, Timo

..................................................................
Prof. Timo Salmi
Moderating at garbo.uwasa.fi anonymous FTP archives 128.214.87.1
Faculty of Accounting & Industrial Management; University of Vaasa
Internet: ts@uwasa.fi Bitnet: salmi@finfun   ; SF-65101, Finland
Newsgroups: comp.compression
From: garys@miki.pictel.com (Gary Sullivan)
Subject: Re: CCIR Recommendation 601
Organization: PictureTel Corporation
Date: Fri, 20 Nov 1992 23:18:46 GMT

In article <1eg3erINN5vn@tik.vtt.fi> mjn@tik.vtt.fi writes:
>Could anyone tell me where I can get the CCIR Recommendation 601 ?
>Is there an electronic version which could be obtained by ftp ?


One easy place to find it is in the Signetics Video Products book,
June 1991, pages 2-29--2-38.


-- 
---------------------------------
Gary Sullivan  (garys@pictel.com)
From: bell@ufp.enet.dec.com
Newsgroups: comp.compression
Subject: Re: Q: How well do random files crunch?  A: Not very.
Date: 19 Nov 92 13:40:38 GMT
Reply-To: bell@ufp.enet.dec.com ()
Organization: Digital Equipment Corporation



Thanks for all e-mail and replies.  I figured as much, but wanted to make sure.

Mike
Newsgroups: comp.compression
From: bill@sees.bangor.ac.uk (Bill Denton (AD))
Subject: Re: Re: Lossless Datacompression Source kit 1.0
Organization: University of Wales (Bangor)
Date: Tue, 17 Nov 1992 14:45:43 GMT

I found the Lossless Datacompression Source kit 1.0 on garbo.uwasa.fi
as /pc/programming/lds_10.zip. It wasn't there when I made the original posting.
Sorry.

Bill Denton.
From: raphe@euler.Berkeley.EDU (Raphe Berry III)
Newsgroups: comp.compression
Subject: The Best Compression around, PD or NOT PD
Date: 17 Nov 92 19:58:40 GMT
Reply-To: raphe@euler.Berkeley.EDU (Raphe Berry III)
Organization: University of California, Berkeley

Hi,

I am new to this group and I was wondering what is the best compression 
program around.  I have been using for most of my like Pkzip, but after
using Lharc on my Unix account and on my Dos machine, I am now in love.
But I want better compression, I don't care if it takes a long time...
I just got to have it!!  Prefer it to be pd too, but doesn't have to 
be.



Raphe Berry III


---------------------------------------------------------------------------
raphe@ocf.berkeley.edu       |  "Liberty is a bitch who must be
raphe@euler.berkeley.edu     |   bedded on a mattress of corpses"
raphe@sodium.berkeley.edu    |           -from the _Sandman_ comic book
---------------------------------------------------------------------------
Newsgroups: comp.compression
From: leslie@sneed.rtp.dg.com (Scott Leslie)
Subject: Error Correction Information Wanted
Date: Wed, 18 Nov 92 14:07:05 GMT
Organization: Data General Corporation. RTP, NC.
Keywords: error correction

Could someone send me some information (or references to information)
about how to provide multi-bit error correction for a given bit stream.
Basically, I'd like to know how to determine the number of bits required
to correct all errors up to n bits in an m bit stream.

Thanks, Scott

+------------------------------------+-----------------------------------+
| Scott P Leslie                     |           (919)248-5802           |
| X Clients & Libraries              |                                   |
Newsgroups: comp.compression
From: compton@plains.NoDak.edu (Curtis M. Compton)
Subject: Re: compressor writing uuencoded files
Date: Thu, 19 Nov 1992 19:17:33 GMT
Organization: University of North Dakota, Grand Forks

In article <1992Nov19.141920.27289@htsa.aha.nl>
	eduard@htsa.aha.nl (Eduard Beijnes) writes:

>Are there compressors who write uuencoded files ?
>The files are for transport via the net.

>Something like:
>arj a -vv60000 -uuencoded postfile program.exe

>(the result is compressed files uuencoded < normal uuencoded files)

Why not simply do the following?

% compress yourprogram.exe | uuencode - yourprogram.exe.Z > output.uue

Or you could pipe it to a mailer (elm, for example) or to a news poster
(Pnews, for another example).

% compress yourprogram.exe | uuencode - yourprogram.exe.Z | elm friend@foobox

-- 
Curtis M. Compton           | Future home of Rookerie Software |
compton@plains.NoDak.edu    +----------------------------------+
560 Carleton Court, Apt #11 | "I am the Walrus"               - John Lennon
Grand Forks, ND 58203       | "I am the Keeper of the Cheese" - Ren Hoek
(701) 795-5099              | "I am the Master of Penguins"   - Curt Compton
Newsgroups: comp.compression