From: edhall@rand.org (Ed Hall)
Newsgroups: comp.compression
Subject: Re: Too much lossy encoding?
Keywords: PASC, audio
Date: 14 Apr 92 02:13:45 GMT
Organization: The RAND Corporation

In article <1992Apr13.212222.7605@tc.fluke.COM> kurt@tc.fluke.COM (Kurt Guntheroth) writes:
>I want to clarify some things I've said in light of the discussion that has
>ensued.  
>
>The following things concerns me about lossy encoding 
>1.  People are "lowering the bar" for what is acceptable signal loss.  The
>    goal of CD encoding (which is only lossy in the Nyquist sense) was to
>    obtain about the same fidelity as an ideal phonograph record.  I have
>    recently seen the PASC scheme compared to the quality of FM radio
>    reception.  It is my opinion that people are focusing more strongly on
>    how mush loss will fit the program material onto the storage or
>    transmission medium than they are on how much loss is, in fact,
>    imperceptible.

You're entitled to your opinion as to what "people are focusing on,"
although I think you're worrying a bit too much about something which
hardly threatens the quality of high-end sound recording.  The proposed
system is an improvement over the mass-market cassettes it is targeted
to replace.  But it was hardly conceived as a no-compromise medium.  If
the right compromises in terms of cost, quality, and capacity have been
made, it will find its market.

And, face it, there is no such thing as a "no-compromise" medium.
[Personally, I feel that two-channel sound is a tremendously compromised
system from the start--but that's another story.]

>2.  A given digital encoding scheme generally defines a single level of
>    quality such that if you can decode the signal at all, you get that
>    level of quality and no more.  Another way of looking at this is that
>    the transmitter/encoder of a digital signal makes all the decisions
>    about the quality of the received signal, and no amount of money or
>    technology at the receiving/decoding end will improve the quality of the
>    signal over this minimum level.  By contrast, in most analog systems,
>    a very fine signal is transmitted/encoded, and can be recovered under
>    ideal conditions.  The quality of the actually received signal can be
>    improved up to a certain limit by spending more on the receiver/decoder.
>    This attribute lets the user determine signal quality.

I have strong doubts of this.  The vast majority of vinyl recordings
emerged from the pressing plant with less capability for superior sound
than the average CD and player.  The average FM transmitter and
transmission path distorts the signal far more than a mid-line consumer
receiver.  Sure, you can refine the playback end to asymptoticly
approach perfection, but only a tiny fraction of signal sources will
provide much return on that investment.  And PASC is no threat to those.
If people will pay enough of a premium for high-quality media--and some
will--the market will provide, just as it does now.

>3.  Multi-format formats are only useful if the receiver can understand them
>    all, at least to some extent.  If each sub-format requires its own
>    receiver hardware, we've hardly improved anything have we?

I'm not sure I understand your complaint.  Hardware can only get
cheaper and more versatile.  Formats don't spring into being
spontaneously.  A better-integrated and more adaptable format will
win out, all other things being equal.  It might take a few years,
and there is a risk that one of the players might badly stumble
(like Sony with Beta), but this is one area where technology seems
to continuously make things better--at least for most people.

>There exist encoding techniques, like some of the HDTV techniques, that
>encode some information at a higher power than other information, so that
>signal quality falls off gracefully with distance from the transmitter.
>Another example would be a storage format with 20 bits of precision, and a
>decoder that could decode 14 bits, or 17, or 20, depending on the needs of
>the users.  These digital techniques have the same desirable user-oriented
>attribute that the user chooses the fidelity of playback.

Desirable to whom?  If medium A can only present half the material of
medium B, just so a few percent of consumers have more tweaking
headroom, it will be a market flop.  Or do you want to have a Government
Standards And Purity Administration force the manufacturers to chose a
marginally higher-quality standard at the cost of consumer convenience?
That won't work in a global market.

>On the other hand, many other proposed systems do not possess this attribute.
>They fix playback fidelity to conform to the instantaneous market and
>technology conditions of a particular day, not to theoretical absolute
>limits or traditional limits like available channel bandwidth.

This has always been true.  The only difference is that with digital
systems it is much easier to derive the upper bound.  It was always
there.

>As a consumer I don't want to have to buy a format that will become obsolete
>in a very short time as storage density and perhaps transmission bandwidth
>increase.  Like the 8-track player, which was rapidly bypassed by better
>technology, I think such systems may not pay back the investment in designing
>them.

That's always the risk.  But on the other hand, who will buy a medium
which is inferior by market standards just so 3-5 years from now we'll
have something better?  The manufacturers have to sell to someone...
and if they don't, someone else will.

> As an engineer, I despise the shortsightedness of some of the designs
>I've seen described.  As electronic display and playback device technology
>advances, the limit of what is perceptible will change.  Setting your sights
>too low as a designer will not pay back your investment.

Neither will setting your sights too high.  Remember the Elcassette?

>I won't throw out my library of software every five years to upgrade
>to a better format.  We all did that once, with records-to-CD's, but we were
>told CD's will last forever (remember that?  Now they say maybe 10 years).
>I won't do it again soon, and certainly not frequently.  Every little
>dead-end compression scheme will make the next one harder to sell, even if
>it is better, and especially if it is not.

I doubt if CDs will go away any time soon, until something better--even
by your refined standards--comes along.  But CDs have problems for the
portable market.  The cassette lacks many of the advantages of the
proposed digital formats (random access, reproducibility, size).  So
there is room for a new medium.  RDAT is expensive overkill for the
consumer market--another example of setting ones sights too high.

>Here's some questions.  These are the reality checks for claims of
>imperceptible signal loss.  This is the test for whether the signal loss is
>REALLY imperceptible, or just a hack to fit today's storage technology.
>
>1.  Anybody designing storage formats where the receiver determines the
>    fidelity in a non-trivial way?

Some people claim CDs do that now. But it really comes down to this:
given a limited budget of bits, where do I put that extra fidelity?
If 5% of the population wants better fidelity, while 95% would rather
not sacrifice program length for this, which makes the most economic
sense?  And unless engineering is a hobby, which is the better
engineering choice?  It is hard to engineer for a bankrupt company.

>2.  How long did the 78 rpm phonograph format last?
>    How long did the 33 1/3 rpm LP format last?

Non sequitur, unless you can show that either was created by explicitly
holding back inferior technologies.  The market did the weeding out, as
it will now.

>    How long did the NTSC TV format last?
>	Color was added in a compatible way.  Even so it took years for a
>	significant number of people to upgrade to color.

This was a matter of cost.  Few people could afford to upgrade.  If
manufacturers make mini disk players cheap enough, and with features
that give them an edge over cassettes, they'll be able to sell them once
a supply of program material is available.  And they're working to
guarantee that.

>    All these formats were replaced by significantly better ones, both in
>    terms of ease of use and signal quality.  All these formats improved
>    during their lifetime, especially the quality of playback equipment.

Indeed.  But how is this germane?  We live in a multi-format world,
now.  Varnish and vinyl records were about the only game in town during
their days, and players were pieces of living room furniture.  As was
furniture for holding records.  Mini disk players will be able to fit in
a shirt pocket, and taken anywhere.

>3.  How many people will give up CDs for mini disks and digital cassettes
>	and MPEG players if the signal quality is less good?  

Lots, if in lets them listen or watch in ways that are more useful to
them, and the quality is "good enough."

>    When CD-I (or whatever) comes along, and data comes on a thing that
>	LOOKS like a CD, what will be users reaction if audio data doesn't
>	SOUND like a CD?

Once again, if having the added capabilities is worth what might be to
the consumer is a small decrease in sound quality, they'll flock to them
if the price is right.

>4.  How many people will replace their broadcast receivers and VCR's if the
>	digital format screws up the images of sports events?  How will
>	advertisers feel about putting their commercials on events that
>       don't broadcast very well?

They won't have to.  Nobody is foolish enough to mass-market a system
with such obvious defects.  A great deal of progress has been made in
dealing with motion artifacts--much of it on the encoding side.

>5.  Have we really exhausted what we can do with lossless compression?  Have
>    we really exhausted what we can do with reconstruction algorithms that
>    look at successive cycles of audio signals to sneak around the Nyquist
>    limits?  Are we solving the wrong problem?

It is a matter of diminishing returns.  Besides, for a given number of
bits, a "lossy" compression scheme is likely to be *better* than a
"lossless" scheme.  Think of it this way: if you have the choice of a
320x240 pixel lossless-encoded image in your 100 kilobytes, or a 960x720
JPEG version of the same scene, which would you choose?  Would you
rather have a "lossless" NTSC TV picture, or a lossy-compressed HDTV
picture in your 6MHz TV channel, given that motion-compensation is
effective (and it will be)?

Sure, in ten or twenty years some of us might have fiber to our homes,
and holographic data storage might be used for consumer recordings.  Do
we sit and wait for this magical bandwidth and data capacity to arrive,
or do we squeeze as much as we can out of the bandwidth and capacity we
have now?  If we don't, someone else will.

		-Ed Hall
		edhall@rand.org