From: cantrell@seqp4.sequoia.com (Paul Cantrell)
Newsgroups: rec.video
Subject: Re: HDTV Myths (again)
Date: 9 Oct 92 20:04:39 GMT
Organization: Sequoia Computer Systems

In article <1992Oct8.051631.15325@foretune.co.jp>, trebor@foretune.co.jp (Robert J Woodhead) writes:
> 2) Each frame should have a "center point" encoded in it, to facilitate
> "pan and scan" on display devices that cannot support the actual
> aspect ratio of the image.

I really like this idea. I can think of a couple ways to improve it, but
the basic idea should definitely be included in an HDTV standard!

A center point doesn't give you all the information you need to pan &
scan, because you need to guarantee certain areas get displayed. Cutting
off a nose during a close up isn't normally acceptable. So you need to
establish an area of important information. So in addition to the "center
of attention" position, we probably want to define a rectangle which says
"make sure all of this information is displayed". Call it the
"frame-important-info" (FII) rectangle. Information outside the FII
rectangle is okay to lose during a P&S operation.

The aspect ratio must be picked by the display such that no information
inside  any FII rectangle will ever be lost. But if you start in the
middle of a movie, the display has no way of knowing what FII rectangles
are going to be encountered. This means that each frame needs to have a
rectangle defined which defines the widest FII rectangle in the entire
film. The display aspect ratio will then be picked so that this worst case
rectangle can be displayed. Call this the "program-widest-important-info"
or PWII. In reality, encode 4, with a distribution of something like 85%,
95%, 99%, 100%, such that at the given setting, that percentage of FII's
will be displayable without any loss. This allows the user to influence
the degree of letterboxing the display will choose, depending on how many
cut-off noses the viewer is willing to put up with. When special sections
of the movie are encountered which must be displayed at a different aspect
ratio (like credits during a P&S movie), the PWII is simply changed, causing
the display to pick a different aspect ratio.

The display picks an aspect ratio based on PWII and a user preference
setting, then proceeds to P&S in order to keep each FII visible, with the
"center point" centered as much as possible. If we include a frame number
with the FII, it can be used to indicate when to start the pan (and how
fast to pan) so that the FII will be totally displayed by the time frame
#n is reached. This allows a slow pan to a new location on the screen. An
FII  with a frame # equal to the current frame # means the display should
just jump to the new area of interest, rather than pan from one area to
the other. Running the movie backward might have some interesting side-effects
with this scheme. :-)

Another related subject which has not been discussed is sound tracks. If
the display is going to P&S, it probably should have control of the stereo
image. Otherwise, an image on stage right which has been P&S so that it
now occupies the center of the display would sound strange if the sound
still comes from stage right. So for the 6 or so separate digital audio
tracks that I think should be included with the video, there should be a
sound location encoded for each track. The tracks are then mixed by the
display to create the correct stereo separation. Including a 'Z' location
would allow you to encode front versus rear if this is necessary.
Otherwise, effects tracks could simply be marked as an effects track while
in use (you would "steal" tracks for effects use. When no effects are in
progress, all tracks are available for location encoded audio
information).

The amount of information to encode all this would be very small - on the
order of 256 bytes per frame. Note that you don't need to use a frame
buffer if all you are doing is selecting a sub-region of a larger picture.
This requires that no zooming is taking place, just discarding of data
which isn't to be displayed. During the transition to HDTV, an HDTV->NTSC
converter which contained this logic would allow wide source material to
be displayed on HDTV monitors, yet allow NTSC aspect ratio monitors to
display in P&S mode for those people who object to letterboxing. Since
the information is encoded, this works not only for pre-recorded material
such as laserdisc or tape, but even for live broadcasts. The additional
hardware cost of these features should be close to zero.
-- 
                  THIS                  | Paul Cantrell "PC"
                  SPACE                 | cantrell@sequoia.com
              INTENTIONALLY             | DoD #1144
               LEFT BLANK               | HELO/ASEL/ZX10 Pilot