Scalable video coding system

A system for coding video data comprised of one or more frames codes a portion of the video data using a frame-prediction coding technique, and generates residual images based on the video data and the coded video data. The system then codes the residual images using a fine-granular scalability coding technique, and outputs the coded video data and at least one of the coded residual images to a receiver.

BACKGROUND OF THE INVENTION
 1. Field of the Invention
 The present invention is directed to a scalable video coding system which
 codes video data using both frame-prediction and fine-granular scalable
 images. The invention has particular utility in connection with
 variable-bandwidth networks and computer systems that are able to
 accommodate different bit rates, and hence different quality images.
 2. Description of the Related Art
 Scalable video coding in general refers to coding techniques which are able
 to provide different levels, or amounts, of data per frame of video.
 Currently, such techniques are used by lead video coding standards, such
 as MPEG-2 and MPEG-4 (i.e., "Motion Picture Experts Group" coding), in
 order to provide flexibility when outputting coded video data.
 In the scalable coding techniques currently employed by MPEG-2 and MPEG-4,
 an encoder codes frames of video data and divides the coded frames into a
 base layer ("BL") and an enhancement layer ("EL"). Typically, the base
 layer comprises a minimum amount of data required to decode the coded
 video data. The enhancement layer, on the other hand, comprises additional
 information which enhances (e.g., improves the quality of) the base layer
 when it is decoded. In operation, the encoder transmits all frames from
 the base layer to a receiving device, which can be a personal computer or
 the like. However, the encoder only transmits frames from the enhancement
 layer in cases where the receiving device has sufficient processing power
 to handle those additional frames and/or the medium over which the frames
 are transmitted has sufficient bandwidth.
 FIGS. 1 and 2 show "scalability structures" which are currently used in
 MPEG-2 and MPEG-4 for the base layer and the enhancement layer. More
 specifically, FIG. 1 shows a scalability structure 1 which employs
 frame-prediction in base layer 2 to generate predicative (or "P") frames
 from an intra (or "I") frame or from a preceding P frame. As shown in the
 figure, frame-prediction is also used in the enhancement layer to generate
 P frames based on frames in the base layer. FIG. 2 shows another
 scalability structure 3 which is currently used in MPEG-2 and MPEG-4. In
 the scalability structure shown in FIG. 2, frame-prediction is again
 employed to determine P frames in the base layer. Unlike scalability
 structure 1, however, scalability structure 3 also uses frame-prediction
 in the enhancement layer to generate bi-directional (or "B") frames which,
 in this case, are interpolated from preceding frames in the enhancement
 layer and contemporaneous frames in the base layer. In general, MPEG-2 and
 MPEG-4 encoders use frame prediction in the manner set forth above to
 increase data compression and thus increase coding efficiency.
 Another well-known scalable video coding technique is called fine-granular
 scalability coding. Fine-granular scalability coding codes the same image
 (e.g., a frame of video) using progressively more data each time coding
 takes place. For example, as shown in FIG. 3, image 4 is initially encoded
 using data sufficient to produce image 5. Thereafter, additional data is
 coded which is sufficient to produce enhanced images 6, 7 and 8 in
 succession.
 Fine-granular scalability coding has several advantages over the
 frame-prediction techniques described above. Specifically, because
 fine-granular scalability coding can provide a wider range of enhanced
 images than frame-prediction techniques, fine-granular scalability coding
 is generally preferred in environments, such as the Internet, which have a
 wide range of available bandwidth. For similar reasons, fine-granular
 scalability coding is also generally preferred when dealing with receiving
 devices that have varying processing capabilities and/or bandwidth. That
 is, because fine-granular scalability coding produces a wide range of
 enhanced images, it is possible to match the appropriate image relatively
 closely to an amount of available bandwidth. As a result, in theory, it is
 possible to obtain the most amount of data for an image for a given amount
 of available bandwidth. On the down-side, fine-granular scalability coding
 does not permit the use of frame-prediction. As a result, it requires more
 data than the frame-prediction techniques described above and,
 consequently, degrades coding efficiency.
 Thus, there exists a need for a scalable video coding technique which
 incorporates the efficiency of frame-prediction coding and the accuracy of
 fine-granular scalability coding.
 SUMMARY OF THE INVENTION
 The present invention addresses the foregoing need by coding a portion
 (e.g., a base layer) of input video data using a frame-prediction coding
 technique and then coding another portion (e.g., residual images in an
 enhancement layer) of the video data using fine-granular scalability
 coding. By coding a base layer using a frame-prediction coding technique,
 the present invention reduces the amount of bits required to code the
 video data and thus maintains coding efficiency. By coding the residual
 images using fine-granular scalability coding, the present invention is
 able to provide a wide range of residual images, one or more of which can
 be selected for transmission based, e.g., on an available bandwidth of a
 receiving device.
 Thus, according to one aspect, the present invention is a system (i.e., a
 method, an apparatus, and computer-executable process steps) for coding
 video data comprised of one or more frames. The system codes a portion
 (e.g., a base layer) of the video data using a frame-prediction coding
 technique, and then generates residual images based on the video data and
 the coded video data. Thereafter, the system codes the residual images
 using a fine-granular scalability coding technique, and outputs the coded
 video data and at least one of the coded residual images to a receiver,
 such as a variable-bandwidth network or a networked device thereon.
 In preferred embodiments of the invention, the system determines a
 bandwidth of the receiver, and then selects which of the coded residual
 images to output based on the bandwidth of the receiver. By doing this,
 the invention is able to output a coded residual image which is most
 appropriate for the available bandwidth.
 In other preferred embodiments, the system codes the portion of the video
 data at a plurality of different bit rates so as to produce multiple
 versions of the coded video data, and generates a plurality of residual
 images for each version of the coded video data. In these embodiments, the
 system codes the residual images using a fine-granular scalability coding
 technique, determines variations in a bandwidth of the receiver over time,
 and then selects which one of the multiple versions and the coded residual
 images to output based on the variations in the bandwidth of the receiver.
 By way of example, for a receiver bandwidth increasing from B.sub.1 to
 B.sub.2, where B.sub.1 &lt;B.sub.2, the system selects a first version of the
 coded video data and successively selects coded residual images
 corresponding to each frame of the first version of the coded video data,
 which are coded at successively higher bit rates. For a receiver bandwidth
 increasing from B.sub.2 to B.sub.3, where B.sub.2 &lt;B.sub.3, the system
 selects a second version of the coded video data and successively selects
 coded residual images corresponding to each frame of the second version of
 the coded video data, which are coded at successively higher bit rates.
 Conversely, for a receiver bandwidth decreasing from B.sub.3 to B.sub.2,
 where B.sub.3 &gt;B.sub.2, the system selects a first version of the coded
 video data and successively selects coded residual images corresponding to
 each frame of the first version of the coded video data, which are coded
 at successively lower bit rates. Likewise, for a receiver bandwidth
 decreasing from B.sub.2 to B.sub.2, where B.sub.2 &gt;B.sub.1, the system
 selects a second version of the coded video data and successively selects
 coded residual images corresponding to each frame of the second version of
 the coded video data, which are coded at successively lower bit rates.
 As is clear from the foregoing, by coding a base layer at a plurality of
 different bit rates and then selecting versions of the base layer and the
 residual images based on a range of available bandwidth, during display
 the present invention is able to provide a relatively smooth transition
 between different versions of the base layer. That is, in conventional
 "simulcast" systems (i.e., systems such as this where a base layer has
 been coded at different bit rates), there is a substantial jump in image
 quality at the transition from a first bit rate to a second bit rate. The
 present invention, however, provides for a smoother transition by
 selecting and outputting fine-granular coded residual images between the
 different versions of the base layer.
 According to another aspect, the present invention is a network system that
 includes an encoder which receives input video data and which outputs
 frames of coded video data therefrom, a variable-bandwidth network over
 which the frames of coded video data are transmitted, a decoder which
 receives the frames of coded video data from the variable-bandwidth
 network and which decodes the coded video data, and a display which
 displays the decoded video data. The encoder includes a processor and a
 memory which stores computer-executable process steps. The processor
 executes process steps stored in the memory so as to produce the frames of
 coded video data by (i) coding a base layer from the input video data
 using a frame-prediction coding technique, (ii) coding an enhancement
 layer from the input video data using a fine-granular scalability coding
 technique, (iii) determining a bandwidth of the variable-bandwidth
 network, and (iv) selecting, for output, the base layer and, in a case
 that the bandwidth of the variable-bandwidth network is greater than a
 predetermined value, a portion of the enhancement layer.
 According to still another aspect, the present invention is a system for
 decoding video data comprised of an enhancement layer bitstream and a base
 layer bitstream, where the base layer bitstream is coded using a
 frame-prediction coding technique and the enhancement layer bitstream is
 encoded using a fine-granular scalability coding technique. The system
 receives the coded video data, decodes the base layer bitstream using a
 frame-prediction decoder, and decodes the enhancement layer bitstream
 using a fine-granular scalability decoder. Thereafter, the system combines
 (e.g., adds) decoded video data from the base layer bitstream and from the
 enhancement layer bitstream to form a video image.
 According to still another aspect, the present invention is a system for
 coding video data and outputting coded video data to a plurality of
 receivers. The system codes a first portion of the video data using a
 frame-prediction coding technique to produce a first bitstream, and then
 codes a second portion of the video data using a fine-granular scalability
 coding technique to produce a second bitstream. The first bitstream is
 output to the plurality of receivers, whereafter the second bitstream is
 divided into two or more sub-streams. Finally, the two or more sub-streams
 are output to the plurality of receivers.
 By virtue of the foregoing aspect of the invention, it is possible to
 multicast video data to a plurality of receivers. In other words, it is
 possible to broadcast coded data to the receivers at multiple bandwidths.
 These receivers may then accept only those bandwidths that they are able
 to process and/or receive. Thus, each receiver is able to receive and
 process as much data as it can handle, thereby resulting in more accurate
 image reproduction thereby.
 This brief summary has been provided so that the nature of the invention
 may be understood quickly. A more complete understanding of the invention
 can be obtained by reference to the following detailed description of the
 preferred embodiments thereof in connection with the attached drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
 FIG. 4 shows a representative embodiment of a computer system 9 on which
 the present invention may be implemented. As shown in FIG. 4, personal
 computer ("PC") 10 includes network connection 11 for interfacing to a
 network, such as a variable-bandwidth network or the Internet, and
 fax/modem connection 12 for interfacing with other remote sources such as
 a video camera (not shown). PC 10 also includes display screen 14 for
 displaying information (including video data) to a user, keyboard 15 for
 inputting text and user commands, mouse 13 for positioning a cursor on
 display screen 14 and for inputting user commands, disk drive 16 for
 reading from and writing to floppy disks installed therein, and CD-ROM
 drive 17 for accessing information stored on CD-ROM. PC 10 may also have
 one or more peripheral devices attached thereto, such as a scanner (not
 shown) for inputting document text images, graphics images, or the like,
 and printer 19 for outputting images, text, or the like.
 FIG. 5 shows the internal structure of PC 10. As shown in FIG. 5, PC 10
 includes memory 20, which comprises a computer-readable medium such as a
 computer hard disk. Memory 20 stores data 23, applications 25, print
 driver 24, and operating system 26. In preferred embodiments of the
 invention, operating system 26 is a windowing operating system, such as
 Microsoft.RTM. Windows95; although the invention may be used with other
 operating systems as well. Among the applications stored in memory 20 are
 scalable video coder 21 and scalable video decoder 22. Scalable video
 coder 21 performs scalable video data encoding in the manner set forth in
 detail below, and scalable video decoder 22 decodes video data which has
 been coded in the manner prescribed by scalable video coder 21. The
 operation of these applications is described in detail below.
 Also included in PC 10 are display interface 29, keyboard interface 30,
 mouse interface 31, disk drive interface 32, CD-ROM drive interface 34,
 computer bus 36, RAM 37, processor 38, and printer interface 40. Processor
 38 preferably comprises a microprocessor or the like for executing
 applications, such those noted above, out of RAM 37. Such applications,
 including scalable video coder 21 and scalable video decoder 22, may be
 stored in memory 20 (as noted above) or, alternatively, on a floppy disk
 in disk drive 16 or a CD-ROM in CD-ROM drive 17. Processor 38 accesses
 applications (or other data) stored on a floppy disk via disk drive
 interface 32 and accesses applications (or other data) stored on a CD-ROM
 via CD-ROM drive interface 34.
 Application execution and other tasks of PC 4 may be initiated using
 keyboard 15 or mouse 13, commands from which are transmitted to processor
 38 via keyboard interface 30 and mouse interface 31, respectively. Output
 results from applications running on PC 10 may be processed by display
 interface 29 and then displayed to a user on display 14 or, alternatively,
 output via network connection 11. For example, input video data which has
 been coded by scalable video coder 21 is typically output via network
 connection 11. On the other hand, coded video data which has been received
 from, e.g., a variable bandwidth-network is decoded by scalable video
 decoder 22 and then displayed on display 14. To this end, display
 interface 29 preferably comprises a display processor for forming video
 images based on decoded video data provided by processor 38 over computer
 bus 36, and for outputting those images to display 14. Output results from
 other applications, such as word processing programs, running on PC 10 may
 be provided to printer 19 via printer interface 40. Processor 38 executes
 print driver 24 so as to perform appropriate formatting of such print jobs
 prior to their transmission to printer 19.
 First Embodiment
 Turning to scalable video coder 21, this module comprises
 computer-executable process steps which code video data comprised of one
 or more successive frames. In brief, these process steps code a portion of
 the video data using a frame-prediction coding technique, generate
 residual images based on the video data and the coded video data, and code
 the residual images using a fine-granular scalability coding technique.
 The steps then output the coded video data and at least one of the coded
 residual images to a receiver which, generally speaking, can comprise a
 network (variable-bandwidth or otherwise), a PC, or other video-supporting
 networkable devices including, but not limited to, digital
 televisions/settop boxes and video concerning equipment.
 FIG. 6 is a block diagram depicting a video source 42, a variable-bandwidth
 network 43, and modules used to effect the foregoing process steps. FIG. 7
 is a flow diagram which explains the functionality of the modules shown in
 FIG. 6. To begin, in step S701 original uncoded video data is input into
 the present invention. This video data may be input via network connection
 11, fax/modem connection 12, or, as shown in FIG. 6, via a video source.
 For the purposes of the present invention, video source 42 can comprise
 any type of video capturing device, an example of which is a digital video
 camera. As shown in FIG. 6, video data from the video source is input to
 both BL encoder 44 and residual image computation block 45. The reason for
 this is apparent below.
 Next, step S702 codes a portion (i.e., a base layer, or BL) of the original
 video data using a standard frame-prediction coding technique. Step S702
 is performed by BL encoder 44, which, in preferred embodiments of the
 invention, is an MPEG-1, an MPEG-2 or an MPEG-4 encoder. A general
 overview of the MPEG standard is provided in "MPEG: A Video Compression
 Standard For Multimedia Applications", by Didier LeGall, Communications of
 the ACM, Vol. 34, No. 4 (April 1991). BL encoder 44 compresses the video
 data at a predetermined bit-rate, R.sub.BL. In preferred embodiments of
 the invention, R.sub.BL is determined by calculation block 48 based on a
 current bandwidth of a receiver, such as variable-bandwidth network 43
 (or, e.g., a computer system having variable processing capabilities).
 More specifically, calculation block 48 measures a minimum bit-rate
 ("R.sub.MIN "), a maximum bit-rate ("R.sub.MAX "), and a current available
 bandwidth ("R") of variable-bandwidth network 43. Calculation block 48
 then sets R.sub.BL to a value between R.sub.MIN and R. In most cases,
 calculation block 48 sets R.sub.BL to R.sub.MIN, so as to ensure that,
 even at its lowest bandwidths, variable-bandwidth network 43 will be able
 to accommodate coded video data output by the present invention. This is
 especially true in cases where base layer encoding takes place off-line.
 FIG. 8 shows an example of a scalability structure which is generated by
 the present invention. As shown in FIG. 8, this scalability structure
 includes both a base layer ("BL") and an enhancement layer ("EL"). Base
 layer 47 includes frames, such as frame 49. These frames are compressed at
 a bit-rate of R.sub.BL by BL encoder 44. Enhancement layer 50, however,
 includes fine-granular coded images corresponding to contemporaneous
 frames in the base layer. The following describes how the invention
 generates enhancement layer 50.
 More specifically, step S703 generates residual images 51 based on the
 original video data input from video source 42 and based on coded video
 data (i.e. the base layer) provided by BL encoder 44. In the block diagram
 shown in FIG. 6, step S703 is performed by residual image computation
 block 45. In operation, residual image computation block 45 receives coded
 video data from BL encoder 44 and then decodes that coded video data.
 Thereafter, residual images 51 are generated based on a difference between
 pixels in this decoded video data and pixels in the original video data.
 Generally speaking, the residual images correspond to the difference
 between frames in the base layer (which comprises the minimum number of
 frames and/or the minimum amount of data required by a decoder to decode a
 video signal) and frames in the original video data.
 Residual image computation block 45 may use one or more of variety of
 different methods to generate residual images 51. For example, in one
 embodiment of the invention, a simple pixel-by-pixel subtraction is
 performed between frames in the base layer and frames in the original
 video data. The resulting difference between these two sets of frames
 (i.e., the residual images) includes differences in the frames'
 resolutions. In cases where the base layer does not include entire frames
 of the original video data, the residual images include these missing
 frames.
 In another embodiment of the invention, residual image computation block 45
 generates residual images 51 by first filtering the decoded video data and
 then determining a difference between this filtered video data and the
 original video data. This technique has the advantage of removing unwanted
 noise and the like from the decoded video data caused, e.g., by the coding
 and decoding processes. In preferred embodiments of the invention, a
 deblocking filter is used to filter the decoded video data; although the
 invention is not limited to the use of this type of filter.
 In still another embodiment of the invention, residual image computation
 block 45 generates residual images 51 by filtering both the decoded video
 and the original video data, and then determining a difference between
 both of these types of filtered data. In this embodiment, the same type of
 filter (e.g., a deblocking filter) may be applied to both the original
 video data and the decoded video data. Alternatively, different types of
 filters may be applied to the original video data and to the decoded video
 data.
 In general, when filtering is used to generate residual images 51, a
 decoder for receiving video data that has been coded in accordance with
 the present invention should be "in synch" with the type of filtering used
 thereby, meaning that substantially the same type of filtering should be
 applied at the decoder in order to compensate for the effects of
 filtering. For example, if residual images 51 are coded based on filtered
 decoded video data, that same filtering should be applied to the residual
 images during decoding thereof.
 Returning to FIG. 7, after step S703, processing proceeds to step S704.
 Step S704 codes the residual images using an embedded fine-granular
 scalability coding technique, as shown in the enhancement layer of the
 scalability structure of FIG. 8. In the embodiment of the invention shown
 in FIG. 6, this step is performed by fine-granular scalable EL encoder 54.
 EL encoder 54 codes residual images 51 at a bit-rate of R.sub.MAX
 -R.sub.BL (i.e., the difference between the base layer bandwidth and
 maximum bandwidth of network 43) using a fine-granular coding technique.
 At this point, it is noted that, since a fine-granular scaling technique
 is used to code frames for the enhancement layer, frame prediction is not
 employed therein.
 As shown in FIG. 6, values for R.sub.MAX and R.sub.BL are provided to EL
 encoder 54 by calculation block 48. Any of a variety of well-known
 fine-granular coding techniques may be used by EL encoder 54. Examples of
 these include an embedded discrete cosine transform ("DCT") technique and
 a scalable matching pursuit ("MP") technique. Preferred embodiments of the
 invention, however, use one of the family of wavelet transforms (e.g.,
 zero tree wavelet transforms) to effect enhancement layer coding. For
 example, the preferred embodiment of the invention uses the still-image
 coding technique provided in MPEG-4 to perform fine-granular scalability
 coding. This approach codes images as whole using wavelet transforms.
 Regardless of what type of fine-granular scalability coding is used by EL
 encoder 54, an EL bitstream is output therefrom which has a bit-rate of
 R.sub.MAX -R.sub.BL. This EL bitstream comprises a plurality of embedded
 fine-granular scalable images, meaning that the bitstream is comprised of
 an initial coarse image and one or more enhancements thereto. For example,
 the EL bitstream may include a coarse image comprised of a predetermined
 number of bits (e.g., the first 100 bits) in the bitstream; an enhancement
 image comprising the coarse image and the next predetermined number of
 bits (e.g., the next 100 bits) in the bitstream; a further enhancement
 image comprising the coarse image, the enhancement image, and the next
 predetermined number of bits (e.g., the next 100 bits) in the bitstream;
 and so on. The number of bits used to enhance these images (100 bits in
 this example) is referred to as the image's granularity.
 At this point, it is noted that the present invention is not limited to
 using 100 bit granularity, or even to using the same number of bits to
 enhance the image. In fact, the granularity used by the invention can vary
 and, in preferred embodiments, can reach down to the byte level or even to
 the single bit level wherein single bits are used to enhance an image.
 As shown in FIG. 6, the EL bitstream is provided to real-time scalable
 video rate controller 55 which performs, in real-time, steps S705 and S706
 shown in FIG. 7. In step S705, controller 55 receives R.sub.BL. R.sub.MAX
 and R from calculation block 48, and then selects, for each frame in the
 base layer, one or more of the coded residual images in enhancement layer
 50 (see FIG. 8) based on these values. In particular, controller 55
 selects image(s) from the enhancement layer which have a bandwidth that
 substantially corresponds to R-R.sub.BL, i.e., the difference between the
 actual bandwidth of network 43 and the bandwidth of the base layer.
 Controller 55 selects these images by transmitting images from the EL
 bitstream (e.g., a coarse image and/or image enhancements) having a
 bandwidth that corresponds to R-R.sub.BL, and blocking transmission of
 those image enhancements which fall outside of that range. By implementing
 the invention using a relatively fine granularity, such as single-bit
 granularity, the invention is able to fill substantially all of the
 bandwidth between R and R.sub.BL. In these cases, the invention is able to
 provide substantially the maximum amount of video data for the given
 amount of available bandwidth. Of course, in cases where the receiver can
 handle only coded images from the base layer, controller 55 will not
 transmit any fine-granular scalable images from the enhancement layer.
 Assuming, however, that these images are to be transmitted, once the
 appropriate fine-granular scalable images (i.e., coded residual images)
 have been selected by controller 55, processing proceeds to step S706. In
 step S706, controller 55 outputs the base layer and the fine-granular
 scalable images selected in step S705. As shown in FIG. 6, the images are
 output to variable-bandwidth network 43 as a BL stream and an EL stream.
 A decoder, a functional block diagram for which is shown in FIG. 9, then
 receives these coded bitstreams and decodes the data therein. Decoder 57
 may comprise a PC, such as that shown in FIG. 4 or, alternatively, any of
 the other receivers mentioned above. As shown in the figure, decoder 57
 includes a scalable video decoder module 58 which is executed by a
 processor therein. This scalable video decoder module is comprised of a
 fine-granular scalable EL decoding module 59 for decoding data in the EL
 bitstream and a frame-prediction BL decoding module 60 for decoding frames
 in the BL bitstream. In preferred embodiments of the present invention, BL
 decoding module 60 comprises an MPEG-1, MPEG-2 or MPEG-4 decoding module.
 Due to the fine granularity of the EL bitstream, the EL decoder can decode
 any appropriate portion of the EL bitstream limited, e.g., by decoder
 processing constraints or the like. Once the respective decoding modules
 have decoded the streams of video data, frames therefrom are added and
 reordered, if necessary, by processing block 61. These frames may then be
 displayed to a user.
 Second Embodiment
 The second embodiment of the present invention generates a scalability
 structure like that shown in FIG. 8 for each of a plurality of "simulcast"
 bitstreams. Briefly, in the second embodiment of the present invention,
 scalable video coder 21 includes computer-executable process steps to code
 a portion (e.g., the base layer) of input video data at a plurality of
 different bit rates so as to produce multiple versions of coded video
 data, to generate a plurality of residual images for each version of the
 coded video data, to code the plurality of residual images for each
 version of the coded video data using a fine-granular scalability coding
 technique, and then to output one version (e.g., one base layer) of the
 coded video data together with one or more coded residual images therefor.
 More specifically, in this embodiment of the invention, BL encoder 44 codes
 the base layer at a plurality of different bit rates R.sub.B1, R.sub.B2,
 R.sub.B3 . . . R.sub.BN, where
 R.sub.MIN &lt;R.sub.B1 &lt;R.sub.B2 &lt;R.sub.B3 . . . &lt;R.sub.BN &lt;R.sub.MAX.
 For each of these resulting simulcast coded bitstrearns, residual image
 computation block 45 generates residual images in the manner described
 above. Thereafter, EL encoder 54 generates corresponding fine-granular
 coded images for each set of residual images. These fine-granular coded
 images have bit-rates of R.sub.E1, R.sub.E2, R.sub.E3 . . . R.sub.EN,
 which are determined in substantially the same manner as those of the EL
 bitstream of the first embodiment. That is,
 ##EQU1##
 where R.sub.EM.epsilon.[R.sub.BM, R.sub.MAX ] and M.epsilon.[1,N]. In a
 case that the maximum EL bit-rate for a particular BL bitstream is set as
 the minimum bit-rate of a next simulcast BL bitstream, equations (1)
 reduce to
 ##EQU2##
 FIG. 10 is an example of a graph of image quality versus bit-rate which
 explains the case corresponding to equations (2). More specifically, as
 shown in FIG. 10, the invention initially selects a scalability structure
 having a base layer with a bit-rate R.sub.B1 (which, in this case is
 R.sub.MIN). The invention then monitors parameters of variable-bandwidth
 network 43 via calculation block 48, and determines a new bandwidth R
 therefor periodically. As the bandwidth of variable-bandwidth network 43
 increases over time, controller 55 selects progressively more detailed
 fine-granular coded residual images for each frame of the selected
 scalability structure/base layer, and outputs those images to the
 receiver. The receiver then provides those image to a display, such as
 display 14 above, thereby leading to the progressive increase in image
 quality shown by line 64 in FIG. 10. However, using the scalability
 structure for R.sub.B1, it is only possible to provide a limited increase
 in image quality, as shown by dotted line 65 in FIG. 10.
 Accordingly, once the bandwidth R of variable bandwidth network 43 reaches
 a predetermined level (which may be pre-set in controller 55), the
 scalability structure for bit-rate R.sub.B2 is selected. As was the case
 above, the invention then continues to monitor variable-bandwidth network
 43 via calculation block 48, and to re-calculate the bandwidth thereof
 over time. As the bandwidth of variable-bandwidth network 43 increases,
 controller 55 selects progressively more detailed fine-granular coded
 residual images for each frame of the selected scalability structure/base
 layer, and outputs those images to the receiver. The receiver then
 provides those image to a display, such as display 14 above, thereby
 leading to the further progressive increase in image quality shown by line
 66 in FIG. 10. A process similar to this is performed up to R.sub.MAX.
 By virtue of the foregoing process, this embodiment of the invention is
 able to use simulcast bitstreams to provide an overall increase image
 quality without large "jumps" at transition points R.sub.B2 and R.sub.B3.
 That is, conventional systems which use simulcast bitstreams to increase
 image quality have a large "jump" at each transition point between two
 simulcast bitstreams.
 This results in an abrupt transition in the displayed image. In contrast,
 because the present invention uses fine-granular images between the
 transition points, the invention is able to provide a gradual transition
 between bitstreams, along with a continuous increase in image quality over
 time.
 Of course, the converse of the foregoing occurs for variable-bandwidth
 networks that have decreasing bandwidth. That is, for a receiver bandwidth
 decreasing from B.sub.3 to B.sub.2, where B.sub.3 &gt;B.sub.2, the invention
 selects a first base layer and successively selects fine-granular coded
 residual images corresponding to each frame of the first base layer that
 are coded at successively lower bit rates. As the bandwidth decreases from
 B.sub.2 to B.sub.1, where B.sub.2 &gt;B.sub.1, the invention selects a second
 base layer and successively selects fine-granular coded residual images
 corresponding to each frame of the second base layer that are coded at
 successively lower bit rates. This results in a relatively smooth decrease
 in image quality, as opposed to an abrupt transition. Of course,
 relatively smooth transitions are also achieved by the present invention
 for variable-bandwidth networks that have neither continuously increasing
 nor continuously decrease bandwidths, but rather have fluctuating or
 oscillating bandwidths. Such is also the case for computer systems or the
 like which have varying processing capabilities
 At this point, it is noted that although the first two embodiments of the
 present invention have been described with respect to a variable-bandwidth
 network, these embodiments can be used outside of a network context. That
 is, rather than measuring network bandwidth, the invention may measure the
 processing capabilities of a receiving device (e.g., a PC) and then vary
 coding accordingly.
 Third Embodiment
 FIG. 11 depicts a third embodiment of the present invention. In brief, this
 embodiment is a method and corresponding apparatus and process steps for
 coding video data and for multicasting coded video data to a plurality of
 receivers. In this embodiment, scalable video coder 21 codes a first
 portion of the video data (e.g., the base layer) using a frame-prediction
 coding technique to produce a first bitstream (e.g., the BL bitstream),
 and then codes a second portion of the video data (e.g., the enhancement
 layer) using a fine-granular scalability coding technique to produce a
 second bitstream (e.g., the EL bitstream). Thereafter, the first bitstream
 is output to one or more of the plurality of receivers, and the second
 bitstream is divided into two or more sub-streams These two or more
 sub-streams are then also output to the plurality of receivers.
 As shown in FIG. 11, the third embodiment of the invention includes video
 source 70, BL encoder 71, residual image computation block 72, and EL
 encoder 73. These features are identical to those described above with
 respect to the first embodiment. Accordingly, detailed descriptions
 thereof are omitted herein for the sake of brevity. As shown in FIG. 11,
 the third embodiment also includes multicast rate controller 74 and
 calculation block 75. Detailed descriptions of these features of the
 invention are as follows.
 Calculation block 75 is similar to calculation block 48 described above in
 that it determines R.sub.MIN, R.sub.MAX and R.sub.BL. In this embodiment,
 however, R.sub.MIN comprises the minimum bandwidth among plural receivers
 (e.g., PCs) on network 76 and R.sub.MAX comprises the maximum bandwidth
 among the plural receivers on network 76. As above, calculation block 75
 sets R.sub.BL to a value between R.sub.MIN and R.sub.MAX, and usually to
 R.sub.MIN so as to ensure that even the lowest bandwidth receiver will be
 able to process coded video data output by the present invention. As shown
 in FIG. 11, in this embodiment of the invention, calculation block 75 also
 determines bandwidths R.sub.1, R.sub.2 . . . R.sub.N for corresponding
 categories of receivers 1, 2 . . . N (not shown) on network 76. This may
 be done by monitoring the network for traffic to and from these receivers
 and/or issuing status inquiries to the respective receivers. Thereafter,
 these values for R.sub.1, R.sub.2 . . . R.sub.N are provided to multicast
 rate controller 74.
 Multicast rate controller 74 uses R.sub.1, R.sub.2 . . . R.sub.N to divide
 the EL bitstreams into sub-streams ranging from 0 bits to R.sub.N bits.
 That is, as shown in FIG. 11, multicast rate controller 74 divides the EL
 bitstream into sub-streams having bandwidths of:
EQU 0.fwdarw.R.sub.1 R.sub.1.fwdarw.R.sub.2 R.sub.N-1.fwdarw.R.sub.N, (3)
 where R.sub.N is less than or equal to R.sub.MAX -R.sub.BL. Each of these
 sub-streams corresponds to embedded fine-granular coded residual images.
 Specifically, the 0 to R.sub.1 bitstream comprises a coarse image; the
 R.sub.1 to R.sub.2 sub-stream comprises an enhancement to the coarse
 image; and so on. The sub-streams described in expression (3) above are
 then output to receivers on network 76, together with the BL bitstream.
 These receivers will then accept the BL bitstream and one, some, all, or
 none of these sub-streams, depending upon the processing capabilities of
 the receiver and/or the network. Decoders, such as that shown in FIG. 9,
 at these receivers may then be used to decode the bitstreams.
 Of course, those skilled in the art will realize that it is also possible
 to combine the second and third embodiments of the invention so as to
 produce an encoder which multicasts sub-streams for a plurality of
 simulcast BL bitstreams. In addition, although this embodiment has been
 described with respect to networked receivers, it is noted that the
 embodiment can be used with non-networked receivers as well. The invention
 can also be used to provide coded data to a plurality of
 variable-bandwidth networks connected, e.g., to a single PC or the like
 via plural network connections.
 Likewise, although the three embodiments of the invention described herein
 are preferably implemented as computer code, all or some of the components
 shown in FIGS. 6 and 11 can be implemented using discrete hardware
 elements and/or logic circuits. The same is true for the decoder shown in
 FIG. 9. Thus, for example, calculation blocks 48 and 75 can comprise a
 workstation, PC or other operator-driven device for inputting and
 selecting required control and command parameters. Lastly, while the
 encoding and decoding techniques of the present invention have been
 described in a PC environment, these techniques can be used in any type of
 video devices including, but not limited to, digital televisions/settop
 boxes, video concerning equipment, and the like.
 In this regard, the present invention has been described with respect to
 particular illustrative embodiments. It is to be understood that the
 invention is not limited to the above-described embodiments and
 modifications thereto, and that various changes and modifications may be
 made by those of ordinary skill in the art without departing from the
 spirit and scope of the appended claims.