Abstract:
A method of decoding and encoding video, the encoding method comprising: receiving a video; performing a shrink operation on the received video, the shrink operation comprising: creating a first lower resolution video from the video; and sequentially creating additional N−1 lower resolution videos, each one of the additional lower resolution videos created from the preceding lower resolution video; compressing the lowest resolution video; creating a lowest resolution reconstructed video by decompressing the lowest resolution compressed video; performing a first raise operation on the lowest resolution reconstructed video, the first raise operation comprising sequentially creating N higher resolutions reconstructed videos, each one of the higher resolution reconstructed videos created from the preceding lower resolution reconstructed video by: creating a higher resolution video from the lower resolution reconstructed video; computing a residual between the respective lower resolution video and the created higher resolution video; compressing the computed residual; decompressing the compressed residual; and combining the decompressed residual with the created higher resolution video, yielding the respective higher resolution reconstructed video; and providing a bit stream comprising the lowest resolution compressed video, said compressed residuals and control information.

Description:
CROSS-REFERENCES TO RELATED APPLICATIONS 
       [0001]    This patent application claims priority from and is related to U.S. Provisional Patent Application Ser. o. 61/710,723, filed Oct. 7, 2012, this U.S. Provisional Patent Application incorporated by reference in its entirety herein. 
     
    
     TECHNOLOGY FIELD 
       [0002]    The present invention is in the field of video compression. 
       BACKGROUND OF THE INVENTION 
       [0003]    Raw video files consume huge amounts of space. For example, a High Definition(HD) movie with 60 frames per second(fps), frame resolution of 1920×1080 pixels, color depth 3, and 8 bits per color, consumes:
       1,920*1,080*60*3=373,248,000 Bytes per second.       
 
         [0005]    And two hours of such movie would consume: 
         [0000]    373,248,000*7,200=2,687,385,600,000 Bytes 3 Tera bytes(Tbytes). 
         [0006]    To store the movie on regular DVD disks, i.e. 4.7 Giga bytes(Gbytes) disks, we need:
       ≈2,687/4.7≈600 DVD disks.       
 
         [0008]    And to transmit the movie over the Internet, say over fast 100 Mbps channels, we need: 
         [0000]    ≈2,687,386*8/100≈2,149,908 seconds 60 hours. 
         [0009]    Video compression is the art of reducing the video size without affecting the perceived quality. 
         [0010]    Video content is not always taken with the best equipment and the best photo shooters. In such cases digital image processing, also known as video enhancement, can substantially improve the visible quality of the video, and help the video compression process. Some of the more known methods for video enhancements use video preprocessing tools such as the following: 
         [0011]    De-interlacing Interlaced movie can be problematic when recording fast moving objects. The moving object can be in one place in the “even” picture, and in another place in the “odd” one, yielding a “stripped” picture which is very disturbing. 
         [0012]    De-blocking Block-like artifacts are the side effect of the current MPEG&#39;s low-quality highly compressed videos. De-blocking greatly increases the quality of such videos. 
         [0013]    Sharpening emphasizes texture and detail, and is critical when post-processing most digital images. An “unsharp mask” is actually used to sharpen an image. 
         [0014]    De-noising Some degree of noise is always present in any electronic device that transmits or receives a “signal”. For television this signal is the broadcast data transmitted over cable or received at the antenna; for digital cameras, the signal is the light which hits the camera sensor. Video De-noising is the process of removing noise from a video signal. 
         [0015]    Stabilization is used to reduce blurring associated with the motion of camera. Specifically, it compensates for pan and tilt of a camera or other imaging devices. With video cameras, camera shake causes visible frame-to-frame jitter in the recorded video. 
         [0016]    Camera Calibration is important in order to get stable and reliable images. Cameras that operate out of sync or are imprecisely adjusted can create blurry or confused images. 
         [0017]    Such tools can greatly improve the video quality and help the compression process. Here, we ignore this issue and assumes that the video has already been preprocessed as required. See Ref. [1] for more details. 
         [0018]    A digital video consists of multiple streams such as video, audio, and control, that are stored together in the same container file. For example, common containers formats are: AVI (Audio Video Interlaced), WMV (Windows Media Video), FLV (Flash Video), MOV (Apple Quick Time Movie). The video stream itself is usually independent of the other streams, or of the container type, and can be represented in many different formats. A media player, such as Apple iTunes, and Microsoft Windows Media Player, displays the video on the screen, using the corresponding Codec (Encoder/Decoder) software. 
         [0019]    The displayed video is usually represented in the raw RGB color space format because the human visual system works in a similar way, i.e., the human eye color vision is based on red, green and blue color sensors. The raw RGB file  100  is schematically depicted in  FIG. 1 , comprising a header section  120  followed by frames  130 . The Header  120  contains the video parameters such as: n—number of rows, m—number of columns, and N—number of frames. A frame  130  contains n*m pixel values, each a triplet for the R, G and B. 
         [0020]    The raw YUV color space format is another very useful format for video representation. Here, Y corresponds to the black and white representation of the video, and U and V to the added color differences. There are many similar formulas for converting RGB to YUV and vice versa. One of them, see Ref. [2], is exemplified in  FIG. 2 , where the RGB to YUV transforming formula is given in unit  210 , and the YUV to RGB transforming formula is given in unit  220 . The raw YUV file  300  is schematically depicted in  FIG. 3 , comprising a header section  310  as in unit  120  of  FIG. 1 , followed by the Y frames  320 , the U frames  330 , and the V frames  340 . Typical frames for the Y, U and V components are shown. In what follows we consider only the video stream part of the container file, and without loss of generality (w.l.g.), we assume a YUV color space representation. 
         [0021]    A camera may change its angle of view many times during the movie. These changes of scenes, also called the movie&#39;s cuts, are distinguished by their shape and contents, see Ref. [3]. In terms of compression this means that we have little redundancy between the cuts. 
         [0022]    The cut file  400  is schematically depicted in  FIG. 4 , comprising a header section  410  followed by the cuts  420 . The header is as follows: 
         [0000]    n is the number of rows, m is the number of columns,
 
N is the number of frames, and M is the number of cuts.
 
         [0023]    Each cut of the file has the same structure as the YUV file format given in unit  300  of  FIG. 3 . For simplicity, we will proceed to consider from now on only one such component of each such cut. A generalization to all components is straightforward. 
         [0024]    Wavelets and multiwavelets, see Ref. [4], are important mathematical tools that we use in the applications that follow. Classical discrete wavelet transform (DWT) filters are depicted in  FIG. 5 ; a pair of low pass and high pass analysis filters are depicted in unit  510 , and a pair of low pass and high pass synthesis filters are depicted in unit  520 . For example, the one dimensional Haar transform is depicted in unit  530 . 
         [0025]    In general, we have m&gt;1 filters, as depicted in  FIG. 6 ; the analysis filters are depicted in unit  610 , and the synthesis filters in unit  620 . For example, a 2D Haar transform is depicted in unit  630 . More generally, the filters may refer to the discrete multiwavelet transform (DMWT). 
         [0026]    The lattice of integers ? n  is the set of n-tuples of integers in the Euclidean space ? n . A frame can be represented as a rectangular grid on the lattice ? 2 , and a video as a cubic grid on ? 3 . A subset of a lattice, which is itself a lattice is called a sub-lattice. Examples of sub-lattices of ? 2  are given in  FIG. 7 . The Quincunx sub-lattices are depicted in unit  710 . The white circled points correspond to the even sub-lattice, and the dark circled points to the odd sub-lattice. The Dyadic sub-lattices are similarly depicted in unit  720 . The Quincunx sub-lattices are determined by the dilation matrix of unit  715 , and the Dyadic sub-lattices by the dilation matrix of unit  725 . The number of sub-lattices is determined by the determinant of the corresponding dilation matrix, 2 in the Quincunx case, and 4 in the Dyadic case. Down-sampling refers to the process of extracting a sub-lattice from a given lattice. For example, we display a dyadic down sampling in  FIG. 8 . The input signal is given in unit  810 , a temporal down sampling in unit  820 , a spatial down sampling in unit  830 , and a combined spatial and temporal down sampling in unit  840 . 
       SUMMARY OF THE INVENTION 
       [0027]    According to an aspect of the present invention there is provided a method of encoding video, comprising: receiving a video; performing a shrink operation on said received video, said shrink operation comprising: creating a first lower resolution video from said video; and sequentially creating additional N−1 lower resolution videos, each one of said additional lower resolution videos created from said preceding lower resolution video; compressing the lowest resolution video; creating a lowest resolution reconstructed video by decompressing said lowest resolution compressed video; performing a first raise operation on said lowest resolution reconstructed video, said first raise operation comprising sequentially creating N higher resolutions reconstructed videos, each one of said higher resolution reconstructed videos created from said preceding lower resolution reconstructed video by: creating a higher resolution video from said lower resolution reconstructed video; computing a residual between said respective lower resolution video and said created higher resolution video; compressing said computed residual; decompressing said compressed residual; and combining said decompressed residual with said created higher resolution video, yielding the respective higher resolution reconstructed video; and providing a bit stream comprising said lowest resolution compressed video, said compressed residuals and control information comprising said N. 
         [0028]    The bit stream may comprise low pass analysis filters and wherein creating a lower resolution video comprises applying a low pass analysis filter to a video. 
         [0029]    The bit stream may comprise blurring and down sampling operators and wherein creating a lower resolution video comprises applying a blurring operator to a video and applying a down sampling operator to the blur operation result. 
         [0030]    The bit stream may comprise blurring and down sampling operators and wherein said low pass analysis filters are computed from said blurring and down sampling operators. 
         [0031]    The bit stream may comprise low pass synthesis filters and wherein creating a higher resolution video comprises applying a low pass synthesis filter to a lower resolution reconstructed video. 
         [0032]    The bit stream may comprise up sampling, interpolation, oracle and deblurring operators and wherein creating a higher resolution video comprises: a. applying an up sampling operator followed by an interpolation operator to a lower resolution reconstructed video; b. applying an oracle operator to the interpolation operation result; and c. applying a deblurring operator to the oracle operation result. 
         [0033]    The bit stream may comprise up sampling, interpolation, oracle and deblurring operators and the low pass synthesis filters may be computed from said up sampling, interpolation, oracle and deblurring operators. 
         [0034]    Computing a residual may comprise calculating the difference between the respective lower resolution video and the higher resolution video. 
         [0035]    The bit stream may comprise high pass analysis filters and computing the residuals may additionally comprise applying high pass analysis filters to the calculated difference. 
         [0036]    The high pass analysis filters may be computed from said calculated difference. 
         [0037]    Computing the higher resolution reconstructed video may comprise adding the decompressed residual to the higher resolution video. 
         [0038]    The bit stream may comprise high pass synthesis filters and creating the higher resolution reconstructed video may comprise applying the high pass synthesis filters to the respective decompressed residuals and adding the results to the higher resolution video. 
         [0039]    The high pass synthesis filters may be computed from low pass and high pass analysis filters and low pass synthesis filters. 
         [0040]    According to another aspect of the present invention there is provided a method of decoding a video encoded according to the above method, comprising: receiving and processing said bit stream comprising said lowest resolution compressed video, said compressed residuals and said control information; creating a lowest resolution reconstructed video by decompressing said lowest resolution compressed video; and performing a second raise operation on said lowest resolution reconstructed video, said second raise operation comprising sequentially creating N higher resolutions reconstructed videos, each one of said higher resolution reconstructed videos created from said preceding lower resolution reconstructed video by: creating a higher resolution video from said lower resolution reconstructed video; decoding the residual between said respective lower resolution video and said created higher resolution video; and combining said decoded residual with said created higher resolution video, yielding the respective higher resolution reconstructed video. 
         [0041]    The bit stream may comprise low pass synthesis filters and creating a higher resolution video may comprise applying a low pass synthesis filter to a lower resolution reconstructed video. 
         [0042]    The bit stream may comprise up sampling, interpolation, oracle and deblurring operators and creating a higher resolution video may comprises: a. applying an up sampling operator followed by an interpolation operator to a lower resolution reconstructed video; b. applying an oracle operator to the interpolation operation result; and c. applying a deblurring operator to the oracle operation result. 
         [0043]    The bit stream may comprise up sampling, interpolation, oracle and deblurring operators and the low pass synthesis filters may be computed from said up sampling, interpolation, oracle and deblurring operators. 
         [0044]    Computing the higher resolution reconstructed video may comprise adding the decompressed residual to the higher resolution video. 
         [0045]    The bit stream may comprise high pass synthesis filters and creating the higher resolution reconstructed video may comprise applying the high pass synthesis filters to the respective decompressed residuals and adding the results to the higher resolution video. 
         [0046]    The high pass synthesis filters may be computed from low pass and high pass analysis filters and low pass synthesis filters. 
         [0047]    According to another aspect of the present invention there is provided a video codec comprising: an encoding unit configured to encode a video; a reconstructing unit configured to reconstruct said encoded video, the reconstructing unit creating a bit stream of decoding data; and a decoding unit configured to receive the bit stream of decoding data and decode the video therewith. 
         [0048]    The encoding unit may be configured to: receive a video; perform a shrink operation on said received video, said shrink operation comprising: creating a first lower resolution video from said video; sequentially creating additional N−1 lower resolution videos, each one of said additional lower resolution videos created from said preceding lower resolution video; and compressing the lowest resolution video. 
         [0049]    The reconstructing unit may be configured to: create a lowest resolution reconstructed video by decompressing said lowest resolution compressed video; 
         [0050]    perform a first raise operation on said lowest resolution reconstructed video, said first raise operation comprising sequentially creating N higher resolutions reconstructed videos, each one of said higher resolution reconstructed videos created from said preceding lower resolution reconstructed video by: creating a higher resolution video from said lower resolution reconstructed video; computing a residual between said respective lower resolution video and said created higher resolution video; compressing said computed residual; decompressing said compressed residual; and combining said decompressed residual with said created higher resolution video, yielding the respective higher resolution reconstructed video; wherein said bit stream comprises said compressed lowest resolution video, said compressed residuals and control information comprising said N. 
         [0051]    The decoding unit may be configured to: receive and process said bit stream comprising said lowest resolution compressed video, said compressed residuals and said control information; create a lowest resolution reconstructed video by decompressing said lowest resolution compressed video; and perform a second raise operation on said lowest resolution reconstructed video, said second raise operation comprising sequentially creating N higher resolutions reconstructed videos, each one of said higher resolution reconstructed videos created from said preceding lower resolution reconstructed video by: creating a higher resolution video from said lower resolution reconstructed video; decoding the residual between said respective lower resolution video and said created higher resolution video; and combining said decoded residual with said created higher resolution video, yielding the respective higher resolution reconstructed video. 
         [0052]    According to another aspect of the present invention there is provided a method of decoding a video encoded according to the method above, wherein said decoding unit is configured to analyze said reconstructed higher resolution videos. 
         [0053]    The decoding unit may be configured to compute the motion field of said reconstructed videos. 
         [0054]    The decoding unit may configured to perform object recognition of said reconstructed videos. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0055]    For better understanding of the invention and to show how the same may be carried into effect, reference will now be made, purely by way of example, to the accompanying drawings. 
           [0056]    With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice. In the accompanying drawings: 
           [0057]      FIG. 1  describes the raw RGB file; 
           [0058]      FIG. 2  describes the transforms between the RGB and YUV formats; 
           [0059]      FIG. 3  describes the raw YUV file; 
           [0060]      FIG. 4  describes the cut file; 
           [0061]      FIG. 5  describes the 2-way Discrete Wavelet Transform Filters; 
           [0062]      FIG. 6  describes the m-way Discrete Wavelet Transform Filters; 
           [0063]      FIG. 7  depicts Lattices and sub-lattices; 
           [0064]      FIG. 8  describes Dyadic Down Sampling; 
           [0065]      FIG. 9  is a flowchart of the new Codec; 
           [0066]      FIG. 10  is a flowchart of the Encoder; 
           [0067]      FIG. 11  depicts the Bit Stream; 
           [0068]      FIG. 12  is a flowchart of the Decoder; 
           [0069]      FIG. 13  describes the M Codec; 
           [0070]      FIG. 14  describes the M Codec; 
           [0071]      FIG. 15  describes the M Codec; 
           [0072]      FIG. 16  describes the O Codec parameters; 
           [0073]      FIG. 17  describes the O Codec; 
           [0074]      FIG. 18  describes the O Codec; 
           [0075]      FIG. 19  describes the OM Codec; and 
           [0076]      FIG. 20  describes the OM Codec. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0077]    The present invention provides a new technology for video compression that unlike the standard MPEG methods such as H.264, benefits from the analysis of the video in both the Encoder and the Decoder, using state of the art computer vision and mathematical methods. For example, motion field calculation and object recognition, see Ref. [1] and [7], may be used to reconstruct the video in both the Encoder and the Decoder. 
         [0078]    A schematic diagram of the new generic Codec is depicted in  FIG. 9 . The Codec consists of two main parts: The Encoder and the Decoder. The Encoder  1000  compresses the video Y into the Bit Stream  1100 , and the Decoder  1200  decompresses the Bit Stream into the reconstructed video Ŷ. The Bit Stream  1100 , which is the output from the Encoder and the input to the Decoder, represents the compressed video. The Bit Stream can be stored on disk or transmitted over a network. Both Y and Ŷ are referred to as “video” throughout the following description. Note that Y is a component of a cut as explained above. 
         [0079]    In what follows we describe the Encoder  1000  see  FIG. 10 , the Bit Stream  1100  see  FIG. 11 , and the Decoder  1200 , see  FIG. 12 . 
         [0080]    The Encoder  1000  has three stages as follows: 
         [0081]    Stage I (E) consists of N iterations as follows: 
         [0082]    Let us define Y 0  ? Y. Then video Y k  is the input to iteration k=0, . . . , N−1, and video Y k+1  is the output of iteration k=0, . . . , N−1. Here, video Y k+1  is a lower resolution, coarser representation, of video Y k . Lower resolution meaning either spatial lower resolution, temporal lower resolution, or both spatial and temporal lower resolutions. We call this operation of lowering the video resolution the Shrink operation. The number of such iterations, namely N, is determined by the Encoder. 
         [0083]    Stage II (E) consists of the Compress and Decompress operations. The Compress operation refers to any method for compressing video as discussed for example in Pat. [1]. The Decompress operation refers to the opposite operation of reconstructing the original video from the compressed video. Let Y N  denote the resulting video after Stage I (E), then we denote the compressed video by F N , and the decompressed video by Ŷ N . Note that video Ŷ N  is a reconstruction of the video Y N  hampered by the quality of the compression method. In case of a lossless compression, the reconstructed video Ŷ N  would be exactly the same as Y N . However, in practice, the compression process is lossy, and Ŷ N  is not the same as Y N . Usually, the stronger the compression is, the more the videos differ, and the more distortion exists. A good compression method keeps the viewing quality of the reconstructed video while minimizing the size of the compressed video. Note further that the compressed video F N  gets included in the Bit Stream  1100 . 
         [0084]    Stage III (E) consists of a first raise operation comprising N backward iterations as follows: 
         [0085]    In iteration k=N−1, . . . , 0, the reconstructed video Ŷ k+1  from the previous iteration (or stage in case of k=N−1), is the input, and the output is the reconstructed video Ŷ k , which is of the same resolution (spatial and temporal) as video Y k . Hence, the final video Ŷ 0  is the reconstruction of the initial video Y 0 , see Stage I (E) above. 
         [0086]    In order to obtain Ŷ k  we do the following steps: 
         [0087]    1) First, we resize video Ŷ k+1  into a higher resolution video Ĉ k , of the same resolution as that of video Y k . We call this operation of raising the video resolution the Expand operation. 
         [0088]    2) Next, we compute a residual R k  between video Y k  and the expanded video Ĉ k . We call this operation of computing the residual the Resultant operation. 
         [0089]    3) Finally, we compress the residual R k  into E k . Here, again, by compression, we mean any valid compression method as was discussed in Stage II (E) above. We call this operation of compressing the residual the Encode operation. Note further that the compressed residual E k  gets included into the Bit Stream  1100 . 
         [0090]    The next steps of the Encoder simulate the operations done at the Decoder: 
         [0091]    4) We decompress the compressed residual E k  into the reconstructed residual {circumflex over (R)} k . We call this operation of decompressing the compressed residual the Decode operation. 
         [0092]      5 ) We combine the expanded video Ĉ k  with the reconstructed residual {circumflex over (R)} k  to get the reconstructed video Ŷ k . We call this operation of combining the expanded video with the reconstructed residual the Assemble operation. The Assemble operation is in a sense the reverse of the Resultant operation. 
         [0093]    The Bit Stream  1100  contains the following components: 
         [0094]    The ctrl, the additional information known to the Encoder but unknown at the Decoder, which is needed in order to decompress the Bit Stream. For example, the number of iterations N, which is determined by the Encoder is part of the ctrl. Note further that ctrl is transmitted in a compressed form. 
         [0095]    The compressed video F N  . 
         [0096]    The compressed residuals: E k , k=N−1, . . . , 0. 
         [0097]    The Decoder  1200  has two stages as follows: 
         [0098]    Stage I (D) consists of the Process and Decompress operations. The 
         [0099]    Process operation retrieves the ctrl information and sets the necessary parameters, such as for example the number of iterations N, required for decoding. The Decompress operation decompresses F N  into the video Ŷ N , as is done in Stage II (E) of the Encoder. 
         [0100]    Stage II (D) consists of a second raise operation comprising N backward iterations as follows: 
         [0101]    In iteration k=N−1, . . . , 0, the reconstructed video Ŷ k+1  from the previous iteration (or stage in case of k=N−1), is the input, and the output is the reconstructed video Ŷ k . This is done in the following way (see also Stage III (E) of the Encoder): 
         [0102]    1) Use the Decode operation to decompress the compressed residual E k  into the reconstructed residual {circumflex over (R)} k . 
         [0103]    2) Use the Assemble operation to combine the expanded video Ĉ k  with the reconstructed residual {circumflex over (R)} k  to get the reconstructed video Ŷ k . 
         [0104]    To further clarify the invention, we describe hereby some possible implementations of the generic Codec. These are the M Codec, the  0  Codec, and the OM Codec. Note, however, that many other implementations are also possible and are not excluded by these examples. 
       EXAMPLE I 
     The Multiwavelet (M) Codec 
       [0105]    The M Codec is governed by a respective set of m k  multiwavelet filters, so called the DMWT filters, as depicted in unit  1310  of  FIG. 13 . Their role is explained in the following. 
         [0106]    In the Encoder  1000 : 
         [0107]    Stage I (E) At iteration k=0, . . . , N−1, the Shrink operation is determined by the low pass analysis filter A 0   (k) . That is, we apply filter A 0   (k)  to Y k  to get Y k+1 , see unit  1320  of  FIG. 13 . 
         [0108]    Stage  11  (E) is general as before. 
         [0109]    Stage III (E) At iteration k=N−1, . . . , 0: 
         [0110]    1) The Expand operation is determined by the low pass synthesis filter S 0   (k) . That is, we apply filter S 0   (k)  to Ŷ k+1  to get Ĉ k , see unit  1330  of  FIG. 13 . 
         [0111]    2) The Resultant operation is determined by the m k  ? 1 high pass analysis filters A 1   (k) , A 2   (k) , ?, A m     k     −   (k) : 
         [0112]    First the difference between Y k  and Ĉ k , namely D k , is computed, see unit  1410  of  FIG. 14 . 
         [0113]    Then, for j=1, . . . , m k  ? 1, we apply A j   (k)  to D k  to get the respective residual component R k   (j) , see unit  1420 . 
         [0114]    3) We Encode the residuals R k   (j)  to E k   (j)  for j=1, . . . , m k  ? 1, see unit  1430 . 
         [0115]    4) We Decode the reconstructed residuals {circumflex over (R)} k   (j)  from E k   (j) , for j=1, . . . , m k  ? 1, see unit  1510  of  FIG. 15 . 
         [0116]    5) The Assemble operation is determined by the m k  ? 1 high pass synthesis filters: S 1   (k) , S 2   (k) , ?, S m     k     −1   (k) : 
         [0117]    For j=1, . . . , m k  ? 1, we apply S j   (k)  to {circumflex over (R)} k   (j)  to get component {circumflex over (D)} k   (j) , see unit  1520 . 
         [0118]    Then we reconstruct Ŷ k  by summing up: 
         [0119]    Ŷ k =Ĉ k   30  Σ j=1   m     k     −1 {circumflex over (D)} k   (j)  see unit  1530 . 
         [0120]    In the Bit Stream  1100 : 
         [0121]    the ctrl information contains the DMWT filters in addition to N. 
         [0122]    In the Decoder  1200 : 
         [0123]    Stage I (D) consists of the Process and Decompress operations, as before. 
         [0124]    Stage II (D) At iteration k=N−1, . . . , 0: 
         [0125]    1) We Decode the reconstructed residuals {circumflex over (R)} k   (j)  from E k   (j) , for j=1, . . . , m k  ? 1, see unit  1510 . 
         [0126]    2) We use the Assemble operation as in Stage III (E) above to reconstruct video Ŷ k . That is, as in step 5 above, we apply S j   (k)  to {circumflex over (R)} k   (j)  to get components {circumflex over (D)} k   (j) , and sum up these components with Ĉ k  to get video Ŷ k  , see units  1520  and  1530 . 
       EXAMPLE II 
     The Oracle (O) Codec 
       [0127]    The O Codec is governed by a respective set of parameters, see  FIG. 16 . The set includes blurring/deblurring inverse operators, see unit  1610 , down sampling/up sampling opposite operators and interpolation operators, see unit  1620 , and oracle operators, see unit  1630 . Their role is described in the following. 
         [0128]    In the Encoder  1000 : 
         [0129]    Stage I (E) At iteration k=0, . . . , N−1: 
         [0130]    The Shrink operation is determined by the respective blurring and down sampling operators. That is, we apply the blurring operator B (k)  to Y k  yielding Z k , and then the down sampling operator ? (k)  to Z k , yielding Y k+1 , see unit  1710  of  FIG. 17 . 
         [0131]    Stage  11  (E) is general as before. 
         [0132]    Stage III (E) At iteration k=N−1 , . . . , 0: 
         [0133]    1) The Expand operation is determined by the respective up sampling, interpolation, oracle and deblurring operators. That is, we apply the up sampling operator ? (k)  followed by the interpolation operator I (k)  to Ŷ k+1  yielding video Z k   0 . Then we apply the oracle operator O (k)  to {circumflex over (Z)} k   0 , yielding {circumflex over (Z)} k , the reconstructed version of video Z k  of Stage I (E) above. The oracle operation, which reconstructs the best approximation {circumflex over (Z)} k  to Z k  based on {circumflex over (Z)} k   0  may use such methods as super resolution see Ref. [5] and compressed sensing see Ref. [6]. Finally, we apply the deblurring operator B inv   (k)  to {circumflex over (Z)} k  yielding Ĉ k , see unit  1720 . 
         [0134]    2) The Resultant operation is simply the difference operation. That is, the residual R k  is the difference between Y k  and Ĉ k . Then, we Encode the residual R k  into E k , see unit  1810  of  FIG. 18 . 
         [0135]    3) We then Decode the reconstructed residuals {circumflex over (R)} k  from E k . The Assemble operation being simply the summation operation, we add {circumflex over (R)} k  to Ĉ k  to get video Ŷ k , see unit  1820 . 
         [0136]    In the Bit Stream  1100 : 
         [0137]    The ctrl information contains the parameters as in  FIG. 16 , in addition to N. 
         [0138]    In the Decoder  1200 : 
         [0139]    Stage I (D) consists of the Process and Decompress operations, as before. 
         [0140]    Stage II (D) At iteration k=N−1, . . . , 0: 
         [0141]    1) We use the Expand operation as in step 1 of Stage III (E) above to reconstruct video Ĉ k  from video Ŷ k+1 , see unit  1720 . 
         [0142]    2) We use the Decode operation to get {circumflex over (R)} k  and then the Assemble operation as in step 3 of Stage III (E) above to reconstruct Ŷ k  from Ĉ k  and {circumflex over (R)} k , see unit  1820 . 
       EXAMPLE III 
     The Oracle Multiwavelet (OM) Codec 
       [0143]    We combine the Oracle Codec together with the Multiwavelet Codec as follows: 
         [0144]    We define the m k  multiwavelet filters, see unit  1310 , based on the O Codec parameters, see  FIG. 16 . We call the resulting method the OM Codec. 
         [0145]    We now describe how to obtain the DMWT filters given the O Codec parameters: 
         [0146]    1) We define the low pass analysis filter A 0   (k)  so that the resulting Shrink method approximates the given O Codec Shrink method, see unit  1910  of  FIG. 19 . 
         [0147]    2) We define the low pass synthesis filter S 0   (k)  so that the resulting Expand method approximates the given O Codec Expand method, see unit  1920 . 
         [0148]    3) We define the set of m k  ? 1 high pass analysis filters A 1   (k) , A 2   (k) , ?, A m     k     −1   (k) , to be the multiwavelet filters that annihilates Ĉ k , see unit  1930 . 
         [0149]    4) We complete the set of DMWT filters, by setting S 1   (k) , ? , S m     k     −1   (k) . This we do using the mathematical theory of wavelets, see Ref. [4] and Pat.[2]. 
         [0150]    The ctrl information contains the parameters as in  FIG. 16 , and the corresponding DMWT filters as in  FIG. 19 , in addition to N. 
         [0151]    A flowchart of the OM Codec is depicted in  FIG. 20 . The Encoder flowchart is depicted in unit  2010 , and the Decoder in unit  2020 . 
         [0000]    The following documents are referenced in the application and are all incorporated by reference herein. 
       Patents 
     [1] Ilan Bar-On and Oleg Kostenko, A Method And A System For Wavelet Based Processing, WO/2008/081459. 
       [0152]    [2] Ilan Bar-On, Method And Apparatus For A Multidimensional Discrete Multiwavelet Transform, U.S. Pat. No. 8,331,708 B2, Dec. 11, 2012. 
       References 
     [1] “Computer Vision, A Modern Approach”, D. Forsyth and J. Ponce, 2012. 
       [0153]    [2] “ITU-R Recommendation BT. 709”, http://en.wikipedia.org/wiki/Rec. — 709 
       [3] “Cut by Cut”, G. Chandler, 2012. 
     [4] “Wavelets and Multiwavelets”, Fritz Keinert, 2004. 
     [5] “Super-Resolution Imaging”, P. Milanfar, Sep. 2010. 
     [6] “Compressed Sensing, Theory and Applications”, Y. C. Eldar et al., June 2012. 
       [0154]    [7] “Optical flow”, http://en.wikipedia.org/wiki/Optical_flow.