Patent Publication Number: US-2012044986-A1

Title: Low complexity adaptive filter

Description:
This application claims the benefit of U.S. Provisional Application No. 61/374,494, filed on Aug. 17, 2010 and U.S. Provisional Application No. 61/389,043, filed on Oct. 1, 2010, the entire contents each of which are incorporated herein by reference. 
    
    
     TECHNICAL FIELD 
     This disclosure relates to block-based digital video coding used to compress video data and, more particularly, techniques for determining filters for use in the filtering of video blocks. 
     BACKGROUND 
     Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless communication devices such as radio telephone handsets, wireless broadcast systems, personal digital assistants (PDAs), laptop computers, desktop computers, tablet computers, digital cameras, digital recording devices, video gaming devices, video game consoles, and the like. Digital video devices implement video compression techniques, such as MPEG-2, MPEG-4, or ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), to transmit and receive digital video more efficiently. Video compression techniques perform spatial and temporal prediction to reduce or remove redundancy inherent in video sequences. New video standards, such as the High Efficiency Video Coding (HEVC) standard being developed by the “Joint Collaborative Team—Video Coding” (JCTVC), which is a collaboration between MPEG and ITU-T, continue to emerge and evolve. This new HEVC standard is also sometimes referred to as H.265. 
     Block-based video compression techniques may perform spatial prediction and/or temporal prediction. Intra-coding relies on spatial prediction to reduce or remove spatial redundancy between video blocks within a given unit of coded video, which may comprise a video frame, a slice of a video frame, or the like. In contrast, inter-coding relies on temporal prediction to reduce or remove temporal redundancy between video blocks of successive coded units of a video sequence. For intra-coding, a video encoder performs spatial prediction to compress data based on other data within the same unit of coded video. For inter-coding, the video encoder performs motion estimation and motion compensation to track the movement of corresponding video blocks of two or more adjacent units of coded video. 
     A coded video block may be represented by prediction information that can be used to create or identify a predictive block, and a residual block of data indicative of differences between the block being coded and the predictive block. In the case of inter-coding, one or more motion vectors are used to identify the predictive block of data from a previous or subsequent coded unit, while in the case of intra-coding, the prediction mode can be used to generate the predictive block based on data within the coded unit associated with the video block being coded. Both intra-coding and inter-coding may define several different prediction modes, which may define different block sizes and/or prediction techniques used in the coding. Additional types of syntax data may also be included as part of encoded video data in order to control or define the coding techniques or parameters used in the coding process. 
     After block-based prediction coding, the video encoder may apply transform, quantization and entropy coding processes to further reduce the bit rate associated with communication of a residual block. Transform techniques may comprise discrete cosine transforms (DCTs) or conceptually similar processes, such as wavelet transforms, integer transforms, or other types of transforms. In a discrete cosine transform process, as an example, the transform process converts a set of pixel values into transform coefficients, which may represent the energy of the pixel values in the frequency domain. Quantization is applied to the transform coefficients, and generally involves a process that limits the number of bits associated with any given transform coefficient. Entropy coding comprises one or more processes that collectively compress a sequence of quantized transform coefficients. 
     Filtering of video blocks may be applied as part of the encoding and decoding loops, or as part of a post-filtering process on reconstructed video blocks. Filtering is commonly used, for example, to reduce blockiness or other artifacts common to block-based video coding. Filter coefficients (sometimes called filter taps) may be defined or selected in order to promote desirable levels of video block filtering that can reduce blockiness and/or improve the video quality in other ways. A set of filter coefficients, for example, may define how filtering is applied along edges of video blocks or other locations within video blocks. Different filter coefficients may cause different levels of filtering with respect to different pixels of the video blocks. Filtering may smooth or sharpen differences in intensity of adjacent pixel values in order to help eliminate unwanted artifacts. 
     SUMMARY 
     This disclosure describes techniques associated with filtering of video data in a video encoding and/or video decoding process. In accordance with this disclosure, filtering is applied at an encoder, and filter information is encoded in the bitstream to enable a decoder to identify the filtering that was applied at the encoder. The decoder receives encoded video data that includes the filter information, decodes the video data, and applies filtering based on the filtering information. In this way, the decoder applies the same filtering that was applied at the encoder. 
     In one example, a method of video coding includes determining a first filter for a first series of video blocks, wherein the first filter is to be applied to a first set of coded units of the first series of video blocks; determining a first interim filter for the first series of video blocks, wherein the first interim filter is determined for a second set of coded units of the first series of video blocks; applying the first interim filter to coded units of a second series of video blocks to determine a filter map that defines a first set of coded units for the second series of video blocks and a second set of coded units for the second series of video blocks; determining a second filter for the first set of coded units of the second series of video blocks; and, applying the second filter for the first set of coded units of the second series of video block. 
     In another example, a video coding device includes a prediction unit that generates a first series of video blocks and a second series of video blocks; and a filter unit that determines a first filter for the first series of video blocks, wherein the first filter is to be applied to a first set of coded units of the first series of video blocks, determines a first interim filter for the first series of video blocks, wherein the first interim filter is determined for a second set of coded units of the first series of video blocks, applies the first interim filter to coded units of the second series of video blocks to determine a filter map that defines a first set of coded units for the second series of video blocks and a second set of coded units for the second series of video blocks, determines a second filter for the first set of coded units of the second series of video blocks, and applies the second filter for the first set of coded units of the second series of video block. 
     In another example, an apparatus for coding video data includes means for determining a first filter for a first series of video blocks, wherein the first filter is to be applied to a first set of coded units of the first series of video blocks; means for determining a first interim filter for the first series of video blocks, wherein the first interim filter is determined for a second set of coded units of the first series of video blocks; means for applying the first interim filter to coded units of a second series of video blocks to determine a filter map that defines a first set of coded units for the second series of video blocks and a second set of coded units for the second series of video blocks; means for determining a second filter for the first set of coded units of the second series of video blocks; and means for applying the second filter for the first set of coded units of the second series of video block. 
     The techniques described in this disclosure may be implemented in hardware, software, firmware, or any combination thereof. If implemented in hardware, an apparatus may be realized as an integrated circuit, a processor, discrete logic, or any combination thereof. If implemented in software, the software may be executed in one or more processors, such as a microprocessor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), or digital signal processor (DSP). The software that executes the techniques may be initially stored in a computer-readable medium and loaded and executed in the processor. 
     Accordingly, this disclosure also contemplates a computer program product that includes a computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors of a device for decoding video data to determine a first filter for a first series of video blocks, wherein the first filter is to be applied to a first set of coded units of the first series of video blocks; determine a first interim filter for the first series of video blocks, wherein the first interim filter is determined for a second set of coded units of the first series of video blocks; apply the first interim filter to coded units of a second series of video blocks to determine a filter map that defines a first set of coded units for the second series of video blocks and a second set of coded units for the second series of video blocks; determine a second filter for the first set of coded units of the second series of video blocks; and apply the second filter for the first set of coded units of the second series of video block. 
     The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a block diagram illustrating an exemplary video encoding and decoding system. 
         FIGS. 2A and 2B  are conceptual diagrams illustrating an example of quadtree partitioning applied to a largest coding unit (LCU). 
         FIGS. 2C and 2D  are conceptual diagrams illustrating an example of a filter map for a series of video blocks corresponding to the example quadtree partitioning of  FIGS. 2A and 2B . 
         FIG. 3  is a block diagram illustrating an exemplary video encoder consistent with this disclosure. 
         FIG. 4  is a block diagram illustrating an exemplary video decoder consistent with this disclosure. 
         FIG. 5  is a conceptual diagram illustrating ranges of values for an activity metric. 
         FIG. 6  is a flow diagram illustrating encoding techniques consistent with this disclosure. 
         FIG. 7  is a flow diagram illustrating encoding techniques consistent with this disclosure. 
     
    
    
     DETAILED DESCRIPTION 
     This disclosure describes techniques associated with filtering of video data in a video encoding and/or video decoding process. In accordance with this disclosure, filtering is applied at an encoder, and filter information is encoded in the bitstream to enable a decoder to identify the filtering that was applied at the encoder. The decoder receives encoded video data that includes the filter information, decodes the video data, and applies filtering based on the filtering information. In this way, the decoder applies the same filtering that was applied at the encoder. 
     According to the techniques of this disclosure, video data, such as a series of video blocks, can be coded in units referred to as coded units (CUs). Coded units can be partitioned into smaller coded units, or sub-units, using a quadtree partitioning scheme. Syntax data identifying the quadtree partitioning scheme for a particular series of video blocks can be transmitted from an encoder to a decoder. Additional filter syntax data, sometimes referred to as a filter map, can also be transmitted from the encoder to the decoder. The filter map identifies which coded units of the series of video blocks are to be filtered by the decoder and which coded units of the series of video blocks are not to be filtered by the decoder. For those coded units of the series of video blocks that are to be filtered, a filter or set of filters is communicated from the encoder to the decoder. 
     The filter or set of filters is determined by the encoder. The process of determining a filter is often very computationally intense, and as a result, can slow the encoding process, which can be undesirable in many situations such as when encoding live video, encoding in real-time, or when using a resource-limited device such as a laptop computer, tablet computer, or smartphone that operates on battery power. The techniques of this disclosure include using an unfiltered portion of a previous series of video blocks to determine an interim filter and use the interim filter to determine a filter map for a current series of video blocks. 
     In particular, for a first series of video blocks, an encoder may determine two filters, a first decoding filter that is to be transmitted to a decoder and a first interim filter that is not to be transmitted to the decoder. The first interim filter is used to determine which coded units of a second series of video blocks are to be filtered. After a decision is made as to which coded units of the second series of video blocks are to be filtered, the encoder determines a second decoding filter for the second series of video blocks and transmits the second decoding filter to the decoder. In addition to determining the second decoding filter, the encoder also determines a second interim filter, which the encoder will use to determine which coded units of a third series of video blocks are to be filtered. This process can repeat for many series of video blocks. This disclosure generally uses the term “decoding filters” to describe filters that are communicated to a decoder to be used as part of a decoding process and generally uses the term “interim filters” to describe filters that are used by an encoder as part of an encoding process but not communicated to a decoder. Except when explicitly identified as an interim filter, references in this disclosure to filters can generally be assumed to be referring to decoding filters. 
     Typically, video encoders use a current series of video blocks to determine the coded units to filter as well as what filter or filters to apply. In particular, the current series of video blocks may be filtered (via one or several different filters), and the filtered results can be compared to the original video data to determine whether the filter improved the video quality for each block. A filter map may be generated for one or several filter possibilities. However, this process often results in a large amount of computational resources being dedicated to attempts to determine filters for coded units, many of which may not ultimately be used as part of the decoding process. By utilizing a previous series of video blocks to determine which coded units of a current series of video should be filtered, the techniques of the present disclosure may reduce the complexity of the encoding process compared to techniques that consider many possible filters, while still maintaining a desired quality level for reconstructed video. 
     Although the techniques of this disclosure will generally be described with reference to in-loop filtering, the techniques may be applied to in-loop filtering, post-loop filtering, and other filtering schemes such as switched filtering. In-loop filtering refers to filtering in which the filtered data is part of the encoding and decoding loops such that filtered data is used for predictive intra- or inter-coding. Post-loop filtering refers to filtering that is applied to reconstructed video data after the encoding loop. With post filtering, the unfiltered data is used for predictive intra- or inter-coding. The techniques of this disclosure are not limited to in-loop filtering or post filtering, and may apply to a wide range of filtering applied during video coding. In some implementations, the type of filtering may switch between post filtering and in-loop filtering on, for example, a frame-by-frame basis, and the decision of whether to use post filtering or in-loop filtering can be signaled from encoder to decoder for each frame. 
     In this disclosure, the term “coding” refers to encoding or decoding. Similarly, the term “coder” generally refers to any video encoder, video decoder, or combined encoder/decoder (codec). Accordingly, the term “coder” is used herein to refer to a specialized computer device or apparatus that performs video encoding or video decoding. 
     Additionally, in this disclosure, the term “filter” generally refers to a set of filter coefficients. For example, a 3×3 filter is defined by a set of 9 filter coefficients, a 5×5 filter is defined by a set of 25 filter coefficients, and so on. Therefore, encoding a filter generally refers to encoding information in the bitstream that will enable a decoder to determine or reconstruct the set of filter coefficients. While encoding a filter may include directly encoding a full set of filter coefficients, it may also include directly encoding only a partial set of filter coefficients or encoding no filter coefficients at all, but rather encoding information that enables a decoder to reconstruct filter coefficients based on other information known or attainable to the decoder. For example, an encoder can encode information describing how to alter a set of existing filter coefficients to create a new set of filter coefficients. 
     The term “set of filters” generally refers to a group of more than one filter. For example, a set of 2 3×3 filters, could include a first set of 9 filter coefficients and a second set of 9 filter coefficients. According to techniques described in this disclosure, for a series of video blocks, such as a frame, slice, or largest coding unit, information identifying sets of filters are transmitted from the encoder to the decoder in a header for the series of the video blocks. 
       FIG. 1  is a block diagram illustrating an exemplary video encoding and decoding system  110  that may implement techniques of this disclosure. As shown in  FIG. 1 , system  110  includes a source device  112  that transmits encoded video data to a destination device  116  via a communication channel  115 . Source device  112  and destination device  116  may comprise any of a wide range of devices. In some cases, source device  112  and destination device  116  may comprise wireless communication device handsets, such as so-called cellular or satellite radiotelephones. The techniques of this disclosure, however, which apply more generally to filtering of video data, are not necessarily limited to wireless applications or settings, and may be applied to non-wireless devices including video encoding and/or decoding capabilities. 
     In the example of  FIG. 1 , source device  112  includes a video source  120 , a video encoder  122 , a modulator/demodulator (modem)  123  and a transmitter  124 . Destination device  116  includes a receiver  126 , a modem  127 , a video decoder  128 , and a display device  130 . In accordance with this disclosure, video encoder  122  of source device  112  may implement a multi-input, multi-filter filtering scheme where video encoder  122  may be configured to select one or more sets of filter coefficients for multiple inputs in a video block filtering process and then encode the selected one or more sets of filter coefficients. Specific filters from the one or more sets of filter coefficients may be selected based on one or more activity metrics for one or more inputs, and the filter coefficients may be used to filter the one or more inputs. In accordance with this disclosure, video encoder  122  may also implement a single input, multi-filter scheme where video encoder  122  identifies a set of filters for a single input, and where specific filters from the set of filters are selected based on one or more activity metrics. In accordance with this disclosure, video encoder  122  may also implement a single input, single filter filtering scheme where video encoder  122  identifies a single filter for an input, and thus no selection based on an activity metric is required. In accordance with this disclosure, video encoder  122  may also implement a multi-input, single filter filtering scheme where video encoder  122  identifies a single filter for each of multiple inputs, and thus no selection based on an activity metric is required. The filtering techniques of this disclosure are generally compatible with any techniques for coding or signaling filter coefficients from an encoder to a decoder. 
     According to the techniques of this disclosure, video encoder  122  can transmit to video decoder  128  one or more sets of filter coefficients for a series of video blocks, such as a frame or slice. More specifically, video encoder  122  of source device  112  may select one or more sets of filters for series of video blocks and apply filters from the set(s) to one or more inputs associated with coded units of the slice or frame during the encoding process, and then encode the sets of filters (i.e. sets of filter coefficients) for communication to video decoder  128  of destination device  116 . In some instances, video encoder  122  may determine an activity metric associated with inputs of coded units coded in order to select which filter(s) from the set(s) of filters to use with that particular coded unit. On the decoder side, video decoder  128  of destination device  116  may also determine the activity metric for one or more inputs associated with the coded unit so that video decoder  128  can determine which filter(s) from the set(s) of filters to apply to the pixel data, or in some instances, video decoder  128  may determine the filter coefficients directly from filter information received in the bitstream. Video decoder  128  may decode the filter coefficients based on direct decoding of the coefficients or predictive decoding of the coefficients relative to previous coefficients, e.g., depending upon how the filter coefficients were encoded and signaled in the bitstream syntax data. The illustrated system  110  of  FIG. 1  is merely exemplary. The filtering techniques of this disclosure may be performed by any encoding or decoding devices. Source device  112  and destination device  116  are merely examples of coding devices that can support such techniques. 
     Video encoder  122  of source device  112  may encode video data received from video source  120  using the techniques of this disclosure. Video source  120  may comprise a video capture device, such as a video camera, a video archive containing previously captured video, or a video feed from a video content provider. As a further alternative, video source  120  may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. In some cases, if video source  120  is a video camera, source device  112  and destination device  116  may form so-called camera phones or video phones. In each case, the captured, pre-captured or computer-generated video may be encoded by video encoder  122 . 
     Once the video data is encoded by video encoder  122 , the encoded video information may then be modulated by modem  123  according to a communication standard, e.g., such as code division multiple access (CDMA) or another communication standard or technique, and transmitted to destination device  116  via transmitter  124 . Modem  123  may include various mixers, filters, amplifiers or other components designed for signal modulation. Transmitter  124  may include circuits designed for transmitting data, including amplifiers, filters, and one or more antennas. 
     Receiver  126  of destination device  116  receives information over channel  115 , and modem  127  demodulates the information. The video decoding process performed by video decoder  128  may include filtering, e.g., as part of the in-loop decoding or as a post filtering step following the decoding loop. The set of filters applied by video decoder  128  for a particular slice or frame may be decoded. In particular, a filter (i.e. a set of the filter coefficients) can be predictively coded as difference values relative to another set of the filter coefficients associated with a different filter. The different filter may, for example, be associated with a different slice or frame. In such a case, video decoder  128  might receive an encoded bitstream comprising video blocks and filter information that identifies the different frame or slice with which the different filter is associated filter. The filter information also includes difference values that define the current filter relative to the filter of the different coded unit. In particular, the difference values may comprise filter coefficient difference values that define filter coefficients for the current filter relative to filter coefficients of a different filter used for a different coded unit. 
     Video decoder  128  decodes the video blocks, generates the filter coefficients, and filters the decoded video blocks based on the generated filter coefficients. The decoded and filtered video blocks can be assembled into video frames to form decoded video data. Display device  130  displays the decoded video data to a user, and may comprise any of a variety of display devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device. 
     Communication channel  115  may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines, or any combination of wireless and wired media. Communication channel  115  may form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. Communication channel  115  generally represents any suitable communication medium, or collection of different communication media, for transmitting video data from source device  112  to destination device  116 . 
     Video encoder  122  and video decoder  128  may operate according to a video compression standard such as the ITU-T H.264 standard, alternatively referred to as MPEG-4, Part 10, Advanced Video Coding (AVC), which will be used in parts of this disclosure for purposes of explanation. However, many of the techniques of this disclosure may be readily applied to any of a variety of other video coding standards, including the newly emerging HEVC standard. Generally, any standard that allows for filtering at the encoder and decoder may benefit from various aspects of the teaching of this disclosure. 
     Although not shown in  FIG. 1 , in some aspects, video encoder  122  and video decoder  128  may each be integrated with an audio encoder and decoder, and may include appropriate MUX-DEMUX units, or other hardware and software, to handle encoding of both audio and video in a common data stream or separate data streams. If applicable, MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP). 
     Video encoder  122  and video decoder  128  each may be implemented as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof. Each of video encoder  122  and video decoder  128  may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in a respective mobile device, subscriber device, broadcast device, server, or the like. 
     In some cases, devices  112 ,  116  may operate in a substantially symmetrical manner. For example, each of devices  112 ,  116  may include video encoding and decoding components. Hence, system  110  may support one-way or two-way video transmission between video devices  112 ,  116 , e.g., for video streaming, video playback, video broadcasting, or video telephony. 
     During the encoding process, video encoder  122  may execute a number of coding techniques or steps. In general, video encoder  122  operates on video blocks within individual video frames in order to encode the video data. In one example, a video block may correspond to a macroblock or a partition of a macroblock. Macroblocks are one type of video block defined by the ITU H.264 standard and other standards. Macroblocks typically refer to 16×16 blocks of data, although the term is also sometimes used generically to refer to any video block of N×N size. The ITU-T H.264 standard supports intra prediction in various block sizes, such as 16×16, 8×8, or 4×4 for luma components, and 8×8 for chroma components, as well as inter prediction in various block sizes, such as 16×16, 16×8, 8×16, 8×8, 8×4, 4×8 and 4×4 for luma components and corresponding scaled sizes for chroma components. In this disclosure, “N×N” refers to the pixel dimensions of the block in terms of vertical and horizontal dimensions, e.g., 16×16 pixels. In general, a 16×16 block will have 16 pixels in a vertical direction and 16 pixels in a horizontal direction. Likewise, an N×N block generally has N pixels in a vertical direction and N pixels in a horizontal direction, where N represents a positive integer value. The pixels in a block may be arranged in rows and columns. 
     The emerging HEVC standard defines new terms for video blocks. In particular, video blocks (or partitions thereof) may be referred to as “coded units” (or CUs). With the HEVC standard, largest coded units (LCUs) may be divided into smaller and CUs according to a quadtree partitioning scheme, and the different CUs that are defined in the scheme may be further partitioned into so-called prediction units (PUs). The LCUs, CUs, and PUs are all video blocks within the meaning of this disclosure. Other types of video blocks may also be used, consistent with the HEVC standard or other video coding standards. Thus, the phrase “video blocks” refers to any size of video block. Separate CUs may be included for luma components and scaled sizes for chroma components for a given pixel, although other color spaces could also be used. 
     Video blocks may have fixed or varying sizes, and may differ in size according to a specified coding standard. Each video frame may include a plurality of slices. Each slice may include a plurality of video blocks, which may be arranged into partitions, also referred to as sub-blocks. In accordance with the quadtree partitioning scheme referenced above and described in more detail below, an N/2×N/2 first CU may comprise a sub-block of an N×N LCU, an N/4×N/4 second CU may also comprise a sub-block of the first CU. An N/8×N/8 PU may comprise a sub-block of the second CU. Similarly, as a further example, block sizes that are less than 16×16 may be referred to as partitions of a 16×16 video block or as sub-blocks of the 16×16 video block. Likewise, for an N×N block, block sizes less than N×N may be referred to as partitions or sub-blocks of the N×N block. Video blocks may comprise blocks of pixel data in the pixel domain, or blocks of transform coefficients in the transform domain, e.g., following application of a transform such as a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video block data representing pixel differences between coded video blocks and predictive video blocks. In some cases, a video block may comprise blocks of quantized transform coefficients in the transform domain. 
     Syntax data within a bitstream may define an LCU for a frame or a slice, which is a largest coding unit in terms of the number of pixels for that frame or slice. In general, an LCU or CU has a similar purpose to a macroblock coded according to H.264, except that LCUs and CUs do not have a specific size distinction. Instead, an LCU size can be defined on a frame-by-frame or slice-by-slice basis, and an LCU be split into CUs. In general, references in this disclosure to a CU may refer to a largest coded unit of a picture or a sub-CU of an LCU. An LCU may be split into sub-CUs, and each sub-CU may be split into sub-CUs. Syntax data for a bitstream may define a maximum number of times an LCU may be split, referred to as CU depth. Accordingly, a bitstream may also define a smallest coding unit (SCU). 
     As introduced above, an LCU may be associated with a quadtree data structure. In general, a quadtree data structure includes one node per CU, where a root node corresponds to the LCU. If a CU is split into four sub-CUs, the node corresponding to the CU includes four leaf nodes, each of which corresponds to one of the sub-CUs. Each node of the quadtree data structure may provide syntax data for the corresponding CU. For example, a node in the quadtree may include a split flag, indicating whether the CU corresponding to the node is split into sub-CUs. Syntax data for a CU may be defined recursively, and may depend on whether the CU is split into sub-CUs. 
     A CU that is not split may include one or more prediction units (PUs). In general, a PU represents all or a portion of the corresponding CU, and includes data for retrieving a reference sample for the PU. For example, when the PU is intra-mode encoded, the PU may include data describing an intra-prediction mode for the PU. As another example, when the PU is inter-mode encoded, the PU may include data defining a motion vector for the PU. The data defining the motion vector may describe, for example, a horizontal component of the motion vector, a vertical component of the motion vector, a resolution for the motion vector (e.g., one-quarter pixel precision or one-eighth pixel precision), a reference frame to which the motion vector points, and/or a reference list (e.g., list 0 or list 1) for the motion vector. Data for the CU defining the PU(s) may also describe, for example, partitioning of the CU into one or more PUs. Partitioning modes may differ between whether the CU is uncoded, intra-prediction mode encoded, or inter-prediction mode encoded. 
     A CU having one or more PUs may also include one or more transform units (TUs). Following prediction using a PU, a video encoder may calculate a residual value for the portion of the CU corresponding to the PU. The residual value may be transformed, quantized, and scanned. A TU is not necessarily limited to the size of a PU. Thus, TUs may be larger or smaller than corresponding PUs for the same CU. In some examples, the maximum size of a TU may be the size of the corresponding CU. The TUs may comprise the data structures that include the residual transform coefficients associated with a given CU. This disclosure also uses the terms “block” and “video block” to refer to any of an LCU, CU, PU, SCU, or TU. 
       FIGS. 2A and 2B  are conceptual diagrams illustrating an example quadtree  250  and a corresponding largest coding unit  272 .  FIG. 2A  depicts an example quadtree  250 , which includes nodes arranged in a hierarchical fashion. Each node in a quadtree, such as quadtree  250 , may be a leaf node with no children, or have four child nodes. In the example of  FIG. 2A , quadtree  250  includes root node  252 . Root node  252  has four child nodes, including leaf nodes  256 A- 256 C (leaf nodes  256 ) and node  254 . Because node  254  is not a leaf node, node  254  includes four child nodes, which in this example, are leaf nodes  258 A- 258 D (leaf nodes  258 ). 
     Quadtree  250  may include data describing characteristics of a corresponding largest coding unit (LCU), such as LCU  272  in this example. For example, quadtree  250 , by its structure, may describe splitting of the LCU into sub-CUs. Assume that LCU  272  has a size of 2N×2N. LCU  272 , in this example, has four sub-CUs  276 A- 276 C (sub-CUs  276 ) and  274 , each of size N×N. Sub-CU  274  is further split into four sub-CUs  278 A- 278 D (sub-CUs  278 ), each of size N/2×N/2. The structure of quadtree  250  corresponds to the splitting of LCU  272 , in this example. That is, root node  252  corresponds to LCU  272 , leaf nodes  256  correspond to sub-CUs  276 , node  254  corresponds to sub-CU  274 , and leaf nodes  258  correspond to sub-CUs  278 . 
     Data for nodes of quadtree  250  may describe whether the CU corresponding to the node is split. If the CU is split, four additional nodes may be present in quadtree  250 . In some examples, a node of a quadtree may be implemented similar to the following pseudocode: 
     
       
         
           
               
               
             
               
                   
                   
               
             
            
               
                   
                 quadtree_node { 
               
               
                   
                   boolean split_flag(1); 
               
               
                   
                   // signaling data 
               
               
                   
                   if (split_flag) { 
               
               
                   
                     quadtree_node child1; 
               
               
                   
                     quadtree_node child2; 
               
               
                   
                     quadtree_node child3; 
               
               
                   
                     quadtree_node child4; 
               
               
                   
                   } 
               
               
                   
                 } 
               
               
                   
                   
               
            
           
         
       
     
     The split_flag value may be a one-bit value representative of whether the CU corresponding to the current node is split. If the CU is not split, the split_flag value may be ‘0’, while if the CU is split, the split_flag value may be ‘1’. With respect to the example of quadtree  250 , an array of split flag values may be 101000000. 
     In some examples, each of sub-CUs  276  and sub-CUs  278  may be intra-prediction encoded using the same intra-prediction mode. Accordingly, video encoder  122  may provide an indication of the intra-prediction mode in root node  252 . Moreover, certain sizes of sub-CUs may have multiple possible transforms for a particular intra-prediction mode. In accordance with the techniques of this disclosure, video encoder  122  may provide an indication of the transform to use for such sub-CUs in root node  252 . For example, sub-CUs of size N/2×N/2 may have multiple possible transforms available. Video encoder  122  may signal the transform to use in root node  252 . Accordingly, video decoder  128  may determine the transform to apply to sub-CUs  278  based on the intra-prediction mode signaled in root node  252  and the transform signaled in root node  252 . 
     As such, video encoder  122  need not signal transforms to apply to sub-CUs  276  and sub-CUs  278  in leaf nodes  256  and leaf nodes  258 , but may instead simply signal an intra-prediction mode and, in some examples, a transform to apply to certain sizes of sub-CUs, in root node  252 , in accordance with the techniques of this disclosure. In this manner, these techniques may reduce the overhead cost of signaling transform functions for each sub-CU of an LCU, such as LCU  272 . 
     In some examples, intra-prediction modes for sub-CUs  276  and/or sub-CUs  278  may be different than intra-prediction modes for LCU  272 . Video encoder  122  and video decoder  128  may be configured with functions that map an intra-prediction mode signaled at root node  252  to an available intra-prediction mode for sub-CUs  276  and/or sub-CUs  278 . The function may provide a many-to-one mapping of intra-prediction modes available for LCU  272  to intra-prediction modes for sub-CUs  276  and/or sub-CUs  278 . 
     A slice may be divided into video blocks (or LCUs) and each video block may be partitioned according to the quadtree structure described in relation to  FIGS. 2A-B . Additionally, as shown in  FIG. 2C , the quadtree sub-blocks indicated by “ON” may be filtered by loop filters described herein, while quadtree sub-blocks indicated by “OFF” may not be filtered. The decision of whether or not to filter a given block or sub-block may be determined at the encoder by comparing the filtered result and the non-filtered result relative to the original block being coded.  FIG. 2D  is a decision tree representing partitioning decisions that results in the quadtree partitioning shown in  FIG. 2C . 
     In particular,  FIG. 2C  may represent a relatively large video block that is partitioned according to a quadtree portioning scheme into smaller video blocks of varying sizes. Each video block is labelled (on or off) in  FIG. 2C , to illustrate whether filtering should be applied or avoided for that video block. The term “filter map” is used in this disclosure to generally describe any data structure that identifies the filter decisions represented by  FIGS. 2C and 2D . The video encoder may define this filter map by comparing filtered and unfiltered versions of each video block to the original video block being coded. 
     Again,  FIG. 2D  is a decision tree corresponding to partitioning decisions that result in the quadtree partitioning shown in  FIG. 2C . In  FIG. 2D , each circle may correspond to a CU. If the circle includes a “1” flag, then that CU is further partitioned into four more CUs, but if the circle includes a “0” flag, then that CU is not partitioned any further. Each circle (e.g., corresponding to CUs) also includes an associated triangle. If the flag in the triangle for a given CU is set to 1, then filtering is turned “ON” for that CU, but if the flag in the triangle for a given CU is set to 0, then filtering is turned off. In this manner,  FIGS. 2C and 2D  may be individually or collectively viewed as a filter map that can be generated at an encoder and communicated to a decoder at least once per slice of encoded video data in order to communicate the level of quadtree partitioning for a given video block (e.g., an LCU) whether or not to apply filtering to each partitioned video block (e.g., each CU within the LCU). 
     Smaller video blocks can provide better resolution, and may be used for locations of a video frame that include high levels of detail. Larger video blocks can provide greater coding efficiency, and may be used for locations of a video frame that include a low level of detail. A slice may be considered to be a plurality of video blocks and/or sub-blocks. Each slice may be an independently decodable series of video blocks of a video frame. Alternatively, frames themselves may be decodable series of video blocks, or other portions of a frame may be defined as decodable series of video blocks. The term “series of video blocks” may refer to any independently decodable portion of a video frame such as an entire frame, a slice of a frame, a group of pictures (GOP) also referred to as a sequence, or another independently decodable unit defined according to applicable coding techniques. Aspects of this disclosure might be described in reference to frames or slices, but such references are merely exemplary. It should be understood that generally any series of video blocks may be used instead of a frame or a slice. 
     Syntax data may be defined on a per-coded-unit basis such that each coded unit includes associated syntax data. The filter information described herein may be part of such syntax data for a coded unit, but might more likely be part of syntax data for a series of video blocks, such as a frame, a slice, a GOP, or a sequence of video frames, instead of for a coded unit. The syntax data can indicate the set or sets of filters to be used with coded units of the slice or frame. The syntax data may additionally describe other characteristics of the filters (e.g., filter types) that were used to filter the coded units of the slice or frame. The filter type, for example, may be linear, bilinear, two-dimensional, bicubic, or may generally define any shape of filter support. Sometimes, the filter type may be presumed by the encoder and decoder, in which case the filter type is not included in the bitstream, but in other cases, filter type may be encoded along with filter coefficient information as described herein. The syntax data may also signal to the decoder how the filters were encoded (e.g., how the filter coefficients were encoded), as well as the ranges of the activity metric for which the different filters should be used. 
     Video encoder  122  may perform predictive coding in which a video block being coded is compared to a predictive frame (or other coded unit) in order to identify a predictive block. The differences between the current video block being coded and the predictive block are coded as a residual block, and prediction syntax data is used to identify the predictive block. The residual block may be transformed and quantized. Transform techniques may comprise a DCT process or conceptually similar process, integer transforms, wavelet transforms, or other types of transforms. In a DCT process, as an example, the transform process converts a set of pixel values into transform coefficients, which may represent the energy of the pixel values in the frequency domain. Quantization is typically applied to the transform coefficients, and generally involves a process that limits the number of bits associated with any given transform coefficient. 
     Following transform and quantization, entropy coding may be performed on the quantized and transformed residual video blocks. Syntax data, such as the filter information and prediction vectors defined during the encoding, may also be included in the entropy coded bitstream for each coded unit. In general, entropy coding comprises one or more processes that collectively compress a sequence of quantized transform coefficients and/or other syntax data. Scanning techniques, such as zig-zag scanning techniques, are performed on the quantized transform coefficients, e.g., as part of the entropy coding process, in order to define one or more serialized one-dimensional vectors of coefficients from two-dimensional video blocks. Other scanning techniques, including other scan orders or adaptive scans, may also be used, and possibly signaled in the encoded bitstream. In any case, the scanned coefficients are then entropy coded along with any syntax data, e.g., via content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), or another entropy coding process. 
     As part of the encoding process, encoded video blocks may be decoded in order to generate the video data used for subsequent prediction-based coding of subsequent video blocks. At this stage, filtering may be employed in order to improve video quality, and e.g., remove blockiness artifacts from decoded video. The filtered data may be used for prediction of other video blocks, in which case the filtering is referred to as “in-loop” filtering. Alternatively, prediction of other video blocks may be based on unfiltered data, in which case the filtering is referred to as “post filtering.” 
     On a frame-by-frame, slice-by-slice, or LCU-by-LCU basis, the encoder may select one or more sets of filters, and on a coded-unit-by-coded-unit basis may select one or more filters from the set(s). In some instances, filters may also be selected on a pixel-by-pixel basis or on a sub-CU basis, such as a 4×4 block basis. Both selection of the set of filters and selection of which filter from the set of filters to apply to any given block (or set of blocks) can be made in a manner that promotes the video quality. Such sets of filters may be selected from pre-defined sets of filters, or may be adaptively defined to promote video quality. As an example, video encoder  122  may select or define several sets of filters for a given frame or slice such that different filters are used for different pixels of coded units of that frame or slice. In particular, for each input associated with a coded unit, several sets of filter coefficients may be defined, and the activity metric associated with the pixels of the coded unit may be used to determine which filter from the set of filters to use with such pixels. In some cases, video encoder  122  may apply several sets of filter coefficients and select one or more sets that produce the best quality video in terms of amount of distortion between a coded block and an original block, and/or the highest levels of compression. In any case, once selected, the set of filter coefficients applied by video encoder  122  for each coded unit may be encoded and communicated to video decoder  128  of destination device  116  so that video decoder  128  can apply the same filtering that was applied during the encoding process for each given coded unit. 
     When an activity metric is used for determining which filter to use with a particular input for a coded unit, the selection of the filter for that particular coded unit does not necessarily need to be communicated to video decoder  128 . Instead, video decoder  128  can also calculate the activity metric for the coded unit, and based on filter information previously provided by video encoder  122 , match the activity metric to a particular filter. 
       FIG. 3  is a block diagram illustrating a video encoder  350  consistent with this disclosure. Video encoder  350  may correspond to video encoder  122  of device  120 , or a video encoder of a different device. As shown in  FIG. 3 , video encoder  350  includes a prediction unit  332 , adders  348  and  351 , and a memory  334 . Video encoder  350  also includes a transform unit  338  and a quantization unit  340 , as well as an inverse quantization unit  342  and an inverse transform unit  344 . Video encoder  350  also includes a deblocking filter  347  and an adaptive filter unit  349 . Video encoder  350  also includes an entropy encoding unit  346 . Filter unit  349  of video encoder  350  may perform filtering operations and also may include a filter selection unit (FSU)  353  for identifying an optimal or preferred filter or set of filters to be used for decoding. Filter unit  349  may also generate filter information identifying the selected filters so that the selected filters can be efficiently communicated as filter information to another device to be used during a decoding operation. 
     During the encoding process, video encoder  350  receives a video block, such as an LCU, to be coded, and prediction unit  332  performs predictive coding techniques on the video block. Using the quadtree partitioning scheme discussed above, prediction unit  332  can partition the video block and perform predictive coding techniques on coding units of different sizes. For inter coding, prediction unit  332  compares the video block to be encoded, including sub-blocks of the video block, to various blocks in one or more video reference frames or slices in order to define a predictive block. For intra coding, prediction unit  332  generates a predictive block based on neighboring data within the same coded unit. Prediction unit  332  outputs the prediction block and adder  348  subtracts the prediction block from the video block being coded in order to generate a residual block. 
     For inter coding, prediction unit  332  may comprise motion estimation and motion compensation units that identify a motion vector that points to a prediction block and generates the prediction block based on the motion vector. Typically, motion estimation is considered the process of generating the motion vector, which estimates motion. For example, the motion vector may indicate the displacement of a predictive block within a predictive frame relative to the current block being coded within the current frame. Motion compensation is typically considered the process of fetching or generating the predictive block based on the motion vector determined by motion estimation. For intra coding, prediction unit  332  generates a predictive block based on neighboring data within the same coded unit. One or more intra-prediction modes may define how an intra prediction block can be defined. 
     After prediction unit  332  outputs the prediction block and adder  48  subtracts the prediction block from the video block being coded in order to generate a residual block, transform unit  38  applies a transform to the residual block. The transform may comprise a discrete cosine transform (DCT) or a conceptually similar transform such as that defined by a coding standard such as the HEVC standard. Wavelet transforms, integer transforms, sub-band transforms or other types of transforms could also be used. In any case, transform unit  338  applies the transform to the residual block, producing a block of residual transform coefficients. The transform may convert the residual information from a pixel domain to a frequency domain. 
     Quantization unit  340  then quantizes the residual transform coefficients to further reduce bit rate. Quantization unit  340 , for example, may limit the number of bits used to code each of the coefficients. After quantization, entropy encoding unit  346  scans the quantized coefficient block from a two-dimensional representation to one or more serialized one-dimensional vectors. The scan order may be pre-programmed to occur in a defined order (such as zig-zag scanning, horizontal scanning, vertical scanning, combinations, or another pre-defined order), or possibly adaptive defined based on previous coding statistics. 
     Following this scanning process, entropy encoding unit  346  encodes the quantized transform coefficients (along with any syntax data) according to an entropy coding methodology, such as CAVLC or CABAC, to further compress the data. Syntax data included in the entropy coded bitstream may include prediction syntax from prediction unit  332 , such as motion vectors for inter coding or prediction modes for intra coding. Syntax data included in the entropy coded bitstream may also include filter information from filter unit  349 , which can be encoded in the manner described herein. 
     CAVLC is one type of entropy coding technique supported by the ITU H.264/MPEG4, AVC standard, which may be applied on a vectorized basis by entropy encoding unit  346 . CAVLC uses variable length coding (VLC) tables in a manner that effectively compresses serialized “runs” of transform coefficients and/or syntax data. CABAC is another type of entropy coding technique supported by the ITU H.264/MPEG4, AVC standard, which may be applied on a vectorized basis by entropy encoding unit  346 . CABAC involves several stages, including binarization, context model selection, and binary arithmetic coding. In this case, entropy encoding unit  346  codes transform coefficients and syntax data according to CABAC. Like the ITU H.264/MPEG4, AVC standard, the emerging HEVC standard may also support both CAVLC and CABAC entropy coding. Furthermore, many other types of entropy coding techniques also exist, and new entropy coding techniques will likely emerge in the future. This disclosure is not limited to any specific entropy coding technique. 
     Following the entropy coding by entropy encoding unit  346 , the encoded video may be transmitted to another device or archived for later transmission or retrieval. Again, the encoded video may comprise the entropy coded vectors and various syntax data, which can be used by the decoder to properly configure the decoding process. Inverse quantization unit  342  and inverse transform unit  344  apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain. Summer  351  adds the reconstructed residual block to the prediction block produced by prediction unit  332  to produce a pre-deblocked reconstructed video block, sometimes referred to as pre-deblocked reconstructed image. De-blocking filter  347  may apply filtering to the pre-deblocked reconstructed video block to improve video quality by removing blockiness or other artifacts. The output of the de-blocking filter  347  can be referred to as a post-deblocked video block, reconstructed video block, or reconstructed image. 
     Filter unit  349  can be configured to receive multiple inputs or receive a single input. In the example of  FIG. 3 , filter unit  349  receives as input the post-deblocked reconstructed image (RI), pre-deblocked reconstructed image (pRI), the prediction image (PI), and the reconstructed residual block (EI). Filter unit  349  can use any of these inputs either individually or in combination to produce a reconstructed image to store in memory  334 . Filtering by filter unit  349  may improve compression in any of several manners, including generating predictive video blocks that more closely match video blocks being coded than unfiltered predictive video blocks, and generating filtered versions of reconstructed video blocks that more closely match original video blocks. After filtering, the reconstructed video block may be used by prediction unit  332  as a reference block to inter-code a block in a subsequent video frame or other coded unit. Although filter unit  349  is shown “in-loop,” the techniques of this disclosure could also be used with post filters, in which case non-filtered data (rather than filtered data) would be used for purposes of predicting data in subsequent coded units. 
     For a series of video blocks, such as a slice or frame, filter unit  349  may select sets of filters for each input in a manner that promotes the video quality. This disclosure will initially describe the process of selecting a single filter for a single input such as the post-deblocked reconstructed image (RI), but as mentioned above, the techniques are generally applicable to filters that receive other inputs or other combinations of inputs. As will be described in more detail below, the techniques are also generally applicable to selecting multiple filters based on an activity metric. 
     Filter unit  349  receives a first series of video blocks, such as a first frame or first slice. The first series of video blocks may, for example, be an RI as shown in  FIG. 3 . As described above in relation to  FIGS. 2A and 2B , the series of video blocks for the RI has an associated quadtree partitioning. For the first series of video block, FSU  353  determines a first decoding filter, and filter unit  349  determines which coded unit of the series of video blocks should be filtered and which coded units should not be filtered. The determination of which coded units to filter and which coded units not to filter is used to generate a filter map, as generally described in  FIGS. 2C and 2D , for the first series of video blocks. Filter unit  349  signals the selection of the decoding filter for the first series of video blocks to entropy encoding unit  346 . Entropy encoding unit  346  encodes the selection of decoding filter into the bitstream which is transmitted to a decoding device. 
     In addition to determining a decoding filter for the first series of video blocks, FSU  353  also determines an interim filter for the first series of video blocks. The interim filter is determined for portions of the first series of video blocks that are not filtered by the decoding filter. Using the filter map of  FIG. 2C  as an example, the coded units identified as “on” are to be filtered by the decoding filter. Thus, FSU  353  determines the interim filter for the coded units identified as “off” in  FIG. 2C . For those coded units identified as “off,” FSU  353  determines an interim filter that improves the quality of those coded units when reconstructed relative to an original image. Unlike the decoding filter, however, the interim filter is not necessarily entropy encoded and transmitted in the bitstream to a decoding device. Instead, the interim filter can be used to help determine the actual filter for a second set of video blocks, possibly without transmission by filter unit  349 . 
     Using this interim filter determined for the first series of video blocks, filter unit  349  can generate a filter map for a second series of video blocks. Filter unit  349  determines the filter map for the second series of video blocks by applying the interim filter determined for the first series of video blocks to the second series of video blocks. Filter unit  349  identifies coded units of the second series of video blocks that are improved by the interim filter as “on” and coded units not improved by the interim filter as “off.” For the coded units of the coded units of the second series of video blocks identified as “on,” FSU  353  determines a new decoding filter. For the coded units of the second series of video blocks identified as “off,” FSU  353  determines a new interim filter. As with the first series of video blocks, filter unit  349  signals the selection of the new decoding filter for the second series of video blocks to entropy encoding unit  346  for inclusion in the bitstream but does not necessarily signal the new interim filter for inclusion in the bitstream. 
     Filter unit  349  uses the new interim filter determined for the second series of video blocks to determine a filter map for the third series of video blocks (a third filter map). For coded units identified in the third filter map as having filtering “on,” FSU  353  determines a new decoding filter (a third decoding filter). For coded units identified in the third filter map as having filtering “off,” FSU  353  determines a new interim filter (a third interim filter). Filter unit  349  signals the selection of the third decoding filter, but not necessarily the selection of the third interim filter, to entropy encoding unit  346  for inclusion in the bitstream. After determining an initial filter and initial filter map, filter unit  349  can repeat this process of using an interim filter determined for a previous frame to determine a filter map for a current frame indefinitely. In this way, the unfiltered blocks of a previous unit of video (e.g., a previous frame or slice) can be used to define the next filter to be applied to the next unit of video (e.g., the next frame or slice). 
     FSU  353  may determine new filters, both decoding filters and interim filters, by analyzing the auto-correlations and cross-correlations between a filtered image and an original image. A new filter or set of filters may, for example, be determined by solving Wienter-Hopt equations based on the auto- and cross-correlations. Regardless of whether a new set of filters is trained or an existing set of filters are selected, filter unit  349  generates syntax data for inclusion in the bit stream that enables a decoder to also identify the set or sets of filters to be used for the particular frame or slice. 
     According to this disclosure, for each pixel of a coded unit within the frame or slice, filter unit  349  may select which filter from a set of filters is to be used based on an activity metric that quantifies activity associated with one or more sets of pixels within the coded unit. Filter unit  349  may select filters on a pixel-by-pixel basis or may select pixels on a group-by-group basis, where each group might be, for example, a 2×2 block, 4×4 block, or M×N block of pixels. In this way, FSU  353  may determine sets of filters for a higher level coded unit such as a frame or slice, while filter unit  349  determines which filter(s) from the set(s) is to be used for a particular pixel or group of pixels of a lower level coded unit based on the activity associated with the pixel or group of pixels of that lower level coded unit. Activity may be indicated in terms of pixel value variance within a coded unit. More variance in the pixel values in the coded unit may indicate higher levels of pixel activity, while less variance in the pixel values may indicate lower levels of pixel activity. Different filters (i.e. different filter coefficients) may result in better filtering (e.g., higher image quality) depending on the level of pixel variance, i.e., activity. The pixel variance may be quantified by an activity metric, which may comprise a sum-modified Laplacian value as discussed in greater detail below. However, other types of activity metrics may also be used. 
     Instead of a single decoding filter, a set of M decoding filters may be used. Depending on design preferences, M may for example be as few as 2 or as great as 16, or even higher. A large number of decoding filters may improve video quality, but also may increase overhead associated with signaling sets of filters from encoder to decoder. A set of M decoding filters can be determined by FSU  353  as described above and transmitted to the decoder for each series of video blocks. A segmentation map can be used to indicate how a coded unit is segmented, and a filter map can be used to indicate whether or not a particular coded unit is to be filtered. The segmentation map, may for example, include for a coded unit an array of split flags as described above as well an additional bit signaling whether each sub-coded unit is to be filtered. For each input associated with a pixel of a coded unit that is to be filtered, a specific filter from the set of filters can be chosen based on the activity metric. The activity metric can be calculated using a sum-modified Laplacian for pixel (i,j) as follows: 
     
       
         
           
             
               var 
                
               
                 ( 
                 
                   i 
                   , 
                   j 
                 
                 ) 
               
             
             = 
             
               
                 
                   ∑ 
                   
                     k 
                     = 
                     
                       - 
                       K 
                     
                   
                   K 
                 
                  
                 
                   
                     ∑ 
                     
                       l 
                       = 
                       
                         - 
                         L 
                       
                     
                     L 
                   
                    
                   
                      
                     
                       
                         2 
                          
                         
                           R 
                            
                           
                             ( 
                             
                               
                                 i 
                                 + 
                                 k 
                               
                               , 
                               
                                 j 
                                 + 
                                 l 
                               
                             
                             ) 
                           
                         
                       
                       - 
                       
                         R 
                          
                         
                           ( 
                           
                             
                               i 
                               + 
                               k 
                               - 
                               1 
                             
                             , 
                             
                               j 
                               + 
                               1 
                             
                           
                           ) 
                         
                       
                       - 
                       
                         R 
                          
                         
                           ( 
                           
                             
                               i 
                               + 
                               k 
                               + 
                               1 
                             
                             , 
                             
                               j 
                               + 
                               l 
                             
                           
                           ) 
                         
                       
                     
                      
                   
                 
               
               + 
               
                 
                    
                   
                     
                       2 
                        
                       
                         R 
                          
                         
                           ( 
                           
                             
                               i 
                               + 
                               k 
                             
                             , 
                             
                               j 
                               + 
                               l 
                             
                           
                           ) 
                         
                       
                     
                     - 
                     
                       R 
                        
                       
                         ( 
                         
                           
                             i 
                             + 
                             k 
                           
                           , 
                           
                             j 
                             + 
                             l 
                             - 
                             1 
                           
                         
                         ) 
                       
                     
                     - 
                     
                       R 
                        
                       
                         ( 
                         
                           
                             i 
                             + 
                             k 
                           
                           , 
                           
                             j 
                             + 
                             l 
                             + 
                             1 
                           
                         
                         ) 
                       
                     
                   
                    
                 
                 . 
               
             
           
         
       
     
     As one example, a 7×7 (K, L=3) group of surrounding pixels may be used for calculation of the sum-modified Laplacian value. The particular filter from the set of M decoding filters to be used for a particular range of sum-modified Laplacian values can also be sent to the decoder with the set of M filters. Filter coefficients can be coded using prediction from coefficients transmitted for previous frames or other techniques. Filters of various shapes and sizes, including for example 1×1, 3×3, 5×5, 7×7, and 9×9 filters with diamond shape support or square shape support might be used. 
     According to the techniques of this disclosure, to determine a set of M decoding filters, filter unit  349  can classify each pixel in the series of video blocks as being in one of M different ranges of an activity metric. Filter unit  349  can then determine a decoding filter using the techniques described above, but instead of determining a single filter for an entire series of video blocks, filter unit  349  determines a decoding filter for each range of the activity metric using the pixels that fall within that particular range. For example, to determine four decoding filters for a first series of video blocks, filter unit  349 , based on an activity metric such as a sum-modified Laplacian value, can classify each pixel in the series of video blocks into one of four different ranges for the activity metric. For pixels in the first range of the activity metric, filter unit  349  can apply a first interim filter determined for pixels of a previous series of video blocks. For pixels in the second range of the activity metric, filter unit  349  can apply a second interim filter determined for pixels of a previous series of video blocks, and so on. The interim filters determined for pixels of the previous series of video blocks can be determined for the same ranges of the activity metric for the previous series of video blocks. Thus, if the first interim filter was determined for a previous series of video blocks for a first range of the activity metric, the first interim filter can be applied to pixels of the current frame within the same first range of the activity metric. 
     Based on applying the set of interim filters to the current series of video blocks, a filter map can be determined for the current series of video blocks. Using the filter map for the current series of video blocks, FSU  353  can determine a decoding filter and an interim filter for each range of the activity metric as described above. Entropy encoding unit can include the set of M decoding filters in the bitstream. 
     In accordance with this disclosure, filter unit  349  performs coding techniques with respect to filter information that may reduce the amount of data needed to encode and convey filter information from encoder  350  to another device. Again, for each series of video blocks, such as a frame or slice, filter unit  349  may define or select one or more sets of filter coefficients to be applied to the pixels of coded units for that frame or slice. Filter unit  349  applies the filter coefficients in order to filter video blocks of reconstructed video frames stored in memory  334 , which may be used for predictive coding consistent with in-loop filtering. Filter unit  349  can encode the filter coefficients as filter information, which is forwarded to entropy encoding unit  346  for inclusion in the encoded bitstream. 
     The techniques of this disclosure may also exploit the fact that some of the filter coefficients defined or selected by FSU  353  may be very similar to other filter coefficients applied with respect to the pixels of coded units of another frame or slice. The same type of filter may be applied for different frames or slices (e.g., the same filter support), but the filters may be different in terms of filter coefficient values associated with the different indices of the filter support. Accordingly, in order to reduce the amount of data needed to convey such filter coefficients, filter unit  349  may predictively encode one or more filter coefficients to be used for filtering based on the filter coefficients of another coded unit, exploiting any similarities between the filter coefficients. In some cases, however, it may be more desirable to encode the filter coefficients directly, e.g., without using any prediction. Various techniques, such as techniques that exploit the use of an activity metric to define when to encode the filter coefficients using predictive coding techniques and when to encode the filter coefficients directly without any predictive coding, can be used for efficiently communicating filter coefficients to a decoder. Additionally, symmetry may also be imposed so that a subset of coefficients (e.g., 5, −2, 10) known by the decoder can be used to define the full set of coefficients (e.g., 5, −2, 10, 10, −2, 5). Symmetry may be imposed in both the direct and the predictive coding scenarios. 
       FIG. 4  is a block diagram illustrating an example of a video decoder  460 , which decodes a video sequence that is encoded in the manner described herein. The received video sequence may comprise an encoded set of image frames, a set of frame slices, a commonly coded group of pictures (GOPs), or a wide variety of types of series of video blocks that include encoded video blocks and syntax data to define how to decode such video blocks. 
     Video decoder  460  includes an entropy decoding unit  452 , which performs the reciprocal decoding function of the encoding performed by entropy encoding unit  346  of  FIG. 3 . In particular, entropy decoding unit  452  may perform CAVLC or CABAC decoding, or any other type of entropy decoding used by video encoder  350 . Entropy decoded video blocks in a one-dimensional serialized format may be inverse scanned to convert one or more one-dimensional vectors of coefficients back into a two-dimensional block format. The number and size of the vectors, as well as the scan order defined for the video blocks may define how the two-dimensional block is reconstructed. Entropy decoded prediction syntax data may be sent from entropy decoding unit  452  to prediction unit  454 , and entropy decoded filter information may be sent from entropy decoding unit  452  to filter unit  459 . 
     Video decoder  460  also includes a prediction unit  454 , an inverse quantization unit  456 , an inverse transform unit  458 , a memory and a summer  464 . In addition, video decoder  460  also includes a de-blocking filter  457  that filters the output of summer  464 . Consistent with this disclosure, filter unit  459  may receive entropy decoded filter information that includes one or more filters to be applied to one or more inputs. Although not shown on  FIG. 4 , de-blocking filter  457  may also receive entropy decoded filter information that includes one or more filters to be applied. 
     The filters applied by filter unit  459  may be defined by sets of filter coefficients. Filter unit  459  may be configured to generate the sets of filter coefficients based on the filter information received from entropy decoding unit  452 . The filter information may include additional signaling syntax data that signals to the decoder the manner of encoding used for any given set of coefficients. Instead of being signaled, the manner of encoding may also be programmed into video decoder  460  or be derivable by video decoder  460  without signaling. In some implementations, the filter information may for example, also include activity metric ranges for which any given set of coefficients should be used. Following decoding of the filters, filter unit  459  can filter the pixel values of decoded video blocks based on the one or more sets of filter coefficients and the signaling syntax data that includes activity metric ranges for which the different sets of filter coefficients should be used. The activity metric ranges may be defined by a set of activity values that define the ranges of activity metrics used to define the type of encoding used (e.g., predictive or direct). 
     Filter unit  459  may receive in the bit stream a set of filters for each frame or slice. For each coded unit within the frame or slice, filter unit  459  can calculate one or more activity metrics associated with the decoded pixels of a coded unit for multiple inputs (i.e. PI, EI, pRI, and RI) in order to determine which filter(s) of the set(s) to apply to each input. For a first range of the activity metric, filter unit  459  may apply a first filter, for a second range of the activity metric filter unit  459  may apply a second filter, and so on. In some implementations four ranges may map to four different filters, although any number of ranges and filters may be used. The filter may generally assume any type of filter support shape or arrangement. The filter support refers to the shape of the filter with respect to a given pixel being filtered, and the filter coefficients may define weighting applied to neighboring pixel values according to the filter support. Sometimes, the filter type may be presumed by the encoder and decoder, in which case the filter type is not included in the bitstream, but in other cases, filter type may be encoded along with filter coefficient information as described herein. The syntax data may also signal to the decoder how the filters were encoded (e.g., how the filter coefficients were encoded), as well as the ranges of the activity metric for which the different filters should be used. 
     Prediction unit  454  receives prediction syntax data (such as motion vectors) from entropy decoding unit  452 . Using the prediction syntax data, prediction unit  454  generates the prediction blocks that were used to code video blocks. Inverse quantization unit  456  performs inverse quantization, and inverse transform unit  458  performs inverse transforms to change the coefficients of the residual video blocks back to the pixel domain. Adder  464  combines each prediction block with the corresponding residual block output by inverse transform unit  458  in order to reconstruct the video block. 
     Filter unit  459  generates the filter coefficients to be applied for each input of a coded unit, and then applies such filter coefficients in order to filter the reconstructed video blocks of that coded unit. In addition to the filtering described herein, filtering may also comprise additional deblock filtering applied to edges of video blocks to smooth the edges and/or eliminate artifacts associated with video blocks. The filtering may also include denoise filtering to reduce quantization noise, or any other type of filtering that can improve coding quality. The filtered video blocks are accumulated in memory  462  in order to reconstruct decoded frames (or other decodable units) of video information. The decoded units may be output from video decoder  460  for presentation to a user, but may also be stored for use in subsequent predictive decoding. 
     In the field of video coding, it is common to apply filtering at the encoder and decoder in order to enhance the quality of a decoded video signal. Filtering can be applied via a post-filter, in which case the filtered frame is not used for prediction of future frames. Alternatively, filtering can be applied “in-loop,” in which case the filtered frame may be used to predict future frames. A desirable filter can be designed by minimizing the error between the original signal and the decoded filtered signal. Typically, such filtering has been based on applying one or more filters to a reconstructed image. For example, a deblocking filter might be applied to a reconstructed image prior to the image being stored in memory, or a deblocking filter and one additional filter might be applied to a reconstructed image prior to the image being stored in memory. Techniques of the present disclosure include the application of filters to inputs other than just a reconstructed image. Additionally, as will be discussed more below, filters for those multiple inputs can be selected based on Laplacian filter indexing. 
     In a manner similar to the quantization of transform coefficients, the coefficients of the filter h(k,l), where k=−K, . . . , K, and l=−L, . . . , L may also be quantized. K and L may represent integer values. The coefficients of filter h(k,l) may be quantized as: 
         f ( k,l )=round(normFact· h ( k,l ))
 
     where normFact is a normalization factor and round is the rounding operation performed to achieve quantization to a desired bit-depth. Quantization of filter coefficients may be performed by filter unit  349  of  FIG. 3  during the encoding, and de-quantization or inverse quantization may be performed on decoded filter coefficients by filter unit  459  of  FIG. 4 . Filter h(k,l) is intended to generically represent any filter. For example, filter h(k,l) could be applied to any one of multiple inputs. In some instances multiple inputs associated with a video block will utilize different filters, in which case multiple filters similar to h(k,l) may be quantized and de-quanitzed as described above. 
     The quantized filter coefficients are encoded and sent from source device associated with encoder  350  to a destination device associated with decoder  460  as part of an encoded bitstream. In the example above, the value of normFact is usually equal to 2n although other values could be used Larger values of normFact lead to more precise quantization such that the quantized filter coefficients f (k, l) provide better performance. However, larger values of normFact may produce coefficients f (k, l) that require more bits to transmit to the decoder. 
     At decoder  460  the decoded filter coefficients f (k,l) may be applied to the appropriate input. For example, if the decoded filter coefficients are to be applied to RI, the filter coefficients may be applied to the post-deblocked reconstructed image RI(i,j), where i=0, . . . , M and j=0, . . . , N as follows: 
     
       
         
           
             
               
                 R 
                 ~ 
               
                
               
                 I 
                  
                 
                   ( 
                   
                     i 
                     , 
                     j 
                   
                   ) 
                 
               
             
             = 
             
               
                 ∑ 
                 
                   k 
                   = 
                   
                     - 
                     K 
                   
                 
                 K 
               
                
               
                 
                   ∑ 
                   
                     l 
                     = 
                     
                       - 
                       L 
                     
                   
                   L 
                 
                  
                 
                   
                     f 
                      
                     
                       ( 
                       
                         k 
                         , 
                         l 
                       
                       ) 
                     
                   
                    
                   
                     
                       RI 
                        
                       
                         ( 
                         
                           
                             i 
                             + 
                             k 
                           
                           , 
                           
                             j 
                             + 
                             l 
                           
                         
                         ) 
                       
                     
                     / 
                     
                       
                         ∑ 
                         
                           k 
                           = 
                           
                             - 
                             K 
                           
                         
                         K 
                       
                        
                       
                         
                           ∑ 
                           
                             l 
                             = 
                             
                               - 
                               L 
                             
                           
                           L 
                         
                          
                         
                           f 
                            
                           
                             ( 
                             
                               k 
                               , 
                               l 
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
             
           
         
       
     
     The variables M, N, K and L may represent integers. K and L may define a block of pixels that spans two-dimensions from −K to K and from −L to L. Filters applied to other inputs can be applied in an analogous manner. 
     The techniques of this disclosure may improve the performance of a post-filter or in-loop filter, and may also reduce number of bits needed to transmit filter coefficients f(k, l). In some cases, a number of different post-filters or in-loop filters are transmitted to the decoder for each series of video block, e.g., for each frame, slice, portion of a frame, group of frames (GOP), or the like. For each filter, additional information is included in the bitstream to identify the coded units, macroblocks and/or pixels for which a given filter should be applied. 
     The frames may be identified by frame number and/or frame type (e.g., I-frames, P-frames or B-frames). I-frames refer to intra-frames that are intra-predicted. P-frames refer to predictive frames that have video blocks predicted based on one list of data (e.g., one previous frame). B-frames refer to bidirectional predictive frames that are predicted based on two lists of data (e.g., a previous and subsequent frame). Macroblocks can be identified by listing macroblock types and/or range of quantization parameter (QP) values use to reconstruct the macroblock. 
     The filter information may also indicate that only pixels for which the value of a given measure of local characteristic of an image, called an activity metric, is within specified range should be filtered with a particular filter. For example, for pixel (i,j) the activity metric may comprise a sum-modified Laplacian value calculated as follows: 
     
       
         
           
             
               var 
                
               
                 ( 
                 
                   i 
                   , 
                   j 
                 
                 ) 
               
             
             = 
             
               
                 
                   ∑ 
                   
                     k 
                     = 
                     
                       - 
                       K 
                     
                   
                   K 
                 
                  
                 
                   
                     ∑ 
                     
                       l 
                       = 
                       
                         - 
                         L 
                       
                     
                     L 
                   
                    
                   
                      
                     
                       
                         2 
                          
                         
                           R 
                            
                           
                             ( 
                             
                               
                                 i 
                                 + 
                                 k 
                               
                               , 
                               
                                 j 
                                 + 
                                 l 
                               
                             
                             ) 
                           
                         
                       
                       - 
                       
                         R 
                          
                         
                           ( 
                           
                             
                               i 
                               + 
                               k 
                               - 
                               1 
                             
                             , 
                             
                               j 
                               + 
                               1 
                             
                           
                           ) 
                         
                       
                       - 
                       
                         R 
                          
                         
                           ( 
                           
                             
                               i 
                               + 
                               k 
                               + 
                               1 
                             
                             , 
                             
                               j 
                               + 
                               l 
                             
                           
                           ) 
                         
                       
                     
                      
                   
                 
               
               + 
               
                  
                 
                   
                     2 
                      
                     
                       R 
                        
                       
                         ( 
                         
                           
                             i 
                             + 
                             k 
                           
                           , 
                           
                             j 
                             + 
                             l 
                           
                         
                         ) 
                       
                     
                   
                   - 
                   
                     R 
                      
                     
                       ( 
                       
                         
                           i 
                           + 
                           k 
                         
                         , 
                         
                           j 
                           + 
                           l 
                           - 
                           1 
                         
                       
                       ) 
                     
                   
                   - 
                   
                     R 
                      
                     
                       ( 
                       
                         
                           i 
                           + 
                           k 
                         
                         , 
                         
                           j 
                           + 
                           l 
                           + 
                           1 
                         
                       
                       ) 
                     
                   
                 
                  
               
             
           
         
       
     
     wherein k represents a value of a summation of pixel values from −K to K and l represents a value of a summation from −L to L for a two-dimensional window that spans from −K to K and −L to L, wherein i and j represent pixel coordinates of the pixel data, R/(i,j) represents a given pixel value at coordinates i and j, and var(i,j) is the activity metric. An activity metric may similarly be found for pRI(i,j), PI(i,j), and EI(i,j). 
     As discussed above, a sum-modified Laplacian value is one commonly used type of activity metric, but it is contemplated that the techniques of this disclosure may be used in conjunction with other types of activity metrics or combinations of activity metrics. Additionally, as discussed above, rather than using an activity metric to select a filter on a pixel-by-pixel basis, an activity metric may also be used to select a filter on a group-by-group basis, where for example, a group of pixels is a 2×2 block of pixels, a 4×4 block of pixels, or an M×N block of pixels. 
     Filter coefficients f(k, l), for any input, may be coded using prediction from coefficients transmitted for previous coded units. For each input of a coded unit m (e.g., each frame, slice or GOP), the encoder may encode and transmit a set of M filters: 
         g   i   m , wherein  i= 0 , . . . , M− 1. 
     For each filter, the bitstream may be encoded to identify a range of values of activity metric value var for which the filter should be used. 
     For example, filter unit  349  of encoder  350  may indicate that filter:
         g o   m  
 
should be used for pixels for which activity metric value var is within interval [0,var 0 ), i.e., var≧0 and var&lt;var 0 . Furthermore, filter unit  349  of encoder  350  may indicate that filter:
       

         g   i   m  where  i= 1 , . . . , M− 2, 
     should be used for pixels for which activity metric value var is within interval [var i-1 ,var i ). In addition, filter unit  349  of encoder  350  may indicate that filter:
         g M-1   m  
 
should be used for pixels for which the activity metric var when var&gt;var M-2 . As described above, filter unit  349  may use one set of filters for all inputs, or alternatively, may use a unique set of filters for each input.
       

     The filter coefficients can be predicted using reconstructed filter coefficients used in a previous coded unit. The previous filter coefficients may be represented as: 
         f   i   n  where  i= 0 , . . . , N− 1, 
     In this case, the number of the coded unit n may be used to identify one or more filters used for prediction of the current filters, and the number n may be sent to the decoder as part of the encoded bitstream. In addition, information can be encoded and transmitted to the decoder to identify values of the activity metric var for which predictive coding is used. 
     For example, assume that for a currently coded frame m, coefficients:
         g r   m  
 
are transmitted for the activity metric values [var r-1 , var r ). The filter coefficients of the frame m are predicted from filter coefficients of the frame n. Assume that filter
   f s   n  
 
is used in frame n for pixels for which the activity metric is within an interval [var s-1 , var s ) where var s-1 ==var r-1 ] and var s &gt;var r . In this case, interval [var r-1 , var r ) is contained within interval [var s-1 , var s ). In addition, information may be transmitted to the decoder indicating that prediction of filter coefficients should be used for activity values [var t−1 , var t ) but not for activity values [var t ,var t+1  ]) where var t−1 ==var r-1  and var t+1 ==var r .
       

     The relationship between intervals [var r-1 −1, var r ), [var s-1 , var s ), [var t−1 , var t ) and [var t , var t+1 ) is depicted in  FIG. 5 . In this case, the final values of the filter coefficients:
         f t   m  
 
used to filter pixels with activity metric in the interval [var t− , var t ) are equal to the sum of coefficients:
   f s   n  and g r   m          

     Accordingly: 
         f   t   m ( k,l )= f   s   n ( k,l )+ g   r   m ( k,l ),  k=−K, . . . , K, l=−L, . . . , L.    
     In addition, filter coefficients:
         f t+1   m  
 
that are used for pixels with activity metric [var t , var t+1 ) are equal to filter coefficients:
   g l   m ·       

     Therefore: 
         f   t+1   m ( k,l )= g   r   m ( k,l ),  k=−K, . . . , K, l=−L, . . . , L.    
     The amplitude of the filter coefficients g(k, l) depends on k and l values. Usually, the coefficient with the biggest amplitude is the coefficient g(0,0). The other coefficients which are expected to have large amplitudes are the coefficients for which value of k or/is equal to 0. This phenomenon may be utilized to further reduce amount of bits needed to transmit the coefficients. The index values k and l may define locations within a known filter support. 
     The coefficients: 
         g   i   m ( k,l ),  i= 0 , . . . , M− 1 
     for each frame m may be coded using parameterized variable length codes such as Golomb or exp-Golomb codes defined according to a parameter p. By changing the value of parameter p that defines the parameterized variable length codes, these codes can be used to efficiently represent wide range of source distributions. The distribution of coefficients g(k,l) (i.e., their likelihood to have large or small values) depends on values of k and l. Hence, to increase coding efficiency, for each frame m, the value of parameter p is transmitted for each pair (k,l). The parameter p can be used for parameterized variable length coding when encoding coefficients: 
         g   i   m ( k,l ) where  k=−K, . . . , K, l=−L, . . . , L.    
       FIG. 4  and this disclosure generally describe filter unit  459  as implementing a multi-input, multi-filter filtering scheme based on an activity metric. As discussed above, however, in some implementations, filter unit  459  may implement a single input, multi-filter filtering scheme based on an activity metric, or may implement filtering schemes that are single input that do not utilize an activity metric. 
       FIG. 6  is a flow diagram illustrating encoding techniques consistent with this disclosure. As shown in  FIG. 3 , video encoder  350  encodes pixel data of a series of video blocks. The series of video blocks may comprise a frame, a slice, a group of pictures (GOP), or another independently decodable unit. The pixel data may be arranged in coded units, and video encoder  350  may encode the pixel data by encoding the coded units in accordance with a video encoding standard such as the HEVC standard. For a first series of video blocks, FSU  353  determines a first filter for a first set of coded units of the first series of video blocks ( 601 ). FSU  353  also determines a first interim filter for a second set of coded units of the first series of video blocks ( 602 ). Determining the first interim filter for the first series of video blocks may, for example, include determining a filter for unfiltered coded units of the first series of video blocks. The first set of coded units for the first series of video blocks might correspond to coded units that are to be filtered by a video decoder, while the second set of coded units of the first series of video blocks might correspond to coded units that are not to be filtered by the video decoder. 
     Filter unit  349  applies the first interim filter to coded units of a second series of video blocks to determine a first set of coded units for the second series of video blocks and a second set of coded units for the second series of video blocks ( 603 ). Applying the first interim filter to coded units of the second series of video blocks to determine the first set of coded units for the second series of video blocks and the second set of coded units for the second series of video blocks might, for example, include comparing filtered versions of coded units of the second series of video blocks to original versions of coded units of the second series of video blocks. The first set of coded units for the second series of video blocks might correspond to coded units that are to be filtered at a decoder, while the second set of coded units for the second series of video blocks might correspond to coded units that are not to be filtered at the decoder. FSU  353  determines a second filter for the first set of coded units of the second series of video block ( 604 ). The first interim filter can be a different filter than the first filter. In some implementations, FSU  353  may also determine a third filter for the first set of coded units for the second series of video block. The second filter can corresponds to a first range of an activity metric, and the third filter can corresponds to a second range of the activity metric. 
     Video encoder  350  outputs an encoded bitstream for the coded unit, which includes encoded pixel data and the encoded filter data. The encoded filter data may include signaling information for identifying the filter or set of filters to be used and may also include signaling information that identifies how the filters were encoded and the ranges of the activity metric for which the different filters should be applied. The encoded pixel data may include among other types of data, a segmentation map and a filter map for a particular coded unit. Entropy encoding unit  346  can include information describing the first filter and the second filter in a bitstream ( 605 ). Information describing the first interim filter, however, may not be included in the bitstream for transmission. 
       FIG. 7  is a flow diagram illustrating encoding techniques consistent with this disclosure. As shown in  FIG. 3 , video encoder  350  encodes pixel data of a series of video blocks, such as a slice or frame. The pixel data may be arranged in coded units, and video encoder  350  may encode the pixel data by encoding the coded units in accordance with a video encoding standard such as the HEVC standard. For a first slice or frame, FSU  353  determines a first decoding filter ( 701 ). A filter map for the first slice or frame identifies which coded units of the first slice or frame are to be filtered with the first decoding filter. FSU  353  also determines a first interim filter for the first slice or frame ( 702 ). The first interim filter is determined based on portions of the first slice or frame that are not to be filtered by the first decoding filter. Filter unit  349  applies the first interim filter to a second slice or frame to generate a filter map for the second slice or frame ( 703 ). The filter map for the second slice or frame generally identifies which coded units of the second slice or frame were improved by the first interim filter relative to an original image and which coded units were not improved. For the coded units that were improved by the first interim filter, FSU  353  determines a second decoding filter ( 704 ). Video encoder  350  outputs an encoded bitstream for the coded unit, which includes encoded pixel data and the encoded filter data. The encoded filter data may include signaling information for identifying the first decoding filter and the second decoding filter ( 705 ). 
     The foregoing disclosure has been simplified to some extent in order to convey details. For example, the disclosure generally describes sets of filters being transmitted on a per-frame or per-slice basis, but sets of filters may also be transmitted on a per-sequence basis, per-group of picture basis, per-group of slices basis, per-CU basis, per-LCU basis, or other such basis. In general, filters may be transmitted for any grouping of one or more coded units. Additionally, in implementation, there may be numerous filters per input per coded unit, numerous coefficients per filter, and numerous different levels of variance with each of the filters being defined for a different range of variance. For example, in some cases there may be sixteen or more filters defined for each input of a coded unit and sixteen different ranges of variance corresponding to each filter. 
     Each of the filters for each input may include many coefficients. In one example, the filters comprise two-dimensional filters with 81 different coefficients defined for a filter support that extends in two-dimensions. However, the number of filter coefficients that are transmitted for each filter may be fewer than 81 in some cases. Coefficient symmetry, for example, may be imposed such that filter coefficients in one dimension or quadrant may correspond to inverted or symmetric values relative to coefficients in other dimensions or quadrants. Coefficient symmetry may allow for 81 different coefficients to be represented by fewer coefficients, in which case the encoder and decoder may assume that inverted or mirrored values of coefficients define other coefficients. For example, the coefficients (5, −2, 10, 10, −2, 5) may be encoded and transmitted as the subset of coefficients (5, −2, 10). In this case, the decoder may know that these three coefficients define the larger symmetric set of coefficients (5, −2, 10, 10, −2, 5). 
     The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, and integrated circuit (IC) or a set of ICs (i.e., a chip set). Any components, modules or units have been described provided to emphasize functional aspects and does not necessarily require realization by different hardware units. 
     Accordingly, the techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in hardware, any features described as modules, units or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable medium comprising instructions that, when executed in a processor, performs one or more of the methods described above. The computer-readable medium may comprise a computer-readable storage medium and may form part of a computer program product, which may include packaging materials. The computer-readable storage medium may comprise random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, magnetic or optical data storage media, and the like. The techniques additionally, or alternatively, may be realized at least in part by a computer-readable communication medium that carries or communicates code in the form of instructions or data structures and that can be accessed, read, and/or executed by a computer. 
     The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, an application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated software modules or hardware modules configured for encoding and decoding, or incorporated in a combined video codec. Also, the techniques could be fully implemented in one or more circuits or logic elements. 
     Various aspects of the disclosure have been described. These and other aspects are within the scope of the following claims.