Patent Publication Number: US-11381846-B2

Title: Image processing device and image processing method

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     The present application is a continuation of U.S. application Ser. No. 16/403,620, filed May 6, 2019, which is a continuation of U.S. application Ser. No. 15/929,006, filed Apr. 4, 2018 (now U.S. Pat. No. 10,334,279), which is a continuation of U.S. application Ser. No. 15/395,471, filed Dec. 30, 2016 (now U.S. Pat. No. 10,003,827), which is a continuation of U.S. application Ser. No. 13/991,007, filed May 31, 2013 (now U.S. Pat. No. 9,912,967), which is the National Stage of International Application No. PCT/JP2011/077954, filed on Dec. 2, 2011, which claimed the benefit of priority from Japanese Application No. 2010-272907, filed Dec. 7, 2010, Japanese Application No. 2011-004392, filed Jan. 12, 2011, Japanese Application No. 2011-045651, filed Mar. 2, 2011, and Japanese Application No. 2011-117558, filed May 26, 2011, the entire contents of each of which are hereby incorporated by reference. 
    
    
     TECHNICAL FIELD 
     The present disclosure relates to an image processing device and an image processing method. 
     BACKGROUND ART 
     H.264/AVC, one of standard specifications for image encoding scheme, applies a deblocking filter to a block boundary in units of blocks each containing 4×4 pixels, for example, in order to prevent image quality degradation due to block distortion while an image is encoded. The deblocking filter requires a large amount of processing and may account for 50% of the entire processing amount in image decoding, for example. 
     The standards work for High Efficiency Video Coding (HEVC), a next-generation image encoding scheme, proposes application of the deblocking filter in units of blocks each containing 8×8 pixels or more according to JCTVC-A119 (see Non-Patent Literature 1). The technique proposed in JCTVC-A119 increases the block size to a minimum unit which allows for applying the deblocking filter to perform filtering processes in parallel on block boundaries in the same direction within one macro block. 
     CITATION LIST 
     Non-Patent Literature 
     
         
         Non-Patent Literature 1: K. Ugur (Nokia), K. R. Andersson (LM Ericsson), A. Fuldseth (Tandberg Telecom), “JCTVC-A119:Video coding technology proposal by Tandberg, Nokia, and Ericsson”, Documents of the first meeting of the Joint Collaborative Team on Video Coding (JCT-VC), Dresden, Germany, 15-23 Apr. 2010. 
       
    
     SUMMARY OF INVENTION 
     Technical Problem 
     Even if the technique proposed in JCTVC-A119 is used, there remains dependency between a process on the vertical block boundary and a process on the horizontal block boundary. Specifically, a process on the vertical boundary for one macro block waits until a process on the horizontal boundary for a neighboring macro block is performed. A process on the horizontal boundary for one macro block waits until a process on the vertical boundary for the same macro block is performed. The above-described technique can just provide a parallel process of the deblocking filter to a very limited extent. Accordingly, the above-described technique may not successfully solve problems of a delay and a decrease in data rates due to a large processing amount while the deblocking filter is applied. 
     The technology according to the disclosure aims at providing an image processing device and an image processing method capable of providing further parallel processing when a deblocking filter is applied. 
     Solution to Problem 
     According to an embodiment of the present disclosure, there is provided an image processing device including a decoding section configured to decode an image from an encoded stream, a horizontal filtering section configured to apply a deblocking filter to a vertical block boundary within an image to be decoded by the decoding section, a vertical filtering section configured to apply a deblocking filter to a horizontal block boundary within an image to be decoded by the decoding section, and a control section configured to cause the horizontal filtering section to filter in parallel a plurality of vertical block boundaries included in a processing unit containing a plurality of coding units and cause the vertical filtering section to filter in parallel a plurality of horizontal block boundaries included in the processing unit. 
     The image processing device can be realized typically as an image decoding device for decoding an image. 
     According to an embodiment of the present disclosure, there is provided an image processing method including decoding an image from an encoded stream, performing horizontal filtering to apply a deblocking filter to a vertical block boundary within an image to be decoded, performing vertical filtering to apply a deblocking filter to a horizontal block boundary within an image to be decoded, and controlling the horizontal filtering and the vertical filtering so as to filter in parallel a plurality of vertical block boundaries included in a processing unit containing a plurality of coding units and filter in parallel a plurality of horizontal block boundaries included in the processing unit. 
     According to an embodiment of the present disclosure, there is provided an image processing device including a horizontal filtering section configured to apply a deblocking filter to a vertical block boundary within an image to be locally decoded when encoding an image to be encoded, a vertical filtering section configured to apply a deblocking filter to a horizontal block boundary within the image, a control section configured to cause the horizontal filtering section to filter in parallel a plurality of vertical block boundaries included in a processing unit containing a plurality of coding units and cause the vertical filtering section to filter in parallel a plurality of horizontal block boundaries included in the processing unit, and an encoding section configured to encode the image to be encoded using an image filtered by the horizontal filtering section and the vertical filtering section. 
     The image processing device can be realized typically as an image encoding device for encoding an image. 
     According to an embodiment of the present disclosure, there is provided an image processing method including performing horizontal filtering to apply a deblocking filter to a vertical block boundary within an image to be locally decoded when encoding an image to be encoded, performing vertical filtering to apply a deblocking filter to a horizontal block boundary within the image, controlling the horizontal filtering and the vertical filtering so as to filter in parallel a plurality of vertical block boundaries included in a processing unit containing a plurality of coding units and filter in parallel a plurality of horizontal block boundaries included in the processing unit, and encoding the image to be encoded using an image filtered by the horizontal filtering and the vertical filtering. 
     Advantageous Effects of Invention 
     As described above, the image processing device and the image processing method according to the present disclosure further improves parallel processing when a deblocking filter is applied. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a block diagram showing an example of a configuration of an image encoding device according to an embodiment. 
         FIG. 2  is a block diagram showing an example of a configuration of an image decoding device according to an embodiment. 
         FIG. 3  is an explanatory diagram showing an example of neighboring pixels around a boundary. 
         FIG. 4  is an explanatory diagram illustrating reference pixels during filtering need determination processes according to an existing technique. 
         FIG. 5  is an explanatory diagram illustrating pixels updated by filtering processes. 
         FIG. 6  is an explanatory diagram illustrating identification of edges for description of the embodiment. 
         FIG. 7  is an explanatory diagram illustrating a parallel process according to an existing technique. 
         FIG. 8  is a first explanatory diagram illustrating dependency between processes according to an existing technique. 
         FIG. 9  is a second explanatory diagram illustrating dependency between processes according to an existing technique. 
         FIG. 10  is an explanatory diagram illustrating a sequence of processes according to an existing technique. 
         FIG. 11  is an explanatory diagram illustrating a sequence of processes according to a first working example. 
         FIG. 12  is a block diagram illustrating a detailed configuration of a deblocking filter according to the first embodiment. 
         FIG. 13  is a block diagram illustrating a detailed configuration of a determination section. 
         FIG. 14  is an explanatory diagram illustrating neighboring blocks around a slice boundary. 
         FIG. 15  is an explanatory diagram illustrating a first example of a sequence of processes for each slice. 
         FIG. 16  is an explanatory diagram illustrating a second example of a sequence of processes for each slice. 
         FIG. 17  is an explanatory diagram illustrating first and second examples of a determination technique provided by a modification. 
         FIG. 18  is an explanatory diagram illustrating third and fourth examples of a determination technique provided by a modification. 
         FIG. 19  is an explanatory diagram illustrating fifth and sixth examples of a determination technique provided by a modification. 
         FIG. 20  is a flowchart illustrating a process flow for the deblocking filter according to the first working example. 
         FIG. 21  is a flowchart illustrating a flow of a filtering need determination process. 
         FIG. 22  is an explanatory diagram illustrating a sequence of processes according to a second working example. 
         FIG. 23  is a block diagram illustrating a detailed configuration of the deblocking filter according to the second working example. 
         FIG. 24  is a flowchart illustrating a process flow for the deblocking filter according to the second working example. 
         FIG. 25  is an explanatory diagram illustrating a process sequence for each LCU. 
         FIG. 26  is a flowchart illustrating a process flow for each LCU. 
         FIG. 27  is an explanatory diagram illustrating an overview of a third working example. 
         FIG. 28  is a block diagram illustrating a detailed configuration of a deblocking filter according to the third working example. 
         FIG. 29  is an explanatory diagram illustrating determination of a weight for weighted average. 
         FIG. 30  is an explanatory diagram illustrating an example of a weight for weighted average. 
         FIG. 31  is an explanatory diagram illustrating an output pixel value from a calculation section according to the third working example. 
         FIG. 32  is an explanatory diagram illustrating a first example of process sequence for comparison. 
         FIG. 33  is an explanatory diagram illustrating a first example of process sequence provided by the third working example. 
         FIG. 34  is an explanatory diagram illustrating a second example of process sequence for comparison. 
         FIG. 35  is an explanatory diagram illustrating a second example of process sequence provided by the third working example. 
         FIG. 36  is a flowchart illustrating a first example of a process flow for the deblocking filter according to the third working example. 
         FIG. 37  is a flowchart illustrating a flow of a pixel value calculation process shown in  FIG. 36 . 
         FIG. 38  is an explanatory diagram illustrating multiview codec. 
         FIG. 39  is an explanatory diagram illustrating an image encoding process according to an embodiment applied to multiview codec. 
         FIG. 40  is an explanatory diagram illustrating an image decoding process according to an embodiment applied to multiview codec. 
         FIG. 41  is an explanatory diagram illustrating scalable codec. 
         FIG. 42  is an explanatory diagram illustrating an image encoding process according to an embodiment applied to scalable codec. 
         FIG. 43  is an explanatory diagram illustrating an image decoding process according to an embodiment applied to scalable codec. 
         FIG. 44  is a block diagram illustrating a schematic configuration of a television apparatus. 
         FIG. 45  is a block diagram illustrating a schematic configuration of a mobile phone. 
         FIG. 46  is a block diagram illustrating a schematic configuration of a recording/reproduction device. 
         FIG. 47  is a block diagram illustrating a schematic configuration of an image capturing device. 
     
    
    
     DESCRIPTION OF EMBODIMENT 
     Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the appended drawings. Note that, in this specification and the drawings, elements that have substantially the same function and structure are denoted with the same reference signs, and repeated explanation is omitted. 
     Description of Embodiment will be described in the following sequence. 
     1. Apparatus Overview
         1-1. Image Encoding Device   1-2. Image Decoding Device       

     2. Existing Technique
         2-1. Basic Configuration of Deblocking Filter   2-2. Dependency Between Processes According to an Existing Technique       

     3. First Working Example
         3-1. Deblocking Filter Configuration Example   3-2. Determination Condition Modifications   3-3. Process Flow       

     4. Second Working Example
         4-1. Deblocking Filter Configuration Example   4-2. Process Flow   4-3. Process Example for Each LCU       

     5. Third Working Example
         5-1. Overview   5-2. Deblocking Filter Configuration Example   5-3. Process Sequence Example   5-4. Process Flow       

     6. Application to Various Codecs
         6-1. Multiview Codec   6-2. Scalable Codec       

     7. Example Applications 
     8. Summing-up 
     1. Apparatus Overview 
     With reference to  FIGS. 1 and 2 , the following describes an overview of an apparatus to which the technology disclosed in this specification is applicable. The technology disclosed in this specification is applicable to an image encoding device and an image decoding device, for example. 
     1-1. Image Encoding Device 
       FIG. 1  is a block diagram showing an example of a configuration of an image encoding device  10  according to an embodiment. Referring to  FIG. 1 , the image encoding device  10  includes an A/D (Analogue to Digital) conversion section  11 , a reordering buffer  12 , a subtraction section  13 , an orthogonal transform section  14 , a quantization section  15 , a lossless encoding section  16 , an accumulation buffer  17 , a rate control section  18 , an inverse quantization section  21 , an inverse orthogonal transform section  22 , an addition section  23 , a deblocking filter  24   a , a frame memory  25 , a selector  26 , an intra prediction section  30 , a motion estimation section  40 , and a mode selection section  50 . 
     The A/D conversion section  11  converts an image signal input in an analogue format into image data in a digital format, and outputs a series of digital image data to the reordering buffer  12 . 
     The reordering buffer  12  reorders the images included in the series of image data input from the A/D conversion section  11 . After reordering the images according to the a GOP (Group of Pictures) structure according to the encoding process, the reordering buffer  12  outputs the image data which has been reordered to the subtraction section  13 , the intra prediction section  30 , and the motion estimation section  40 . 
     The image data input from the reordering buffer  12  and predicted image data selected by the mode selection section  50  described later are supplied to the subtraction section  13 . The subtraction section  13  calculates predicted error data which is a difference between the image data input from the reordering buffer  12  and the predicted image data input from the mode selection section  50 , and outputs the calculated predicted error data to the orthogonal transform section  14 . 
     The orthogonal transform section  14  performs orthogonal transform on the predicted error data input from the subtraction section  13 . The orthogonal transform to be performed by the orthogonal transform section  14  may be discrete cosine transform (DCT) or Karhunen-Loeve transform, for example. The orthogonal transform section  14  outputs transform coefficient data acquired by the orthogonal transform process to the quantization section  15 . 
     The transform coefficient data input from the orthogonal transform section  14  and a rate control signal from the rate control section  18  described later are supplied to the quantization section  15 . The quantization section  15  quantizes the transform coefficient data, and outputs the transform coefficient data which has been quantized (hereinafter, referred to as quantized data) to the lossless encoding section  16  and the inverse quantization section  21 . Also, the quantization section  15  switches a quantization parameter (a quantization scale) based on the rate control signal from the rate control section  18  to thereby change the bit rate of the quantized data to be input to the lossless encoding section  16 . 
     The quantized data input from the quantization section  15  and information described later about intra prediction or inter prediction generated by the intra prediction section  30  or the motion estimation section  40  and selected by the mode selection section  50  are supplied to the lossless encoding section  16 . The information about intra prediction may include prediction mode information indicating an optimal intra prediction mode for each block, for example. Also, the information about inter prediction may include prediction mode information for prediction of a motion vector for each block, difference motion vector information, reference image information, and the like, for example. 
     The lossless encoding section  16  generates an encoded stream by performing a lossless encoding process on the quantized data. The lossless encoding by the lossless encoding section  16  may be variable-length coding or arithmetic coding, for example. Furthermore, the lossless encoding section  16  multiplexes the information about intra prediction or the information about inter prediction mentioned above to the header of the encoded stream (for example, a block header, a slice header or the like). Then, the lossless encoding section  16  outputs the generated encoded stream to the accumulation buffer  17 . 
     The accumulation buffer  17  temporarily stores the encoded stream input from the lossless encoding section  16  using a storage medium, such as a semiconductor memory. Then, the accumulation buffer  17  outputs the accumulated encoded stream at a rate according to the band of a transmission line (or an output line from the image encoding device  10 ). 
     The rate control section  18  monitors the free space of the accumulation buffer  17 . Then, the rate control section  18  generates a rate control signal according to the free space on the accumulation buffer  17 , and outputs the generated rate control signal to the quantization section  15 . For example, when there is not much free space on the accumulation buffer  17 , the rate control section  18  generates a rate control signal for lowering the bit rate of the quantized data. Also, for example, when the free space on the accumulation buffer  17  is sufficiently large, the rate control section  18  generates a rate control signal for increasing the bit rate of the quantized data. 
     The inverse quantization section  21  performs an inverse quantization process on the quantized data input from the quantization section  15 . Then, the inverse quantization section  21  outputs transform coefficient data acquired by the inverse quantization process to the inverse orthogonal transform section  22 . 
     The inverse orthogonal transform section  22  performs an inverse orthogonal transform process on the transform coefficient data input from the inverse quantization section  21  to thereby restore the predicted error data. Then, the inverse orthogonal transform section  22  outputs the restored predicted error data to the addition section  23 . 
     The addition section  23  adds the restored predicted error data input from the inverse orthogonal transform section  22  and the predicted image data input from the mode selection section  50  to thereby generate decoded image data. Then, the addition section  23  outputs the generated decoded image data to the deblocking filter  24   a  and the frame memory  25 . 
     A deblocking filter  24   a  performs filtering processes to decrease block distortion that occurs during image encoding. For example, the deblocking filter  24   a  determines necessity of filtering for each block boundary of decoded image data supplied from an addition section  23  and applies the deblocking filter to a boundary that is determined to require the filter. The deblocking filter  24   a  is also supplied with information used for the determination of filtering necessity (e.g., mode information, transform coefficient information, and motion vector information) as well as decoded image data from the addition section  23 . After the filtering, the block distortion is eliminated from the decoded image data and the deblocking filter  24   a  outputs the decoded image data to frame memory  25 . The process for the deblocking filter  24   a  will be described in detail later. 
     The frame memory  25  stores, using a storage medium, the decoded image data input from the addition section  23  and the decoded image data after filtering input from the deblocking filter  24   a.    
     The selector  26  reads, from the frame memory  25 , the decoded image data before filtering that is to be used for the intra prediction, and supplies the decoded image data which has been read to the intra prediction section  30  as reference image data. Also, the selector  26  reads, from the frame memory  25 , the decoded image data after filtering to be used for the inter prediction, and supplies the decoded image data which has been read to the motion estimation section  40  as reference image data. 
     The intra prediction section  30  performs an intra prediction process in each intra prediction mode, based on the image data to be encoded that is input from the reordering buffer  12  and the decoded image data supplied via the selector  26 . For example, the intra prediction section  30  evaluates the prediction result of each intra prediction mode using a predetermined cost function. Then, the intra prediction section  30  selects an intra prediction mode by which the cost function value is the smallest, that is, an intra prediction mode by which the compression ratio is the highest, as the optimal intra prediction mode. Furthermore, the intra prediction section  30  outputs, to the mode selection section  50 , prediction mode information indicating the optimal intra prediction mode, the predicted image data, and the information about intra prediction such as the cost function value. 
     A motion estimation section  40  performs an inter prediction process (prediction process between frames) based on image data for encoding supplied from a reordering buffer  12  and decoded image data supplied via a selector  26 . For example, the motion estimation section  40  evaluates the prediction result of each prediction mode using a predetermined cost function. Then, the motion estimation section  40  selects an optimal prediction mode, namely, a prediction mode that minimizes the cost function value or maximizes the compression ratio. The motion estimation section  40  generates predicted image data according to the optimal prediction mode. The motion estimation section  40  outputs information about the inter prediction such as prediction mode information indicating the optimal intra prediction mode, the predicted image data, and the cost function value to a mode selection section  50 . 
     The mode selection section  50  compares the cost function value related to the intra prediction input from the intra prediction section  30  and the cost function value related to the inter prediction input from the motion estimation section  40 . Then, the mode selection section  50  selects a prediction method with a smaller cost function value, from the intra prediction and the inter prediction. In the case of selecting the intra prediction, the mode selection section  50  outputs the information about intra prediction to the lossless encoding section  16 , and also, outputs the predicted image data to the subtraction section  13  and the addition section  23 . Also, in the case of selecting the inter prediction, the mode selection section  50  outputs the information about inter prediction described above to the lossless encoding section  16 , and also, outputs the predicted image data to the subtraction section  13  and the addition section  23 . 
     1-2. Image Decoding Device 
       FIG. 2  is a block diagram showing an example of a configuration of an image decoding device  60  according to an embodiment. With reference to  FIG. 2 , the image decoding device  60  includes an accumulation buffer  61 , a lossless decoding section  62 , an inverse quantization section  63 , an inverse orthogonal transform section  64 , an addition section  65 , a deblocking filter  24   b , a reordering buffer  67 , a D/A (Digital to Analogue) conversion section  68 , a frame memory  69 , selectors  70  and  71 , an intra prediction section  80 , and a motion compensation section  90 . 
     The accumulation buffer  61  temporarily stores an encoded stream input via a transmission line using a storage medium. 
     The lossless decoding section  62  decodes an encoded stream input from the accumulation buffer  61  according to the encoding method used at the time of encoding. Also, the lossless decoding section  62  decodes information multiplexed to the header region of the encoded stream. Information that is multiplexed to the header region of the encoded stream may include information about intra prediction and information about inter prediction in the block header, for example. The lossless decoding section  62  outputs the information about intra prediction to the intra prediction section  80 . Also, the lossless decoding section  62  outputs the information about inter prediction to the motion compensation section  90 . 
     The inverse quantization section  63  inversely quantizes quantized data which has been decoded by the lossless decoding section  62 . The inverse orthogonal transform section  64  generates predicted error data by performing inverse orthogonal transformation on transform coefficient data input from the inverse quantization section  63  according to the orthogonal transformation method used at the time of encoding. Then, the inverse orthogonal transform section  64  outputs the generated predicted error data to the addition section  65 . 
     The addition section  65  adds the predicted error data input from the inverse orthogonal transform section  64  and predicted image data input from the selector  71  to thereby generate decoded image data. Then, the addition section  65  outputs the generated decoded image data to the deblocking filter  24   b  and the frame memory  69 . 
     The deblocking filter  24   b  performs filtering processes to decrease block distortion appearing on a decoded image. The deblocking filter  24   b  determines the necessity of filtering at each block boundary for decoded image data input from the addition section  65 , for example, and applies the deblocking filter to a boundary that is determined to require the filter. The deblocking filter  24   b  is also supplied with information used for the determination of filtering necessity as well as decoded image data from the addition section  65 . After the filtering, the block distortion is eliminated from the decoded image data and the deblocking filter  24   b  outputs the decoded image data to the reordering buffer  67  and the frame memory  69 . The process for the deblocking filter  24   b  will be described in detail later. 
     The reordering buffer  67  generates a series of image data in a time sequence by reordering images input from the deblocking filter  24   b . Then, the reordering buffer  67  outputs the generated image data to the D/A conversion section  68 . 
     The D/A conversion section  68  converts the image data in a digital format input from the reordering buffer  67  into an image signal in an analogue format. Then, the D/A conversion section  68  causes an image to be displayed by outputting the analogue image signal to a display (not shown) connected to the image decoding device  60 , for example. 
     The frame memory  69  uses a storage medium to store the decoded image data input from the addition section  65  before filtering and the decoded image data input from the deblocking filter  24   b  after filtering. 
     The selector  70  switches the output destination of the image data from the frame memory  69  between the intra prediction section  80  and the motion compensation section  90  for each block in the image according to mode information acquired by the lossless decoding section  62 . For example, in the case the intra prediction mode is specified, the selector  70  outputs the decoded image data before filtering that is supplied from the frame memory  69  to the intra prediction section  80  as reference image data. Also, in the case the inter prediction mode is specified, the selector  70  outputs the decoded image data after filtering that is supplied from the frame memory  69  to the motion compensation section  90  as the reference image data. 
     The selector  71  switches the output source of predicted image data to be supplied to the addition section  65  between the intra prediction section  80  and the motion compensation section  90  for each block in the image according to the mode information acquired by the lossless decoding section  62 . For example, in the case the intra prediction mode is specified, the selector  71  supplies to the addition section  65  the predicted image data output from the intra prediction section  80 . In the case the inter prediction mode is specified, the selector  71  supplies to the addition section  65  the predicted image data output from the motion compensation section  90 . 
     The intra prediction section  80  performs in-screen prediction of a pixel value based on the information about intra prediction input from the lossless decoding section  62  and the reference image data from the frame memory  69 , and generates predicted image data. Then, the intra prediction section  80  outputs the generated predicted image data to the selector  71 . 
     The motion compensation section  90  performs a motion compensation process based on the information about inter prediction input from the lossless decoding section  62  and the reference image data from the frame memory  69 , and generates predicted image data. Then, the motion compensation section  90  outputs the generated predicted image data to the selector  71 . 
     2. Existing Technique 
     2-1. Basic Configuration of Deblocking Filter 
     Generally, processes using the deblocking filter in an existing image encoding scheme such as H.264/AVC or HEVC include two types of processes, namely, filtering need determination processes and filtering processes. The following describes these two processes in HEVC, for example. 
     (1) Filtering Need Determination Processes 
     The filtering need determination processes determine whether the deblocking filter needs to be applied to each boundary of blocks within an input image. Block boundaries include a vertical boundary between blocks horizontally adjacent to each other and a horizontal boundary between blocks vertically adjacent to each other. JCTVC-A119 uses a block size of 8×8 pixels as a minimum processing unit. For example, a macro block of 16×16 pixels includes four blocks of 8×8 pixels. The process is applied to one (left) vertical boundary and one (top) horizontal boundary for each block, namely, four boundaries plus four boundaries equal to eight boundaries in total. The specification assumes that the macro block as a technical term includes an coding unit (CU) in the context of HEVC. 
       FIG. 3  is an explanatory diagram showing an example of pixels in two blocks (neighboring blocks) Ba and Bb adjacent to each other around a boundary. The following describes the vertical boundary as an example and the description is obviously applicable to the horizontal boundary. The example in  FIG. 3  uses symbol p ij  to represent a pixel in block Ba. In this symbol, i denotes a column index and j denotes a row index. The column index i is numbered as 0, 1, 2, and 3 in order (from right to left) from the column nearest to the vertical boundary. The row index j is numbered as 0, 1, 2, . . . , 7 from the top to the bottom. The left half of block Ba is omitted from the drawing. Symbol q kj  is used to represent a pixel in block Bb. In this symbol, k denotes a column index and j denotes a row index. The column index k is numbered as 0, 1, 2, and 3 in order (from left to right) from the column nearest to the vertical boundary. The right half of block Bb is omitted from the drawing. 
     The following conditions can be used to determine the necessity of applying the deblocking filter to the vertical boundary between blocks Ba and Bb shown in  FIG. 3 . 
     Determination condition of luma component (Luma) . . . . The deblocking filter is applied if conditions A and B are both true. 
     Condition A:
 
Block  Ba  or  Bb  enters the intra prediction mode;  (A1)
 
Block  Ba  or  Bb  has a nonzero orthogonal transform coefficient; or  (A2)
 
| MVAx−MVBx |≥4 or | MVAy−MVBy |≥4  (A3)
 
Condition B:
 
| p   22 −2 p   12   +p   02   |+|q   22 −2 q   12   +q   02   |+|p   25 −2 p   15   +p   05   |+|q   25 −2 q   15   +q   05 |&lt;β
 
     Condition A3 assumes a motion vector for block Ba to be (MVAx,MVAy) and a motion vector for block Bb to be (MVBx,MVBy) according to the Qpel (¼ pixel) accuracy. Condition B uses β as an edge determination threshold value. An initial value of β is given according to a quantization parameter. The value for β is user-specifiable using a parameter within the slice header. 
     Determination condition of chroma component (Chroma) . . . . The deblocking filter is applied if condition A1 is true. 
     Condition A1: Block Ba or Bb enters the intra prediction mode. 
     As indicated by broken-line frames L3 and L6 in  FIG. 4 , the filtering need determination processes on general vertical boundaries (particularly under determination condition B of luma component) reference pixels on the third and sixth rows (assuming the top row to be the first) in each block. Similarly, the filtering need determination processes on horizontal boundaries reference pixels (not shown in  FIG. 4 ) on the third and sixth columns in each block. The above-described determination conditions are used to determine that the deblocking filter needs to be applied to a boundary on which the filtering processes described below are performed. 
     (2) Filtering Processes 
     If it is determined that the deblocking filter needs to be applied to a boundary, the filtering processes are performed on pixels to the right and the left of the vertical boundary and on pixels above and below the horizontal boundary. On luma components, the filter strength is switched between a strong filter and a weak filter according to pixel values. 
     Filtering Luma Components 
     Selecting the strength . . . . The filter strength is selected for each row or column. The strong filter is selected if all of the following conditions C1 through C3 are satisfied. The weak filter is selected if even any one of the conditions is not satisfied.
 
 d &lt;(β&gt;&gt;2)  (C1)
 
(| p   31   −p   0j   |+|q   0j   −q   3j |)&lt;(β&gt;&gt;3)  (C2)
 
| p   0j   −q   0j |&lt;((5 t   C +1)&gt;&gt;1)  (C3)
 
     where j denotes a row index for the vertical boundary or a column index for the horizontal boundary. d=|p 22 −2p 12 +p 02 |+|q 22 −2q 12 +q 02 |+|p 25 −2p 15 +p 05 |+|q 25 −2q 15 +q 05 | 
     Weak Filtering
 
Δ=Clip(− t   C   ,t   C ,(13( q   0j   −p   0j )+4( q   1j   −p   1j )−5( q   2j   −p   2j )+16)&gt;&gt;5))
 
 p   0j =Clip 0-255 ( p   0j +Δ)
 
 q   0j =Clip 0-255 ( q   0j −Δ)
 
 p   1j =Clip 0-255 ( p   1j +Δ/2)
 
 q   1j =Clip 0-255 ( q   1j −Δ/2)
 
     Strong Filtering
 
 p   0j =Clip 0-255 (( p   2j +2 p   1j +2 p   0j +2 q   0j   +q   1j +4)&gt;&gt;3)
 
 q   0j =Clip 0-255 (( p   1j +2 p   0j +2 q   0j +2 q   1j   +q   2j +4)&gt;&gt;3)
 
 p   1j =Clip 0-255 (( p   2j   +p   1j   +p   0j   +q   0j +2)&gt;&gt;2)
 
 q   1j =Clip 0-255 (( p   0j   +q   0j   +q   1j   +q   2j +2)&gt;&gt;2)
 
 p   2j =Clip 0-255 ((2 p   3j +3 p   2j   +p   1j   +p   0j   +q   0j +4)&gt;&gt;3)
 
 q   2j =Clip 0-255 (( p   0j   +q   0j   +q   1j +3 q   2j +2 q   3j +4)&gt;&gt;3)
 
     where Clip(a,b,c) denotes a process to clip value c within the range of a≤c≤b and Clip 0-255 (c) denotes a process to clip value c within the range of 0≤c≤255. 
     Filtering Chroma Components
 
Δ=Clip(− t   C   ,t   C ,(((( q   0j   −p   0j )&lt;&lt;2)+ p   1j   −q   1j +4)&gt;&gt;3))
 
 p   0j =Clip 0-255 ( p   0j +Δ)
 
 q   0j =Clip 0-255 ( q   0j −Δ)
 
     As indicated by broken-line frames C6 through C8 and C1 through C3 in  FIG. 5 , the filtering processes (particularly strong filtering on luma components) on general vertical boundaries update pixel values on the first through third and sixth through eighth columns in each block. Similarly, the filtering processes on horizontal boundaries update pixel values on the first through third and sixth through eighth rows in each block. 
     2-2. Dependency Between Processes According to an Existing Technique 
     For the purpose of description, as shown in  FIG. 6 , macro block MBx (MB0, MB1 . . . ) each having the size of 16×16 pixels includes the top left vertical boundary represented as Vx,0, the top center vertical boundary represented as Vx,1, the bottom left vertical boundary represented as Vx,2, the bottom center vertical boundary represented as Vx,3, the top left horizontal boundary represented as Hx,0, the top right horizontal boundary represented as Hx,1, the left center horizontal boundary represented as Hx,2, and the right center horizontal boundary represented as Hx,3. Concerning boundary Z, for example, the filtering need determination process is represented as J Z  and the filtering process is represented as F Z . 
     The above-described existing technique causes no dependency between processes on boundaries in the same direction within one macro block. Therefore, the technique can perform parallel filtering on vertical boundaries and horizontal boundaries within one macro block, for example. As an example,  FIG. 7  makes it clear that there is no dependency among four filtering processes F V0,0 , F V0,1 , F V0,2 , and F V0,3  (no pixel updated redundantly) within macro block MB0 and the filtering processes can be performed in parallel. 
     However, the above-described existing technique leaves the dependency between the filtering processes on vertical boundaries and the filtering need determination processes on horizontal boundaries. The existing technique also leaves the dependency between the filtering processes on horizontal boundaries and the filtering need determination processes on vertical boundaries. If a vertical boundary is processed prior to a horizontal boundary, for example, the filtering need determination processes need to be performed on horizontal boundaries within a given macro block after termination of the filtering processes on vertical boundaries. As an example,  FIG. 8  shows that, within macro block MB0, filtering need determination process J H0,0  depends on results of filtering processes F V0,0  and F V0,1  and filtering need determination process J H0,1  depends on a result of filtering processes F V0,1 . Similarly, the filtering need determination processes need to be performed on vertical boundaries within a given macro block after termination of the filtering process on the horizontal boundary for the adjacent macro block. As an example,  FIG. 9  shows that filtering need determination process J V1,0  for macro block MB1 depends on results of filtering processes F H0,1  and F H0,3  for macro block MB0 and filtering need determination process J V1,2  for macro block MB1 depends on a result of filtering process F H0,3  for macro block MB0. 
     The existing technique involves the dependency between processes and therefore provides parallel processing of the deblocking filter to a very limited extent even if the technique proposed in JCTVC-A119 is used. 
       FIG. 10  is an explanatory diagram illustrating a sequence of deblocking filter processes according to an existing technique. The example assumes that the deblocking filter is supplied with an image having the size of 32×32 pixels. The input image includes four macro blocks MB0 through MB3 each having the size of 16×16 pixels. 
     In  FIG. 10 , each broken-line frame represents a process to be performed in parallel. For example, the first step performs, in parallel, filtering need determination processes J V0,0 , J V0,2  and J V0,3  on four vertical boundaries in macro block MB0. The second step performs, in parallel, filtering processes F V0,0 , F V0,1 , F V0,2  and F V0,3  on four vertical boundaries in macro block MB0. After termination of the second step, the third step performs, in parallel, filtering need determination processes J H0,0 , J H0,1 , J H0,2  and J H0,3  on four horizontal boundaries in macro block MB0. The third step uses a pixel value after the filtering process on the vertical boundary at the second step for the filtering need determination process on the horizontal boundary. The fourth step performs, in parallel, filtering processes F H0,0 , F H0,1 , F H0,2  and F H0,3  on four horizontal boundaries in macro block MB0. After termination of the fourth step, processes (fifth to eighth steps) for macro block MB1 are performed successively. The fifth step uses a pixel value after the filtering process on the horizontal boundary of the macro block MB0 at the fourth step for the filtering need determination process on the vertical boundary of the macro block MB1. After termination of the process on the macro block MB1, processes (ninth to twelfth steps) for macro block MB2 are performed successively. After termination of the process on the macro block MB2, processes (thirteenth to sixteenth steps) for macro block MB3 are performed successively. 
     Such parallel processing within the limited extent cannot satisfactorily solve the problem of delay or data rate degradation due to a large processing amount when the deblocking filter is applied. Three working examples described below further improve parallel processing when the definition is applied. 
     3. First Working Example 
     3-1. Deblocking Filter Configuration Example 
     The following describes example configurations of the deblocking filter  24   a  for the image encoding device  10  shown in  FIG. 1  and the deblocking filter  24   b  for the image decoding device  60  shown in  FIG. 2  according to the first working example. The configurations of the deblocking filter  24   a  and the deblocking filter  24   b  may be common to each other. In the following description, the deblocking filter  24   a  and the deblocking filter  24   b  are generically referred to as a deblocking filter  24  when there is no need for distinction between them. 
     (1) Dependency Between New Processes 
     According to the working example, processes using the deblocking filter  24  also include two types of processes, namely, a filtering need determination process and a filtering process. The deblocking filter  24  uses pixel values for an image input to the deblocking filter for the determination across a plurality of macro blocks in the filtering need determination process on one of the vertical boundary and the horizontal boundary. If the vertical boundary is processed prior to the horizontal boundary, for example, the deblocking filter  24  can perform the filtering need determination process on the vertical boundary for a given block without waiting for the filtering process on the horizontal boundary for the neighboring blocks. If the horizontal boundary is processed prior to the vertical boundary, for example, the deblocking filter  24  can perform the filtering need determination process on the horizontal boundary for a given block without waiting for the filtering process on the horizontal boundary for the neighboring blocks. The result is to relieve the dependency of processes between macro blocks. 
     Relieving the dependency of processes between macro blocks can parallelize processes between the plurality of macro blocks within an image. For example, this enables to perform filtering need determination processes in parallel on vertical boundaries for all blocks within an input image. This also enables to perform filtering need determination processes in parallel on horizontal boundaries for all blocks within an input image. 
       FIG. 11  is an explanatory diagram illustrating a sequence of processes available for the working example. The example also assumes that the deblocking filter is supplied with an image having the size of 32×32 pixels. The input image includes four macro blocks MB0 through MB3 each having the size of 16×16 pixels. 
     In  FIG. 11 , each broken-line frame represents a process to be performed in parallel. While the example in  FIG. 10  requires 16 process steps for a sequence of processes, the example in  FIG. 11  aggregates the same number of processes into four process steps. The first step performs, in parallel, filtering need determination processes J V0,0  through J V3,3  and J H0,0  through J H3,3  on all vertical boundaries and all horizontal boundaries of all macro blocks MB0 through MB3. The second step performs, in parallel, filtering processes F V0,0  through F V3,3  on 16 vertical boundaries of all macro blocks MB0 through MB3. The third step performs, in parallel, filtering need determination processes F H0,0  through F H3,3  on all horizontal boundaries of all macro blocks MB0 through MB3. The fourth step performs, in parallel, filtering processes F H0,0  through F H3,3  on 16 horizontal boundaries of all macro blocks MB0 through MB3. The third and fourth steps may precede the first and second steps if the horizontal boundary is processed prior to the vertical boundary. 
       FIG. 11  provides the example of maximizing the parallelism (processes performed in parallel) by parallelizing processes over all macro blocks in an image. While not limited to the example, processes may be parallelized over some macro blocks instead of all macro blocks in an image. 
     (2) Detailed Configuration of Deblocking Filter 
       FIG. 12  is a block diagram illustrating a detailed configuration of the deblocking filter  24  according to the first working example for performing the above-described parallel processes. With reference to  FIG. 12 , the deblocking filter  24  includes a vertical determination block  110 , a horizontal determination block  114 , a horizontal filtering block  130 , a vertical filtering block  140 , and a parallelization control section  150 . 
     (2-1) Vertical Determination Block 
     The vertical determination block  110  includes a plurality of vertical boundary determination sections  112 - 1  through  112 - n . Each vertical boundary determination section  112  is supplied with images input to the deblocking filter  24  and determination information used to determine whether filtering is needed. 
     The vertical boundary determination sections  112 - 1  through  112 - n  determine whether to apply the deblocking filter to vertical boundaries using pixel values for an image input to the deblocking filter  24  across the plurality of macro blocks within the image. Each vertical boundary determination section  112  supplies the horizontal filtering block  130  with information indicating a determination result about each vertical boundary such as binary information indicating a determination result that value “1” forces application of the deblocking filter. 
     (2-2) Horizontal Filtering Block 
     The horizontal filtering block  130  includes a plurality of horizontal filtering sections  132 - 1  through  132 - n . Each horizontal filtering section  132  is supplied with an input image and the determination result about each vertical boundary from the vertical determination block  110 . 
     A determination result from the corresponding vertical boundary determination section  112  may indicate that the filter needs to be applied. In such a case, each horizontal filtering section  132  applies the deblocking filter for vertical boundary to right and left elements with reference to the vertical boundary. Each horizontal filtering section  132  supplies the horizontal determination block  114  and the vertical filtering block  140  with pixel values after filtering for filter-applied pixels and pixel values of the input image for the other pixels. 
     (2-3) Horizontal Determination Block 
     The horizontal determination block  114  includes a plurality of horizontal boundary determination sections  116 - 1  through  116 - n . Each horizontal boundary determination section  116  is supplied with pixel values after the filtering performed by the horizontal filtering block  130  and the determination information used to determine whether filtering is needed. 
     The horizontal boundary determination section  116 - 1  through  116 - n  determine whether to apply the deblocking filter to horizontal boundaries using pixel values after the filtering performed by the horizontal filtering block  130  across the plurality of macro blocks within the image. Each horizontal boundary determination section  116  supplies the vertical filtering block  140  with information indicating a determination result about each horizontal boundary. 
     (2-4) Vertical Filtering Block 
     The vertical filtering block  140  includes a plurality of vertical filtering sections  142 - 1  through  142 - n . Each vertical filtering section  142  is supplied with pixel values after the filtering performed by the horizontal filtering block  130  and a determination result about each horizontal boundary from the horizontal determination block  114 . 
     A determination result from the corresponding horizontal boundary determination section  116  may indicate that the filter needs to be applied. In such a case, each vertical filtering section  142  applies the deblocking filter for horizontal boundary to top and bottom elements with reference to the horizontal boundary. Each vertical filtering section  142  supplies filter-applied pixels with pixel values after the filtering and the other pixels with pixel values supplied from the horizontal filtering block  130 . An output from each vertical filtering section  142  may configure an output image from the deblocking filter  24 . 
     (3) More Detailed Configuration of the Determination Section 
       FIG. 13  is a block diagram illustrating a detailed configuration of each of the vertical boundary determination sections  112  and the horizontal boundary determination sections  116 . With reference to  FIG. 13 , each determination section includes a tap constitution section  121 , a calculation section  122 , a threshold comparison section  123 , a distortion evaluation section  124 , and a filtering determination section  125 . 
     The tap constitution section  121  acquires a reference pixel value from pixel values of two blocks neighboring across a focused boundary in the input image and constitutes a tap (a set of reference pixel values) for determining determination condition B for the above-described luma component. For example, a vertical boundary may be focused in the blocks each of which has the size of 8×8 pixels. In this case, the tap constitution section  121  constitutes a tap from pixel values belonging to the third and sixth rows of two blocks at the right and left. If a horizontal boundary is focused, the tap constitution section  121  constitutes a tap from pixel values belonging to the third and sixth columns of two blocks at the top and bottom. The calculation section  122  assigns the tap constituted by the tap constitution section  121  to the left-hand side of the determination expression in determination condition B and calculates an edge value to be compared with edge determination threshold value β. The threshold comparison section  123  compares the value calculated by the calculation section  122  with edge determination threshold value β and outputs a comparison result to the filtering determination section  125 . 
     The distortion evaluation section  124  evaluates determination condition A of the above-described luma component using mode information (MB mode), transform coefficient information, and motion vector information supplied as the determination information. The distortion evaluation section  124  outputs an evaluation result to the filtering determination section  125 . The distortion evaluation section  124  evaluates only determination condition A1 of a chroma component based on the mode information. 
     The filtering determination section  125  determines whether to apply the deblocking filter to a focused boundary based on a comparison result of determination condition B supplied from the threshold comparison section  123  and an evaluation result of determination condition A supplied from the distortion evaluation section  124 . The filtering determination section  125  outputs information indicating the determination result. 
     (4) Parallelization Control Section 
     The parallelization control section  150  shown in  FIG. 12  controls the parallelism of filtering need determination processes in the vertical determination block  110  and the horizontal determination block  114 , and the parallelism of filtering processes in the horizontal filtering block  130  and the vertical filtering block  140 . 
     For example, the parallelization control section  150  may control the parallelism of processes for each block based on an input image size. More specifically, the parallelization control section  150  increases the parallelism of processes for each block if the input image size is relatively large. This can adaptively prevent delay or data rate degradation due to a processing amount that increases according to image sizes. For example, the parallelization control section  150  may control the parallelism of processes for each block based on a sequence parameter set, a picture parameter set, or parameters contained in the slice header. This enables to flexibly configure the parallelism according to requirements of users who develop apparatuses. For example the parallelism may be configured according to restrictions on the installation environment such as the number of processor cores or the number of software threads. 
     The working example can parallelize processes between macro blocks. This signifies that any sequence of processes on blocks within an image has no effect on a finally output result. Accordingly, the parallelization control section  150  can control a sequence of filtering need determination processes in the vertical determination block  110  and the horizontal determination block  114 , and a sequence of filtering processes in the horizontal filtering block  130  and the vertical filtering block  140  on a block basis. 
     More specifically, the parallelization control section  150  may control a sequence of filtering processes according to the dependency of the filtering processes between macro blocks. According to an existing technique, for example, the dependency of processes between neighboring macro blocks around a slice boundary may delay parallel processes on each slice within an image. However, the parallelization control section  150  according to the working example can perform filtering processes on neighboring macro blocks around the slice boundary prior to the other macro blocks. 
     For example,  FIG. 14  illustrates eight macro blocks MB10 through MB13 and MB20 through MB23 around a slice boundary. Macro blocks MB10 through MB13 belong to slice SL1. Macro blocks MB20 through MB23 belong to slice SL2. Concerning these macro blocks, the filtering processes for horizontal boundaries on macro block MB20 in slice SL2 depend on the filtering processes for vertical boundaries on macro block MB12 in slice SL1. Similarly, the filtering processes for horizontal boundaries on macro block MB21 in slice SL2 depend on the filtering processes for vertical boundaries on macro block MB13 in slice SL1. 
     According to an example in  FIG. 15  under these conditions, the parallelization control section  150  performs filtering processes on the vertical boundaries of macro blocks MB12 and MB13 out of filtering processes for slice SL1 in preference to processes on the other boundaries. The result is to prevent a large delay from occurring in filtering processes on the horizontal boundaries of macro blocks MB20 and MB21 out of filtering processes for slice SL2. An example in  FIG. 16  initially performs filtering processes in parallel on vertical boundaries for all macro blocks included in slice SL1. Also in this case, no delay occurs in the filtering process on the horizontal boundaries of macro blocks MB20 and MB21 in slice SL2. 
     3-2. Determination Condition Modifications 
     As described above, each vertical boundary determination section  112  references pixels corresponding to the third and sixth rows in a block and determines for vertical boundaries of each block whether filtering is needed similarly to the existing technique as illustrated in  FIG. 4 . Likewise, each horizontal boundary determination section  116  references pixels corresponding to the third and sixth columns in a block and determines for horizontal boundaries of each block whether filtering is needed. In such a case, the configuration according to the working example can be easily embodied without changing determination conditions for the filtering need determination process provided for the existing apparatus. 
     Each vertical boundary determination section  112  and each horizontal boundary determination section  116  may perform the determination using determination conditions different from the existing technique. For example, each vertical boundary determination section  112  may reference pixels corresponding to three or more columns in a block. Each horizontal boundary determination section  116  may reference pixels corresponding to three or more columns in a block. In addition, each vertical boundary determination section  112  and each horizontal boundary determination section  116  may use determination condition expressions different from the existing technique. With reference to  FIGS. 17 through 19 , the following describes six examples of the determination technique according to the working example. 
     (1) First Example 
       FIG. 17  is an explanatory diagram illustrating first and second examples of the determination technique. In the first and second examples, the filtering need determination processes (particularly the determination using determination condition B for luma components) for vertical boundaries references pixels of all rows L1 through L8 from the first to the eighth in each block. The filtering need determination processes for horizontal boundaries also references pixels of all columns from the first to the eighth in each block. 
     The first example may define determination conditions for luma components as follows. 
     Determination condition of luma component (Luma) . . . . The deblocking filter is applied if conditions A and B are both true. 
     Condition A:
 
Block  Ba  or  Bb  enters the intra prediction mode;  (A1)
 
Block  Ba  or  Bb  has a nonzero orthogonal transform coefficient; or  (A2)
 
| MVAx−MVBx |≥4 or | MVAy−MVBy |≥4  (A3)
 
Condition B:
 
 iD   0   =|p   20 −2 p   10   +p   00   |+|q   20 −2 q   10   +q   00   |+|p   27 −2 p   17   +p   07   |+|q   27   −q   17   +q   07 |
 
 iD   1   =|p   21 −2 p   11   +p   01   |+|q   21 −2 q   11   +q   01   |+|p   26 −2 p   16   +p   06   |+|q   26 −2 q   16   +q   06 |
 
 iD   2   =|p   22 −2 p   12   +p   02   |+|q   22 −2 p   12   +q   02   |+|p   25 −2 p   15   +p   05   |+|q   25 −2 q   15   +q   05 |
 
 iD   3   =|p   23 −2 p   13   +p   03   |+|q   23 −2 q   13   +q   03   |+|p   24 −2 p   14   +p   04   |+|q   24 −2 q   14   +q   04 |
 
 iD   ave =( iD   0   +iD   1   +iD   2   +iD   3 )&gt;&gt;2
 
     Under this condition, iD ave &lt;β 
     The determination condition for chroma components may be equal to the above-described existing technique. A weighted average may be calculated to calculate average iD ave  for four determination parameters iD 0  through iD 3 . 
     (2) Second Example 
     The second example may define determination condition B for luma components as follows. 
     Condition B:
 
 iD   0   =|p   20 −2 p   10   +p   00   |+|q   20 −2 q   10   +q   00   |+|p   27 −2 p   17   +p   07   |+|q   27 −2 q   17   +q   07 |
 
 iD   1   =|p   21 −2 p   11   +p   01   |+|q   21 −2 q   11   +q   01   |+|p   26 −2 p   16   +p   06   |+|q   26 −2 q   16   +q   06 |
 
 iD   2   =|p   22 −2 p   12   +p   02   |+|q   22 +2 q   12   +q   02   |+|p   25 −2 p   15   +p   05   |+|q   25 −2 q   15   +q   05 |
 
 iD   3   =|p   23 −2 p   13   +p   03   |+|q   23 −2 q   13   +q   03   |+|p   24 −2 p   14   +p   04   |+|q   24 −2 q   14   +q   04 |
 
Under this condition,  iD   0 &lt;β and  iD   1 &lt;β and  iD   2 &lt;β and  iD   3 &lt;β
 
     An equation to calculate four determination parameters iD 0  through iD 3  is equal to that of the first example. An available condition is that not all of, but at least three, two, or one of four determination parameters iD 0  through iD 3  is smaller than edge determination threshold value β. 
     (3) Third Example 
       FIG. 18  is an explanatory diagram illustrating third and fourth examples of the determination technique. In the third and fourth examples, the filtering need determination processes (particularly the determination using determination condition B for luma components) for vertical boundaries references pixels of four rows L1, L3, L6, and L8 in each block. The filtering need determination processes for horizontal boundaries also references pixels of four columns in each block. 
     The third example may define determination conditions for luma components as follows. 
     Determination condition of luma component (Luma) . . . The deblocking filter is applied if conditions A and B are both true. 
     Condition A:
 
Block  Ba  or  Bb  enters the intra prediction mode;  (A1)
 
Block  Ba  or  Bb  has a nonzero orthogonal transform coefficient; or  (A2)
 
| MVAx−MVBx |≥4 or | MVAy−MVBy |≥4  (A3)
 
Condition B:
 
 iD   0   =|p   20 −2 p   10   +p   00   |+|q   20 −2 q   10   +q   00   |+|p   27 −2 p   17   +p   07   |+|q   27 −2 q   17   +q   07 |
 
 iD   2   =|p   22 −2 p   12   +p   02   |+|q   22 −2 q   12   +q   02   |+|p   25 −2 p   15   +p   05   |+|q   25 −2 q   15   +q   05 |
 
 iD   ave ( iD   0   +iD   2 )&gt;&gt;1
 
Under this condition,  iD   ave &lt;β
 
     The determination condition for chroma components may be equal to the above-described existing technique. A weighted average may be calculated to calculate average iD ave  for two determination parameters iD 0  and iD 2 . 
     (4) Fourth Example 
     The fourth example may define determination condition B for luma components as follows. 
     Condition B:
 
 iD   0   =|p   20 −2 p   10   +p   00   |+|q   20 −2 q   10   +q   00   |+|p   27 −2 p   17   +p   07   |+|q   27 −2 q   17   +q   07 |
 
 iD   2   =|p   22 −2 p   12   +p   02   |+|q   22 −2 q   12   +q   02   |+|p   25 −2 p   15   +p   05   |+|q   25 −2 q   15   +q   05 |
 
Under this condition,  iD   0 &lt;β and  iD   2 &lt;β
 
     An equation to calculate two determination parameters iD 0  and iD 2  is equal to that of the third example. An available condition is that not both of, but either of two determination parameters iD 0  and iD 2  is smaller than edge determination threshold value β. 
     While there has been described the example of referencing the first, third, sixth, and eighth rows (or columns) L1, L3, L6, and L8 in a block during the determination, the other combinations of rows or columns may be referenced. 
     (5) Fifth Example 
       FIG. 19  is an explanatory diagram illustrating fifth and sixth examples of the determination technique. In the fifth and sixth examples, the filtering need determination processes for vertical boundaries references pixels of four rows L1, L3, L5, and L7 in each block. The filtering need determination processes for horizontal boundaries also references pixels of four columns in each block. 
     The fifth example may define determination conditions for luma components as follows. 
     Determination condition of luma component (Luma) . . . The deblocking filter is applied if conditions A and B are both true. 
     Condition A:
 
Block  Ba  or  Bb  enters the intra prediction mode;  (A1)
 
Block  Ba  or  Bb  has a nonzero orthogonal transform coefficient; or  (A2)
 
| MVAx−MVBx |≥4 or | MVAy−MVBy |≥4  (A3)
 
Condition B:
 
 iD   0   =|p   20 −2 p   10   +p   00   |+|q   20 −2 q   10   +q   00   |+|p   26 −2 p   16   +p   06   |+|q   26 −2 q   16   +q   06 |
 
 iD   2   =|p   22 −2 p   12   +p   02   |+|q   22 −2 q   12   +q   02   |+|p   24 −2 p   14   +p   04   |+|q   24 −2 q   14   +q   04 |
 
 iD   ave ( iD   0   +iD   2 )&gt;&gt;1
 
Under this condition,  iD   ave &lt;β
 
     The determination condition for chroma components may be equal to the above-described existing technique. A weighted average may be calculated to calculate average iD ave  for two determination parameters iD 0  and iD 2 . 
     (6) Sixth Example 
     The sixth example may define determination condition B for luma components as follows. 
     Condition B:
 
 iD   0   =|p   20 −2 p   10   +p   00   |+|q   20 −2 q   10   +q   00   |+|p   26 −2 p   16   +p   06   |+|q   26 −2 q   16   +q   06 |
 
 iD   2   =p   22 −2 p   12   +p   02   |+|q   22 −2 q   12   +q   02   |+|p   24 −2 p   14   +p   04   |+|q   24 −2 q   14   +q   04 |
 
Under this condition,  iD   0 &lt;β and  iD   2 &lt;β
 
     An equation to calculate two determination parameters iD 0  and iD 2  is equal to that of the fifth example. An available condition is that not both of, but either of two determination parameters iD 0  and iD 2  is smaller than edge determination threshold value β. 
     Generally, increasing the number of rows and columns to be referenced for the determination improves the determination accuracy. Therefore, the first and second examples of referencing eight rows and columns can minimize a possibility of filtering a block originally not targeted for the deblocking filter to be applied and a possibility of not filtering a block originally targeted for the deblocking filter to be applied. The result is to improve the quality of an image to be encoded and decode. On the other hand, decreasing the number of rows and columns to be referenced for the determination can reduce processing costs. Since there is trade-off between the image quality and the processing cost, it may be advantageous to adaptively select the number of rows and columns to be referenced for the determination depending on the use of the apparatus or restrictions on the installation. It may be advantageous to adaptively select combinations of rows and columns to be referenced. 
     As described in the first, third, and fifth examples, average value iD ave  of determination parameters can be compared with edge determination threshold value β to appropriately perform the determination on a block basis without an excess effect of parameter variations for each row or column. 
     3-3. Process Flow 
     With reference to  FIGS. 20 and 21 , a process flow for the deblocking filter  24  will be described. 
       FIG. 20  is a flowchart illustrating a process flow for the deblocking filter  24  according to the first working example. With reference to  FIG. 20 , the vertical boundary determination sections  112 - 1  through  112 - n  determine in parallel for all vertical boundaries included in a plurality of macro blocks within an input image whether filtering is needed (step S 110 ). The horizontal filtering sections  132 - 1  through  132 - n  apply in parallel the deblocking filter to all vertical boundaries determined at step S 110  to require the deblocking filter to be applied (step S 120 ). The horizontal boundary determination sections  116 - 1  through  116 - n  determine in parallel for all horizontal boundaries included in a plurality of macro blocks within an input image whether filtering is needed (step S 130 ). The vertical filtering sections  142 - 1  through  142 - n  apply in parallel the deblocking filter to all horizontal boundaries determined at step S 130  to require the deblocking filter to be applied (step S 140 ). 
     The above-described process flows are mere examples. For example, the deblocking filter  24  may parallelize processes on two or more macro blocks. The sequence of processes may be changed. 
       FIG. 21  is a flowchart illustrating a process flow of the filtering need determination process corresponding to steps S 110  and S 130  in  FIG. 20 . With reference to  FIG. 21 , the distortion evaluation section  124  evaluates boundaries for distortion based on mode information, transform coefficient information, and motion vector information (step S 150 ). The process proceeds to step S 154  if the evaluation results in the presence of distortion (determination condition A is true). The process proceeds to step S 160  if the evaluation results in the absence of distortion (step S 152 ). 
     At step S 154 , the calculation section  122  calculates an edge value based on a reference pixel tap constituted by the tap constitution section  121  (step S 154 ). The threshold comparison section  123  compares the calculated value with edge determination threshold value β (step S 156 ). The process proceeds to step S 158  if the edge value is not smaller than threshold value β (determination condition B is true). The process proceeds to step S 160  if the edge value is not smaller than threshold value β. 
     At step S 158 , the filtering determination section  125  determines to apply the deblocking filter to a boundary to be determined (step S 158 ). At step S 140 , the filtering determination section  125  determines not to apply the deblocking filter to a boundary to be determined (step S 160 ). 
     4. Second Working Example 
     4-1. Deblocking Filter Configuration Example 
     The following describes example configurations of the deblocking filter  24  according to the second working example. 
     (1) Dependency Between New Processes 
     According to the working example, the deblocking filter  24  performs the filtering need determination process on vertical boundaries of each block without waiting for application of the deblocking filter to the other blocks in the macro block to which the block belongs. The deblocking filter  24  performs the filtering need determination process on horizontal boundaries of each block without waiting for application of the deblocking filter to the other blocks in the macro block to which the block belongs. This can relieve the dependency of processes within a macro block. 
     As described above, relieving the dependency of processes can consequently parallelize filtering need determination processes on vertical boundaries and horizontal boundaries in a macro block. 
       FIG. 22  is an explanatory diagram illustrating a first example of process sequence available on the deblocking filter  24 . The example also assumes that the deblocking filter is supplied with an image having the size of 32×32 pixels. The input image includes four macro blocks MB0 through MB3 each having the size of 16×16 pixels. 
     In  FIG. 22 , each broken-line frame represents a process to be performed in parallel. While the example in  FIG. 10  requires 16 process steps for a sequence of processes, the example in  FIG. 22  aggregates the same number of processes into 12 process steps. The first step performs, in parallel, filtering need determination processes J V0,0  through J V0,3  and J H0,0  through J H0,3  on four vertical boundaries and four horizontal boundaries of macro block MB0. The second step performs, in parallel, filtering processes F V0,0  through F V0,3  on four vertical boundaries in macro block MB0. The third step performs, in parallel, filtering need determination processes J V1,0  through J V1,3  and J V1,0  through J H1,3  on four vertical boundaries and four horizontal boundaries of macro block MB1. The fourth step performs, in parallel, filtering processes F V1,0  through F V1,3  on four vertical boundaries in macro block MB1. The fifth step performs, in parallel, filtering processes F H0,0  through F H0,3  on four horizontal boundaries in macro block MB0. The sixth step performs, in parallel, filtering need determination processes J V1,0  through J V2,3  and J H2,0  through J H2,3  on four vertical boundaries and four horizontal boundaries of macro block MB2. The seventh step performs, in parallel, filtering processes F V2,0  through F V2,3  on four vertical boundaries in macro block MB2. The eighth step performs, in parallel, filtering processes F H1,0  through F H1,3  on four horizontal boundaries in macro block MB1. The ninth step performs, in parallel, filtering need determination processes J V1,0  through J V3,3  and J H3,0  through J H3,3  on four vertical boundaries and four horizontal boundaries of macro block MB3. The tenth step performs, in parallel, filtering processes F V3,0  through F V3,3  on four vertical boundaries in macro block MB3. The eleventh step performs, in parallel, filtering processes F H2,0  through F H2,3  on four horizontal boundaries in macro block MB2. The twelfth step performs, in parallel, filtering processes F H3,0  through F H3,3  on four horizontal boundaries in macro block MB3. In this case, the deblocking filter  24  can perform a process on the entire input image using process steps fewer than those of the existing technique. 
     (2) Detailed Configuration of Deblocking Filter 
       FIG. 23  is a block diagram illustrating a detailed configuration of the deblocking filter  24  according to the second working example for performing the above-described parallel processes. With reference to  FIG. 23 , the deblocking filter  24  includes a vertical determination block  210 , a horizontal determination block  214 , the horizontal filtering block  130 , the vertical filtering block  140 , and the parallelization control section  150 . 
     (2-1) Vertical Determination Block 
     The vertical determination block  210  includes a plurality of vertical boundary determination sections  212 - 1  through  212 - n . Each vertical boundary determination section  212  determines whether to apply the deblocking filter to vertical boundaries of each block without waiting for application of the deblocking filter to the other blocks in the macro block to which the block belongs. Each vertical boundary determination section  212  supplies the horizontal filtering block  130  with information indicating a determination result about each vertical boundary such as binary information indicating a determination result that value “1” forces application of the deblocking filter. 
     (2-2) Horizontal Determination Block 
     The horizontal determination block  214  includes a plurality of horizontal boundary determination sections  216 - 1  through  216 - n . Each horizontal boundary determination section  216  determines whether to apply the deblocking filter to horizontal boundaries of each block without waiting for application of the deblocking filter to the other blocks in the macro block to which the block belongs. Each horizontal boundary determination section  216  supplies the vertical filtering block  140  with information indicating a determination result about each horizontal boundary. 
     Also according to the working example, each vertical boundary determination section  212  and each horizontal boundary determination section  216  may determine for each boundary whether filtering is needed by referencing pixels at positions similarly to the existing technique. Instead, each vertical boundary determination section  212  and each horizontal boundary determination section  216  may determine for each boundary whether filtering is needed according to the technique described in “3-2. Determination Condition Modifications.” 
     3-2. Process Flow 
       FIG. 24  is a flowchart illustrating a process flow for the deblocking filter  24  according to the second working example. With reference to  FIG. 24 , the vertical boundary determination sections  212 - 1  through  212 - n  determine in parallel for all vertical boundaries included in a focused macro blocks within an input image whether filtering is needed (step S 202 ). The horizontal boundary determination sections  214 - 1  through  214 - n  determine in parallel whether filtering is needed for all horizontal boundaries included in the focused macro block(step S 204 ). Steps S 202  and S 204  are also performed in parallel. 
     The horizontal filtering sections  132 - 1  through  132 - n  apply the deblocking filter in parallel to vertical boundaries in the focused macro block determined at step S 202  to require the deblocking filter to be applied (step S 210 ). 
     The process at step S 220  aims at a focused macro block in the most recent loop. The process at step S 220  may be skipped for the first focused macro block. The vertical filtering sections  142 - 1  through  142 - n  apply the deblocking filter in parallel to horizontal boundaries determined, at step S 204  in the most recent loop, to require the deblocking filter to be applied (step S 220 ). 
     The process at steps S 202  through S 220  is repeated for a newly focused macro block if focused macro blocks remain unprocessed in the input image (step S 230 ). 
     If there remains no focused macro block unprocessed, the vertical filtering sections  142 - 1  through  142 - n  apply the deblocking filter in parallel to horizontal boundaries determined to require the deblocking filter to be applied in the focused macro block for the last loop (step S 240 ). 
     The flow of processes described above is also a mere example. The parallelism and sequence of processes may be changed. Further, the parallelization control section  150  may control the parallelism and sequence of processes. 
     4-3. Process Example for Each LCU 
     As already mentioned, the technology according to various working examples described in this specification may be provided as a process based on an HEVC coding unit (CU). According to HEVC, a coding unit having the largest size is referred to as a largest coding unit (LCU) that can be selected as 64×64 pixels, for example. The minimum selectable CU size is 8×8 pixels. Normally, an image is encoded and decoded corresponding to each LCU in accordance with a raster scan sequence from the LCU at the top left of a picture (or a slice). The following describes process examples corresponding to LCUs in the deblocking filter  24 . 
       FIG. 25  is an explanatory diagram illustrating a process sequence for each LCU according to the second working example described above. The example assumes the LCU size to be 16×16 pixels and the CU size to be 8×8 pixels. 
     With reference to  FIG. 25 , the first stage is shown at the top left of the drawing and indicates that the filtering on LCUs has completed up to the (n−1)th LCU. Shaded pixels are targeted for filtering on vertical boundaries. Filled pixels are targeted for filtering on horizontal boundaries. 
     In  FIG. 25 , the second process at the top right and the third process at the bottom left are targeted for the nth LCU. Prior to the second process, filtering need determination processes are performed in parallel on all vertical boundaries and horizontal boundaries belonging to the nth LCU. Namely, the filtering need determination process on boundaries belonging to CUs in the nth LCU is performed without waiting for application of the deblocking filter to the other CUs in the nth LCU. The second process performs filtering processes in parallel on vertical boundaries that belong to the nth LCU and are determined to require the deblocking filter to be applied. The second process performs filtering processes in parallel on horizontal boundaries that belong to the nth LCU and are determined to require the deblocking filter to be applied. 
     A process for the fourth stage at the bottom right of  FIG. 25  is targeted for the (n+1)th LCU. At the fourth stage, the filtering process is performed in parallel on a vertical boundary determined to require the deblocking filter to be applied after the filtering need determination processes are performed in parallel on boundaries belonging to all CUs in the (n+1)th LCU. 
     While the example assumes the LCU size to be 16×16 pixels, it may be set to 32×32 or 64×64 pixels. The effect of shortening the processing time according to the parallelization is further improved because increasing the size of an LCU to be selected also increases the number of vertical boundaries and horizontal boundaries belonging to one LCU. 
       FIG. 26  is a flowchart illustrating a process flow of the deblocking filter  24  for each LCU. 
     With reference to  FIG. 26 , the vertical boundary determination sections  212 - 1  through  212 - n  determine in parallel whether filtering is needed for all vertical boundaries included in a focused LCU within an input image (step S 252 ). The horizontal boundary determination sections  216 - 1  through  216 - n  determine in parallel whether filtering is needed for all horizontal boundaries included in the focused LCU (step S 254 ). Steps S 252  and S 254  are also performed in parallel. 
     The horizontal filtering sections  132 - 1  through  132 - n  apply the deblocking filter in parallel to vertical boundaries in the focused LCU determined at step S 252  to require the deblocking filter to be applied (step S 260 ). 
     The vertical filtering sections  142 - 1  through  142 - n  apply the deblocking filter in parallel to horizontal boundaries in the focused LCU determined at step S 254  to require the deblocking filter to be applied (step S 270 ). 
     The process at steps S 252  through S 270  is repeated for a newly focused LCU if an LCU remains unprocessed in the input image (step S 280 ). The process terminates if there remains no LCU unprocessed. 
     5. Third Working Example 
     5-1. Overview 
     The first and second working examples change the existing sequence of processes for the deblocking filter to improve the parallelism of processes. Particularly, the first working example relieves the process dependency by extending the scope of pixel values for an input image supplied to the deblocking filter when the necessity of filtering is determined. The third working example to be described enhances this concept. The third working example further parallelizes processes by filtering input pixel values supplied to the deblocking filter during a filtering process on vertical boundaries and horizontal boundaries. 
       FIG. 27  is an explanatory diagram illustrating an overview of the working example. At the bottom left of  FIG. 27 , there is shown a shape representing input pixels (also referred to as reconstruct pixels) before being processed by the deblocking filter. The working example allows pixel values input to the deblocking filter to be referenced from a filtering need determination process for vertical boundaries and horizontal boundaries as well as a filtering process for vertical boundaries and a filtering process for horizontal boundaries. Accordingly, this enables to solve the dependency between the two filtering need determination processes and the dependency between the two filtering processes. 
     The filtering processes for vertical boundaries and the filtering processes for horizontal boundaries may update values of duplicate pixels. Filled pixels in  FIG. 27  illustrate positions of the pixels likely to be duplicated. The deblocking filter according to the working example calculates one output pixel value from two filter outputs in terms of pixels that are duplicately updated by two filters operating in parallel. 
     5-2. Deblocking Filter Configuration Example 
       FIG. 28  is a block diagram illustrating a detailed configuration of the deblocking filter  24  according to the third working example. With reference to  FIG. 28 , the deblocking filter  24  includes the line memory  308 , the determination block  310 , a horizontal filtering block  330 , a vertical filtering block  340 , the parallelization control section  150 , and a calculation section  360 . 
     The line memory  308  stores pixel values for an input image supplied to the deblocking filter  24 . Filtering processes in the horizontal filtering block  330  and the vertical filtering block  340  do not update pixel values stored in the line memory  308 . Filtering need determination processes performed by sections described below in the determination block  310  reference pixel values stored in the line memory  308 . The apparatus includes another memory for purposes different from processes of the deblocking filter  24 . This memory may be reused (shared) as the line memory  308 . 
     The determination block  310  includes vertical boundary determination sections  312 - 1  through  312 - n  and horizontal boundary determination sections  314 - 1  through  314 - n . The vertical boundary determination sections  312  and the horizontal boundary determination sections  314  are supplied with pixel values stored in the line memory  308  for an image input to the deblocking filter  24  and determination information used to determine the need for filtering. 
     The vertical boundary determination sections  312  use pixel values input to the deblocking filter  24  to determine whether to apply the deblocking filter to each vertical boundary. The vertical boundary determination sections  312  output, to the horizontal filtering block  330 , information indicating a determination result about each vertical boundary. 
     The horizontal boundary determination sections  314  also use pixel values input to the deblocking filter  24  to determine whether to apply the deblocking filter to each horizontal boundary. The horizontal boundary determination sections  314  perform determination processes in parallel to determination processes performed by the vertical boundary determination sections  312 . The horizontal boundary determination sections  314  output, to the vertical filtering block  340 , information indicating a determination result about each horizontal boundary. 
     Also according to the working example, each vertical boundary determination section  312  and each horizontal boundary determination section  314  may determine for each boundary whether filtering is needed by referencing pixels at positions similarly to the existing technique. Instead, each vertical boundary determination section  312  and each horizontal boundary determination section  314  may determine for each boundary whether filtering is needed according to the technique described in “3-2. Determination Condition Modifications.” 
     The horizontal filtering block  330  includes horizontal filtering sections  332 - 1  through  332 - n . The horizontal filtering sections  332  are supplied with an input image value from the line memory  208  and a determination result concerning each vertical boundary from the determination block  310 . 
     The horizontal filtering sections  332  apply the deblocking filter for vertical boundaries to right and left pixels around the corresponding vertical boundary if the determination result from the vertical boundary determination section  312  indicates that the filter needs to be applied. The horizontal filtering sections  332  output, to the calculation section  360 , a pixel value after the filtering in terms of the filtered pixel or an input pixel value in terms of the other pixels. The vertical filtering block  340  includes vertical filtering sections  342 - 1  through  342 - n . The vertical filtering sections  342  are supplied with an input pixel value from the line memory  308  and a determination result concerning each horizontal boundary from the determination block  310 . 
     The vertical filtering sections  342  apply the deblocking filter for horizontal boundaries to top and bottom pixels around the corresponding horizontal boundary if the determination result from the horizontal boundary determination section  314  indicates that the filter needs to be applied. Filtering processes of the vertical filtering sections  342 - 1  through  342 - n  are performed in parallel to filtering processes of the horizontal filtering sections  332 - 1  through  332 - n . The vertical filtering sections  342  output, to the calculation section  360 , a pixel value after the filtering in terms of the filtered pixel or an input pixel value in terms of the other pixels. 
     The calculation section  360  is supplied with an output pixel value from the horizontal filtering block  330  and an output pixel value from the vertical filtering block  340  in parallel. Further, the calculation section  360  is supplied with determination results from the vertical boundary determination section  312  and the horizontal boundary determination section  314 . According to a determination result, the calculation section  360  calculates output pixel values for pixels filtered from the horizontal filtering block  330  and the vertical filtering block  340  based on filter outputs from the horizontal filtering block  330  and the vertical filtering block  340 . 
     According to the working example, for example, the calculation section  360  calculates an average of two filter outputs for duplicately filtered pixels. The calculation section  360  may calculate a simple average of two filter outputs. Instead, the calculation section  360  may calculate a weighted average of two filter outputs. For example, the calculation section  360  may determine a weight for weighted averages of pixels according to the distance from each pixel to the vertical boundary and to the horizontal boundary. 
       FIG. 29  is an explanatory diagram illustrating determination of a weight for weighted average calculated by the calculation section  360 .  FIG. 29  shows focused pixel P Z  in black corresponding to one of the duplicated positions illustrated in  FIG. 27 . There are three pixels corresponding to distance D V  between focused pixel P Z  and nearest vertical boundary V Z . There are two pixels corresponding to distance D H  between focused pixel P Z  and nearest horizontal boundary H Z . Distance D H  is smaller than distance D V . In this case, the calculation section  360  may set a weight for output from the deblocking filter applied to horizontal boundary H Z  to be larger than a weight for output from the deblocking filter applied to vertical boundary V Z . The example in  FIG. 29  assumes that a ratio of filter output V out . for vertical boundary V Z  to filter output H out . for horizontal boundary H Z  is 2:3. 
     As seen from  FIG. 29 , calculating a weighted average of two filter outputs can consequently provide each focused pixel with an output pixel value similar to the case of applying one two-dimensional filter having a filter tap along the horizontal direction and a filter tap along the vertical direction. Parallelizing filtering processes on the vertical boundary and the horizontal boundary can also appropriately reduce block distortion appearing on the vertical boundary and the horizontal boundary. As another working example, the deblocking filter  24  may include one two-dimensional filter that simultaneously calculates horizontal filtering, vertical filtering, and a weighted average. In this case, however, the installation is very complicated because filter coefficients need to be variously changed correspondingly to pixels. On the other hand, the third working example performs two one-dimensional filters in parallel and then calculates a weighted average. This can easily provide processes substantially equal to a two-dimensional filter while ensuring the functionality of existing deblocking filters. 
       FIG. 30  is an explanatory diagram illustrating an example of the weight for weighted averages calculated based on the example in  FIG. 29 .  FIG. 30  shows 36 pixels (6×6) around an intersection between the vertical boundary and the horizontal boundary. The pixels correspond to the above-described duplicated positions. The ratio of the weight for filter output V out . to the weight for filter output H out  is 1:1 (2:2 or 3:3) for pixels positioned at an equal distance from the vertical boundary and the horizontal boundary. The weight for filter output V out  is larger than the weight for filter output H out  for pixels nearer to the vertical boundary. For example, the ratio of weights for pixel P 1  is V out :H out =3:1. The weight for filter output V out  is smaller than the weight for filter output H out  for pixels nearer to the horizontal boundary. For example, the ratio of weights for pixel P 2  is V out :H out =1:3. 
     The block distortion can be more effectively suppressed and the image quality can be improved by varying the weight for weighted averages depending on the distance between each pixel and the boundary. 
     The above-described weights are mere examples. For example, the calculation section  360  may determine the weight of weighted averages for pixels according to the edge strengths of the vertical boundary and the horizontal boundary corresponding to each pixel instead of or in addition to the distance between each pixel and the boundary. The edge strength may be represented with a parameter such as an edge value calculated from the calculation section  122  as shown in  FIG. 13 , for example. In this case, the weight for filer output on a boundary having a stronger edge may be set to be larger than the weight for filer output on a boundary having a weaker edge. Varying the weight of weighted averages according to the edge strength can adaptively improve the effect of the deblocking filter at a boundary that remarkably causes block distortion. 
     The calculation section  360  selects outputs from actually filtered blocks in terms of pixels filtered by one of the horizontal filtering block  330  and the vertical filtering block  340 . The calculation section  360  directly outputs an input pixel value to be output to the deblocking filter  24  in terms of pixels not filtered by the horizontal filtering block  330  or the vertical filtering block  340 . A table in  FIG. 31  lists output pixel values from the calculation section  360  according to results of the determination whether to require the filtering. 
     5-3. Process Sequence Example 
     The following describes two examples of process sequences available for the deblocking filter  24  according to the working example. The example also assumes that the deblocking filter is supplied with an image having the size of 32×32 pixels. The input image includes four macro blocks MB0 through MB3 each having the size of 16×16 pixels. 
     (1) First Example 
     For comparison,  FIG. 32  illustrates a process sequence when the dependency remains between a filtering process on the vertical boundary and a filtering process on the horizontal boundary. In  FIG. 32 , the first step performs, in parallel, filtering need determination processes J V0,0  through J V3,3  and J H0,0  through J H3,3  on all vertical boundaries and all horizontal boundaries of all the four macro blocks MB0 through MB3. The second step performs filtering processes F V0,0  through F V3,3  on 16 vertical boundaries of the four macro blocks MB0 through MB3. The third step performs filtering processes F H0,0  through F H3,3  on 16 horizontal boundaries of the four macro blocks MB0 through MB3. The fourth step stores pixel values after the filtering process on the horizontal boundary in the memory used for outputs from the deblocking filter  24 . 
       FIG. 33  illustrates a first example of process sequence provided by the working example. In  FIG. 33 , the first step performs, in parallel, filtering need determination processes J V0,0  through J V3,3  and J H0,0  through J H3,3  on all vertical boundaries and all horizontal boundaries of the four macro blocks MB0 through MB3. The second step performs, in parallel, filtering processes F V0,0  through F V3,3  and F H0,0  through F H3,3  on all vertical boundaries and all horizontal boundaries of the four macro blocks MB0 through MB3. Actually, the second step filters only a boundary determined to require the filtering. The third step stores pixel values in the memory used for outputs from the deblocking filter  24 . A weighted average of two filter outputs may be calculated as an output pixel value in terms of pixels filtered by the horizontal filtering block  330  and the vertical filtering block  340 . 
     (2) Second Example 
     While the first example maximizes the parallelism, the deblocking filter  24  according to the second example can also perform a process for each macro block. 
     For comparison,  FIG. 34  illustrates a process sequence for each macro block when the dependency remains between a filtering process on the vertical boundary and a filtering process on the horizontal boundary. The process sequence in  FIG. 34  substantially equals the process sequence in  FIG. 22  according to the first working example.  FIG. 36  explicitly shows the four process steps (sixth, tenth, 14th, and 16th) that store pixel values in the memory for output and are omitted from  FIG. 22  for simplicity. Sixteen process steps including the four process steps configure the process in  FIG. 34 . 
       FIG. 35  illustrates a second example of process sequence provided by the working example. In  FIG. 35 , the first step performs, in parallel, filtering need determination processes J V0,0  through J V0,3  and J H0,0  through J H0,3  on four vertical boundaries and four horizontal boundaries of macro block MB0. The second step performs, in parallel, filtering processes F V0,0  through F V0,3  and F H0,0  through F H0,3  on four vertical boundaries and four horizontal boundaries of macro block MB0. The third step stores pixel values of macro block MB0 in the memory used for outputs from the deblocking filter  24 . A weighted average of two filter outputs may be calculated as an output pixel value in terms of pixels duplicately filtered by two filters. The fourth to sixth steps similarly process macro block MB1. The seventh to ninth steps similarly process macro block MB2. The tenth to twelfth steps similarly process macro block MB3. The process in  FIG. 35  includes twelve process steps fewer than those of the process in  FIG. 34 . 
     The third working example eliminates the dependency between filtering processes for vertical boundaries and a filtering process for horizontal boundaries. The process of the deblocking filter  24  can be performed using fewer process steps than those used for the first and second working examples. One of advantages of allowing a filtering process to reference only pixels input to the deblocking filter is that any configuration of filter taps causes no dependency between filtering processes for vertical boundaries and filtering processes for horizontal boundaries. The third working example can improve the image quality by configuring a filter tap using more pixels than used for existing techniques. For example, the existing technique uses a filter tap of three pixels for each side of each boundary as described with reference to  FIG. 7 . The working example causes no dependency between processes even if a filter tap of five pixels or more is used at each boundary. No dependency occurs between processes even by further decreasing the block size as a process unit for the deblocking filter. 
     Also in the third working example as well as the first and second working examples, the parallelization control section  150  may control the parallelism and sequence of processes in the deblocking filter  24 . 
     5-4. Process Flow 
       FIG. 36  is a flowchart illustrating an example of a process flow for the deblocking filter according to the third working example.  FIG. 37  is a flowchart illustrating a flow of the pixel value calculation process shown in  FIG. 36 . 
     With reference to  FIG. 36 , the vertical boundary determination sections  312 - 1  through  312 - n  determine in parallel whether filtering is needed for all vertical boundaries within an input image or a macro block (step S 302 ). The horizontal boundary determination sections  314 - 1  through  314 - n  determine in parallel whether filtering is needed for all horizontal boundaries within the input image or the macro block (step S 304 ). Steps S 302  and S 304  are also performed in parallel. 
     The horizontal filtering sections  332 - 1  through  332 - n  apply the deblocking filter in parallel to all vertical boundaries determined at step S 302  to require the deblocking filter to be applied (step S 306 ). The vertical filtering sections  342 - 1  through  342 - n  apply the deblocking filter in parallel to all horizontal boundaries determined at step S 304  to require the deblocking filter to be applied (step S 308 ). Steps S 306  and S 308  are also performed in parallel. 
     The calculation section  360  then performs the pixel value calculation process as shown in  FIG. 37  (step S 310 ). With reference to  FIG. 37 , the process from step S 314  to step S 326  loops for each pixel to be processed (step S 312 ). 
     At step S 314 , the calculation section  360  determines whether two filters for vertical boundaries and horizontal boundaries have filtered a focused pixel (step S 314 ). The process proceeds to step S 322  if the two filters have filtered the focused pixel. The process proceeds to step S 316  if the two filters have not filtered the focused pixel. 
     At step S 316 , the calculation section  360  determines whether one of the two filters for vertical boundaries and horizontal boundaries has filtered the focused pixel (step S 316 ). The process proceeds to step S 320  if one of the two filters has filtered the focused pixel. The process proceeds to step S 318  if none of the filters has filtered the focused pixel. 
     At step S 318 , the calculation section  360  acquires an input pixel value to the deblocking filter  24  (step S 318 ). At step S 320 , the calculation section  360  acquires a filter output from the filter that actually filters the focused pixel (step S 320 ). 
     At step S 322 , the calculation section  360  determines weight values for calculating a weighted average of filter outputs from the two filters concerning the focused pixel according to distances from the focused pixel to the vertical boundary and the horizontal boundary or the edge strengths of the vertical boundary and the horizontal boundary corresponding to the focused pixel (step S 322 ). The calculation section  360  calculates a weighted average of filter outputs from the two filters using the determined weight (step S 324 ). 
     The calculation section  360  stores the pixel value of the focused pixel in the memory while the pixel value is acquired at step S 318  or S 320  or is calculated at step S 324  (step S 326 ). The sequences of processes as shown in  FIGS. 36 and 37  terminate when the process is performed on all pixels to be processed. 
     6. Application to Various Codecs 
     The technology according to the disclosure is applicable to various codecs related to image encoding and decoding. The following describes examples of applying the technology according to the disclosure to multiview codec and scalable codec. 
     6-1. Multiview Codec 
     The multiview codec is an image encoding scheme that encodes and decodes multiple-perspective video.  FIG. 38  is an explanatory diagram illustrating the multiview codec.  FIG. 38  illustrates sequences of frames for three views captured at three observing points. Each view is provided with a view ID (view_id). One of the views is specified as a base view. Views other than the base view are referred to as non-base views. The example in  FIG. 38  represents a base view with view ID “0” and two non-base views with view ID “1” or “2.” Encoding multiview image data may compress the data size of the encoded stream as a whole by encoding frames of the non-base view based on encoding information about frames of the base view. 
     The deblocking filter may be applied to each view during the encoding process and the decoding process according to the multiview codec described above. Application of the deblocking filter to each view may parallelize horizontal filtering and vertical filtering in units of processes including multiple CUs for each view according to the technology of the disclosure. The above-described process unit may represent several CUs, LCUs, or pictures. The parameter (such as the one described in the preceding paragraph 0092) to control a parallel process may be set for each view. The parameter set for the base view may be reused for the non-base view. 
     The horizontal filtering and the vertical filtering may be parallelized over a plurality of views. The plurality of views may share the parameter (such as the one described in the preceding paragraph 0092) to control the parallel process. It may be advantageous to additionally specify a flag indicating whether the plurality of views share the parameter. 
       FIG. 39  is an explanatory diagram illustrating the image encoding process applied to the multiview codec described above.  FIG. 39  shows a configuration of a multiview encoding device  710  as an example. The multiview encoding device  710  includes a first encoding section  720 , a second encoding section  730 , and a multiplexing section  740 . 
     The first encoding section  720  encodes a base view image and generates an encoded stream for the base view. The second encoding section  730  encodes a non-base view image and generates an encoded stream for the non-base view. The multiplexing section  740  multiplexes an encoded stream for the base view generated from the first encoding section  720  and one or more encoded streams for the non-base view generated from the second encoding section  730  to generate a multiplexed stream for multiview. 
     The first encoding section  720  and the second encoding section  730  illustrated in  FIG. 39  are configured similarly to the image encoding device  10  according to the above-described embodiment. Applying the deblocking filter to views enables to parallelize the horizontal filtering and the vertical filtering in units of processes containing multiple CUs. A parameter to control these processes may be inserted into a header area of the encoded stream for each view or into a common header area in the multiplexed stream. 
       FIG. 40  is an explanatory diagram illustrating an image decoding process applied to the multiview codec described above.  FIG. 40  shows a configuration of a multiview decoding device  760  as an example. The multiview decoding device  760  includes a demultiplexing section  770 , a first decoding section  780 , and a second decoding section  790 . 
     The demultiplexing section  770  demultiplexes a multiplexed stream for multiview into an encoded stream for the base view and an encoded stream for one or more non-base views. The first decoding section  780  decodes a base view image from an encoded stream for the base view. The second decoding section  730  decodes a non-base view image from an encoded stream for the non-base view. 
     The first decoding section  780  and the second decoding section  790  illustrated in  FIG. 40  are configured similarly to the image decoding device  60  according to the above-described embodiment. Applying the deblocking filter to views enables to parallelize horizontal filtering in units of processes including multiple CUs and parallelize vertical filtering. A parameter to control these processes may be acquired from a header area of the encoded stream for each view or from a common header area in the multiplexed stream. 
     6-2. Scalable Codec 
     The scalable codec is an image encoding scheme to provide hierarchical encoding.  FIG. 41  is an explanatory diagram illustrating the scalable codec.  FIG. 41  illustrates frame sequences for three layers of different space resolutions, time resolutions, or image qualities. Each layer is provided with a layer ID (layer_id). These layers include a base layer having the lowest resolution (or image quality). Layers other than the base layer are referred to as enhancement layers. The example in  FIG. 41  represents a base layer with layer ID “0” and two enhancement layers with layer ID “1” or “2.” Encoding multi-layer image data may compress the data size of the encoded stream as a whole by encoding frames of the enhancement layer based on encoding information about frames of the base layer. 
     The deblocking filter may be applied to each layer during the encoding process and the decoding process according to the scalable codec described above. Application of the deblocking filter to each layer may parallelize horizontal filtering and vertical filtering in units of processes including multiple CUs for each view according to the technology of the disclosure. The above-described process unit may represent several CUs, LCUs, or pictures. The parameter (such as the one described in the preceding paragraph 0092) to control a parallel process may be set for each layer. The parameter set for the base layer view may be reused for the enhancement layer. 
     The horizontal filtering and the vertical filtering may be parallelized over a plurality of layers. The plurality of layers may share the parameter (such as the one described in the preceding paragraph 0092) to control the parallel process. It may be advantageous to additionally specify a flag indicating whether the plurality of layers share the parameter. 
       FIG. 42  is an explanatory diagram illustrating the image encoding process applied to the scalable codec described above.  FIG. 42  shows a configuration of a scalable encoding device  810  as an example. The scalable encoding device  810  includes a first encoding section  820 , a second encoding section  830 , and a multiplexing section  840 . 
     The first encoding section  820  encodes a base layer image and generates an encoded stream for the base layer. The second encoding section  830  encodes an enhancement layer image and generates an encoded stream for the enhancement layer. The multiplexing section  840  multiplexes an encoded stream for the base layer generated from the first encoding section  820  and one or more encoded streams for the enhancement layer generated from the second encoding section  830  to generate a multiplexed stream for multi-layer. 
     The first encoding section  820  and the second encoding section  830  illustrated in  FIG. 42  are configured similarly to the image encoding device  10  according to the above-described embodiment. Applying the deblocking filter to layers enables to parallelize horizontal filtering in units of processes including multiple CUs and parallelize vertical filtering. A parameter to control these processes may be inserted into a header area of the encoded stream for each layer or into a common header area in the multiplexed stream. 
       FIG. 43  is an explanatory diagram illustrating an image decoding process applied to the scalable codec described above.  FIG. 43  shows a configuration of a scalable decoding device  860  as an example. The scalable decoding device  860  includes a demultiplexing section  870 , a first decoding section  880 , and a second decoding section  890 . 
     The demultiplexing section  870  demultiplexes a multiplexed stream for multi-layer into an encoded stream for the base layer and an encoded stream for one or more enhancement layers. The first decoding section  880  decodes a base layer image from an encoded stream for the base layer. The second decoding section  830  decodes an enhancement layer image from an encoded stream for the enhancement layer. 
     The first decoding section  880  and the second decoding section  890  illustrated in  FIG. 43  are configured similarly to the image decoding device  60  according to the above-described embodiment. Applying the deblocking filter to layers enables to parallelize horizontal filtering in units of processes including multiple CUs and parallelize vertical filtering. A parameter to control these processes may be acquired from a header area of the encoded stream for each layer or from a common header area in the multiplexed stream. 
     7. Example Application 
     The image encoding device  10  and the image decoding device  60  according to the embodiment described above may be applied to various electronic appliances such as a transmitter and a receiver for satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, distribution to terminals via cellular communication, and the like, a recording device that records images in a medium such as an optical disc, a magnetic disk or a flash memory, a reproduction device that reproduces images from such storage medium, and the like. Four example applications will be described below. 
     7-1. First Example Application 
       FIG. 44  is a block diagram showing an example of a schematic configuration of a television adopting the embodiment described above. A television  900  includes an antenna  901 , a tuner  902 , a demultiplexer  903 , a decoder  904 , an video signal processing section  905 , a display section  906 , an audio signal processing section  907 , a speaker  908 , an external interface  909 , a control section  910 , a user interface  911 , and a bus  912 . 
     The tuner  902  extracts a signal of a desired channel from broadcast signals received via the antenna  901 , and demodulates the extracted signal. Then, the tuner  902  outputs an encoded bit stream obtained by demodulation to the demultiplexer  903 . That is, the tuner  902  serves as transmission means of the televisions  900  for receiving an encoded stream in which an image is encoded. 
     The demultiplexer  903  separates a video stream and an audio stream of a program to be viewed from the encoded bit stream, and outputs each stream which has been separated to the decoder  904 . Also, the demultiplexer  903  extracts auxiliary data such as an EPG (Electronic Program Guide) from the encoded bit stream, and supplies the extracted data to the control section  910 . Additionally, the demultiplexer  903  may perform descrambling in the case the encoded bit stream is scrambled. 
     The decoder  904  decodes the video stream and the audio stream input from the demultiplexer  903 . Then, the decoder  904  outputs video data generated by the decoding process to the video signal processing section  905 . Also, the decoder  904  outputs the audio data generated by the decoding process to the audio signal processing section  907 . 
     The video signal processing section  905  reproduces the video data input from the decoder  904 , and causes the display section  906  to display the video. The video signal processing section  905  may also cause the display section  906  to display an application screen supplied via a network. Further, the video signal processing section  905  may perform an additional process such as noise removal, for example, on the video data according to the setting. Furthermore, the video signal processing section  905  may generate an image of a GUI (Graphical User Interface) such as a menu, a button, a cursor or the like, for example, and superimpose the generated image on an output image. 
     The display section  906  is driven by a drive signal supplied by the video signal processing section  905 , and displays a video or an image on an video screen of a display device (for example, a liquid crystal display, a plasma display, an OLED, or the like). 
     The audio signal processing section  907  performs reproduction processes such as D/A conversion and amplification on the audio data input from the decoder  904 , and outputs audio from the speaker  908 . Also, the audio signal processing section  907  may perform an additional process such as noise removal on the audio data. 
     The external interface  909  is an interface for connecting the television  900  and an external appliance or a network. For example, a video stream or an audio stream received via the external interface  909  may be decoded by the decoder  904 . That is, the external interface  909  also serves as transmission means of the televisions  900  for receiving an encoded stream in which an image is encoded. 
     The control section  910  includes a processor such as a CPU (Central Processing Unit), and a memory such as an RAM (Random Access Memory), an ROM (Read Only Memory), or the like. The memory stores a program to be executed by the CPU, program data, EPG data, data acquired via a network, and the like. The program stored in the memory is read and executed by the CPU at the time of activation of the television  900 , for example. The CPU controls the operation of the television  900  according to an operation signal input from the user interface  911 , for example, by executing the program. 
     The user interface  911  is connected to the control section  910 . The user interface  911  includes a button and a switch used by a user to operate the television  900 , and a receiving section for a remote control signal, for example. The user interface  911  detects an operation of a user via these structural elements, generates an operation signal, and outputs the generated operation signal to the control section  910 . 
     The bus  912  interconnects the tuner  902 , the demultiplexer  903 , the decoder  904 , the video signal processing section  905 , the audio signal processing section  907 , the external interface  909 , and the control section  910 . 
     In the television  900  configured in this manner, the decoder  904  has a function of the image decoding device  60  according to the embodiment described above. Accordingly, also in the case of the image decoding in the television  900 , it is possible to enhance the parallelism of deblocking filter processes and ensure high-speed processing. 
     7-2. Second Example Application 
       FIG. 45  is a block diagram showing an example of a schematic configuration of a mobile phone adopting the embodiment described above. A mobile phone  920  includes an antenna  921 , a communication section  922 , an audio codec  923 , a speaker  924 , a microphone  925 , a camera section  926 , an image processing section  927 , a demultiplexing section  928 , a recording/reproduction section  929 , a display section  930 , a control section  931 , an operation section  932 , and a bus  933 . 
     The antenna  921  is connected to the communication section  922 . The speaker  924  and the microphone  925  are connected to the audio codec  923 . The operation section  932  is connected to the control section  931 . The bus  933  interconnects the communication section  922 , the audio codec  923 , the camera section  926 , the image processing section  927 , the demultiplexing section  928 , the recording/reproduction section  929 , the display section  930 , and the control section  931 . 
     The mobile phone  920  performs operation such as transmission/reception of audio signal, transmission/reception of emails or image data, image capturing, recording of data, and the like, in various operation modes including an audio communication mode, a data communication mode, an image capturing mode, and a videophone mode. 
     In the audio communication mode, an analogue audio signal generated by the microphone  925  is supplied to the audio codec  923 . The audio codec  923  converts the analogue audio signal into audio data, and A/D converts and compresses the converted audio data. Then, the audio codec  923  outputs the compressed audio data to the communication section  922 . The communication section  922  encodes and modulates the audio data, and generates a transmission signal. Then, the communication section  922  transmits the generated transmission signal to a base station (not shown) via the antenna  921 . Also, the communication section  922  amplifies a wireless signal received via the antenna  921  and converts the frequency of the wireless signal, and acquires a received signal. Then, the communication section  922  demodulates and decodes the received signal and generates audio data, and outputs the generated audio data to the audio codec  923 . The audio codec  923  extends and D/A converts the audio data, and generates an analogue audio signal. Then, the audio codec  923  supplies the generated audio signal to the speaker  924  and causes the audio to be output. 
     Also, in the data communication mode, the control section  931  generates text data that makes up an email, according to an operation of a user via the operation section  932 , for example. Moreover, the control section  931  causes the text to be displayed on the display section  930 . Furthermore, the control section  931  generates email data according to a transmission instruction of the user via the operation section  932 , and outputs the generated email data to the communication section  922 . Then, the communication section  922  encodes and modulates the email data, and generates a transmission signal. Then, the communication section  922  transmits the generated transmission signal to a base station (not shown) via the antenna  921 . Also, the communication section  922  amplifies a wireless signal received via the antenna  921  and converts the frequency of the wireless signal, and acquires a received signal. Then, the communication section  922  demodulates and decodes the received signal, restores the email data, and outputs the restored email data to the control section  931 . The control section  931  causes the display section  930  to display the contents of the email, and also, causes the email data to be stored in the storage medium of the recording/reproduction section  929 . 
     The recording/reproduction section  929  includes an arbitrary readable and writable storage medium. For example, the storage medium may be a built-in storage medium such as an RAM, a flash memory or the like, or an externally mounted storage medium such as a hard disk, a magnetic disk, a magneto-optical disk, an optical disc, an USB memory, a memory card, or the like. 
     Furthermore, in the image capturing mode, the camera section  926  captures an image of a subject, generates image data, and outputs the generated image data to the image processing section  927 , for example. The image processing section  927  encodes the image data input from the camera section  926 , and causes the encoded stream to be stored in the storage medium of the recording/reproduction section  929 . 
     Furthermore, in the videophone mode, the demultiplexing section  928  multiplexes a video stream encoded by the image processing section  927  and an audio stream input from the audio codec  923 , and outputs the multiplexed stream to the communication section  922 , for example. The communication section  922  encodes and modulates the stream, and generates a transmission signal. Then, the communication section  922  transmits the generated transmission signal to a base station (not shown) via the antenna  921 . Also, the communication section  922  amplifies a wireless signal received via the antenna  921  and converts the frequency of the wireless signal, and acquires a received signal. These transmission signal and received signal may include an encoded bit stream. Then, the communication section  922  demodulates and decodes the received signal, restores the stream, and outputs the restored stream to the demultiplexing section  928 . The demultiplexing section  928  separates a video stream and an audio stream from the input stream, and outputs the video stream to the image processing section  927  and the audio stream to the audio codec  923 . The image processing section  927  decodes the video stream, and generates video data. The video data is supplied to the display section  930 , and a series of images is displayed by the display section  930 . The audio codec  923  extends and D/A converts the audio stream, and generates an analogue audio signal. Then, the audio codec  923  supplies the generated audio signal to the speaker  924  and causes the audio to be output. 
     In the mobile phone  920  configured in this manner, the image processing section  927  has a function of the image encoding device  10  and the image decoding device  60  according to the embodiment described above. Accordingly, also in the case of encoding and decoding an image in the mobile phone  920 , it is possible to enhance the parallelism of deblocking filter processes and ensure high-speed processing. 
     7-3. Third Example Application 
       FIG. 46  is a block diagram showing an example of a schematic configuration of a recording/reproduction device adopting the embodiment described above. A recording/reproduction device  940  encodes, and records in a recording medium, audio data and video data of a received broadcast program, for example. The recording/reproduction device  940  may also encode, and record in the recording medium, audio data and video data acquired from another device, for example. Furthermore, the recording/reproduction device  940  reproduces, using a monitor or a speaker, data recorded in the recording medium, according to an instruction of a user, for example. At this time, the recording/reproduction device  940  decodes the audio data and the video data. 
     The recording/reproduction device  940  includes a tuner  941 , an external interface  942 , an encoder  943 , an HDD (Hard Disk Drive)  944 , a disc drive  945 , a selector  946 , a decoder  947 , an OSD (On-Screen Display)  948 , a control section  949 , and a user interface  950 . 
     The tuner  941  extracts a signal of a desired channel from broadcast signals received via an antenna (not shown), and demodulates the extracted signal. Then, the tuner  941  outputs an encoded bit stream obtained by demodulation to the selector  946 . That is, the tuner  941  serves as transmission means of the recording/reproduction device  940 . 
     The external interface  942  is an interface for connecting the recording/reproduction device  940  and an external appliance or a network. For example, the external interface  942  may be an IEEE 1394 interface, a network interface, an USB interface, a flash memory interface, or the like. For example, video data and audio data received by the external interface  942  are input to the encoder  943 . That is, the external interface  942  serves as transmission means of the recording/reproduction device  940 . 
     In the case the video data and the audio data input from the external interface  942  are not encoded, the encoder  943  encodes the video data and the audio data. Then, the encoder  943  outputs the encoded bit stream to the selector  946 . 
     The HDD  944  records in an internal hard disk an encoded bit stream, which is compressed content data of a video or audio, various programs, and other pieces of data. Also, the HDD  944  reads these pieces of data from the hard disk at the time of reproducing a video or audio. 
     The disc drive  945  records or reads data in a recording medium that is mounted. A recording medium that is mounted on the disc drive  945  may be a DVD disc (a DVD-Video, a DVD-RAM, a DVD-R, a DVD-RW, a DVD+, a DVD+RW, or the like), a Blu-ray (registered trademark) disc, or the like, for example. 
     The selector  946  selects, at the time of recording a video or audio, an encoded bit stream input from the tuner  941  or the encoder  943 , and outputs the selected encoded bit stream to the HDD  944  or the disc drive  945 . Also, the selector  946  outputs, at the time of reproducing a video or audio, an encoded bit stream input from the HDD  944  or the disc drive  945  to the decoder  947 . 
     The decoder  947  decodes the encoded bit stream, and generates video data and audio data. Then, the decoder  947  outputs the generated video data to the OSD  948 . Also, the decoder  904  outputs the generated audio data to an external speaker. 
     The OSD  948  reproduces the video data input from the decoder  947 , and displays a video. Also, the OSD  948  may superimpose an image of a GUI, such as a menu, a button, a cursor or the like, for example, on a displayed video. 
     The control section  949  includes a processor such as a CPU, and a memory such as an RAM or an ROM. The memory stores a program to be executed by the CPU, program data, and the like. A program stored in the memory is read and executed by the CPU at the time of activation of the recording/reproduction device  940 , for example. The CPU controls the operation of the recording/reproduction device  940  according to an operation signal input from the user interface  950 , for example, by executing the program. 
     The user interface  950  is connected to the control section  949 . The user interface  950  includes a button and a switch used by a user to operate the recording/reproduction device  940 , and a receiving section for a remote control signal, for example. The user interface  950  detects an operation of a user via these structural elements, generates an operation signal, and outputs the generated operation signal to the control section  949 . 
     In the recording/reproduction device  940  configured in this manner, the encoder  943  has a function of the image encoding device  10  according to the embodiment described above. Also, the decoder  947  has a function of the image decoding device  60  according to the embodiment described above. Accordingly, also in the case of encoding and decoding an image in the recording/reproduction device  940 , it is possible to enhance the parallelism of deblocking filter processes and ensure high-speed processing. 
     7-4. Fourth Example Application 
       FIG. 47  is a block diagram showing an example of a schematic configuration of an image capturing device adopting the embodiment described above. An image capturing device  960  captures an image of a subject, generates an image, encodes the image data, and records the image data in a recording medium. 
     The image capturing device  960  includes an optical block  961 , an image capturing section  962 , a signal processing section  963 , an image processing section  964 , a display section  965 , an external interface  966 , a memory  967 , a media drive  968 , an OSD  969 , a control section  970 , a user interface  971 , and a bus  972 . 
     The optical block  961  is connected to the image capturing section  962 . The image capturing section  962  is connected to the signal processing section  963 . The display section  965  is connected to the image processing section  964 . The user interface  971  is connected to the control section  970 . The bus  972  interconnects the image processing section  964 , the external interface  966 , the memory  967 , the media drive  968 , the OSD  969 , and the control section  970 . 
     The optical block  961  includes a focus lens, an aperture stop mechanism, and the like. The optical block  961  forms an optical image of a subject on an image capturing surface of the image capturing section  962 . The image capturing section  962  includes an image sensor such as a CCD, a CMOS or the like, and converts by photoelectric conversion the optical image formed on the image capturing surface into an image signal which is an electrical signal. Then, the image capturing section  962  outputs the image signal to the signal processing section  963 . 
     The signal processing section  963  performs various camera signal processes, such as knee correction, gamma correction, color correction and the like, on the image signal input from the image capturing section  962 . The signal processing section  963  outputs the image data after the camera signal process to the image processing section  964 . 
     The image processing section  964  encodes the image data input from the signal processing section  963 , and generates encoded data. Then, the image processing section  964  outputs the generated encoded data to the external interface  966  or the media drive  968 . Also, the image processing section  964  decodes encoded data input from the external interface  966  or the media drive  968 , and generates image data. Then, the image processing section  964  outputs the generated image data to the display section  965 . Also, the image processing section  964  may output the image data input from the signal processing section  963  to the display section  965 , and cause the image to be displayed. Furthermore, the image processing section  964  may superimpose data for display acquired from the OSD  969  on an image to be output to the display section  965 . 
     The OSD  969  generates an image of a GUI, such as a menu, a button, a cursor or the like, for example, and outputs the generated image to the image processing section  964 . 
     The external interface  966  is configured as an USB input/output terminal, for example. The external interface  966  connects the image capturing device  960  and a printer at the time of printing an image, for example. Also, a drive is connected to the external interface  966  as necessary. A removable medium, such as a magnetic disk, an optical disc or the like, for example, is mounted on the drive, and a program read from the removable medium may be installed in the image capturing device  960 . Furthermore, the external interface  966  may be configured as a network interface to be connected to a network such as a LAN, the Internet or the like. That is, the external interface  966  serves as transmission means of the image capturing device  960 . 
     A recording medium to be mounted on the media drive  968  may be an arbitrary readable and writable removable medium, such as a magnetic disk, a magneto-optical disk, an optical disc, a semiconductor memory or the like, for example. Also, a recording medium may be fixedly mounted on the media drive  968 , configuring a non-transportable storage section such as a built-in hard disk drive or an SSD (Solid State Drive), for example. 
     The control section  970  includes a processor such as a CPU, and a memory such as an RAM or an ROM. The memory stores a program to be executed by the CPU, program data, and the like. A program stored in the memory is read and executed by the CPU at the time of activation of the image capturing device  960 , for example. The CPU controls the operation of the image capturing device  960  according to an operation signal input from the user interface  971 , for example, by executing the program. 
     The user interface  971  is connected to the control section  970 . The user interface  971  includes a button, a switch and the like used by a user to operate the image capturing device  960 , for example. The user interface  971  detects an operation of a user via these structural elements, generates an operation signal, and outputs the generated operation signal to the control section  970 . 
     In the image capturing device  960  configured in this manner, the image processing section  964  has a function of the image encoding device  10  and the image decoding device  60  according to the embodiment described above. Accordingly, in the case of encoding and decoding an image in the image capturing device  960 , it is possible to enhance the parallelism of deblocking filter processes and ensure high-speed processing. 
     8. Summing-Up 
     With reference to  FIGS. 1 through 47 , there have been described three working examples of the deblocking filter for the image encoding device  10  and the image decoding device  60  according to an embodiment. The three working examples relieve the dependency of deblocking filter processes inherent to the existing technique. This can improve the parallelism of processes when the deblocking filter is applied. As a result, it is possible to avoid delay or data rate degradation due to a large processing amount of the deblocking filter and ensure high-speed processing. The parallelism and sequences of deblocking filter processes can be flexibly set according to various conditions such as image sizes or installation environment. 
     According to the first working example, pixel values of an input image supplied to the deblocking filter are referred to across a plurality of macro blocks within the image when determining for one of a vertical boundary and a horizontal boundary whether filtering is needed. The result is to relieve the dependency of processes between macro blocks or coding units. Therefore, it is possible to parallelize filtering need determination processes across the plurality of macro blocks or all macro blocks within an image if the parallelism is maximized. 
     According to the second working example, determinations for vertical boundaries and horizontal boundaries of each block whether filtering is needed are made without waiting for applying deblocking filters to the other blocks in the macro block to which the block belongs. This relieves the dependency of processes between a vertical boundary and a horizontal boundary in a macro block or a coding unit. Accordingly, it is possible to parallelize filtering need determination processes on vertical boundaries and horizontal boundaries in a macro block. 
     According to the third working example, filtering processes for vertical boundaries and horizontal boundaries filter pixels input to the deblocking filter. This configuration can parallelize filtering processes for vertical boundaries and horizontal boundaries each other. This can further accelerate processes of the deblocking filter. An output pixel value is calculated based on two filter outputs in terms of a pixel updated by two filtering processes performed in parallel. Parallelizing two filtering processes can also appropriately reduce block distortion appearing on the vertical boundary and the horizontal boundary. An output pixel value can be calculated as a weighted average of two filter outputs. This can allow the deblocking filter to more effectively eliminate the block distortion and further improve the image quality. 
     The specification has mainly described examples where filtering processes for vertical boundaries precedes filtering processes for horizontal boundaries. In addition, the above-described effects of the technology according to the disclosure are equally available to a case where filtering processes for horizontal boundaries precede filtering processes for vertical boundaries. The deblocking filter processing unit or the macro block may be sized otherwise than described in the specification. An available technique may omit the filtering need determination processes and parallelize application of the deblocking filter to vertical boundaries and horizontal boundaries. 
     A technique of transmitting information used for deblocking filter process parallelization from the encoding side to the decoding side is not limited to the technique of multiplexing the information into the encoded stream header. For example, the information may not be multiplexed into an encoded bit stream but may be transmitted or recorded as separate data associated with the encoded bit stream. The term “association” signifies ensuring possibility of linking an image (or part of an image such as a slice or a block) contained in the bit stream with information corresponding to the image. Namely, the information may be transmitted over a transmission path different from that used for images (or bit streams). The information may be recorded on a recording medium (or a different recording area on the same recording medium) different from that used for images (or bit streams). The information and the image (or bit stream) may be associated with each other based on any units such as multiple frames, one frame, or part of a frame. 
     The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, whilst the present invention is not limited to the above examples, of course. A person skilled in the art may find various alternations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present invention. 
     The specification represents filtering processes for vertical boundaries as “horizontal filtering” and filtering processes for horizontal boundaries as “vertical filtering.” Generally, filtering processes for vertical boundaries uses horizontally positioned filter taps. Filtering processes for horizontal boundaries uses vertically positioned filter taps. For this reason, the above-described nomenclature is used for the filtering processes. 
     REFERENCE SIGNS LIST 
     
         
           10 ,  60  image processing device 
           112 ,  212  first determination section (vertical boundary determination section) 
           116 ,  216  second determination section (horizontal boundary determination section) 
           132  first filtering section (horizontal filtering section) 
           142  second filtering section (vertical filtering section) 
           150  parallelization control section