Patent Publication Number: US-2017365070-A1

Title: Encoding program media, encoding method, encoding apparatus, decoding program media, decoding method, and decoding apparatus

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2016-122973, filed on Jun. 21, 2016, the entire contents of which are incorporated herein by reference. 
     FIELD 
     The embodiments discussed herein are related to an encoding program, an encoding method, an encoding apparatus, a decoding program, a decoding method, and a decoding apparatus. 
     BACKGROUND 
     In streaming a video or the like, part of the stream data is sometimes lost over a network due to packet loss. Packet loss is more likely to occur, for example, over mobile networks for mobile phones and the like, which are prone to variation in communication bandwidth. If the frequency of occurrence of packet loss increases, the receiver of the stream will frequently experience disturbance of the reproduced video. Further, in a case where encoding using inter-frame prediction is performed, not only the frame experiencing packet loss but also other frames used by this frame for the inter-frame prediction are affected. 
     As techniques for handling such packet loss, there have been ones as below, For example, there has been a technique in which a key frame such as an intra picture (I picture) is regularly inserted so that, when packet loss occurs in the middle, the frame experiencing the packet loss may be restored using the key frame. There has also been a technique in which, when packet loss occurs, that packet is retransmitted and the retransmitted packet is used to reconstruct the data stream, which is then decoded and displayed, Besides these, there has also been a technique in which a piece of moving-image data is encoded using a plurality of compression ratios, and the pieces of encoded data with the lowest compression ratio and therefore the highest image quality are selected from the pieces of encoded data properly received by the reception side and then decoded on a frame-to-frame basis or a packet-to-packet basis. Related techniques are disclosed in, for example, Japanese Laid-open Patent Publication No. 2001-339722 and International Publication Pamphlet No. WO 2003/092295. 
     However, the above techniques might not be able to suppress variation in image quality resulting from packet loss. For example, in the case of inserting key frames, the occurrence of packet loss in a key frame makes it impossible to suppress decrease in the image, quality of the video until the, next key frame is received. Also, in the case of retransmitting a packet, the reception side has to wait until the packet is retransmitted and therefore experiences a transfer delay. In this case too, the image quality is deteriorated since the reproduction of the retransmitted frame is delayed by the transfer delay. Further, in the case of encoding a piece of moving-image data by using a plurality of compression ratios, it is possible that a piece of moving-image data with frames at different compression ratios might be decoded, In this case, a video with low image quality is reproduced when packet loss is present and a video with high image quality is reproduced when no packet loss is present, thereby causing variation in image quality between frames. 
     According to one aspect, an object of the embodiments is to provide an encoding program, an encoding method, an encoding apparatus, a decoding program, a decoding method, and a decoding apparatus which are capable of suppressing variation in image quality resulting from packet loss. 
     SUMMARY 
     According, to an aspect of the invention, an encoding method includes; acquiring a first image; separating the first image into a plurality of second images by extracting a pixel in the first image after every predetermined number of pixels in each of horizontal and vertical directions of the first image; and encoding each of the separated second images. By transmitting those pieces of encoded data, even if a packet loss occurs in one of the second images, the missing pixel can be re-generated based on corresponding neighboring pixels in other second images. 
     The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims. 
     It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a block diagram illustrating the functional configurations of apparatuses included in a transfer system according to embodiment 1; 
         FIG. 2  is a diagram illustrating an example of a transmission sequence; 
         FIG. 3  is a diagram illustrating an example of a reception sequence; 
         FIG. 4  is a diagram illustrating an example of a unit of encoding; 
         FIG. 5  is a diagram illustrating an example of a unit of encoding; 
         FIG. 6  is a flowchart illustrating the procedure of an encoding process according to embodiment 1; 
         FIG. 7  is a flowchart illustrating the procedure of a decoding process according to embodiment 1; and 
         FIG. 8  is a diagram illustrating an example hardware configuration of a computer that executes an encoding program according to embodiment 1 and embodiment 2. 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     Hereinafter, an encoding program, an encoding method, an encoding apparatus, a decoding program, a decoding method, and a decoding apparatus according to the present application will be described with reference to the accompanying drawings. It is to be noted that the following embodiments do not limit the disclosed technique. Moreover, the embodiments may be combined as appropriate as long as the contents of processes do not contract each other. 
     Embodiment 1 
       FIG. 1  is a block diagram illustrating the functional configurations of apparatuses included in a transfer system according to embodiment 1. A transfer system  1  illustrated in  FIG. 1  is configured to provide a transfer service for transferring video data streams from a transmission apparatus  10  to a reception apparatus  30 . 
     As illustrated in  FIG. 1 , the transfer system  1  includes the transmission apparatus  10  and the reception apparatus  30 . The transmission apparatus  10  and the reception apparatus  30  are connected by a network  2 . While the network  2  may be constructed using any communication network, it may also partly include a mobile communication network, or a mobile network. For example, one or both of the transmission apparatus  10  and the reception apparatus  30  may be connected to the network  2  through a base station or the like. 
     The transmission apparatus  10  is a computer from which a video is transmitted, and the reception apparatus  30  at which the video is received. The transmission apparatus  10  and the reception apparatus  30  may each be implemented using any computer such as an embedded microcomputer, a general-purpose computer, or a workstation. Besides these, mobile communication terminals such as mobile phones including smartphones and personal handy-phone systems (PHS) and also mobile terminals in general including slates and tablets may be employed as the transmission apparatus  10  and the reception apparatus  30 . 
     Thus, any mode of implementation and any mode of connection may be employed for the transmission apparatus  10  and the reception apparatus  30  in accordance with the content of the above-mentioned transfer service. Examples of the transfer service include a video surveillance service for security, rivers, or the like, a video transfer service for broadcasting, and a maintenance and management service for roads, bridges, or the like. However, these are mere examples, and the range of application is not limited to these examples. 
     Here, as part of the above transfer service, the transmission apparatus  10  provides an encoding service involving separating a first image to be transferred into a plurality of second images by extracting a pixel in the first image after every predetermined number of pixels in each of the horizontal and vertical directions of the first image, and encoding each second image. In the following, an original image of a video stream will be presented as an example of the “first image”, and sub-samples obtained by separating the original image will be presented as an example of the “second images”. 
       FIG. 2  is a diagram illustrating an example of a transmission sequence.  FIG. 2  illustrates one original image of a video to be streamed from the transmission apparatus  10  to the reception apparatus  30  and exemplarily illustrates a case where the original image is separated into two sub-samples. As illustrated in  FIG. 2 , the transmission apparatus  10  extracts a pixel in the original image after every one pixel in each of the horizontal and vertical directions of the original image. Thus, among the pixels in the original image illustrated in  FIG. 2 , the pixels illustrated as the white circles are extracted and also the black-circle pixels, which remain as a result of extracting the white-circle pixels, are extracted as well. Consequently, the original image is separated into a sub-sample A and a sub-sample B with a ½ resolution in both the horizontal and vertical directions. Thus, by performing the above separation upon every input of an original-image frame, a video sequence of sub-samples A and a video sequence of sub-samples B are obtained from a video sequence of original images. 
     Thereafter, the transmission apparatus  10  performs video compression encoding on each of the sub-samples obtained by the above separation. Consequently, from the video sequence of sub-samples A, encoded data a of each frame is obtained and, from the video sequence of sub-samples B, encoded data b of each frame is obtained. 
     While the encoded data a and the encoded data b may be transmitted using the same transmission method, they may be transmitted using different transmission methods in view of achieving both real-time performance and improved image quality. For example, the transmission apparatus  10  transmits a stream of encoded data a and a stream of encoded data b to the reception apparatus  30  with a protocol such as the Realtime Transport Protocol (RTP). Here, the transmission apparatus  10  additionally uses at least one of forward error correction (FEC) and retransmission control using Automatic Repeat-reQuest (ARQ) for the encoded data a. Further, the transmission apparatus  10  may send the streams of encoded data a and encoded data b at the same timing to the network  2  but may send them at different timings to the network  2 . For example, the transmission apparatus  10  may start sending the encoded data a, which is to be transferred with FEC or the retransmission control, to the network  2  and then start sending the encoded data b, which is to be transferred with neither FEC nor the retransmission control, to the network  2  with a predetermined time delay. 
     With such an encoding service, the reception apparatus  30  may provide a decoding service as below. Specifically, in a case where packet loss occurs in any of sub-samples sent from the transmission apparatus  10  to the network  2 , the reception apparatus  30  may restore the original image after performing interpolation for the missing part of the sub-sample experiencing the packet loss by using the pixels of the region corresponding to the missing part on a sub-sample experiencing no packet loss. 
       FIG. 3  is a diagram illustrating an example of a reception sequence.  FIG. 3  illustrates a case where the reception apparatus  30  receives the encoded data a and the encoded data b illustrated in  FIG. 2  and illustrates an example where packet loss has occurred in part of the encoded data b. As illustrated in  FIG. 3 , upon receipt of the encoded data a and the encoded data b from the transmission apparatus  10 , the reception apparatus  30  performs decoding for each sub-sample. Consequently, the encoded data a is decoded into the sub-sample A, and the encoded data b is decoded into the sub-sample B. 
     Here, as illustrated in  FIG. 3 , due to the packet loss occurring in part of the encoded data b, the sub-sample B is missing the pixel values of the pixels in the part surrounded by a dotted line in  FIG. 3 . Hereinafter, a pixel missing its pixel value will also be referred to as a “missing pixel” and a collection of missing pixels will also be referred to as a “missing part”. When a missing part is present in the sub-sample B as described above, the reception apparatus  30  uses, in the interpolation, the pixel values of the pixels in the region corresponding to the missing part of the sub-sample B on the sub-sample A, which experienced no packet loss. 
     Specifically, the sub-sample A and the sub-sample B are collections of pixels obtained by alternately extracting one pixel in the original image in the horizontal and vertical directions thereof. Thus, in a situation where a missing part is present in the sub-sample B but no missing part is present in the sub-sample A, the pixels in the original image on the left, right, upper, and lower sides of each missing pixel of the sub-sample B are pixels of the sub-sample A. Thus, even when packet loss occurs in a sub-sample, the possibility of missing the pixel values of pixels sitting adjacent to each other on the original image is reduced. This enhances the accuracy of interpolation using a pixel(s) near a missing pixel, for example, the pixels on the left, right, upper, and lower sides of the missing pixel. The reception apparatus  30  then unites the sub-sample A and the sub-sample B to thereby restore the original image. 
     [Configuration of Transmission Apparatus  10 ] 
     As illustrated in  FIG. 1 , the transmission apparatus  10  includes an acquisition part  11 , a separation part  12 , a first encoding part  13 - 1  to an M-th encoding part  13 -M, and a transmission processing part  14 . It is needless to say that the transmission apparatus  10  may include functional parts which the computer implemented as the transmission apparatus  10  is equipped with as standard features, for example, a communication interface and the like besides the functional parts illustrated in  FIG. 1 . 
     The acquisition part  11  is a processing part that acquires a first image. In one embodiment, the acquisition part  11  acquires a live video captured by a camera not illustrated as original images. In another example, the acquisition part  11  may acquire original images from an auxiliary storage device such as a hard disk drive (HDD) or an optical disc or a removable medium such as a memory card or a universal serial bus (USB) memory that stores videos therein. In still another example, the acquisition part  11  may acquire original images by receiving them from an external device through a network. As described above, how the transmission apparatus  10  acquires a video sequence of original images may be any way and is not limited to a particular way. 
     The separation part  12  is a processing part that separates a first image into a plurality of second images. In one embodiment, each time the acquisition part  11  acquires an original-image frame, the separation part  12  separates the original image into a plurality of sub-samples by repeating a process of extracting a pixel in the original image after every predetermined number of pixels in each of the horizontal and vertical directions of the original image. For example, when the number of divisions of the original image is 2, the separation part  12  extracts pixels in the original image after every one pixel (or every other pixel) in each of the horizontal lines, while the extract starting position is shifted for one pixel in every other horizontal line, as a first divided image. Then, the separation part  12  extracts the remaining pixels in the original image as a second divided image. Consequently, one original image is separated into M sub-samples. 
     When the number of divisions of the original image is 4, the separation part  12  extracts pixels in the original image every other pixel in every other horizontal line as a first divided image, extracts the remaining pixels in those every other horizontal lines as a second divided image, extracts pixels every other pixel in every remaining horizontal lines as a third divided image, and extracts the remaining pixels in the every remaining horizontal lines as a fourth divided image. To generalize this, when the number of divisions is M, the separation part  12  extracts a pixel from the original image after every (square root of M)−1 pixels in after every (square root of M)−1 horizontal line, and repeat this process, with shifting the extract starting position for one pixel for both of the horizontal and the vertical directions, for M times in total. (Please note that the value M here should be a square of an integer of 2 or more, For other value of M, some specific process will be required, depending on the specific value of M.) Consequently, one original image is separated into M sub-samples. 
     The following assumes, as a mere example, a case where the process of extracting sub-samples and the processing of encoding the sub-samples are performed in parallel and the sub-samples are inputted into the first encoding part  13 - 1  to the M-th encoding part  13 -M, respectively. Specifically, for each single sub-sample, extracted pixels are sequentially inputted into the encoding part before the completion of extraction of all the pixels of the sub-sample. Also, the extraction and encoding of a sub-sample and the extraction and encoding of the other sub-sample(s) may be processed in parallel. 
     The first encoding part  13 - 1  to the M-th encoding part  13 -M are processing parts that encodes respective sub-samples. 
     In one embodiment, the first encoding part  13 - 1  to the M-th encoding part  13 -M carries out reduction of the amount of information by using the difference from a predicted image, removal of high-frequency components via orthogonal transform, entropy encoding, and the like based on a predetermined unit(s) of encoding. For example, when H.264 is employed as the encoding scheme, the unit of encoding is macroblock (MB) which is an image region with a fixed size of 16×16 pixels. Alternatively, when the high efficiency video coding (HEVC) is employed as the encoding scheme, the unit of encoding has a hierarchical quadtree structure and includes coding unit (CU), variable from 8×8 to 64×64 pixels, and prediction unit (PU) or transform unit (TU), 
     Here, the first encoding part  13 - 1  to the M-th encoding part  13 -M share a common encoding mode for use in encoding their encoding-unit blocks on a block-by-block basis. The “encoding mode” here includes whether or not to perform inter-frame prediction, such as “Inter/Intra”. Further, the “encoding mode” includes a reference direction such as predicted frame (P frame) or bi-directional predicted frame (B frame). Furthermore, the “encoding mode” includes a type such as inter- and intra-frame predictions or inter- and intra-field predictions. In addition, the first encoding part  13 - 1  to the M-th encoding part  13 -M also share a common predicted motion vector for use in encoding their encoding-unit blocks on a block-by-block basis. 
     In the case where the first encoding part  13 - 1  to the M-th encoding, part  13 -M share a common encoding mode for their encoding-unit blocks as mentioned above, the sharing may be realized through a procedure as below, For example, after encoding a sub-sample allocated to the first encoding part  13 - 1 , the first encoding part  13 - 1  notifies the second encoding part  13 - 2  of the unit of encoding and the encoding mode. Thereafter, the second encoding part  13 - 2  encodes a sub-sample allocated to the second encoding part  13 - 2  by using the unit of encoding and the encoding mode notified of from the first encoding part  13 - 1 , and then notifies the third encoding part  13 - 3  of the unit of encoding and the encoding mode. This is repeated and lastly the M-th encoding part  13 -M encodes a sub-sample allocated to the M-th encoding part  13 -M by using the unit of encoding and the encoding mode notified of from the (M−1)-th encoding part  13 -(M−1). Alternatively, after encoding the sub-sample allocated to the first encoding part  13 - 1 , the first encoding part  13 - 1  may notify the second encoding part  13 - 2  to the M-th encoding part  13 -M of the unit of encoding and the encoding mode. 
     Further, the first encoding part  13 - 1  to the M-th encoding part  13 -M match the amounts of information of pieces of encoded data in blocks in their sub-samples, the blocks corresponding to each other in position on the original image, and further match each of these amounts of information and the amount of information of a piece of encoded data obtainable by encoding a block in the original image. These are done so that the decoded image will not appear unnatural due to blocks with different image qualities. 
       FIGS. 4 and 5  are diagrams illustrating example units of encoding.  FIGS. 4 and 5  illustrate a case where an original image is separated into two sub-samples A and B, as in  FIG. 2 .  FIG. 4  illustrates encoding-unit blocks  1  to  4  in the original image illustrated in  FIG. 2 .  FIG. 5  illustrates encoding-unit blocks A 1  to A 4  and encoding-unit blocks B 1  to B 4  in the sub-sample A and the sub-sample B illustrated in  FIG. 2 , The amounts of information of pieces of encoded data obtained by encoding the blocks  1 ,  2 ,  3 , and  4  in the original image illustrated in  FIG. 4  will be denoted as “J 1 ”, “J 2 ”, “J 3 ”, and “J 4 ”, respectively, and the sum of these will be denoted as “ΣJ”. Further, the amounts of information of pieces of encoded data obtained by encoding the blocks A 1 , A 2 , A 3 , and A 4  in the sub-sample A illustrated in  FIG. 5  will be denoted as “J A1 ”, “J A2 ”, “J A3 ”, and “J A4 ”, respectively, and the sum of these will be denoted as “ΣJ A ”. Furthermore, the amounts of information of pieces of encoded data obtained by encoding the blocks B 1 , B 2 , B 3 , and B 4  in the sub-sample B illustrated in  FIG. 5  will be denoted as “J B1 ”, “J B2 ”, “J B3 ”, and “J B4 ”, respectively, and the sum of these will be denoted as “ΣJ B ”. 
     Here, the first encoding part  13 - 1  to the M-th encoding part  13 -M encode the encoding-unit blocks in their respective sub-samples so as to satisfy equations (1) to (4) given below. In this way, when the sub-samples are united after decoding, the encoding-unit blocks are less likely to differ in image quality in the united original image. In one example, to obtain matched amounts of information satisfying equations (1) to (4), the amount of information of the piece of encoded data in each encoding-unit block in the original image may be estimated, and the first encoding part  13 - 1  to the M-th encoding part  13 -M may then perform encoding based on the estimation. 
       J A1 /ΣJ A ≅J B1 /ΣJ B ≅J 1 /ΣJ   (1)
 
       J A2 /ΣJ A  ≅J B2 /ΣJ B ≅J 2 /ΣJ   (2)
 
       J A3 /ΣJ A ≅J B3 /ΣJ B ≅J 3 /ΣJ   (3)
 
       J A4 /ΣJ A ≅J B4 /ΣJ B ≅J 4 /ΣJ   (4)
 
     The transmission processing part  14  is a processing part that transmits pieces of encoded data of sub-samples. 
     In one embodiment, the transmission processing part  14  transmits pieces of encoded data obtained by the first encoding part  13 - 1  to the M-th encoding part  13 -M by encoding respective sub-samples to the reception apparatus  30 . In doing so, the transmission processing part  14  may transmit the pieces of encoded data using the same transmission method but may transmit them using different transmission methods in view of achieving both real-time performance and improved image quality. For example, in the case of transferring encoded data a and encoded data b, as illustrated in  FIG. 2 , the transmission processing part  14  transmits a stream of encoded data a and a stream of encoded data b to the reception apparatus  30  by using a protocol such as RTP. In doing so, the transmission processing part  14  additionally uses at least one of FEC and retransmission control using ARQ for the encoded data a. Further, the transmission processing part  14  may send the streams of encoded data a and encoded data b at the same timing to the network  2  but may send them at different timings to the network  2 . For example, the transmission processing part  14  may start sending the encoded data a, which is to be transferred with FEC or the retransmission control, to the network  2  and then start sending the encoded data b, which is to be transferred with neither FEC nor the retransmission control, to the network  2 . Here, in one example, in the case where an original image is separated into a plurality of sub-samples, the first half of the pieces of encoded data to be outputted from the first encoding part  13 - 1  to the M-th encoding part  13 -M may be transmitted with a combination of RTP and FEC or the retransmission control, and the last half may be transmitted with RTP. 
       FIG. 1  has exemplarily illustrated the case where sub-samples are encoded in parallel. Sub-samples may be sequentially encoded and transferred. In this case, it is more efficient to firstly encode the sub-sample to be transmitted with the combination of RTP and FEC or the retransmission control. 
     Also, the above-described processing parts such as the acquisition part  11 , the separation part  12 , the first encoding part  13 - 1  to the M-th encoding part  13 -M, and the transmission processing part  14  may be implemented as below. For example, the acquisition part  11 , the separation part  12 , the first encoding part  13 - 1  to the M-th encoding part  13 -M, and the transmission processing part  14  may be implemented by causing a central processing device such as a central processing unit (CPU) to expand and execute a process that functions similarly to these processing parts on a memory. These processing parts may not necessarily be run by a central processing device but may be run by a micro-processing unit (MPU). Alternatively, the above-described processing parts may be implemented using hard-wired logics. 
     Also, in one example, various semiconductor memory elements such as a random access memory (RAM) and a flash memory are employable as a main storage device to be used by each of the above-described processing parts. Meanwhile, the storage device to be referred to by each of the above-described processing parts may not necessarily be a main storage device but may be an auxiliary storage device. In this case, an HDD, an optical disc, a solid state drive (SSD), or the like may be employed. 
     [Configuration of Reception Apparatus  30 ] 
     As illustrated in  FIG. 1 , the reception apparatus  30  includes a reception processing part  31 , a first decoding part  33 - 1  to an M-th decoding part  33 -M, an interpolation part  34 , and a uniting part  35 . It is needless to say that the reception apparatus  30  may include functional parts which the computer implemented as the reception apparatus  30  is equipped with as standard features, for example, a communication interface and the like besides the functional parts illustrated in  FIG. 1 . 
     The reception processing part  31  is a processing part that receives pieces of encoded data of sub-samples. 
     In one embodiment, each time video packets of pieces of encoded data are received through the network  2 , the reception processing part  31  refers to header information in the video packets and inputs the pieces of encoded data into the corresponding decoders for the sub-samples of the pieces of encoded data, namely, the first decoding part  33 - 1  to the M-th decoding part  33 -M Here, the reception processing part  31  may change a threshold for determining timeout for video packets, in accordance with a command inputted from an input unit or external device not illustrated. For example, when a display priority mode is selected, the reception processing part  31  may set the timeout determining threshold shorter than when an image-quality priority mode is selected. On the other hand, when the image-quality priority mode is selected, the reception processing part  31  may set the timeout determining threshold longer than when the display priority mode is selected, In this case, the reception processing part  31  may permit retransmission control and use retransmitted packets to reconstruct the pieces of encoded data of sub-samples. 
     The first decoding part  33 - 1  to the M-th decoding part  33 -M are processing parts that perform decoding for respective sub-samples. 
     In one embodiment, the first decoding part  33 - 1  to the M-th decoding part  33 -M decode the pieces of encoded data of encoding-unit blocks on a block-by-block basis. When all the blocks in a single frame are decoded, that sub-sample is decoded. 
     The interpolation part  34  is a processing part that performs interpolation for the pixel values of missing pixels. 
     In one embodiment, when packet loss is detected upon timeout or the like, the interpolation part  34  identifies the positions of the missing pixels on the sub-sample from the sequence number of the packet detected to be lost and the positions of the pixels in the preceding and following received packets. Then, from among the sub-samples decoded by the first decoding part  33 - 1  to the M-th decoding part  33 -M, the interpolation part  34  extracts the other sub-samples which include no missing pixels at the same positions as the identified missing pixels. After this, the interpolation part  34  performs interpolation for the pixel values of the missing pixels by using the pixel values of pixels on the extracted sub-samples near the missing pixels, specifically, pixels located at nearby positions on the original image such as the upper, lower, right, and left sides and the corners of the missing pixels. For example, the interpolation part  34  may set the pixel value of each missing pixel simply to, the pixel value of the neighboring pixel on one of the upper, lower, left, and right sides of the missing pixel or on one of the upper left, lower left, upper right, and lower right corners of the missing pixel. Alternatively, the interpolation part  34  may set the pixel value of each missing pixel to a representative value such as an average or middle value of the two pixels on the upper and lower sides or the left and right sides of the missing pixel, the four pixels on the upper, lower, left, and right sides of the missing pixel, or the eight pixels including the upper, lower, left, and right pixels plus those on the upper left, lower left, upper right, and lower right corners of the missing pixel. Besides these, the interpolation part  34  may use a method such as bilinear interpolation or a bicubic interpolation. 
     The uniting part  35  is a processing part that unites sub-samples. 
     In one embodiment, if no packet loss has not been detected, the uniting part  35  unites the sub-samples decoded by the first decoding part  33 - 1  to the M-th decoding part  33 -M. On the other hand, if packet loss has been detected, the uniting part  35  unites, among the sub-samples decoded by the first decoding part  33 - 1  to the M-th decoding part  33 -M, the sub-samples from which no packet loss has been detected and the sub-sample which has undergone the interpolation by the interpolation part  34  for the pixel values of the missing pixels. Consequently, the original image is restored from the sub-samples. The original image thus obtained may be outputted to any output destination not illustrated, for example, a moving-image record part, a moving-image reproduction processing part, a processing part that identifies a monitoring target in a moving image, a processing part that detects a moving object in a moving image, or the like. 
     Note that the above-described processing parts such as the reception processing part  31 , the first decoding part  33 - 1  to the M-th decoding part  33 -M, the interpolation part  34 , and the uniting part  35  may be implemented as below. For example, the reception processing part  31 , the first decoding part  33 - 1  to the M-th decoding part  33 -M, the interpolation part  34 , and the uniting part  35  may be implemented by causing a central processing device such as a CPU to expand and execute a process that functions similarly to these processing, parts on a memory. These processing parts may not necessarily be run by a central processing device but may be run by an MPU. Alternatively, the above-described processing parts may be implemented using hard-wired logics. 
     Also, in one example, various semiconductor memory elements such as a RAM and a flash memory are employable as a main storage device to be used by each of the above-described processing parts. Meanwhile, the storage device to be referred to by each of the above-described processing parts may not necessarily be a main storage device but may be an auxiliary storage device. In this case, an HDD, an optical disc, an SSD, or the like may be employed. 
     [Procedure of Processing] 
     Next, the procedure of processing by the transfer system  1  according to this embodiment will be described. Here, the description will be given of (1) the encoding process executed by the transmission apparatus  10  and then (2) the decoding process executed by the reception apparatus  30 . 
     (1) Encoding Process 
       FIG. 6  is a flowchart illustrating the procedure of the encoding process according to embodiment 1. This process is executed each time the acquisition part  11  acquires an original-image frame. As illustrated in  FIG. 6 , the separation part  12  initializes the value of a counter m that counts the number of divisions of the original image to 1 (step S 101 ). Then, the separation part  12  extracts a pixel in the original image acquired by the acquisition part  11  after every predetermined number of pixels in each of the horizontal and vertical directions of the original image (step S 102 ). Consequently, one sub-sample is separated from the original image. 
     Then, the separation part  12  increments the value of the counter m by 1 and repeats the process of step S 102  until the value of the counter m exceeds the number M of divisions (No in step S 103 ). 
     Then if the value of the counter m exceeds the number M of divisions (Yes in step S 103 ), the first encoding, part  13 - 1  to the M-th encoding part  13 -M encode in parallel the sub-samples obtained by repeating step S 102 . 
     Specifically, the first encoding part  13 - 1  initializes a counter i that counts the encoding-unit blocks in the sub-sample A to “1” (step S 104 A). Then, the first encoding part  13 - 1  encodes a block in the sub-sample A (step S 105 A). In doing so, the first encoding part  13 - 1  notifies the second encoding part  13 - 2  of the encoding mode used for the encoding in step S 105 A and the amount of information of the piece of encoded data obtained in step S 105 A. The first encoding part  13 - 1  then increments the value of the counter i by 1 and repeats the process of step S 105 A until the value of the counter i exceeds the total number N of encoding-unit blocks (No in step S 106 A). Then if the value of the counter i exceeds the total number N of encoding-unit blocks (Yes in step S 106 A), the transmission processing part  14  transmits the pieces of encoded data of the sub-sample A, obtained via the encoding by the first encoding part  13 - 1 , to the reception apparatus  30  (step S 107 A). 
     Likewise, the second encoding part  13 - 2  initializes a counter i that counts the encoding-unit blocks in the sub-sample B to “1” (step S 104 B). Then, the second encoding part  13 - 2  encodes a block in the sub-sample B in accordance with the amount of information and the encoding mode notified of from the first encoding part  13 - 1  (step S 105 B). In doing so, the second encoding part  13 - 2  notifies the third encoding part  13 - 3  of the encoding mode used for the encoding in step S 105 B and the amount of information of the piece of encoded data obtained in step S 105 B. The second encoding part  13 - 2  then increments the value of the counter i by 1 and repeats the process of step S 105 B until the value of the counter i exceeds the total number N of encoding-unit blocks (No in step S 106 B). Then if the value of the counter i exceeds the total number N of encoding-unit blocks (Yes in step S 106 B), the transmission processing part  14  transmits the pieces of encoded data of the sub-sample B, obtained via the encoding by the second encoding part  13 - 2 , to the reception apparatus  30  (step S 107 B). 
     In the case of the M-th encoding part  13 -M, which is the last encoding part, the M-th encoding part  13 -M initializes a counter, i that counts the encoding-unit blocks in the sub-sample M to “1” (step S 104 M). Then, the M-th encoding part  13 -M encodes a block in the sub-sample M in accordance with the amount of information and the encoding mode notified of from the (M−1)-th encoding part  13 -(M−1) (step S 105 M). The M-th encoding part  13 -M then increments the value of the counter i by 1 and repeats the process of step S 105 M until the value of the counter i exceeds the total number N of encoding-unit blocks (No in step S 106 M). Then if the value of the counter i exceeds the total number N of encoding-unit blocks (Yes in step S 106 M), the transmission processing part  14  transmits the pieces of encoded data of the sub-sample M, obtained via the encoding by the M-th encoding part  13 -M, to the reception apparatus  30  (step S 107 M). 
     (2) Decoding Process 
       FIG. 7  is a flowchart illustrating the procedure of the decoding process according to embodiment 1. In one example, this process is executed when pieces of encoded data of sub-samples are received. As illustrated in  FIG. 7 , the pieces of encoded data are decoded in parallel for the respective sub-samples. 
     Specifically, when the reception processing part  31  receives the pieces of encoded data of the sub-sample allocated to the first decoding part  33 - 1  (step S 301 A), the first decoding part  33 - 1  initializes a counter j that counts the blocks in the sub-sample each defined as a unit of encoding during the encoding of the sub-sample, to “1” (step S 302 A). 
     Then, the first decoding part  33 - 1  decodes the piece of encoded data of a block in the sub-sample allocated to the first decoding part  33 - 1  (step S 303 A). Here, if a missing pixel is present in the block (Yes in step S 304 A), the interpolation part  34  performs interpolation for the pixel value of the missing pixel by referring to a neighboring pixel(s) in that block with no missing pixel in each of the sub-samples whose encoded data have been decoded by the decoding parts other than the first decoding part  33 - 1  (step S 305 A). 
     Then, the first decoding part  33 - 1  increments the value of the counter j by 1 and repeats the processes of steps S 303 A to S 305 A until the value of the counter j exceeds the total number N of encoding-unit blocks (No in step S 306 A). 
     The second decoding part  33 - 2  executes its processes as below in parallel with these processes of steps S 301 A to S 306 A. Specifically, when the reception processing part  31  receives the pieces of encoded data of the sub-sample allocated to the second decoding part  33 - 2  (step S 301 B), the second decoding part  33 - 2  initializes a counter j that counts the blocks in the sub-sample each defined as a unit of encoding during the encoding of the sub-sample, to “1” (step S 302 B). 
     Then, the second decoding part  33 - 2  decodes the piece of encoded data of a block in the sub-sample allocated to the second decoding part  33 - 2  (step S 303 B). Here, if a missing pixel is present in the block (Yes in step S 304 B), the interpolation part  34  performs interpolation for the pixel value of the missing pixel by referring to a neighboring pixel(s) in that block with no missing pixel in each of the sub-samples whose encoded data have been decoded by the decoding parts other than the second decoding part  33 - 2  (step S 305 B). 
     Then, the second decoding part  33 - 2  increments the value of the counter j by 1 and repeats the processes of steps S 303 B to S 305 B until the value of the counter exceeds e total number N of encoding-unit blocks (No in step S 306 B). 
     Likewise, when the reception processing part  31  receives the pieces of encoded data of the sub-sample allocated to the M-th decoding part  33 -M, which is the last decoding part (step S 301 M), the M-th decoding part  33 -M initializes a counter j that counts the blocks in the sub-sample each defined as a unit of encoding during the encoding of the sub-sample, to “1” (step S 302 M). 
     Then, the M-th decoding part  33 -M decodes the piece of encoded data of a block in the sub-sample allocated to the M-th decoding part  33 -M (step S 303 M). Here, if a missing pixel is present in the block (Yes in step S 304 M), the interpolation part  34  performs interpolation for the pixel value of the missing pixel by referring to a neighboring pixel(s) in that block with no missing pixel in each of the sub-samples whose encoded data have been decoded by the decoding parts other than the M-th decoding part  33 -M (step S 305 M). 
     Then, the M-th decoding part  33 -M increments the value of the counter j by 1 and repeats the processes of steps S 303 M to S 305 M until the value of the counter j exceeds the total number N of encoding-unit blocks (No in step S 306 M). 
     Then, in the case where the value of the counter j exceeds the total number N of encoding-unit blocks, in each of steps S 306 A, S 306 B, . . . , and S 306 M, it represents that the sub-samples have been decoded. In this case, if no packet loss has been detected, the uniting part  35  unites the sub-samples decoded by the first decoding part  33 - 1  to the M-th decoding part  33 -M. On the other hand, if packet loss has been detected, the uniting part  35  unites, among the sub-samples decoded by the first decoding part  33 - 1  to the M-th decoding part  33 -M, each sub-sample from which no packet loss has been detected and each sub-sample which has undergone the interpolation by the interpolation part  34  for the pixel values of missing pixels. Consequently, the original image is restored (step S 307 ) The process is then terminated. 
     [One Aspect of Advantageous Effect] 
     As described thus far, the transmission apparatus  10  according to this embodiment separates a first image to be transferred into a plurality of second images by extracting a pixel in the first image after every predetermined number of pixels in each of the horizontal and vertical directions of the first image, and encodes each of the second images. In this way, if packet loss occurs in any of these sub-samples sent to the network  2  from the transmission apparatus  10 , the reception apparatus  30  restores the original image after performing interpolation for the missing part of the sub-sample experiencing the packet loss by using pixels in the region corresponding to the missing part on the sub-sample not experiencing the packet loss. Thus, even if packet loss occurs in a sub-sample, performing the above encoding reduces the possibility that pixels sitting adjacent to each other on the original image miss their pixel values. This enhances the accuracy of the interpolation using a pixel(s) near a missing pixel, for example, the pixels on the left, right, upper, and lower sides of the missing pixel. Accordingly, the transmission apparatus  10  according to this embodiment may suppress variation in image quality resulting from packet loss. 
     Embodiment 2 
     An embodiment has been described above regarding the disclosed apparatuses, However, various other embodiments other than the above embodiment may be carried out. In view of this, another embodiment will be described below. 
     [Use of Different Transmission Methods] 
     In embodiment 1, the case has been described in which different transmission methods are used for two sub-samples—a combination of RTP and FEC or retransmission control is used for one and RTP is used for the other. However, different protocols may be used for the two-samples, for example. In one example, the encoded data a may be transferred using the Transmission Control Protocol/Internet Protocol (TCP-IP), and the encoded data b may be transferred using the User Datagram Protocol (UDP). 
     [Spreading and Integration] 
     Also, the illustrated apparatuses&#39; constituent elements may not necessarily be physically configured as illustrated. In other words, the specific configurations of the apparatuses regarding spreading and integration of their constituent elements are not limited to the illustrated ones. All or some of the constituent elements may be functionally or physically spread or integrated based on any unit in accordance with various types of loads, use conditions, and the like. For example, the acquisition part  11 , the separation part  12 , the first encoding part  13 - 1  to the M-th encoding part  13 -M, or the transmission processing part  14  of the transmission apparatus  10  may be connected as an external device of the transmission apparatus  10  through a network. The reception processing part  31 , the first decoding part  33 - 1  to the M-th decoding part  33 -M, the interpolation part  34 , or the uniting part  35  of the reception apparatus  30  may be connected as an external device of the reception apparatus  30  through a network. The acquisition part  11 , the separation part  12 , the first encoding part  13 - 1  to the M-th encoding part  13 -M, and the transmission processing part  14  of the transmission apparatus  10  may be included in other devices, and these other devices may be connected to a network and operate in cooperation with each other to implement the above-described functions of the transmission apparatus  10 . The reception processing part  31 , the first decoding part  33 - 1  to the M-th decoding part  33 -M, the interpolation part  34 , and the uniting part  35  of the reception apparatus  30  may be included in other devices, and these other devices may be connected to a network and operate in cooperation with each other to implement the above-described functions of the reception apparatus  30 . 
     [Encoding Program] 
     Also, the various processes described in the above embodiment may be implemented by executing a program prepared in advance on a computer such as a personal computer or a workstation. Now, description will be given below of an example of a computer that executes an encoding program having functions similar to those in the above embodiment. 
       FIG. 8  is a diagram illustrating an example hardware configuration of a computer that executes an encoding program according to an embodiment. As illustrated in  FIG. 8 , a computer  100  includes an operation unit  110   a,  a speaker  110   b,  a camera  110   c,  a display  120 , and a communication unit  130 . This computer  100  further includes a CPU  150 , a ROM  160 , an HDD  170 , and a RAM  180 . These elements  110  to  180  are connected by a bus  140 . 
     As illustrated in  FIG. 8 , the HDD  170  stores an encoding program  170   a  that functions similarly to the acquisition part  11 , the separation part  12 , the first encoding part  13 - 1  to the M-th encoding part  13 -M, and the transmission processing part  14 , which have been discussed in embodiment 1. This encoding program  170   a  may be integrated or spread as in the constituent elements illustrated in  FIG. 1 , namely, the acquisition part  11 , the separation part  12 , the first encoding part  13 - 1  to the M-th encoding part  13 -M, and the transmission processing part  14 . Specifically, the HDD  170  may not necessarily store all the pieces of data mentioned in embodiment 1 but may just store pieces of data to be used in the processes. 
     In this environment, the CPU  150  reads the encoding program  170   a  from the HDD  170  and expands it onto the RAM  180 . Consequently, the encoding program  170   a  functions as an encoding process  180   a,  as illustrated in  FIG. 8 . This encoding process  180   a  expands various pieces of data read from the HDD  170  onto a region allocated to the encoding process  180   a  in the storage region of the RAM  180 , and executes various processes by using the various pieces of data thus expanded. Examples of the processes executed by the encoding process  180   a  may include the process illustrated in  FIG. 6  and the like. Note that not all the processing parts illustrated in embodiment 1 may necessarily operate on the CPU  150 . Only those processing parts corresponding to target processes to be executed may be virtually implemented. 
     Note that the encoding program  170   a  may not necessarily be stored in the HDD  170  or the ROM  160  from the beginning. For example, the encoding program  170   a  may be stored in a “portable physical medium” insertable into the computer  100  such as a flexible disk, or a so-called FD, a CD-ROM, a DVD, a magneto-optical disc, or an IC card. The computer  100  may then acquire the encoding program  170   a  from this potable physical medium and execute it. Alternatively, the encoding program  170   a  may be stored in a different computer or a server apparatus connected to the computer  100  by a public line, the Internet, a LAN, a WAN, or the like, and the computer  100  may acquire the encoding program  170   a  from the different computer or the server apparatus and executes it. 
     Also,  FIG. 8  exemplarily illustrates the case where the HDD  170  stores the encoding program  170   a.  It is needless to say that the HDD  170  may store a decoding program that functions similarly to the reception processing part  31 , the first decoding part  33 - 1  to the Meth decoding part  33 -M, the interpolation part  34 , and the uniting part  35 , and a decoding process may be expanded onto the RAM  180  to implement the process illustrated in  FIG. 7  and the like. 
     All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.