Abstract:
The present invention relates to a displaying method for video image coded data which prevents deterioration of image quality of a video image as much as possible, and conceals, from an observer, deteriorated video quality caused due to a transmission error. The displaying method selects target image frames to be decoded and target image frames to be discarded, by performing error determination ( 32 ) in parallel with an error recovery process and other image restoration processes, and decodes ( 33 ) only the selected target image frames. A predetermined display device sequentially displays ( 34 ) only the decoded target image frames while adjusting a frame rate.

Description:
TECHNICAL FIELD 
       [0001]    The present invention relates to a technology for displaying video image coded data distributed through an existing wired or wireless network. More particularly, the present invention relates to a method, device, and communication system capable of concealing a deteriorated image quality caused due to an error in transmitting a video to be displayed on a display device. 
       BACKGROUND ART 
       [0002]    With the development of the high-speed data communication technologies in recent years, large volumes of data such as video image data can be transmitted/received through an existing network to/from users using the network. However, transmission and reception of a large volume of data increases the traffic, allowing some of the network users to occupy the network resource. This situation makes it difficult for all the network users to benefit from the network resource. For this reason, the communications carriers, the service providers and the like limit the bandwidth of each transmission channel and thereby implement the operations for preventing the network resource from being occupied by some of the network users. 
         [0003]    Meanwhile, with the development of the digital signal processing technologies, active studies have been performed on video compression technologies in order to enable more efficient data transmission using a transmission channel of a limited bandwidth. Motion-JPEE, Motion-JPEG2000, H264 and the like are known as the video compression technologies, wherein while each image frame data to be transmitted is compressed (coded), the compressed image frame data is decoded by a receiver, in order to display a video image on a monitor and the like. 
         [0004]    However, a transmission error caused due to deteriorated communication quality and the like occurs frequently in a video image distribution service that uses the network. In such a case, block noise and the like occur in the video on the monitor regenerated by the receiver, causing obvious image deterioration. Patent Documents 1 to 3 described below disclose technologies that prevent the deterioration of image quality as much as possible when regenerating the video image data, by executing a re-transmission process for re-transmitting the video image data from the transmitter side to the receiver side and an error recovery process on video image data that is not received successfully, even when a transmission error occurs in the video image distribution service. 
       CITATION LIST 
     Patent Document 
       [0000]    
       
         Patent Document 1: Japanese Translation of PCT Application No. 2007-519338 
         Patent Document 2: Japanese Patent Application Laid-Open No. 2007-211511 
         Patent Document 3: Japanese Patent Application Publication Laid-Open No. 2008-203597 
       
     
       SUMMARY OF INVENTION 
     Problems that the Invention is to Solve 
       [0008]    The present inventors have examined the conventional technologies for displaying video image coded data, especially the error compensation technologies (also known as error concealment technologies), and as a result, have discovered the following problems. That is, the technologies described in the abovementioned Patent Documents 1 to 3 are primarily intended to prevent deterioration of the quality of a video image to be displayed on a monitor, as much as possible, by executing a re-transmission process, an error recovery process (error correction, compensation by inter-frame prediction, etc.), and other image restoration processes, even when a transmission error occurs when transmitting/receiving video image coded data. However, when the video image cannot be restored after all, the distributed video image with deteriorated image quality is displayed on the monitor and the like. As a result, block noise and the like occur in the video on the monitor, allowing an observer to recognize obvious image deterioration. In particular, when transmitting/receiving the actual video image coded data, the data becomes deteriorated more than one can handle. Therefore, unfortunately, deterioration of the quality of the displayed image is inevitable. 
         [0009]    The present invention has been developed to eliminate the problems described above. It is an object of the present invention to provide a displaying method, device and communication system for video image coded data that can avoid, as much as possible, the occurrence of image quality deterioration upon regeneration of video image data using an image restoration process, and at the same time conceal from an observer a deteriorated image quality of a displayed video that is caused due to a transmission error. 
       Means for Solving the Problems 
       [0010]    It is generally known that, when the quality of video image data with a certain or higher level of frame rate deteriorates, the deterioration of the image quality itself is recognized by an observer more easily than the decrease of the frame rate. The present invention was completed based on such visual characteristics of an observer and is capable of effectively concealing missing data and the like caused due to a transmission error, without allowing the observer to recognize deterioration of quality of a video to be displayed, while executing the re-transmission process, the error recovery process, and other image restoration processes as much as possible. More specifically, the communication system according to the present invention comprises receiving means such as a server that is connected to a network, and receiving means such as a terminal device, and processes video image coded data that is configured by coded data of a plurality of compressed image frames and distributed through predetermined transmission means. Particularly a displaying method that is implemented by the receiving means (the displaying method for video image coded data according to the present invention) has a receiving step of sequentially receiving the coded data of the plurality of image frames to be delivered, a selection step of sequentially selecting image frames to be decoded, from among a plurality of target image frames to be displayed per unit time out of the plurality of received image frames, a decoding step, and a display step. The present invention may be applied alone or in combination with a conventional error concealment technology, when displaying video image coded data. 
         [0011]    The selection step calculates a missing data rate corresponding to missing data caused due to a transmission error (error determination), for each of the coded data of the target image frames that are displayed per unit time and disposed at a predetermined interval, and then selects, based on the obtained calculation result, target image frames to be decoded out of the target image frames. The decoding step decodes only the coded data of the target image frames selected in the selection step. Therefore, the target image frames that are not selected in the selection step (these target image frames are not decoded because the data qualities thereof are deteriorated to the certain level or lower) are discarded. The display step causes a predetermined display device to display the target image frames decoded in the decoding step, while adjusting a frame rate based on the number of target image frames. 
         [0012]    The present invention described above can effectively conceal deteriorated image quality of a displayed video by causing the predetermined display device to display the image frames of a certain quality level or higher while adjusting the frame rate. Furthermore, when the displaying method according to the present invention is used in combination with a conventional error concealment technology, even the target frames that normally are not decoded as a result of the error determination can be decoded within an allowance by performing the error recovery process using the conventional error concealment technology. In this case, the number of video image frames to be discarded can be reduced, further improving the concealment effect of the present invention. 
         [0013]    It is preferred that the displaying method for video image coded data according to the present invention request for re-transmission of missing data included in the received coded data, prior to completion of reception of the coded data of the target image frames. If decodable data can be received by sending the re-transmission request for removing the missing data, then it is not necessary to perform the image restoration process or to determine whether to discard the target image frames. This is because the number of missing data can be eventually reduced as soon as the decodable data are received. Normally, a single image frame is transmitted/received in units of packets; however, the time of “completion of reception of the coded data of the target image frames” may be determined based on, for example, the time when a final packet of a target image frame is received, the time when a first packet of the next image frame is received, or at the time of timeout. Moreover, the coded data that correspond to the missing data requested to be re-transmitted are preferably distributed prior to the distribution of other coded data that are scheduled to be transmitted. This is because when there is a delay in the re-transmission of the coded data that correspond to the missing data requested to be re-transmitted (the normal coded data that can be decoded prior to the distribution thereof), a delay is caused in the decoding process to be performed on the subsequent image frames. 
         [0014]    The selection step in the displaying method for video image coded data according to the present invention may select a target image frame to be decoded out of the target image frames, based on the calculation result of the missing data rate and the continuity of the received data. Even with the coded data having the missing data rate outside the allowance, it is possible to acquire information that is enough to conceal or compensate the missing data by means of other received data of the same frame or inter-frame prediction, as long as the continuity of the received data is kept at a certain level. 
         [0015]    The selection step in the displaying method for video image coded data according to the present invention may also request for re-transmission of at least any of the target image frames that are not the subject of the decoding. Even when the target image frames are to be destroyed, it is sometimes appropriate to issue the re-transmission request, depending on the image characteristics thereof, until the missing data rate falls within the allowance again. 
         [0016]    When adjusting the frame rate in the display step, the displaying method for video image coded data according to the present invention discards only the target image frames that are not selected in the selection step, without changing the time-axial positions of the target image frames to be displayed per unit time. However, continuously discarding the plurality of image frames in the selection step spreads the frame interval between the target image frames to be displayed, allowing the observer to recognize the target image frames being displayed intermittently. When adjusting the frame rate in such a case, the target image frames that are decoded in the decoding step are preferably disposed evenly within a unit time in a manner that the frame interval therebetween is constant. As described above, because the quality of each target image frame to be displayed does not at all or hardly becomes deteriorated by the frame rate adjustment, the deterioration of the quality of the video to be displayed can be effectively concealed by taking advantage of the visual characteristics of the observer. 
         [0017]    The displaying method for video image coded data according to the present invention can detect pixel movement by comparing adjacent target image frames out of the target image frames, and perform, on a space region in the frame data of a target image frame in which the pixel movement is detected, weighting for determining a selection reference value in the selection step. The space region in which the pixel movement is detected is a region that is likely to be recognized by the observer; thus, the target image frames to be displayed are preferably prevented from being discarded as much as possible. For this reason, it is preferred to make a difference between selection reference values such that the selection reference value of the target image frame where the pixel movement is detected is lower than the selection reference value of other target image frame. 
         [0018]    In the displaying method for video image coded data according to the present invention, the coded data of the target image frames received in the receiving step each include a plurality of types of information arranged a priority order. In addition, the coded data of each of the target image frames are distributed sequentially in descending order of priority of the plurality of types of information. When a single image frame is divided into a plurality of transmission packets, highly important information items (with high degrees of contribution to image decoding) are preferentially transmitted, increasing the likelihood of restoring the data. The order of priority is, for example, an order of image quality, resolution, color component, position (space region) and the like, which is determined beforehand by the transmitter side. 
         [0019]    The display device according to the present invention is a device for realizing the displaying method for video image coded data configured as described above, and has, at least, an input/output unit, a memory, a controller, and a display unit. Specifically, the input/output unit imports video image coded data configured by the coded data of the plurality of compressed image frames, through predetermined transmission means. The memory temporarily stores the video image coded data imported through the input/output unit. The controller executes the displaying method for video image coded data configured as described above (the displaying method for video image coded data according to the present invention). The display unit sequentially displays the decoded target image frames in accordance with a frame rate adjusted by the controller. 
         [0020]    The present invention will be more fully understood from the detailed description given hereinbelow and the accompanying drawings, which are given by way of illustration only and are not to be considered as limiting the present invention. 
         [0021]    Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the scope of the invention will be apparent to those skilled in the art from this detailed description. 
       Effects of the Invention 
       [0022]    The displaying method, device and communication system for video image coded data according to the present invention can discard only deteriorated image frames while performing the error recovery process and the like to prevent deterioration of image quality as much as possible when regenerating the video image data, and causes a predetermined display device to sequentially display decoded image frames while adjusting the frame rates. Therefore, the present invention can effectively conceal the deteriorated quality of a displayed video while minimizing the number of deteriorated image frames to be discarded. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0023]      FIG. 1  is a schematic view for showing an example of a system configuration (the communication system according to the present invention) for realizing an embodiment of the displaying method for video image coded data according to the present invention; 
           [0024]      FIG. 2  is a view showing a specific structure of a terminal device (a display device according to the present invention) shown in  FIG. 1 ; 
           [0025]      FIG. 3  is a view for showing a structure of video image coded data to be displayed; 
           [0026]      FIG. 4  is a flowchart for showing a primary step according to the embodiment of the video image coded data displaying method according to the present invention; 
           [0027]      FIG. 5  is a flowchart for showing a destruction determination process in the video image coded data displaying method according to the present invention; 
           [0028]      FIG. 6  is a view for showing a weighting step in the video image coded data displaying method according to the present invention; 
           [0029]      FIG. 7  is a view for showing a first example of frame rate adjustment in the video image coded data displaying method according to the present invention; and 
           [0030]      FIG. 8  is a view for showing a second example of the frame rate adjustment in the video image coded data displaying method according to the present invention. 
       
    
    
     REFERENCE SINGS LIST 
       [0031]      10  . . . network;  20  . . . server;  30  . . . terminal device (display device);  300  . . . information processor;  310 ,  340  . . . input/output unit (I/O);  320  . . . controller; and  400  . . . display monitor. 
       DESCRIPTION OF EMBODIMENTS 
       [0032]    Embodiments of the displaying method, device and communication system for video image coded data according to the present invention are described hereinafter in detail with reference to  FIGS. 1 to 8 . The same reference numerals are used to explain the corresponding elements in the views; thus the overlapping explanations are omitted accordingly. 
         [0033]      FIG. 1  is a schematic view for showing an example of a system configuration (the communication system according to the present invention) for realizing an embodiment of the video image coded data displaying method according to the present invention. In a communication system shown in  FIG. 1 , video image coded data to be displayed in the present invention is distributed from a server  20 , which is video image coded data transmission means, through a network  10  to each terminal device  30  (a terminal A and a terminal B in the view), which is video image coded data receiving means, and is configured by coded data of a plurality of compressed image frames. In the communication system shown in  FIG. 1 , either one of the terminal devices connected to the network  10  functions as a content distribution server, and the other terminal device  30  functions as a reception terminal device (the display device according to the present invention). 
         [0034]    The server  20  has a database  21  (D/B in the view) in which a plurality of types of distribution contents are stored. The video image coded data is generated by compressing video data, which are read from the D/B  21 , with respect to each image frame (coding  22 ). Video image coded data generated in the manner described above (a plurality of divided transmission packets) are distributed from the server  20  to the terminal device  30  (e.g., the terminal A) via the network  10  (transmission  23 ). The server  20  receives a re-transmission request from the terminal A (request reception  24 ) and adjusts an order of transmitting the transmission packets. 
         [0035]    The video image coded data distributed from the server  20  have a previously-determined priority order, i.e., an order of image quality, resolution, color component, and position. The server  20  distributes the video image coded data to the terminal A in descending order of transmission packets (transmission  23 ). 
         [0036]    The terminal A, on the other hand, receives the video image coded data from the server  20  via the network  10  (reception  31 ), and selects and discards target image frames to be displayed per unit time (error determination  32 ). After decoding the selected target image frames (decoding  33 ), the terminal A causes a predetermined display device to sequentially display these target image frames while adjusting a display frame rate (display  34 ). Note that the terminal B performs the same display control as the terminal A. The error determination  32  may be carried out prior to the decoding  33 , during the decoding  33 , or after completion of the decoding  33 . 
         [0037]    Moreover, in order to reduce the number of missing data, the terminal A requests for re-transmission of the missing data included in the received coded data, prior to completion of reception of the coded data of the target image frames (request transmission  35 ). The time of “completion of reception of the coded data of the target image frames” is determined based on, for example, the time when a final packet of a target image frame is received, the time when a first packet of the next image frame is received, or at the time of timeout. Meanwhile, the server  20  distributes the transmission packet that includes the missing data requested to be re-transmitted, before distributing the other transmission packets (transmission  23 ). 
         [0038]      FIG. 2  is a view showing a specific structure of the terminal device  30  shown in  FIG. 1  (the display device according to the present invention, corresponding to both the terminals A and B shown in  FIG. 1 ). The terminal device shown in  FIG. 2  is configured by an information processor  300  and a display monitor  400 . The information processor  300  has an input/output unit  310  (I/O), a controller  320 , a rendering unit  330 , an input/output unit  340 , and a memory  350 . 
         [0039]    Specifically, the I/O  310  is an interface for importing the video image coded data via the network  10 . The I/O  340  is an interface for external input devices such as a keyboard  410  and a mouse (pointing device)  420 . The memory  350  stores the video image coded data imported through the I/O  310 , a program for executing the video image coded data displaying method according to the present invention, and various types of control data. The controller  320  executes the program stored in the memory  350  (the program for executing the video image coded data displaying method according to the present invention). The rendering unit  330  is an image processor that causes the display monitor  400  to sequentially display the image frames decoded by the controller  320 , wherein the display monitor  400  displays, in response to a display control signal from the controller  330 , the image data decoded according to a predetermined frame rate. 
         [0040]    A structure of the video image coded data to be displayed and controlled in the terminal device  30  having the above-mentioned structure is shown in  FIG. 3 .  FIG. 3  is a view for showing a structure of the video image coded data to be displayed. 
         [0041]    As shown in the area (a) of  FIG. 3 , video image data itself is configured by a plurality of image frames f n  (n=0, 1, 2, . . . ) that are disposed on a time axis at a certain interval (frame interval). Video image coded data is obtained by compressing (coding) each of the plurality of image frames f 0  to f n+1 . The frame rate is defined based on the number of image frames to be expressed that are disposed in a unit time T. 
         [0042]    The area (b) of  FIG. 3  shows a structure of frame data of, for example, a Motion-JPEG2000 image frame, which is an image frame f n . The image frame f n  is configured by a plurality of types of hierarchical scalabilities. Color components A, B, C, image qualities A 1 , B 1 , C 1 , resolutions A 2 , B 2 , C 2 , and space regions A 3 , B 3 , C 3  are designated as the scalabilities. In the error determination  32  in the terminal A shown in  FIG. 1 , a weighted missing data rate is set for each of the scalabilities (the color component, image quality component, resolution component, and space region component) of the image frame f n , and then image frames to be decoded are selected with reference to the set missing data rate. The missing data rate here is defined for each scalability as a ratio between a total number of components to be subjected to the error determination and the number of components that are missing due to a transmission error or the like. 
         [0043]    A selection step performed between the error determination  32  and the decoding  33  is described in detail using the flowcharts shown in  FIGS. 4 and 5 . 
         [0044]    First, in this selection step, mainly the error determination is performed on the image frames to be displayed per unit time T while changing the target image frames sequentially (steps ST 440 , ST 441 ). In the flowchart shown in  FIG. 4 , an embedded structure is adopted as an error determination step for each scalability; however, for the sake of explanation, the error determination is executed independently on each target scalability. Therefore, there are no substantial differences between the detail of the following description and the flowchart shown in  FIG. 4 . 
         [0045]    In each of the target image frames, the missing data rate of a target color component is determined (step ST 401 ). The determination in this step ST 401  is performed sequentially on all of the color components that configure the frame data of the image frames to be subjected to the error determination (steps ST 414 , ST 424 ). Meanwhile, data continuity is checked when the calculated missing data rate is equal to or greater than a previously weighted determination value L 1  (step ST 401   a ). When the missing data is less than an acceptable value in the error determination performed on the target color component, or when a certain level of data continuity that is enough to ensure information required for data interpolation is confirmed even when the missing data rate is equal to or greater than the acceptable value, the error determination is performed on the image quality component of each target image frame. On the other hand, as the error determination regarding the target color component of each target image frame, a destruction determination process S 100  is performed when it is determined based on both the missing data rate and the data continuity that the image cannot be restored (J 1 ). 
         [0046]    Missing data rate determination regarding the image quality components (step ST 402 ) is also performed sequentially on all of the image quality components of the frame data configuring the image frames to be subjected to the error determination (step ST 413 , ST 423 ). Meanwhile, the data continuity is checked when the calculated missing data rate is equal to or greater than a previously weighted determination value L 2  (step ST 402   a ). When the missing data is less than an acceptable value in the error determination performed on the target quality component, or when a certain level of data continuity that is enough to ensure information required for data interpolation is confirmed even when the missing data rate is equal to or greater than the acceptable value, the error determination is performed on the resolution component of each target image frame. On the other hand, as the error determination regarding the target image quality component, the destruction determination process S 100  is performed when it is determined based on both the missing data rate and the data continuity that the image cannot be restored (J 2 ). 
         [0047]    Missing data rate determination regarding the resolution components (step ST 403 ) is also performed sequentially on all of the resolution components of the frame data configuring the image frames to be subjected to the error determination (steps ST 412 , ST 422 ). Meanwhile, the data continuity is checked when the calculated missing data rate is equal to or greater than a previously weighted determination value L 3  (step ST 403   a ). When the missing data is less than an acceptable value in the error determination performed on the target resolution component, or when a certain level of data continuity that is enough to ensure information required for data interpolation is confirmed even when the missing data rate is equal to or greater than the acceptable value, the error determination is performed on the space region component of each target image frame. On the other hand, as the error determination regarding the target resolution component, the destruction determination process S 100  is performed when it is determined based on both the missing data rate and the data continuity that the image cannot be restored (J 3 ). 
         [0048]    Missing data rate determination regarding the space region (position) components (step ST 404 ) is also performed sequentially on all of the resolution components of the frame data configuring the image frames to be subjected to the error determination (steps ST 411 , ST 421 ). Meanwhile, the data continuity is checked when the calculated missing data rate is equal to or greater than a previously weighted determination value L 4  (step ST 404   a ). When the missing data is less than an acceptable value in the error determination performed on the target space region component, or when a certain level of data continuity that is enough to ensure information required for data interpolation is confirmed even when the missing data rate is equal to or greater than the acceptable value, the error determination is performed on the space region component of each target image frame. On the other hand, as the error determination regarding the target resolution component, the destruction determination process S 100  is performed when it is determined based on both the missing data rate and the data continuity that the image cannot be restored (J 4 ). 
         [0049]    As shown in  FIG. 5 , the state of the missing data is determined in the destruction determination process S 100  (step ST 501 ). When step ST 501  is executed based on any of the determination results J 1  to J 4 , at least any of the color component, the image quality component, the resolution component, and the space region component satisfies a destruction criterion. However, when the image quality component and the resolution component clear the error determination, then an image of practically sufficient quality can be decoded. In this case, the process is shifted to a junction J 10 . Furthermore, even when it is determined in step ST 501  that the image cannot be restored, sometimes it is better to make a request for re-transmission of the missing data prior to completion of reception of the frames or until each component reaches the acceptable value when each component is not at the acceptable value, depending on the characteristics of each image frame. Step ST 502 , therefore, determines the need for issuance of the re-transmission request (step ST 502 ). When it is determined that the re-transmission request is necessary, request transmission is performed (request transmission  35  in  FIG. 1 ) (step ST 503 ), and the process is shifted to the junction. When it is determined in step ST 502  that the re-transmission request is not necessary, the relevant target image frames are destroyed (step ST 504 ), and the process is shifted to the junction J 10 . 
         [0050]    Only the image frames, which have the missing data rates falling within the allowances (less than the set values L 1 , L 2 , L 3 , L 4 ), have the data continuity confirmed, and satisfy the other conditions in the error determination performed on each scalability, are selected and subsequently decoded (step ST 450 ). 
         [0051]    In the error determination described above, the previously weighted determination values (L 1 , L 2 , L 3 , L 4 ) are set with respect to the scalabilities, but the weighted values can be changed dynamically. For example, as shown in the area (a) of  FIG. 6 , movement of a pixel A may be detected by comparing the continuous image frames f n  and f n−1 , and then weighting for determining a selection reference value in the selection step may be performed on a space region of the frame data of the target image frame f n  in which the pixel movement is detected. 
         [0052]    More specifically, as shown by the flowchart in the area (b) of  FIG. 6 , when it is detected that the pixel A moves by V 1  as a result of comparing the continuous image frames f n  and f n−1 , this distance V 1  is compared with a previously set determination value L 5  (step ST 601 ). When the distance V 1  is equal to or greater than the determination value L 5 , a weighting operation is performed by changing the determination value L 4  that is used when the error determination is performed on the space region configuring the frame data of the target image frame (step ST 602 ). The space region in which the pixel movement is detected is a region that is likely to be recognized by the observer; thus, it is particularly preferred that the target image frames to be displayed be prevented from being discarded as much as possible. Therefore, executing the weighting step shown by the flowchart in the area (b) of  FIG. 6  makes a difference between the determination values (selection reference values) such that the determination value of the target image frame f n  where the movement of the pixel A is detected is lower than the determination values of the other target image frames. 
         [0053]    Besides the error determination described above (error determination performed in the order of the color, quality, resolution, and space components, see  FIG. 4 ), for example, determination of the scalabilities may be performed in a different order as the error determination method. The error determination is not necessarily performed with respect to each scalability; thus, a cumulative value of the missing data rates corresponding to the scalabilities may be obtained, and then the error determination may be performed based on this cumulative missing data rate. In addition, an image quality evaluation index such as PSNR or SSIM may be used. 
         [0054]    Next, a frame rate adjustment method of the video image coded data displaying method according to the present invention is described in detail with reference to  FIGS. 7 and 8 . As described above, the video image coded data displaying method according to the present invention was completed based on the knowledge that the observer is more sensitive to deterioration of image quality of a displayed image than increase/decrease of the frame rate. The various steps of frame adjustment are performed based on fluctuations of the frame interval (occurring simultaneously with fluctuations of the number of image frames that are actually selected from among the image frames to be displayed per unit time T). 
         [0055]    For instance,  FIG. 7  shows an example of frame rate adjustment that is performed when the maximum value of the frame interval obtained when discarding some image frames is relatively small. More specifically, as shown in the area (a) of  FIG. 7 , when the image frames f 1  and f 6  are discarded by the selection step during the execution of display control on the plurality of image frames f 0  to f 10  (frame rate: 11/T) disposed evenly at a frame interval of Δt within the unit time T, the maximum frame interval per unit time T between the image frames f 0 , f 2  to f 5 , f 7  to f 10  to be decoded is 2Δt, which is relatively small. Thus, as shown in the area (b) of  FIG. 7 , the decoded image frames f 0 , f 2  to f 5 , f 7  to f 10  are sequentially displayed on the display monitor  400  (see  FIG. 2 ) (frame rate: 9/T) without the simply discarded image frames f 1  and f 6 . 
         [0056]    On the other hand, as shown in the area (a) of  FIG. 8 , discarding the plurality of continuous image frames increases the maximum frame interval between the image frames to be displayed per unit time T. For instance, in the case of the area (a) of  FIG. 8 , when the image frames f 1  to f 3 , f 6  to f 7  are discarded in the selection step during the execution of display control on the plurality of image frames f 0  to f 10  (frame rate: 11/T) disposed evenly at a frame interval of Δt within the unit time T, the maximum frame interval per unit time T between the image frames f 0 , f 4  to f 5 , f 8  to f 10  to be decoded is 3Δt, which is relatively large. When frame adjustment shown in the area (b) of  FIG. 7  is performed in such a case, the observer recognizes the target image frames being displayed intermittently. For this reason, when the frame interval between the image frames to be decoded is large as shown in the area (a) of  FIG. 8 , it is preferred that the decoded target image frames be disposed evenly within the unit time (frame interval: Δt′), in a manner that the frame interval therebetween is constant (frame rate: 6/T), as shown in the area (b) of  FIG. 8 . Any of the frame rate adjustment steps shown in  FIGS. 7 and 8  adjusts the frame rate only, and, since the quality of each target image frame to be displayed does not at all or hardly becomes deteriorated by the frame rate adjustment, deterioration of the quality of the video to be displayed can effectively be concealed by taking advantage of the visual characteristics of the observer. The frame rate adjustment can be carried out prior to or after the decoding, as long as the frame rate adjustment is performed after the error determination step. 
         [0057]    It is clear from the description of the present invention above that various modifications can be made on the present invention. Such modifications shall not be excluded from the ideas and scope of the present invention, and improvements that will be obvious to those skilled in the art are intended to be included in the scope of the following claims.