# Patent

## Title
Video processing system, video processing method, and program

## Abstract

To make it difficult to acquire an original moving image based on display data. A moving image processing system includes a moving image data acquiring unit for acquiring moving image data, a frame image set specifying unit for specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially, and a modified frame image generating unit for generating, by modifying a key frame image that is based on at least one of the frame images included in the image set, a same number of modified frame images as the frame images included in the image set, wherein an average of pixel values of corresponding pixels of the modified frame images corresponds to a pixel value of a corresponding pixel of the key frame image, and a display control unit for sequentially display the modified frame images generated by modified frame image generating unit, instead of the frame images included in the image set.

## Background
CROSS REFERENCE TO RELATED APPLICATIONS
This application is a National Stage of International Application No. PCT/JP2013/059731 filed on Mar. 29, 2013. The contents of the above document is incorporated herein by reference in its entirety.
TECHNICAL FIELD
The present invention relates to a moving image processing system, a moving image processing method, and a program.
BACKGROUND ART
An image displayed on a screen of an information processor can be copied by taking a screen shot. That is, data of the image displayed on the screen can be obtained by capturing display data stored in the display memory.
In an attempt to prevent such fraudulent acts, Patent Literature 1 below discloses a technology for randomly selecting S for each pixel of an original image, generating a first converted image obtained by adding S to each pixel value and a second converted image obtained by subtracting S from each pixel value, and displaying these two converted images while switching them at a high speed. In accordance with this technology, even though a viewer recognizes as if the original image is displayed, it is possible to prevent fraudulent acquisition of original image data because the data obtained by a screen shot is the first or the second converted image.
CITATION LIST
Patent Document
Patent Literature 1: JP2002-72994A
SUMMARY OF INVENTION
Technical Problem
However, the technology described in the Patent Literature 1 is directed to a still image, and could not be applied to a moving image.
One or more embodiments of the present invention have been conceived in view of the above, and an object thereof is to provide a moving image processing system, a moving image processing method, and a program that make it difficult to obtain an original moving image based on display data.
Solution to Problem
In order to solve the above described problems, a moving image processing system according to the present invention includes acquiring means for acquiring moving image data, specifying means for specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially, and generating means for generating, by modifying a key frame image that is based on at least one of the frame images included in the image set, a same number of modified frame images as the frame images included in the image set. An average of pixel values of corresponding pixels of the modified frame images corresponds to a pixel value of a corresponding pixel of the key frame image. The modified frame images are sequentially displayed instead of the frame images included in the image set.
The moving image processing system according to the present invention includes acquiring means for acquiring moving image data, specifying means for specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially, and generating means for generating a same number of noise images as the frame images included in the image set, the noise images being combined with a key frame image that is based on at least one of the frame images included in the image set. An average of pixel values of corresponding pixels of the noise images is zero. The modified frame images, which are obtained by combining the key frame image with the noise images respectively, are sequentially displayed instead of the frame images included in the image set.
The specifying means may specify the image set based on a change in the frame images indicated by the moving image data. Further, the specifying means may specify the image set based on content of the frame images indicated by the moving image data.
The moving image processing system may further include pausing means for repeating, in a case where reproducing the moving image data is paused while the modified frame images are sequentially displayed, displaying a plurality of pause modified frame images for use in pausing in order, the pause modified frame images being obtained by modifying a pause key frame image for use in pausing that is based on at least one of the frame images included in the image set, wherein an average of pixel values of corresponding pixels of the pause modified frame images corresponds to a pixel value of a corresponding pixel of the pause key frame image.
The pause modified frame images may be the modified frame images generated by the generating means.
A moving image processing method according to the present invention includes an acquiring step of acquiring moving image data, a specifying step of specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially, and a generating step of generating, by modifying a key frame image that is based on at least one of the frame images included in the image set, a same number of modified frame images as the frame images included in the image set. An average of pixel values of corresponding pixels of the modified frame images corresponds to a pixel value of a corresponding pixel of the key frame image. The modified frame images are sequentially displayed instead of the frame images included in the image set.
A moving image processing method according to the present invention includes an acquiring step of acquiring moving image data, a specifying step of specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially, and a generating step of generating a same number of noise images as the frame images included in the image set, the noise images being combined with a key frame image that is based on at least one of the frame images included in the image set. An average of pixel values of corresponding pixels of the noise images is zero. The modified frame images, which are obtained by combining the key frame image with the noise images respectively, are sequentially displayed instead of the frame images included in the image set.
A program according to the present invention program causes a computer to function as acquiring means for acquiring moving image data, specifying means for specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially, and generating means for generating, by modifying a key frame image that is based on at least one of the frame images included in the image set, a same number of modified frame images as the frame images included in the image set. An average of pixel values of corresponding pixels of the modified frame images corresponds to a pixel value of a corresponding pixel of the key frame image. The modified frame images are sequentially displayed instead of the frame images included in the image set. Further, a program according to the present invention causes a computer to function as acquiring means for acquiring moving image data, specifying means for specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially, and generating means for generating a same number of noise images as the frame images included in the image set, the noise images being combined with a key frame image that is based on at least one of the frame images included in the image set. An average of pixel values of corresponding pixels of the noise images is zero. The modified frame images, which are obtained by combining the key frame image with the noise images respectively, are sequentially displayed instead of the frame images included in the image set. These programs may be stored in a computer-readable information storage medium, such as a CD-ROM and a DVD-ROM.



BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 is a diagram illustrating a moving image display device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a hardware configuration of the moving image display device.
FIG. 3 is a diagram illustrating conversion of moving images.
FIG. 4 illustrates an example of a key frame image.
FIG. 5 illustrates an example of an intermediate noise image.
FIG. 6 illustrates an example of a noise image.
FIG. 7 illustrates an example of a modified frame image.
FIG. 8 is a functional block diagram of the moving image display device.
FIG. 9 is a flow chart illustrating processing for specifying a frame image set.
FIG. 10 is a flow chart illustrating processing for generating an intermediate noise image.
FIG. 11 is a flow chart illustrating processing for generating a noise image.



DESCRIPTION OF EMBODIMENTS
An embodiment of the present invention will be described below in detail with reference to the accompanying drawings.
FIG. 1 is a diagram illustrating a moving image display device according to an embodiment of the present invention. The moving image display device 10 is a computer in various forms, such as a personal computer, a smartphone, and a tablet. As shown in FIG. 2, the moving image display device 10 has a hardware configuration in which a CPU 10a, a main memory 10b, a communication interface 10c, a display unit 10d, an input unit 10e, and a storage unit 10f are connected to a bus. The moving image display device 10 is one form of a moving image processing system according to the present invention. The CPU 10a controls each section of the device and executes various information processing. The main memory 10b is volatile storing means and used for operations of the CPU 10a. The communication interface 10c accesses a data communication network 30, such as the Internet, through a communication line, such as a wireless LAN and a radio telephone line. The display unit 10d is display means such as a liquid crystal display and an organic EL display. The display unit 10d includes a display memory. When the CPU 10a stores display data (raster style) in the display memory, the display unit 10d displays an image indicated in the display data. The input unit 10e is, for example, mechanical keys provided to the moving image display device 10, and a touch panel mounted on the display surface of the display unit 10d. Data input by the input unit 10e is used by the CPU 10a. The storage unit 10f is non-volatile storage means, and stores an Operating System (OS) and a moving image display program. The moving image display program may be downloaded from the data communication network 30, or read from a computer-readable information storage medium, such as a CD-ROM. An audio output unit 10g has speakers and reproduces audio data.
In this embodiment, as shown in FIG. 1, the moving image display device 10 is connected to the data communication network 30, to which the moving image transmission device 20 is also connected. The moving image transmission device 20 is a computer that has a hardware configuration similar to the moving image display device 10, and sends moving image data to the moving image display device 10. For example, the moving image display device 10 and the moving image transmission device 20 are computers each executing a video chat program and configured as a video chat device. That is, the moving image transmission device 20 captures moving image of a user with a camera, and generates moving image data of the user in a stream format. Subsequently, the moving image transmission device 20 streams the generated moving image data to the moving image display device 10. The moving image display device 10 receives the moving image data, and displays the received data on the display unit 10d. The moving image transmission device 20 may have a microphone for recording a user's voice, and send the moving image data with audio to the moving image display device 10. Further, when displaying moving image indicated by the moving image data with audio on the display unit 10d, the moving image display device 10 may output, from the audio output unit 10g, the audio indicated by the moving image data with audio in synchronization with the moving image. The moving image transmission device 20 may have functions of the moving image display device 10, and the moving image display device 10 may have the functions of the moving image transmission device 20. This can implement two-way video chatting. Applying the present invention to a video chat device can be very valuable for a user to prevent a chat partner from copying the user's image.
The moving image transmission device 20 may send not only moving image data with audio for video chatting, but also moving image data with other audio, such as recorded audio of lectures. Alternatively, moving image data may not be with audio. Further, the moving image data may be generated not only in a stream format, but also in a downloadable format. The moving image transmission device 20 may also send moving image data in an encrypted form.
An OS runs on the moving image display device 10, and the OS provides a screen shot function. With the screen shot function, when a user performs a specific operation using the input unit 10e, the display data stored in a memory for display, which is integral to the display unit 10d, is moved to the storage unit 10f. In this way, the display data stored in the storage unit 10f can be reviewed on the display unit 10d later. When various types of moving image data sent from the moving image transmission device 20 is displayed on the display unit 10d, display data of frame images constituting the moving image is stored in the memory for display. With the screen shot function provided by the OS, the display data can be moved to the storage unit 10f, and then sent to other moving image display devices.
In order to prevent such fraudulent acts, the moving image display program according to this embodiment stores display data (modified frame image), which is obtained by adding noise to a key frame image of an original moving image, in the display memory of the display unit 10. As such, even though the screen shot function provided by the OS is used, only the modified frame image can be moved to the storage unit 10f. In this regard, a plurality of types of modified frame images are prepared, and an average value of the modified frame images matches the key frame image. As such, the plurality types of modified frame images are switched at short intervals and sequentially displayed, whereby the user recognizes as if the original key frame image without noise is displayed on the display unit 10d. 
FIG. 3 is a diagram illustrating conversion of moving images by the moving image display program. FIG. 3(a) shows a series of frame images indicated by the moving image data sent from the moving image transmission device 20, and FIG. 3(b) shows a series of modified frame images generated by the moving image display program. As shown in FIG. 3(a), the moving image data sent from the moving image transmission device 20 indicates a large number of frame images V1, V2, V3, V4 . . . that are sequentially displayed at predetermined time intervals. The moving image display program selects a frame image Vj with a noticeable change from these frame images as a key frame image K. Here, frame images V1, V5, and V7 are selected as key frame images K. Subsequently, successive frame images from a key frame image K to a frame image immediately before a next key frame image K are specified as a frame image set. For example, in FIG. 3(a), the frame images V1 to V4 are specified as a frame image set 1, and the frame image V1 at the top is specified as the key frame image K. The frame images belonging to one frame image set have no prominent differences, and are recognized as similar images by the user.
As shown in FIG. 3(b), the moving image display program converts a key frame images K into the same number of modified frame images as the frame image number R of the frame image set to which the key frame image K belongs. For example, the frame image number R of the frame image set 1 is four, and the frame image set 1 including the frame images V1 to V4 is converted into the modified frame image set 1 including the same number of modified frame images C1(1) to C4(1). These four modified frame images C1(1) to C4(1) are respectively generated based on the frame image V1, which is the key frame image K. Similarly, the frame image number R of the frame image set 2 is two, and the frame image set 2 including the frame images V5 to V6 is converted into the modified frame image set 2 including the same number of modified frame images C1(2) and C2(2). These two modified frame images C1(2) and C2(2) are respectively generated based on the frame image V5, which is the key frame image K. A frame image set is converted into a modified frame image set that includes the same number of modified frame images as the frame images of the frame image set, and thus the length of the moving image is not changed. As discussed above, the frame images belonging to one frame image set have no prominent differences, and are recognized as the similar images by the user. On the other hand, the user recognizes the modified frame images that belong to the corresponding modified frame set image set as if the key frame image is displayed for the same length as the modified frame images. These recognitions are similar to each other, which does not provide a feeling of strangeness to the user.
In the following, a method for generating a given number of modified frame images Cj based on a key frame image K will be described in detail.
FIG. 4 illustrates an example of a key frame image K. Here, the key frame image K is a 256-level gray scale image, and a person is displayed on a gray background. For the sake of description, FIG. 4 illustrates large alphabet letters and numbers instead of a person.
If a pixel value of a position (x,y) on the key frame image K is K (x,y), such pixel value is in the range of 0 to 255 inclusive. A pixel value K (x,y) at a position in the background is 1 or more and less than 255, and may be, for example, 30 or so. On the other hand, a pixel value K (x,y) at a position of the person is greater than such value, and may be, for example, 100 to 200.
The moving image display program of the moving image display device 10 specifies a frame image set, and determines a key frame image K and a frame image number R of the frame image set to which the key frame image K belongs. Subsequently, a modified frame image set S is generated based on the key frame image K. Here, the image set S includes R (R≧2) types of modified frame images Cj (j=1 to R).
Specifically, the moving image display program generates R types of noise images Nj (j=1 to R), and adds the generated noise images to the key frame image K, thereby obtaining R types of modified frame images Cj (j=1 to R). The moving image display program first generates R types of intermediate noise images Mj (j=1 to R) in order to generate R types of noise images Nj, and then converts the generated intermediate noise images into R types of noise images Nj. In this regard, the moving image display program randomly determines the intermediate noise images Mj and the noise images N under the condition described later.
FIGS. 5 to 7 respectively show examples of intermediate noise images M1 to M4, noise images N1 to N4, and modified frame images C1 to C4, where R=4. Here, horizontal and vertical sizes (dimensions) of the noise image Nj, the intermediate noise image Mj, the modified image Cj, and the key frame image K are the same, and each image is a 256-level gray scale image.
When the pixel value of the noise image Nj at the position (x,y) is Nj (x,y), and the pixel value of the modified frame image Cj at the position (x,y) is Cj (x,y), the following expression (1) is satisfied.

Cj(x,y)=K(x,y)|Nj(x,y)  (1)

When all of the R types of noise images Nj are viewed while being switched at short intervals, viewers perceive as if no image is displayed due to the human visual system. That is, the following expression (2) is satisfied at any position (x,y) of the noise image Nj.

ΣNj(x,y)=0  (2)

Here, Σ is a sum from j=1 to j=R. Since the equations (1) and (2) are satisfied, the average of pixel values Cj (x,y) at any position (x,y) of the modified frame image Cj matches the pixel value K (x,y) of the position (x,y) of the key frame image K. In this regard, there is an upper limit of R in order that the human visual system works effectively. According to the experiment of the present inventors, the upper limit of R is about 2 to 4.
As described above, the modified frame image Cj also has 256 levels, and needs to be represented by pixel values displayable on the display unit 10d. As such, the following expression (3) must be satisfied.

α≦K(x,y)+Nj(x,y)≦β  (3)

Here, α is the lower limit value of a pixel value displayable by the display unit 10d, and β is the upper limit value of a pixel value displayable by the moving image display device 10. The display unit 10d performs 256-level gray-scale display, and thus the lower limit value α is 0, and the upper limit value β is 255.
Upon generating a modified frame image set S, the moving image display program of the moving image display device 10 displays modified frame images C1 to Cn included in the modified frame image set S in order.
FIG. 8 is a functional block diagram of the moving image display device 10. The functional block diagram shown in FIG. 8 indicates the functions that the moving image display device 10 has and are related to display of moving images. The functions shown in FIG. 8 are implemented by the moving image display device 10, which is a computer, executing the moving image display program.
As shown in FIG. 8, the moving image display device 10 includes a moving image data acquiring unit 11, a frame image set specifying unit 12, an intermediate noise image generating unit 13, a noise image generating unit 14, a modified frame image generating unit 15, a display control unit 16, and a display unit 17. The moving image data acquiring unit 11 includes, for example, a FIFO (First In First Out) frame image buffer 11a reserved in the main memory 10b. The display control unit 16 includes, for example, a FIFO modified frame image buffer 16a reserved in the main memory 10b. 
The moving image data acquiring unit 11 receives moving image data sent from the moving image transmission device 20. The moving image data acquiring unit 11 then acquires a series of frame images indicated in the moving image data, and stores the acquired frame images in order in the frame image buffer 11a. The frame image set specifying unit 12 specifies a plurality of frame image sets, each including successive frame images of similar content, based on the frame images stored in the frame image buffer 11a. Each frame image set is temporally adjacent to another frame image set, and all the frame images stored in the frame image buffer 11a respectively belong to the frame image sets.
The frame image set specifying unit 12 then outputs key frame images K, each being a representative image of each frame image set, and frame image numbers R of respective frame image sets. The intermediate noise image generating unit 13, the noise image generating unit 14, and the modified frame image generating unit 15 sequentially generate the modified frame image sets S. A modified frame image set S includes a frame image number R, which is output by the frame image set specifying unit 12, of modified frame images Cj, which are obtained by converting the key frame image K output from the frame image set specifying unit 12.
That is, the intermediate noise image generating unit 13 generates the frame image number R of intermediate noise images Mj based on the key frame image K. The noise image generating unit 14 generates noise images Nj based on the intermediate noise images Mj. As shown in the expression (1), the modified frame image generating unit 15 adds noise images Nj generated by the noise image generating unit 14 to the key frame image K, thereby generating the frame image number R of modified frame images Cj. The modified frame images Cj according to respective frame image sets S are stored in the modified frame image buffer 16a in order of display. The display control unit 22 sequentially displays the modified frame images Cj of the respective modified frame image sets S stored in the modified frame image buffer 16a. When reproducing the frame images is paused using the input unit 10e, the modified frame image set S being reproduced at that time is repeatedly displayed. In this way, the original frame image is not stored in the memory for display, and fraudulent acts can be prevented while reproducing the frame images is paused.
Here, processing of the frame image set specifying unit 12 will be discussed in detail. FIG. 9 is a flow chart showing the processing of the frame image set specifying unit 12. In FIG. 9, K represents a key frame image, and Vj is jth frame image. Further, a gradient evaluation value ε (x,y) indicates a gradient at a position (x,y) of the key frame image K, and a gradient evaluation value Wj indicates a gradient at a position (x,y) of the frame image Vj, and are respectively shown by the following expressions (4) and (5).

ε(x,y)={(∂V0/∂x)2+(∂V0/∂y)2}1/2  (4)

Wj(x,y)={(∂Vj/∂x)2+(∂Vj/∂y)2}1/2  (5)

A difference evaluation value D (x,y) indicates a difference between a pixel value at a position (x,y) of the key frame image K and a pixel value at a position (x,y) of the frame image Vj, and is shown by the following expression (6). The difference evaluation value D (x,y) indicates how the frame image Vj is different from the key frame image K at the position (x,y).

D(x,y)=[{K(x,y)−Vj(x,y)}2]1/2  (6)

Further, a gradient evaluation value Q (x,y) is shown by the following expression (7). The value Q (x,y) indicates an average value of gradients of respective key frame image K and frame image Vj at the position (x,y). When the value Q is larger, an image frequency at the position (x,y) is higher.

Q(x,y)={ε(x,y)+Wj(x,y)}/2  (7)

An index φ is an index for determining whether a frame image Vj is noticeably changed from a key frame image K preceding the frame image Vj, and defined by the following expression (8).

φ=Σ{D(x,y)×Q(x,y)}  (8)

Here, Σ represents the sum of the positions (x,y). The value of index φ is larger as the number of positions (x,y), at which both of a difference evaluation value D (x,y) and a gradient evaluation value Q (x,y) are large, is increased. That is, when the number of positions in a key frame image K and a frame image Vj, at which image frequency is high, is greater and a change from the key frame image K to the frame image Vj in these positions is greater, the value of index φ is larger.
In FIG. 9, the frame image set specifying unit 12 selects a first frame image V0 as a key frame image K, and then calculates gradient evaluation values ε (x,y) for all of positions (x,y) (S101). Alternatively, the frame image set specifying unit 12 initializes variables i and j to 0.
Next, the frame image set specifying unit 12 increments the variable j, and acquires a subsequent frame image V from the frame image buffer 11a (S102). The frame image set specifying unit 12 then determines whether j−i is equal to or more than 2 (S103). If NO in S103, the processing returns to S102. If Yes in S103, the frame image set specifying unit 12 calculates a gradient evaluation value Wj (x,y) (S104). Subsequently, the frame image set specifying unit 12 determines whether j−i is equal to or more than m (S105). Here, m is the maximum number of frame image number R included in the frame image sets, and may be set to a number in the range of, for example, 2 or more to 4 or less as discussed above.
If Yes in S105, the processing proceeds to S110. If No in S105, the frame image set specifying unit 12 calculates difference evaluation values D (x,y), gradient evaluation values Q (x,y), and indexes φ for all of positions (x,y) (S106, S107, S108). Subsequently, the frame image set specifying unit 12 determines whether the index φ is equal to or more than τ (S109). τ may be determined by an experiment, for example.
If No in S109, the processing proceeds to S113. If Yes in S109, the frame image set specifying unit 12 sets the frame image number R to j−i (S110), and passes the frame image number R and the key frame image K selected at S101 to the intermediate noise image generating unit 13 (S111). Subsequently, the frame image set specifying unit 12 selects a frame image V as the next key frame image K, and sets gradient evaluation values Wj (x,y) to the gradient evaluation value ε (x,y) for all positions (x,y). Further, the frame image set specifying unit 12 sets the variable i to the same value as the variable j (S112). The frame image set specifying unit 12 then deletes the frame image Vj from the frame image buffer 11a (S113), and returns to S102.
As a result of the foregoing processing, the frame image set specifying unit 12 can preferably specify the frame image set based on D (x,y) indicating a change in the frame image Vj and Q (x,y) indicating content characteristics. Here, although the index φ is used for specifying the key frame image K, other methods may be employed for specifying the key frame image K. For example, the frame image set specifying unit 12 may take a difference between a frame image Vj and a previous frame image Vj−1, and when the difference is a predetermined value or more, select the frame image Vj as a key frame image K. In the above example, although a first frame image of each frame image set is selected as a key frame image K, other frame images, such as a center frame image, may be selected as a key frame image K. Alternatively, images generated based on the frame image included in the frame image set, such as an average image of frame images included in the frame image set, may be selected as a key frame image K.
Next, processing performed by the intermediate noise image generating unit 13 will be discussed in detail. FIG. 10 is a flow chart illustrating the processing performed by the intermediate noise image generating unit 13. As shown in FIG. 10, upon receiving the key frame image K and the frame image number R from the frame image set specifying unit 12, the intermediate noise image generating unit 13 sets the variable j to 1 (S201). The intermediate noise image generating unit 13 then calculates L0 (x,y) and U0 (x,y), which respectively are initial values of the lower limit value Lj (x,y) and the upper limit value Uj (x,y), in the allowable range of pixel values Mj (x,y) of intermediate images Mj described later, for all positions (x,y) based on the following expressions (9) and (10) (S202).

L0(x,y)=α−K(x,y)  (9)

U0(x,y)=β−K(x,y)  (10)

Subsequently, if the variable j is not equal to or more than R (S203), the intermediate noise image generating unit 13 calculates the lower limit values Lj (x,y) and the upper limit values Uj (x,y) of pixel values Mj (x,y) for all positions (x,y) based on the following expressions (11) and (12) (S204). Here, ε is a sum from k=1 to j−1.

Lj(x,y)=−ΣMk(x,y)−(n−j)Uj−1(x,y)  (11)

Uj(x,y)=−ΣMk(x,y)−(n−j)Lj−1(x,y)  (12)

The intermediate noise image generating unit 13 then randomly determines M (x,y) in the range of the lower limit value Lj (x,y) and the upper limit value Uj (x,y) (S205). Specifically, the intermediate noise image generating unit 13 generates random numbers γ in the range of 0 to 1 for all positions (x,y), and calculates pixel values Mj (x,y) by substituting the γ for the following expression (13).

Mj(x,y)=γ×Lj(x,y)+(1−γ)×Uj(x,y)  (13)

Subsequently, the intermediate noise image generating unit 13 increments the variable j by 1 (S206), and returns to the processing of S203. If it is determined that the variable j is R or more in S203, the intermediate noise image generating unit 13 calculates Mn (x,y) for all positions (x,y) based on the following expression (14) (S207), and finishes the processing. Here, Σ is a sum from k=1 to n−1.

Mn(x,y)=ΣMk(x,y)  (14)

The pixel value Mj (x,y) of the intermediate noise image Mj satisfies the following expression (15) similarly to the pixel value Nj (x,y) of the noise image Nj. Here, Σ is a sum from j=1 to n.

ΣMj(x,y)=0  (15)

The expression (14) is derived from the expression (15). The M (x,y) is a value obtained by inverting the sign of the sum of M1 (x,y) to Mj−1 (x,y), which have been determined in the processing of S205, and Mj−1 (x,y) to Mn (x,y), which are to be determined in the subsequent processing of S205. Here, assuming that the upper limit value and the lower limit value of Mj+1 (x,y) to Mn (x,y), which are to be determined in the processing of S205, respectively are Uj−1 (x,y) and Lj−1 (x,y), and this leads to the expressions (11) and (12).
Next, processing of the noise image generating unit 14 will be explained in detail. FIG. 11 is a flow chart illustrating processing executed by the noise image generating unit 14. The processing in FIG. 11 shows that the pixel values M1 (x,y) to Mn (x,y) at the positions (x,y) of the intermediate noise images M1 to Mn are randomly exchanged and thereby the pixel values N1 (x,y) to Nn (x,y) at the positions (x,y) of the noise images N1 to Nn are obtained. The processing shown in FIG. 11 is repeated with respect to all the positions (x,y). In this processing, the noise image generating unit 14 sets the variable t to 1 (S301), generates two random numbers in the range of 1 to R unless the variable t is not equal to or more than T, which is the predetermined number of repetition times (S302), and sets variable δ and variable ε to the generated numbers (S303). Subsequently, the noise image generating unit 14 exchanges the pixel value Mδ (x,y) of the δth intermediate noise image Mδ and the pixel value Mε (x,y) of the εth intermediate noise image Mε (S304). The noise image generating unit 14 then increments the variable t by 1 (S305), and returns to the processing of S302. In the processing of S302, if it is determined that the variable t is equal to or more than the repetition times T, the pixel values M1 (x,y) to Mn (x,y) of the intermediate noise images M1 to Mn are set to the pixel values N1 (x,y) to Nn (x,y) of the noise images N1 to Nn.
Subsequently, the processing of S301 to S306 is repeated for all of the positions (x,y). The number of the repetition times T may be any number, but preferably equal to or more than R.
As discussed above, the modified frame image generating unit 15 adds noise images N generated by the frame image set specifying unit 12 to key frame images K output from the noise image generating unit 14, thereby generating the frame image number R of modified frame images Cj. Subsequently, the modified frame image generating unit 15 stores the generated modified frame images Cj in the modified frame image buffer 16a in order of display. The display control unit 22 stores the modified frame images Cj of the modified frame image sets S, which are stored in the modified frame image buffer 16a, in the memory for display provided to the display unit 10d in order of display, thereby sequentially displaying the modified frame images Cj.
According to the moving image display device 10 described above, the modified frame image sets S are sequentially generated, and the modified frame images Cj included in the modified frame image sets S are displayed in time order. The original frame image Vj is not displayed on the display unit 10d, and thus it is possible to prevent a fraudulent copy of data of frame images Vj.
The present invention is not limited to the above described embodiment, and various modifications can be made. For example, while in the above discussion a 256-level gray scale image is used as a frame image Vj, other type of image (e.g., color image) may of course be used. Further, a method for generating the modified frame images Cj is not limited to the above-described examples, and various generation methods can be used. For example, according to the embodiment described above, a modified frame image Cj is acquired by adding noise to an entire key frame image K. However, a modified frame image Cj may be acquired by adding noise to a part of a key frame image K. For example, an area showing particular content, such as a human face, may be specified in the key frame image K, and only such area may be added with noise. In this case, pixel values Mj (x,y) and pixel values Nj (x,y) are values greater than 0 only at the positions (x,y) in the specified area, and are set to 0 at the other positions (x,y).
Further, in the above discussion, the moving image processing system according to the present invention is implemented in the moving image display device 10, but may be implemented in the moving image transmission device 20. In this case, the moving image transmission device 20 generates modified frame images, and sends the generated modified frame images in a streaming form or a downloading form to the moving image display device 10. The moving image display device 10 sequentially displays the received modified frame images. Alternatively, the moving image transmission device 20 may generate noise images, and send the generated noise images together with the original moving image data to the moving image display device 10 in a streaming form or a downloading form. In this case, the moving image display device 10 generates modified frame images by combining the frame images indicated by the received moving image data with the received noise images, and sequentially displays the modified frame images. The functions of the moving image processing system according to the present invention may be shared by some computers in various ways.

## Claims

1. A moving image processing system comprising:
at least one processor; and
at least one memory device that stores a plurality of instructions, which when executed by the at least one processor, causes the at least one processor to execute operations comprising:
acquiring moving image data;
specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially; and
generating, by modifying a key frame image that is based on at least one of the frame images included in the image set, a same number of modified frame images as the frame images included in the image set, an average of pixel values of corresponding pixels of the modified frame images being corresponding to a pixel value of a corresponding pixel of the key frame image, and the modified frame images being sequentially displayed instead of the frame images included in the image set.

2. The moving image processing system according to claim 1, wherein
the operations comprise specifying the image set based on a change in the frame images indicated by the moving image data.

3. The moving image processing system according to claim 1, wherein
the operations comprise specifying the image set based on content of the frame images indicated by the moving image data.

4. The moving image processing system according to claim 1, wherein the operations further comprise:
repeating, in a case where reproducing the moving image data is paused while the modified frame images are sequentially displayed, displaying a plurality of pause modified frame images for use in pausing in order, the pause modified frame images being obtained by modifying a pause key frame image for use in pausing that is based on at least one of the frame images included in the image set, an average of pixel values of corresponding pixels of the pause modified frame images being corresponding to a pixel value of a corresponding pixel of the pause key frame image.

5. The moving image processing system according to claim 4, wherein
the paused modified frame images are the modified frame images.

6. The moving image processing system according to claim 1, wherein
when a pixel value at a position (x, y) of a key frame image K is K (x,y), a pixel value at a position (x,y) of jth modified image Cj is Cj (x,y), and a pixel value at a position (x,y) of jth noise image Nj is Nj (x,y), the following expression is established:


K(x,y)|Nj(x,y)=Cj(x,y)



ΣNj(x,y)=0



where n is a number of the modified frame images, and Σ is a sum from j=1 to n.

7. The moving image processing system according to claim 6, wherein the following expression is established:


α≦Nj(x,y)+K(x,y)≦β


where α is a lower limit value of a pixel value displayable by a display, and β is a upper limit value of a pixel value displayable by the display.

8. The moving image processing system according to claim 7, wherein
when a pixel value at a position (x,y) of jth intermediate noise image Mj is Mj (x,y), a lower limit value Lj (x,y) and a upper limit value Uj (x,y) of the pixel value are sequentially calculated by


Lj(x,y)=−ΣMk(x,y)−(n−j)Uj−1(x,y)



Uj(x,y)=−ΣMk(x,y)−(n−j)Lj−1(x,y)



the pixel value Mj (x,y) is determined within a range of the lower limit value Lj (x,y) and the upper limit value Uj (x,y), and the plurality of modified frame images Cj are generated based on the pixel value Mj (x,y), where Σ is a sum from k=1 to j−1, U0 (x,y) is β−K (x,y), and L0 (x,y) is α−K (x,y).

9. The moving image processing system according to claim 8, wherein
a pixel value Mδ (x,y) at a position (x,y) of δth intermediate noise image Mδ and a pixel value Mε (x,y) at a position (x,y) of εth intermediate noise image Mε are interchanged so as to generate the plurality of noise images Nj, and whereby the plurality of modified frame images Cj are generated, where δ ands ε are random numbers in a range of 1 to n inclusive.

10. A moving image processing method comprising:
acquiring using a first processor, moving image data;
specifying using the first processor an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially; and
generating using the first processor, by modifying a key frame image that is based on at least one of the frame images included in the image set, a same number of modified frame images as the frame images included in the image set, an average of pixel values of corresponding pixels of the modified frame images being corresponding to a pixel value of a corresponding pixel of the key frame image, and the modified frame images being sequentially displayed instead of the frame images included in the image set.

11. A non-transitory computer readable storage medium storing a program for causing a computer to function as:
acquiring means for acquiring moving image data;
specifying means for specifying an image set that includes, among a plurality of frame images indicated by the moving image data, frame images to be displayed sequentially; and
generating means for generating, by modifying a key frame image that is based on at least one of the frame images included in the image set, a same number of modified frame images as the frame images included in the image set, an average of pixel values of corresponding pixels of the modified frame images being corresponding to a pixel value of a corresponding pixel of the key frame image, and the modified frame images being sequentially displayed instead of the frame images included in the image set.