Abstract:
There are provided an image processing apparatus, an image processing method, and a computer readable memory medium storing a program for such image processing method, which are featured by inputting image data, discriminating the motion of the image data, and superposing watermark information with the image data or the predetermined information data such as a motion vector employed in the encoding of the image data, according to the result of discrimination, thereby enabling concealing of the watermark information with the increased resistance against the attacking and with the reduced deterioration of the image.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    The present invention relates to an image processing apparatus, an image processing method and a computer readable memory medium storing an image processing program, for embedding watermark information in image data.  
           [0003]    2. Related Background Art  
           [0004]    For image recording/reproduction, there has conventionally been known the video cassette recorder which records analog image signal on a magnetic tape, but the copyright issue relating to the illegal or improper use of image data has not been raised seriously with such apparatus because the image is deteriorated by repeated copying.  
           [0005]    On the other hand, digital image data, formed by digitizing the image signal with eventual compression encoding, are easy for handling and can be not only recorded on a magnetic tape in a digital VCR but also stored in a computer memory or transmitted through a communication channel. Illegal use is a serious concern for such digitized image data because such data are mostly free from image deterioration even after recording or transmission.  
           [0006]    For avoiding such illegal use, it is already known to incorporate, in the image, a digital watermark constituting ID information data for the author, distribution source, management number, distribution channel etc. of the image data. The digital watermarking is to secretly conceal the watermark data into the image, undiscernibly to the user observing the image, in order to later confirm the source or channel of distribution of the illegally used image. In such watermarking, it is important to minimize the image deterioration resulting from the concealing of the watermark data, to conceal the watermark data in such a manner that such watermark alone cannot be extracted from the image, and to avoid a significant increase in the amount of information by the concealing of the watermark data.  
           [0007]    In the digital watermarking in a moving image, it is proposed to conceal the watermark data only in a part of the frames, since the amount of the watermark data becomes very large if such data are concealed in every frame, However, if the interval of the watermarked frames is made excessively large, the effect of prevention for illegal use becomes lower in case a part of the moving image is used.  
           [0008]    In the above-mentioned method of concealing the digital watermark in a part of the frames of the moving image, if the digital watermark is concealed over several frames in an area that does not deteriorate the image, there will result a drawback that not only the concealing of such digital watermark but also the position thereof become obvious through the comparison of the image data with those of the preceding or succeeding frame clarifies, because of the change in the image data though the image itself is not deteriorated. Consequently it also becomes easy to extract only the watermark, for example by eliminating the watermarked frame.  
         SUMMARY OF THE INVENTION  
         [0009]    In consideration of the foregoing, the object of the present invention is to provide an image processing apparatus, an image processing method and a computer readable memory medium storing a program for such image processing, capable of concealing watermark information into image data with an increased resistance to the attack while suppressing the image deterioration and the increase in the data amount.  
           [0010]    To attain the above-mentioned object according to one aspect of the present invention, there is provided an image processing apparatus/method in which image data is input, the motion of such image data is discriminated and watermark information is superposed with predetermined information data according to the result of the discrimination.  
           [0011]    Also according to an another aspect of the present invention, there is provided a computer readable memory medium storing a program for executing an input step of entering image data, a discrimination step of discriminating the motion of the image data, and a superposing step of superposing watermark information with predetermined information data according to the result of the discrimination.  
           [0012]    Other objects of the invention, and features and advantages thereof, will become fully apparent from the following detailed description to be taken in conjunction with the accompanying drawings. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0013]    [0013]FIG. 1 is a block diagram showing the configuration of a first embodiment in which the present invention is applied to a video camera;  
         [0014]    [0014]FIGS. 2A, 2B,  2 C and  2 D are views showing examples of image display in the present invention;  
         [0015]    [0015]FIG. 3 is a detailed block diagram showing a compression encoding circuit  103  in the first embodiment;  
         [0016]    [0016]FIG. 4 is a view showing a discrimination process for a motion in block;  
         [0017]    [0017]FIG. 5 is a view showing a block with motion within a frame;  
         [0018]    [0018]FIG. 6 is a detailed block diagram showing a compression encoding circuit  103  in a second embodiment;  
         [0019]    [0019]FIG. 7 is a block diagram showing the configuration of an image processing apparatus constituting a third embodiment of the present invention;  
         [0020]    [0020]FIG. 8 is a block diagram showing the configuration of an encoding circuit for generating image data encoded and recorded on a recording medium  401 ;  
         [0021]    [0021]FIG. 9 is a detailed block diagram of a watermarking circuit  403 ;  
         [0022]    [0022]FIG. 10 is a detailed block diagram showing a compression encoding circuit  103  in a fourth embodiment; and  
         [0023]    [0023]FIG. 11 is a view showing a example of arrangement of image types in MPEG encoding. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0024]    In the following there will be explained a first embodiment of the present invention, with reference to the accompanying drawings.  
         [0025]    [0025]FIG. 1 is a block diagram showing the configuration of the first embodiment in which the present invention is applied to a video camera capable of recording and reproducing a moving image.  
         [0026]    Referring to FIG. 1, an object image focused in an image pickup device  101  such as a CCD (charge coupled device) through an image taking lens  100  is converted into an electrical signal, which is converted by predetermined signal processing in a camera signal processing circuit  102  into digital image data. The digital image data are subjected, in a compression encoding circuit  103 , to compression encoding to be explained later, and recorded by a recording/reproducing circuit  104  on a recording medium  105 . The recording medium  105  can be composed, for example, of a hard disk, a memory card or a video tape.  
         [0027]    A watermark information generation circuit  106  generates watermark information such as copyright information on the author, editor, copyright owner or user or information for clarifying the distribution channel, and such watermark information can be prepared by the manipulation of an operation unit  107 .  
         [0028]    The watermark information generated by the watermark information generation circuit  106  is concealed into the image data by the compression encoding circuit  103  in the course of encoding process, as will be explained later in more details.  
         [0029]    The encoded image data, recorded on the recording medium  105 , are reproduced by the recording/reproducing circuit  104 , and decoded by a decoding circuit  108 . The decoded image data are supplied to a watermark information separation circuit  109 .  
         [0030]    The watermark information separation circuit  109  separates the watermark information from the image data outputted from the decoding circuit  108 . The image data processed in the watermark information separation circuit  109  are supplied to a display unit  110  and displayed thereon. The watermark information separation circuit  109  sends the separated watermark information to the display unit  110 , according to an instruction from the operation unit  107 .  
         [0031]    More specifically, the watermark information separation circuit  109  can send either the entire watermark information or necessary information (for example author information) among the watermark information, according to the instruction from the operation unit  107 , to the display unit  110  for display thereon. Also it may be also possible not to send any watermark information to the display unit  110 , thereby not displaying the watermark information thereon.  
         [0032]    If the watermark information is supplied from the watermark information separation circuit  109 , the display unit  110  displays such watermark information together with the image data on a monitor.  
         [0033]    [0033]FIGS. 2A to  2 D show examples of the display on the display unit  110 .  
         [0034]    [0034]FIG. 2A shows an example of on-screen display of the author information, among the watermark information concealed as character information. In this example, since the watermark information is character information, the displayed characters are generated by a character generator in the display unit  110 .  
         [0035]    [0035]FIG. 2B shows an example in which the watermark information is concealed not as character information as in the case of FIG. 2A but as image information. In this example, the author information is displayed in superposition with the image.  
         [0036]    [0036]FIG. 2C shows an example of display of the distribution channel of the image. This example indicates that the image is distributed by television broadcasting.  
         [0037]    [0037]FIG. 2D shows an example of display in which the watermark information is turned off.  
         [0038]    In the following there will be explained in detail the compression encoding circuit  103  with reference to FIG. 3, which is a detailed block diagram thereof in the first embodiment.  
         [0039]    Referring to FIG. 3, the image data entered into the compression encoding circuit  103  are divided by a block forming circuit  201  into blocks of a size of 8×8 pixels in the horizontal and vertical directions, and the image data divided into such block are supplied to a motion detection circuit  206  and an adder  202 .  
         [0040]    The motion detection circuit  206  discriminates the motion of the image data in the unit of the block. As shown in FIG. 4, the motion of image data of each block is discriminated by comparing the block data between the current frame (k-th frame) and the immediately preceding frame ((k-1)th frame). In this embodiment, the difference between the current frame and the preceding frame is calculated in the block unit basis and the motion of the current block is discriminated from such difference. More specifically, the motion is discriminated as present or absent respectively if the difference is large or small.  
         [0041]    The block identified by the motion detection circuit  206  as showing motion, as shown in FIG. 5, is selected by a block selection circuit  207 , and the watermark information generated by the watermark information generation circuit  106  is superposed, in the adder  202 , to the image data of such block showing motion. However, the block selection circuit  207  conceals the watermark information only in a frame in which the motion is detected in a predetermined number of blocks, based on a predetermined frame interval obtained by a frame counter  208 . Thus the watermark information is not concealed in the image data of all the frames but is superposed with the image data of the blocks with motion with a fixed frame interval or with a random frame interval.  
         [0042]    In the following there will be explained the method of superposing the watermark information with the image data in the first embodiment.  
         [0043]    The concealing of the watermark information in the image data is achieved by the concealing of encoded watermark information, in consideration of the redundancy (noise margin) of the image.  
         [0044]    In the present embodiment, one bit of the watermark information is concealed in each block showing motion. The watermark information of plural bits in total is concealed by concealing into plural blocks. Also the order of concealing of the watermark information has to be determined in advance in order to enable decoding of the watermark information. In the present embodiment, the watermark information is concealed in succession into the block showing motion, starting from the block at upper left corner of the image. FIG. 5 shows an example of the order of concealing of the watermark information, and the numbers in the blocks indicate the order of concealing of the watermark information.  
         [0045]    Again referring to FIG. 3, a DCT circuit  203  executes DCT (discrete cosine transformation) on the image data outputted from the adder  202 , in the unit of the block. The DCT circuit  203  has a frame DCT mode (for effecting DCT in the block of 8×8 pixels) and a field DCT mode (dividing the block of 8×8 pixels into blocks of 4×8 pixels respectively in the first and second fields and effecting DCT in thus divided blocks), and switches these modes according to the result of motion detection by the motion detection circuit  206 . More specifically, it selects the field mode in case the motion is present, and the frame mode in case the motion is absent.  
         [0046]    The image data transformed by the DCT circuit  203  are quantized by a quantization circuit  204 , and then subjected to data compression by variable-length encoding such as Huffman encoding in a variable length encoding circuit  205 .  
         [0047]    The watermark information can be code information obtained by encoding information, or an image signal represented for example by luminance or color information.  
         [0048]    In the present embodiment, the watermark information is concealed in a part of the space of the image with motion, whereby the watermark information is scarcely detectable in the decoded image but the watermark information continues to remain in the encoded image.  
         [0049]    Also, as the watermark information is concealed into the moving image only in a part thereof in which the motion is detected, it is rendered possible to prevent the increase in the data amount. Also as the still image portion does not contain the watermark information, it is not possible to extract the watermark information through the comparison of the successive images, so that any third person is unable to find the presence of the watermark or the location thereof. Furthermore, as the watermark information cannot be extracted even after the moving image is cut out or modified, there can be obtained advantages of preventing illegal use of the moving image and tracing such illegal use.  
         [0050]    Furthermore, the watermark information is concealed only in an image area showing motion, so that the image deterioration resulting from the concealing of the watermark information can be rendered inconspicuous visually.  
         [0051]    In the following there will be explained a second embodiment with reference to the accompanying drawings.  
         [0052]    The entire configuration of the second embodiment is similar to that of the first embodiment and is as already explained with reference to FIG. 1. However, the internal configuration of the compression encoding circuit  103  is different from that in the first embodiment. In the following there will be explained the configuration and the process of the compression encoding circuit  103 .  
         [0053]    [0053]FIG. 6 is a detailed block diagram of the compression encoding circuit  103  of the second embodiment.  
         [0054]    Referring to FIG. 6, the image data entered into the-compression encoding circuit  103  are divided by a block formation circuit  301  into blocks of a size of 8×8 pixels in the horizontal and vertical directions. The image data of thus divided block are supplied to a motion detection circuit  306  and a DCT circuit  302 .  
         [0055]    The motion detection circuit  306  discriminates the motion of the image in each block, through a process similar to that in the motion detection circuit  206  of the first embodiment.  
         [0056]    The image data are converted in the DCT circuit  302  into DCT coefficients in the unit of the above-mentioned block, and are quantized in a quantization circuit  303 . The DCT circuit  302  has a frame transformation mode and a field transformation mode as in the first embodiment, and these modes are selected according to the output of the motion detection circuit  306 .  
         [0057]    The block identified by the motion detection circuit  306  as showing motion, as shown in FIG. 5, is selected by a block selection circuit  307 , and the watermark information generated by the watermark information generation circuit  106  is superposed, in an adder  304 , to the image data of such block showing motion. However, as in the first embodiment, the block selection circuit  307  conceals the watermark information only in a frame in which the motion is detected in a predetermined number of blocks, based on a predetermined-frame interval obtained by a frame counter  308 . Thus the watermark information is not concealed in the image data of all the frames but is superposed with the image data of the blocks showing motion at a fixed frame interval or with a random frame interval.  
         [0058]    The image data from the adder  304  are subjected to data compression by variable-length encoding, such as Huffman encoding, in a variable-length encoding circuit  305 , and are supplied to the recording/reproducing circuit  104 .  
         [0059]    In the following there will be explained the method of superposing the watermark information with the image data in the second embodiment.  
         [0060]    The concealing of the watermark information in the image data is achieved by the concealing of encoded watermark information, in consideration of the redundancy (noise margin) of the image.  
         [0061]    In the present embodiment, one bit of the watermark information is concealed in each block showing motion. In the present embodiment, the watermark information is concealed into the data of low frequency AC components after DCT conversion, in order to provide resistance to any attack, such as the encoding efficiency of Huffman encoding employed in the succeeding variable-length encoding circuit  305  or the enlarging or reducing conversion.  
         [0062]    The watermark information of plural bits in total is concealed by the above-described concealing into plural blocks. Also the order of concealing of the watermark information has to be determined in advance in order to enable decoding of the watermark information. In the present embodiment, the watermark information is concealed in succession into the block showing motion, starting from the block at the upper left corner of the image. FIG. 5 shows an example of the order of concealing of the watermark information, and the numbers in the blocks indicate the order of concealing of the watermark information.  
         [0063]    In contrast to the first embodiment, the present embodiment conceals the watermark information into the spatial frequency components after DCT in the blocks showing motion, and is therefore superior in resistance to the attack in comparison with the first embodiment. The present embodiment employs DCT for conversion into frequency components, but such method is not restrictive and the conversion into the frequency components may also be achieved by FFT (fast Fourier transformation).  
         [0064]    In the present embodiment, the watermark information is concealed in a part of the frequency space of the image with motion, whereby the watermark information is scarcely detectable in the decoded image but the watermark information continues to remain in the encoded image. Also, as the watermark information is concealed into the moving image only in a part thereof in which the motion is detected, it is not possible to extract the watermark information through the comparison of the successive images, so that any third person is unable to find the presence of the watermark or the location thereof. Furthermore, as the watermark information cannot be extracted even after the moving image is cut out or modified, there can be obtained advantages of preventing illegal use of the moving image and tracing such illegal use.  
         [0065]    Furthermore, the watermark information is concealed only in an image area showing motion, so that the image deterioration resulting from the concealing of the watermark information can be rendered inconspicuous visually.  
         [0066]    In the following there will be explained a third embodiment of the present invention with reference to the accompanying drawings.  
         [0067]    [0067]FIG. 7 is a block diagram showing the configuration of an image processing apparatus constituting the third embodiment of the present invention.  
         [0068]    Referring to FIG. 7, the compression encoded image data are read from a recording medium  401  by a reproducing circuit  402 . The recording medium  401  can be composed for example-of a hard disk, a RAM, a video tape or a DVD.  
         [0069]    The image data recorded on the recording medium  401  are encoded by an encoding circuit constructed as shown in FIG. 8.  
         [0070]    [0070]FIG. 8 is a block diagram showing the configuration of the encoding circuit for generating encoded image data, to be recorded on the recording medium  401 .  
         [0071]    Referring to FIG. 8, a block forming circuit  501  divides the entered image data into blocks of a size of 8×8 pixels in the horizontal and vertical direction. The image data divided into blocks by the block forming circuit  501  are subjected to DCT conversion in the unit of such block, then quantized in a quantization circuit  503  and subjected to variable-length encoding such as Huffman encoding in a variable-length encoding circuit  504 .  
         [0072]    Again referring to FIG. 7, the encoded image data reproduced by the reproducing circuit  402  are subjected, by a watermarking circuit  403 , to the concealing of the watermark information generated by a watermark information generation circuit  406 , according to an instruction from an operation unit  402 , as will be explained later in more details. A distribution server  404  transmits the image data, outputted from the watermarking circuit  403 , to a digital network  405 .  
         [0073]    In the following there will be explained the details of the watermarking circuit  403  with reference to FIG. 9, which is a detailed block diagram thereof.  
         [0074]    Referring to FIG. 9, a variable-length decoding circuit  601  executes variable-length decoding of the encoded image data reproduced by the reproducing circuit  402 . The image data outputted from the variable-length decoding circuit  601  are the image data composed of frequency components. The image data of the frequency components, decoded in the variable-length decoding circuit  601 , are supplied to an adder  602  for superposing the watermark information and an inverse quantization circuit  604 .  
         [0075]    The inverse quantization circuit  604  executes inverse quantization of the image data decoded in the variable-length decoding circuit  601 . The inverse quantized image data are subjected to inverse DCT conversion in the unit of the above-mentioned block by an inverse DCT circuit  605 , whereby the data of the frequency components are converted into in the image data prior to the encoding.  
         [0076]    A motion detection circuit  606  detects, in the unit of the block, motion in the image data from the inverse DCT circuit  605 , through a process similar to that in the foregoing first embodiment.  
         [0077]    The block identified by the motion detection circuit  606  as showing motion, as shown in FIG. 5, is selected-by a block selection circuit  607 , and the watermark information generated by the watermark information generation circuit  606  is superposed, in the adder  602 , to the decoded image data. However, the block selection circuit  607  conceals the watermark information only in a frame in which the motion is detected in a predetermined number of blocks, based on a predetermined frame interval obtained by a frame counter  608 . Thus the watermark information is not concealed in the image data of all the frames but is superposed with the image data of the blocks showing motion with a fixed frame interval or with a random frame interval. In this context, the watermark information concealing method is the same as the second embodiment.  
         [0078]    The image data outputted from the adder  602  are again variable-length encoded by the variable-length encoding circuit  603 , and are supplied to the distribution server  404 .  
         [0079]    In the present embodiment, the watermark information (for example copyright information) can be concealed anew into the encoded image data for distribution through the network, so that it is rendered possible to protect the copyright of the distributed image data.  
         [0080]    Also, the watermark information is concealed in a part of the frequency space of the image with motion, whereby the watermark information is scarcely detectable in the decoded image but the watermark information continues to remain in the encoded image.  
         [0081]    Also, as the watermark information is concealed into the moving image only in a part thereof in which the motion is detected, it is rendered possible to prevent the increase in the data amount. Also as the still image portion does not contain the watermark information, it is not possible to extract the watermark information through the comparison of the successive images, so that any third person is unable to find the presence of the watermark or the location thereof. Furthermore, as the watermark information cannot be extracted even after the moving image is cut out or modified, there can be obtained advantages of preventing illegal use of the moving image and tracing such illegal use.  
         [0082]    Furthermore, the watermark information is concealed only in an image area showing motion, so that the image deterioration resulting from the concealing of the watermark information can be rendered inconspicuous visually.  
         [0083]    In the following there will be explained a fourth embodiment with reference to the accompanying drawings.  
         [0084]    The entire configuration of the fourth embodiment is similar to that of the first embodiment and is as already explained with reference to FIG. 1. However, the internal configuration of the compression encoding circuit  103  is different from that in the first embodiment. In the following there will be explained the configuration and the process of the compression encoding circuit  103 .  
         [0085]    [0085]FIG. 10 is a detailed block diagram of the compression encoding circuit  103  in the fourth embodiment, and this circuit executes compression encoding based on MPEG-1 or MPEG-2 standard, in the fourth embodiment.  
         [0086]    The MPEG-1 or MPEG-2 standard executes the encoding process for example with an arrangement of the types of the pictures as shown in FIG. 11.  
         [0087]    In the MPEG standard, there are defined pictures of three types, namely I-picture, P-picture and B-picture.  
         [0088]    For the I-picture (frames n and n+6 in FIG. 11), there is executed an in-frame encoding (intra encoding). For the P-picture (frame n+3 in FIG. 11), the encoding is executed with interframe estimation in the forward direction. For example, for the frame n+3 there is conducted motion compensation encoding with reference to the frame n+6.  
         [0089]    For the B-picture (frames n+1, n+2, n+4, n+5 in FIG. 11), the encoding is executed with interframe estimation in both directions. For the B-picture, there is conducted motion compensation encoding with Preference to the I- and P-pictures positioned in front and at rear.  
         [0090]    In the present embodiment, the watermark information is concealed into the P-picture. In the following there will only be explained the processing of the P-picture. However, the watermark information may also be concealed similarly into the B-picture.  
         [0091]    Referring to FIG. 10, the entered digital image data are divided by a block forming circuit  701  into blocks of a size of 8×8 pixels in the horizontal and vertical directions. The image data divided into blocks are subjected to the calculation, by a subtractor  702 , of the difference from image data from a motion compensation circuit  707  to be explained later, and the obtained difference is supplied to a DCT circuit  703 , which executes DCT conversion in the unit of the block.  
         [0092]    The image data after DCT process is quantized by a quantization circuit  704 , and the quantized image data are supplied to an inverse quantization circuit  708  for partial decoding to be explained later, and are subjected to variable-length encoding such as Huffman encoding by a variable-length encoding circuit  705 .  
         [0093]    On the other hand, the inverse quantization circuit  708  executes inverse quantization on the image data outputted from the quantization circuit  704 , and an inverse DCT circuit  709  executes partial decoding to the image data prior to the encoding.  
         [0094]    The partially decoded image data are supplied to a motion compensation circuit  707 , then retained for a predetermined period, and compared with the currently entered image data to detect the motion (detection of motion vector). The motion compensation circuit  707  supplies the subtractor  702  with image data of a corresponding part for effecting compensation of the motion.  
         [0095]    The motion compensation circuit  707  detects the motion in the unit of a block of 16×16 pixels in the present embodiment, thereby detecting the data of the above-mentioned motion vector. The motion vector is supplied to a gate circuit  710  and an adder  711 .  
         [0096]    The gate circuit  710  transmits the watermark information, generated by the watermark information generation circuit  106 , when the motion vector exceeds a predetermined level. Thus, when the motion vector exceeds the predetermined level, the watermark information is transmitted by the gate circuit  710 , and is superposed with the motion vector data in the adder  711 . When the motion vector does not exceed the predetermined level, the watermark information is not transmitted by the gate circuit  710  and is therefore not superposed with the motion vector data.  
         [0097]    The motion vector data, outputted from the adder  711 , are multiplexed, in a multiplexing circuit  706 , with the encoded image data and are outputted to the recording/reproducing circuit  104 .  
         [0098]    In the present embodiment, the movement of a motion vector is made to correspond to a bit in the bit train of the watermark information, and the watermark information is composed of a combination of plural motion vectors. The number of the motion vectors to be employed and the order of addition of the bits of the watermark information are naturally determined in advance, since these information are required for restoring the watermark information.  
         [0099]    In the present embodiment, the watermark information is concealed, among the motion vectors obtained from the detection of motion, in those of a portion showing a large motion, whereby the watermark information can be incorporated in the unobservable portion of the image, without deterioration of the image.  
         [0100]    Furthermore, the present embodiment provides an effect that any third person is unable to find the presence of the watermark or the location thereof. Furthermore, as the watermark information cannot be extracted even after the moving image is cut out or modified, there can be obtained advantages of preventing illegal use of the moving image and tracing such illegal use.  
         [0101]    The present invention may also be applied to a system consisting of-plural equipment (for example a host computer, interface devices, a reader, a printer etc.) or an apparatus consisting of a single equipment (such as a digital video cassette recorder, a digital camera or a digital television).  
         [0102]    Also the present invention includes a case where program codes of a software for realizing the functions of the foregoing embodiments are supplied to a computer of the apparatus or the system connected to various devices so as to function such devices for achieving such functions, and the computer (MPU or CPU) of such system or apparatus operates such devices according to the stored program.  
         [0103]    In such case the program codes themselves of the software realize the functions of the aforementioned embodiments, and the program cods themselves, and the mans for supplying the computer with such program codes such as a memory medium storing the program codes constitutes the present invention.  
         [0104]    The memory medium storing such program codes can be, for example, a floppy disk, a hard disk, an optical disk, a magnetooptical disk, a CD-ROM, a magnetic tape, a non-volatile memory card or a ROM.  
         [0105]    The present invention also includes not only a case where the functions of the aforementioned embodiments are realized by the execution of the read program codes by the computer but also a case where an operating system or the like functioning on the computer executes all or a part of the actual processes under the control of such program codes thereby realizing the functions of the foregoing embodiments.  
         [0106]    The present invention further includes a case wherein the program codes read from the memory medium are once stored in a function expansion board concealed into the computer or a function expansion unit connected to the computer, and a CPU provided in the function expansion board or the function expansion unit executes all the process or a part thereof according to the instructions of such program codes, thereby realizing the functions of the aforementioned embodiments.  
         [0107]    In other words, the foregoing description of embodiment has been given for illustrative purposes only and not to be construed as imposing any limitation in every aspect.  
         [0108]    The scope of the invention is, therefore, to be determined solely by the following claims and not limited by the text of the specifications and alterations made with a scope equivalent to the scope of the claims fall within the true spirit and scope of the invention.