Patent Publication Number: US-2004046749-A1

Title: Image recording and replay apparatus

Description:
INCORPORATED BY REFERENCE  
       [0001] The disclosure of the following priority application is herein incorporated by reference: Japanese Patent Application No. 8-272413, filed Oct. 15, 1996.  
       BACKGROUND OF THE INVENTION  
       [0002] 1. Field of Invention  
       [0003] The present invention relates to an image recording and replay apparatus that records and replays images, and in particular, to an image recording and replay apparatus that records and replays line drawings.  
       [0004] 2. Description of Related Art  
       [0005] In general, image recording and replay apparatus are known that input a line drawing using a tablet (e.g., a touch tablet). In this type of tablet input image recording and replay apparatus, the line drawing can be drawn on the screen by writing on the tablet with, e.g., a light pen or the like.  
       [0006] However, in the conventional image recording and replay apparatus which uses a tablet, only the completed line drawing is recorded as an image in memory. Because of this, the process of drawing a line drawing from start to completion cannot be reproduced.  
       [0007] In order to produce (and display) the process of drawing the line drawing using a conventional image recording and replay apparatus, the line drawing being produced had to be completed little by little (i.e., in stages), with each stage being saved, one by one as separate files. Consequently, the number of image files created was equal to the number of times the line drawing was saved. This would require a series of user input commands to save the image (one for each stage).  
       [0008] Additionally, the replay operation had to be performed one by one for each of these image files, with a separate user input command to display each image. The line drawing process could not be replayed smoothly from the beginning of the drawing until the end of the drawing.  
       SUMMARY OF THE INVENTION  
       [0009] In order to solve the above and other problems, an image recording and replay apparatus of the invention not only records and replays a completed line drawing, but also reproduces the intermediate stages of the image drawing process. A position input device, such as, for example, a touch panel or a mouse, allows an operator to input position coordinates on a screen. The position input device sequentially takes in these position coordinates over time. A recording device sequentially records the position coordinates that are taken in via the position input device to a memory where they are stored along with information (e.g., pointers) relating to the take-in order. Thus, the memory stores information relating to the position coordinates in a time sequence. An image drawing device reads out the position coordinates recorded in the memory. Then, following the take-in order of the position coordinates, the image drawing device sequentially performs image drawing by drawing the points corresponding to those position coordinates on the screen.  
       [0010] Accordingly, on the screen, the intermediate stages of the line drawing process performed by the operator is realized (illustrated, or displayed) as if it is being drawn at that moment.  
       [0011] Preferably, the recording device automatically completes the recording of the position coordinates for a particular line drawing (storing these coordinates as a particular file) when a new position coordinate is not taken in (input) after a predetermined time period.  
       [0012] The image recording and replay apparatus also can include a sound replay device that reads out sound data recorded in the memory to replay the sound. The image drawing device can perform the read-out operation of the position coordinates in conjunction with the read-out operation of the sound replay device. Accordingly, the image drawing operation is performed along with and the sound replay operation.  
       [0013] Preferably, the image drawing device controls the replay period of the sound and the image drawing period so as to have approximately the same duration. Accordingly, the image drawing speed is automatically adjusted to conform with the length of the sound replay period. 
     
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
     [0014] The invention will be described in conjunction with the following drawings in which like reference numerals designate like elements and wherein:  
     [0015]FIG. 1 is a high-level functional diagram of aspects of the invention;  
     [0016]FIG. 2 is a block diagram of an embodiment of the invention;  
     [0017]FIG. 3 is a first flow chart explaining the operation of the embodiment;  
     [0018]FIG. 4 is a second flow chart explaining the operation of the embodiment;  
     [0019] FIGS.  5 A- 5 B demonstrate a line drawing process and the recording of information according to an embodiment of the invention;  
     [0020] FIGS.  6 A- 6 B show display screens;  
     [0021] FIGS.  7 A- 7 B show display screens and demonstrate a line drawing process in which an error is corrected;  
     [0022] FIGS.  8 A- 8 B show display screens of a completed line drawing;  
     [0023] FIGS.  9 A- 9 B show display screens in which a line drawing is replayed; and  
     [0024]FIG. 10 shows a display screen of a completed, replayed line drawing. 
    
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS  
     [0025]FIG. 1 is a high-level representation of aspects of the invention. The image recording and replay apparatus includes a position input means  1  that takes in a position coordinate externally provided to a screen. A recording means  2  records the position coordinates taken in by the position input means  1  onto a recording medium corresponding to the take-in order. An image drawing means  3  performs image drawing by sequentially rewriting on a display screen points corresponding to the position coordinates taken in by the recording medium  2 .  
     [0026] According to one aspect of the invention, the recording of the position coordinates for a particular line drawing is completed when position coordinates have not been taken in for a predetermined time period.  
     [0027] According to another aspect of the image recording and replay apparatus of the invention, a sound replay means  4  is provided to read out sound data recorded in the recording medium and replay the sound data as sound. The image drawing means  3  can read out the position coordinates from the recording medium in parallel with the operation of the sound replay means  4  reading out the sound data from the recording medium. Preferably, the image drawing means  3  is capable of controlling the image drawing speed so that a sound replay period, during which sound is replayed by the sound replay means  4 , matches the image drawing period. Thus, the line drawing will be completed at the same time that the sound reproduction is completed.  
     [0028] Hereafter, embodiments of the present invention are explained referring to the drawings. According to the illustrated embodiment, the image recording and replay apparatus is implemented in an electronic camera that is capable of recording and replaying still images and moving images.  
     [0029] In the interior of the main body  21  of the FIG. 2 camera, a magnetooptical recording medium  22  is detachably provided. A data access component  23 , which reads from and writes to the magnetooptical recording medium  22 , is connected with a frame memory  25  via a data compression/decompression component  24 . The output of the frame memory  25  is input to a liquid-crystal display component  27  via a display image creating component  26 .  
     [0030] A touch panel  28  is provided for the liquid-crystal display component  27 . The position information which is detected by the touch panel is input to a controller  30  via a touch panel input component  29 . The output of the controller  30  is input to the data compression/decompression component  24  and to the display image creating component  26 . The output of the controller  30  is also connected to the data access component  23 .  
     [0031] The functions of the data compression/decompression component  24 , the display image creating component  26 , the touch panel input component  29  and the controller  30  are realized through a microprocessor  31 .  
     [0032] A sound input/output component  37 , which comprises a microphone and a speaker, also is provided in the main body  21 . The output terminal of the sound input/output component  37  is connected to the input terminal of A/D converter  38 . The input terminal of the sound input/output component  37  is connected to the output terminal of D/A converter  39 .  
     [0033] The A/D converter  38  and the D/A converter  39  are connected to a sound signal processor  40 . The sound signal processor  40  also is connected to the data access component  23 .  
     [0034] A camera component  32 , which is attached to the main body  21 , is provided with a photograph lens  33 , An imaging element  34  is arranged in a position to receive light from the object, which is obtained via the photograph lens  33 . The imaging component  34  is a CCD (Charge Coupled Device), which is a photoelectric converter in that it converts light images to electrical (image) signals. Other photoelectric converters include CMOS devices and PSDs (Photo-Sensitive-Diodes).  
     [0035] The output signal of the imaging element  34  is input to the frame memory  25  via A/D converter  35  and a digital signal processor (DSP)  36 .  
     [0036] With respect to the high-level representation of FIG. 1, the position input means  1  includes the touch panel  28 . The recording means  2  includes the data access component  23 , the touch panel input component  29  and the image drawing data creation function performed by the controller  30 . The image drawing means  3  includes the data access component  23 , the display image creation component  26  and the reading out function of the image drawing data performed by the controller  30 .  
     [0037] The sound replay means  4  includes the data access component  23 , the reading out function of the sound data performed by the controller  30 , the sound input/output component  37 , the D/A converter  39  and the sound signal processor  40 .  
     [0038]FIG. 3 and FIG. 4 are flow charts explaining the operation of the embodiment. The operations of the embodiment are now explained referring to FIGS.  2 - 10 .  
     [0039] The magnetooptical recording medium  22  is provided in the main body  21 .  
     [0040]FIG. 6A shows the initial screen displayed in the liquid crystal display component  27 .  
     [0041] The operation button for recording and replaying is arranged in the lower part of the screen of the liquid crystal display component  27 . The following buttons are provided, starting on the left: a reverse button  42 , a fast forward button  43 , a reverse slow-motion button  44 , a pause button  45 , a slow-motion button  46 , a reverse replay button  47 , a stop button  48 , a replay button  49  and a recording button  50 .  
     [0042] At the right side of the screen of the liquid crystal display component  27  are arranged, from the top, a still image replay button  51 , a moving image replay button  52 , a replay-with-sound button  53  and a clock counter component  54 .  
     [0043] When these operation buttons are operated, the buttons are switched to a highlighted display, and their respective conditions are displayed.  
     [0044] The operations of all the operation buttons are performed via the touch panel  28  or the like; which uses well known GUI (Graphical User Interface) technology. In short, in the liquid crystal display component  27 , the operation buttons are displayed as objects, and the touch panel  28  and the touch panel input component  29  sense the contact operation of the operator with respect to these objects. As opposed to a touch panel, the invention also can be implemented using a mouse or other input device to move a cursor over desired icons, which are then selected by clicking, for example.  
     [0045] The controller  30 , corresponding to this contact (or selection) operation, generates a set message relating to the object on the screen. The display image creating component  26  receives that message, and, for example, changes the screen display. The data access component  23  performs a reading operation that corresponds to that message.  
     [0046] When the operator touches a button via the touch panel  28 , a voltage is generated by the touch panel  28  being pressed. The signal representing that voltage is conveyed to the controller  30  via the touch panel input component  29 . As illustrated in FIG. 6B, the controller  30  recognizes the fact that the recording button  50  is pressed based upon that signal, and starts the recording of the line drawing. When the operator touches the touch panel  28 , the touched positions are input to the controller  30  via the touch panel input component  29  as point information such as a position coordinate data (X, Y).  
     [0047] If it is determined in step S 1  that coordinate data has been input within a specified time, flow proceeds to step S 2 . In step S 2 , the display image creating component  26  sequentially displays points or combinations of points (such as lines) on the liquid crystal display component  27  based upon the coordinate data which is obtained via the controller  30 . At this time, the points are sequentially displayed on the liquid crystal display component  27  at exactly the same position as the position that the operator touched on the touch panel  28 . By repeating this operation, the operator can draw the line drawing via the touch panel  28 . The time interval by which the position coordinates are taken from the touch panel  28  (hereafter, “sampling interval time”) is, for example, a time interval of 5 ms.  
     [0048] The controller  30  creates the image drawing data from the obtained coordinate data. As illustrated in FIG. 5A, assume that the controller  30  took in the coordinate data of the positions (X 1 , Y 1 ) through (X 9 , Y 9 ). Since the sampling interval time is a certain time, if there is a distance between the data of (X 8 , Y 8 ) and (X 9 , Y 9 ), the control is able to determine that the operator drew a line quickly in that interval.  
     [0049] As illustrated in FIG. 5B, the controller  30 , with the coordinate data as an element, creates bi-directional list data by adding a pointer area which indicates the address (pointer) of the coordinate data input at the time interval that is one before and one after that coordinate data (step S 3 ). Thus, for example, the coordinate position (X 2 , Y 2 ) includes a first pointer pointing to positions (X 1 , Y 1 ) as the preceding position data and a second pointer pointing to position (X 3 , Y 3 ) as the subsequent position data.  
     [0050] The controller  30  records this bi-directional list data as image drawing data in the magnetooptical recording medium  22  via the data access part  23  (step S 4 ). Furthermore, between each coordinate data, the display image creating component  26  performs interpolation to draw a line segment.  
     [0051] By the above mentioned operation, as shown in FIG. 7A, the image drawing data of the line drawing is recorded one by one in the magnetooptical recording medium  22  while the operator is drawing the line drawing.  
     [0052] When the operator wishes to re-draw the line drawing during recording of the line drawing, as illustrated in FIG. 7B (e.g., to make a correction to the line drawing), the operator touches the pause button  45  and the reverse slow-motion button  44  (the operation  1  and the operation  2 ). When the controller  30  recognizes that the pause button  45  is pressed (YES in step S 5 ), the controller  30  stops the creation and the recording of the image drawing data (step S 7 ). Additionally, when the controller  30  recognizes that the reverse slow motion button  44  is pressed, it reads out the image drawing data which is recorded in the magnetooptical recording medium  22  in the reverse direction rather than in the recorded order. Since the image drawing data is a bi-directional direction list, it is possible to sequentially read out the data starting from the end-point data by following the pointers. The controller  30  seeks the latest recorded data, and reads out in the reverse direction rather than in the recorded direction.  
     [0053] The controller  30  outputs the coordinate data to the display image creating component  26  from the readout image drawing data. At this time, the controller  30  orders the display image creating component  26  to sequentially (in reverse order) delete the points displayed for the coordinate data. The display image creating component  26  sequentially deletes the displayed points, in accordance with the read out coordinate data. Accordingly, the reverse replay is performed, and the points sequentially disappear from the locations that the operator just drew (step S 8 ).  
     [0054] The above-mentioned reverse replay operation continues until the pause button  45  is pressed again. When the operator returns to the point that he wants to amend, and the pause button  45  is pressed again, the controller  30  recognizes that operation, and ends the reverse replay operation. Then, the controller  30  again returns to the condition in which the line drawing recording is possible. At this time, the end point of the data of the image drawing data at the time of the reverse replay becomes the last readout data from the magnetooptical recording medium  22 .  
     [0055] When the operator again starts the image drawing (the operation  3  in FIG. 7B), the controller  30  starts taking in the coordinate data again via the touch panel  28  and the touch panel input component  29 , and outputs that coordinate data to the display image creating component  26 . Further, the controller  30  creates the image drawing data from the coordinate data, and records them in the magnetooptical recording medium  22  via the data access part  23 .  
     [0056] It is possible to connect the newly created image drawing data to the last image drawing data by rewriting the pointer that indicates the address of the following data in the last image drawing data (step S 9 ).  
     [0057] The controller  30  continues the taking in operation of the coordinate data. The creating and recording operation of the image drawing data are repeated until the stop button  48  is pressed (YES in step S 6 ). The controller  30 , as shown in FIG. 8A, automatically completes the creation and recording of the image drawing data when the stop button  48  is pressed (step S 6 ) or when coordinate data is not taken in for a predetermined period (NO in step S 1 ). When the creation or recording of the image drawing data is completed, the image drawing data is controlled as a single file.  
     [0058] Next, the replay operation is explained. The controller  30  determines which one of the still image replay button  51 , the moving image replay button  52 , or the replay button with sound  53  is pressed (step S 11 ).  
     [0059] When the still image replay button is pressed, and the replay button  49  is pressed, the controller  30  recognizes that operation and reads out the image drawing data from the magnetooptical recording medium  22  via the data access part  23 . The controller  30  converts the image drawing data to the coordinate data, and outputs it to the display image creating component  26 . The display image creating component  26  reads in all the coordinate data, and creates the line drawing in a completed condition, and displays it on the liquid crystal display part  27  (step S 12 ).  
     [0060] When the moving image replay button  52  is pressed, and the replay button  49  is pressed, the controller  30  recognizes that operation and reads out the image drawing data from the magnetooptical medium  22  via the data access part  23 . At this time, the controller  30  reads out starting from the top data of the image drawing data in accordance with the pointers (step S 13 ). The controller  30  converts the read out image drawing data to the coordinate data, and sequentially outputs this data to the display image creation component  26 . The display image creation component  26  displays the points in accordance with the coordinate data, and performs the image drawing by writing additional successive points (step S 14 ). Accordingly, the line drawing is replayed as if the operator is drawing.  
     [0061] As shown in FIG. 8B and FIG. 9A, when the replay button  53  with sound is pressed, and the replay button  49  is pressed, the controller  30  recognizes that operation, and reads out the pre-recorded sound data via the data access part  23  (step S 15 ). In this sound data, the replay time is recorded as an attribute of the file. The image drawing time is adjusted by the controller  30  based upon this replay time.  
     [0062] The controller  30  reads out the number of the image drawing data recorded in the magnetooptical recording medium  22 , and calculates the point number of the image drawing (an image drawing speed) per second by dividing the number of image drawing data (the number of coordinates) by the sound replay time (step S 16 ).  
     [0063] When performing a normal image drawing, since the sampling interval time is 5 ms, the display image creating component  26  performs the image drawing at 200 dots per second. When, for example, the image drawing time is longer than the sound replay time, the image drawing speed is made faster than 200 dots per second. When the image drawing time is shorter than the sound replay time, the image drawing speed is made slower than 200 dots per second. By performing these processes, the image drawing time and the sound replay time are made to match.  
     [0064] The controller  30  reads out the image drawing data from the magnetooptical recording medium  22  via the data access part  23 . At this time, the controller  30  reads out the data from the top data of the image drawing data by sequentially following the pointers (step S 17 ) When a certain amount of the image drawing data is read out, the controller  30  converts that data to the coordinate data, and sequentially outputs to the display image creating component  26 . The display image creating component  26  displays the points by following the coordinate data being sent, and performs the image drawing with the calculated image drawing speed by writing additional successive points (step S 18 ).  
     [0065] Next, the controller  30  seeks the sound data via the data access part  23 , and performs the reading out of the sound data (step S 19 ). The sound data which is read out for a certain amount by the data access part  23  is sent to the sound signal processing component  40 , and then is passed through a filter and is amplified by the sound signal processing component  40  and the D/A converter  39 , and is replayed as sound (step S 20 ).  
     [0066] The image drawing operation and the sound replay operation are simultaneously performed by performing, in a time-divided manner, the reading operation of the image drawing data and the reading operation of the sound data described above as shown in FIG. 9B and FIG. 10. When all the image drawing data and the sound data are read out, the image drawing operation and the sound replay operation are completed (step S 21 ).  
     [0067] In the described embodiment, external sound can be recorded. The sound obtained via the sound input/output component  37  is passed through a low pass filter, and is digitally converted at the A/D converter  38  and the sound signal processor  40 . At this time, it is converted with a sampling frequency of 48 khz and 16 bits as the quantization bit number.  
     [0068] The sound data is recorded in the magnetooptical recording medium  22  with a file attribute, such as the file size, the date made and the replay time. This enables the performance of the replay of this sound data and the replay of the line drawing simultaneously.  
     [0069] The main body  21  includes the camera component  32 , which has the imaging element  34  that photoelectrically converts the image obtained via the photographic lens  33 . That signal is converted to a digital signal at the A/D converter  35 . The digital signal processor  36  performs signal processing such as white balance adjustment, gamma correction and interpolation of pixels. The image signal is compressed at the data compression/decompression component  24  via the frame memory  25 . The compressed data is recorded in the magnetooptical recording medium  22  via the data access part  23 . When it is desired to display a previously stored image, it is read from the magnetooptical recording medium  22 , decompressed by component  24 , and displayed. In this way, the photographic data of the frame memory  25  is monitor displayed at the liquid crystal display component  27  via the display image creating component  26 .  
     [0070] In the present embodiment, the line drawing, which was drawn by the operator using the touch panel  28 , can be sequentially replayed. Accordingly, the image is drawn out on the screen as if the operator is drawing the line drawing at that moment, and an interesting picture image can be obtained.  
     [0071] Moreover, a variety of image drawing replays can be performed since not only the sound is replayed simultaneously at the time of replay of the image drawing, but also the image drawing speed can be adjusted to correspond to the sound replay time.  
     [0072] Furthermore, since the touch panel can be formed integrally with the apparatus, an image recording and replay apparatus having a superior portability can be realized.  
     [0073] In the present embodiment, with respect to the magnetooptical recording medium  22 , the reading-out operation of the image drawing data and the reading-out operation of the sound data occur in parallel by accessing the data in a time-divided manner. However, the present invention is not limited to this technique. For example, it is possible to provide another data access component (another reading device) for accessing the sound data.  
     [0074] In the present embodiment it is possible to reproduce how much time it took to draw the line drawing by making the sampling interval time constant. However, the present invention is not limited to this example. For example, the length of time required to draw the line drawing can be accurately reproduced by adding and recording to the coordinate data the time required for taking in the coordinate data.  
     [0075] Further, in the present embodiment, the image drawing speed is changed uniformly in correspondence with the sound replay time. However, the present invention is not limited to this example. For example, the image drawing speed can be made faster by thinning out the data from any parts that take too much image drawing time due to the high number of samples.  
     [0076] In the present embodiment, a touch panel was used as an input component. However, the present invention is not limited to this example. For example, a mouse can be used.  
     [0077] Furthermore, in the present embodiment, a magnetooptical recording medium is used as a memory. However, the present invention is not limited to this example. It is also possible to use other recording media such as, for example, magnetic recording medium, optical recording medium and semiconductor recording medium (i.e., flash memories). Moreover, the memory need not be removable from the apparatus.  
     [0078] In the present embodiment, the replay of a line drawing is described. However, the present invention is not limited to this example. The editing described below can be performed. For example, a line drawing can be added as a title image to the picture image obtained via the camera component  32 . The picture image with an attached title image can be produced by performing combination editing or performing the composed editing of the line drawing file and the photographed image file. Thus, a line drawing can be displayed superimposed on a moving image or on a still image input via the camera component  32 .  
     [0079] In the image recording and replay apparatus of the invention, a line drawing that is drawn by the operator also includes mid-process line drawings so that it appears as if the operator is actually drawing directly on the screen. Accordingly, for example, when the line drawing is used as a title image, the line drawing is gradually drawn on the screen, so that dramatic expression is possible in which anticipation is heightened. Text, including typed letters (e.g., input by a keyboard) can also be sequentially drawn using the invention.  
     [0080] By making the recording of the position coordinates automatically end when position coordinates have not been taken in for a certain period, the condition in which the apparatus is waiting for input for a long time is prevented.  
     [0081] When the image drawing operation and the sound replay operation are performed simultaneously, for example, with respect to a line drawing that depicts an image of summer, when that line drawing is replayed, a greater realism (ambiance, or presence) can be offered by simultaneously playing the sound of waves, producing a high level of dramatic effect that makes the user have a feeling of anticipation.  
     [0082] The image drawing speed can be changed automatically in correspondence with the replay period of the sound. For example, when the sound replay period is long, the image drawing speed becomes slow, and when the sound replay period is short, the image drawing speed becomes faster as if performing fast forward replay. Accordingly, a single line drawing can be image-drawn at various speeds, and an interesting product having a highly expressive effect can be obtained freely.  
     [0083] While this invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the preferred embodiments of the invention set forth herein are intended to be illustrative, not limiting. Various changes may be made without departing from the spirit and scope of the invention as defined in the following claims.