Information transmitting apparatus

Abstract: Image information to be transmitted is formed, audio information and coordinates information which is made correspond to the audio information and to a position of the image information are formed. The image information, audio information, and coordinates information are transmitted by a single communication. When receiving, the image information, audio information, and coordinates information which is made correspond to the audio information and to the position of the image information are received. The image information and audio information are reproduced. The position indicated by the coordinates information is displayed on the image. The contents of the image can be further easily understood on the reception side of the image. Such a display is realized by a simple system.

Description: The present invention relates to an information transmitting apparatus for transmitting information through a communication line or the like and, more particularly, to an information transmitting apparatus for transmitting an image to be transmitted by a facsimile signal or the like. In recent years, the costs and size of a facsimile apparatus have been being more and more reduced and is being rapidly spread to various fields from a small-scale office to a general home. In addition to a thermal paper, a normal paper can be also used as a recording paper and a using method is also made variable. For example, a supplementary school and a preparatory school uses the facsimile apparatus to instruct and provide information, thereby enabling a home study to be performed. Although the lesson using a satellite communication has already been put into practical use, an attention is paid to the facsimile communication from viewpoints such that the teaching can be performed every person and a large equipment investment is unnecessary. Consequently, for example, a good instruction can be immediately received even at a remote location. An information presentation by the facsimile apparatus is fairly spread by the introduction of a facsimile server on the enterprise side. Even from the user side, although it is troublesome to request references by a telephone, since by merely calling an information presenting station by the facsimile apparatus and dialing the necessary information numbers, a desired reference can be transmitted via the facsimile, it is very convenient. In future, there will be many cases where it is sufficient to receive various information for admission or the like by merely transmitting and receiving via the facsimile apparatus. It is one of advantages of the facsimile apparatus that an instantaneousness is unnecessary in a manner similar to an answering telephone or an electronic mail. The user can receive information any time and it is sufficient for the user to read it as necessary. On the other hand, a telewriting, electronic conference, cooperative work, or the like is effective for explanation, conference, or the like which requires an instantaneousness because information such as voice, image, locus, and the like are simultaneously transmitted to each terminal. According to the conventional apparatus, the facsimile apparatus merely transmits a still image such as a drawing or the like and, in case of needing a description, it is necessary to make sentences of such an explanation and to transmit them together with the still image. In future, even in case of an information presentation to many unspecified or specified persons as well as the communication between the specified persons, it is insufficient to raise an understanding degree of references by only the image information and it is effective to add a voice or information indicative of a specified portion. Such a system has already been put into practice use as a television telephone, television conference system, video conference system, video mail, or the like. However, since they need a very expensive information processing apparatus of a high performance, they cannot be generally easily used. European patent application EP-A-0574138 discloses a remote conferencing system, which includes a network of local computers and a host computer adapted to generate a common image for display by the local computers, and to allow users of the local computers to modify the common image. The conferencing system also includes index cameras and speaker-type telephones to allow the parties to see and speak with each other. HyperCard Reference Manual, Apple Computer, Inc., 1993 discloses a system whereby clicking on a particular part of an image displayed by a display means causes further images and information relating to that particular part of the image to be displayed. It is an object of the invention to solve the above problems. Another object of the invention is to provide an information transmitting apparatus which can effectively transmit both of image information as main information and information to effectively use such an image. According to a first aspect of the present invention there is provided a data processing apparatus for processing data to be transmitted comprising: means for generating image data representative of an image; means for inputting locus data representative of at least one locus on the image; means for inputting audio data; and means for producing control data for causing the locus data to be reproduced in synchronisation with the audio data by a receiving means arranged to receive the transmitted image data, locus data, audio data and control data. According to a second aspect of the present invention there is provided a data processing apparatus for processing received data comprising: receiving means for receiving transmitted image data, locus data and audio data; and reproduction means for reproducing said image data, locus data and audio data; characterised in that: said receiving means is arranged to receive control data transmitted with said image data; and said reproduction means is arranged to respond to said control data to cause said audio data, and locus data to be synchronously reproduced. According to a third aspect of the present invention there is provided a data processing method comprising the steps of: generating image data representative of an image; inputting locus data representative of at least one locus on the image; inputting audio data; and producing control data for causing the locus data to be reproduced in synchronisation with the audio data by a receiving means arranged to receive the transmitted image data, locus data, audio data and control data. According to a fourth aspect of the present invention there is provided a data processing method for processing received data comprising the steps of: receiving transmitted image data, locus data and audio data; and reproducing said image data, locus data and audio data; characterised by the steps of: receiving control data transmitted with said image data, locus data and audio data; and responding to said control data to cause said audio data and locus data to be synchronously reproduced. The above and other objects and features of the present invention will become apparent from the following description of an embodiment of the invention, which are described by way of example only with reference to the accompanying drawings in which: Fig. 1 is a block diagram showing a schematic construction of a whole facsimile apparatus according to an embodiment of the invention; Fig. 2 is a front view showing an external view of the facsimile apparatus of Fig. 1; Fig. 3 is a block diagram in which main functions in the transmitting mode and receiving mode of the facsimile apparatus of Fig. 1 are shown as circuit blocks; Figs. 4A and 4B are diagrams showing examples of a display picture plane in the facsimile apparatus of Fig. 1; Fig. 5 is a diagram showing another example of the display picture plane in the facsimile apparatus of Fig. 1; Fig. 6 is a diagram showing still another example of the display picture plane in the facsimile apparatus of Fig. 1; Fig. 7 is a time chart showing the correspondence relation between a voice and a locus which are inputted in the facsimile apparatus of Fig. 1; Fig. 8 is a diagram showing a table constructed to transmit the voice and locus in Fig. 7; Fig. 9 is a diagram showing a format of transmission data including an image, a voice, and a locus in the apparatus of Fig. 1; Fig. 10 is a flowchart showing the fundamental operation on the transmission side in the apparatus of Fig. 1; Fig. 11 is a flowchart showing the fundamental operation on the reception side in the apparatus of Fig. 1; Figs. 12A, 12B, and 12C are diagrams showing examples of picture planes showing locus displays in the apparatus of Fig. 1; Fig. 13 is a diagram showing an example of a display picture plane including a sub-image in the apparatus of Fig. 1; Fig. 14 is a diagram showing a format of transmission data including a main image, a sub-image, a voice, and a locus in the apparatus of Fig. 1; Fig. 15 is a flowchart showing the fundamental operation in case of transmitting the sub-image also in the apparatus of Fig. 1; and Fig. 16 is a flowchart showing the fundamental operation in case of receiving the sub-image also in the apparatus of Fig. 1. An embodiment of the invention will now be described in detail hereinbelow with reference to the drawings. Fig. 1 is a block diagram showing a schematic construction of a facsimile apparatus according to an embodiment of the invention. Reference numeral 10 denotes a CPU for controlling the whole apparatus; 12 a peripheral input/output (I/O) circuits for performing a memory control and an I/O control; 14 a memory comprising an ROM and an RAM; 16 a hard disk as a secondary memory; and 18 a well-known digitizer serving as a coordinates input device which can solely or continuously input arbitrary coordinates by a pen, a stylus, or the like. In the embodiment, the digitizer 18 is arranged so as to be overlapped to a liquid crystal display (LCD) panel 20, so that an arbitrary locus can be inputted like a feeling such as to be overlapped to an image that is displayed on the liquid crystal display panel 20. Reference numeral 22 denotes a display control circuit for displaying a desired image to the LCD panel 20 under control of the CPU 10. Reference numeral 24 denotes an audio I/O device (for example, handset) comprising a microphone and a speaker; 26 an audio control for controlling an audio input/output by the audio I/O device 24; 28 a read-out output device 28 (namely, an image scanner and a printer) for a facsimile transmission; 30 a facsimile control for controlling the read-out output device 28; 32 a modulating/demodulating apparatus (modem) for frequency modulating a signal to be transmitted to a frequency band that is suitable for a communication line and for demodulating the modulated signal from the communication line; and 34 a communication control circuit for controlling the connection and communication with the communication line such as a setting of a communication protocol or the like. Fig. 2 is a front view showing an external appearance of the facsimile apparatus of the embodiment. The LCD panel 20 and digitizer 18 are arranged at the front surface. The handset 24 is put on the left side. A ten-key 36 to input telephone numbers is arranged on the right side of the front surface. Reference numeral 38 denotes an input pen of the digitizer 18. Fig. 3 is a block diagram in which main functions in the transmitting mode and receiving mode of the apparatus according to the embodiment are shown as circuit blocks. Image input or image formation means 40 generates data of an image read by the read-out output device 28 or an image of a document formed by a word processor or a graphics creating software which operates on the present apparatus. Coordinates input means 42 is the digitizer 18 and calibrates the coordinates which are inputted by the pen 38, adjusts its timing, and generates as clear coordinate values. Audio input means 44 converts an analog audio signal inputted from the handset 24 into a digital signal and generates. Outputs of those means 40, 42, and 44 are supplied to input control/storage means 46 and stored as data. Although the details will be explained hereinlater, when a message by a voice and a locus is added to the inputted image, the input control/storage means 46 displays the inputted image on display means 48 (namely, LCD panel 20). The user inputs an arbitrary locus by the digitizer 18 and inputs a voice by the handset 24 while looking at the displayed image. Data construction table formation means 50 numbers the locus data every stroke and also numbers the audio data while setting a sound without a no-voice state of 0.5 second or more to one data group in a manner such that the inputted voice and coordinate train (locus) can be synchronously reproduced on the reception side, thereby forming a table to set a time-dependent flow. Each data of the inputted image, voice, and locus is transmitted through communication control means 52. Communication control means 54 on the reception side receives information through the telephone line and writes the reception data into storage means 56. In response to a disclosure request of the user on the reception side, output control means 58 reads out the data of the reception image from the storage means and supplies to image output means 62 (printer of the read-out output device 28) and/or display means 64 (LCD panel 20 and a display control circuit 22). When the audio/locus data has been added, the output control means 58 first supplies the reception image to the display means and allows the reception image to be displayed and also promotes the user to reproduce the audio/locus. In response to an audio/locus reproduction request of the user, audio/locus synchronization control means 60 reads out the data regarding the audio and locus from the storage means 56, reproduces and processes the data in accordance with a time set of the table formed by the data construction table formation means 50, supplies the locus data to the display means 64, and supplies the audio data to audio output means 66. Thus, the locus inputted on the transmission side is displayed at the same position as that inputted on the transmission side so as to be overlapped to the reception image which is displayed by the display means 64. A voice is generated synchronously with it. It will be obviously understood that in case of outputting the reception image by a paper, the image output means 62 is used. Figs. 4A and 4B show examples of outputs (the display image of the display means 64 and the audio output of the audio output means 66) which are obtained on the reception side by the facsimile apparatus of the embodiment. Fig. 4A shows a picture plane of a written application which was transmitted. "Information" corresponds to a reproduction button of the locus and audio. By depressing the reproduction button on the reception side, a message "please don't forget to put your seal here" is reproduced by a voice. Synchronously with it, a locus to indicate a portion to put a seal on the display screen is reproduced and displayed. Fig. 5 shows an example of a display on the picture plane for teaching of a supplementary school or a correspondence course or school. Fig. 6 shows an example of a display on the picture plane for guiding complicated roads. Voice and locus as shown in the diagrams are added, respectively. An example of a table forming procedure in the data construction table formation means 50 will now be described. Fig. 7 is a timing chart showing the correspondence relation between the audio and locus to be transmitted. Fig. 8 shows an example of the table for the example shown in Fig. 7. As shown in Fig. 7, audio and locus outputs are generally inputted mutually intermittently. For convenience of explanation, those data groups are referred to as audio 1, audio 2, ..., locus 1, locus 2, .... Reference characters a, b, c, d, and e denote triggers to start the reproduction of the other data and their contents indicate the periods of time from a timing when the reproduction of each data group has been started until a timing when the reproduction of the other data is started. Namely, the audio (or locus) which continues long activates the locus (or audio). When both of the audio and locus are interrupted, the apparatus pauses. For a pause duration, the kind of output is set to either one of the audio and the locus. Reference character E indicates that the reproduction of the data group is finished and there is no data. By alternately performing a trigger, the time control can be easily performed and the data for such a period of time can be also deleted. Such a control contributes to a reduction of a data amount. Fig. 9 shows a format of transmission data including image data, audio data, and locus data. First, in a manner similar to the conventional facsimile apparatus, an image is transmitted and, after that, an audio table and an audio data group are transmitted and a locus table and a locus data group are subsequently transmitted. Fig. 10 is a flowchart showing the fundamental operation on the transmission side of the apparatus according to the embodiment. Whether audio/locus are added after the image was read or formed or not is asked to the user (step S1). When no audio/locus is added (S1), the normal facsimile transmission is executed as it is (S2). When audio/locus are added, an image to be transmitted is displayed on the picture plane of the display means 48 (S3). The apparatus waits for an instruction to start the input of the audio/locus (S4). When the audio/locus input is started, the audio/locus input data is stored (S5). A data group as shown in Fig. 8 is constructed and a table showing the relation between the time and the trigger is formed (S6) and is transmitted together with the input image (S7). Fig. 11 is a flowchart showing the fundamental operation on the reception side of the apparatus of the embodiment. After the reception image was displayed on the display means 64 by an instruction of the user, a check is made to see if the audio/locus data has been added or not (S11). If NO, the processing routine is finished there. If YES (S11), a start button indicative of the existence of the audio/locus data is displayed (S12). The apparatus waits for the depression of the start button (S13). The audio and locus are synchronously reproduced in accordance with the received table (S14). With respect to the locus, it can be reconstructed at a high fidelity as shown in Fig. 12A. However, instead of reconstructing the locus data as it is, as shown in Fig. 12B, an arrow cursor or icon can be also moved so as to reconstruct only its motion. Further, as shown in Fig. 12C, a thickness (and color) of locus can be also originally changed on the reception side. Although the facsimile transmission has been described as an example, the invention can be also obviously applied to a transmitting system of an electronic mail or the like. According to the embodiment as mentioned above, an audio and a locus for compensating it are transmitted as a message for explaining together with a transmission image and, on the reception side, the image is displayed and the audio and locus can be synchronously reproduced, so that an understanding degree of the contents to be transmitted is further improved. The transmitter doesn't need to make sentences of the description as well and can input and output by the audio and locus in a manner similar to the case of normally explaining, so that a use efficiency is improved. Although the audio and locus data have been added to the image in the above embodiment, another image can be also added to them. An embodiment in which an image synchronized with the audio/locus data can be also added will now be described hereinbelow. The apparatus of the construction of Fig. 1 can be applied as it is as a whole construction itself of such an embodiment. The embodiment is useful in, for example, a scene such as an information presentation or the like. Fig. 13 shows an example of a display picture plane including such a sub-image. A main image is a map of a restaurant and its location is described by a voice. After that, an image of a recommended menu (healthy bento of a lunch menu) is displayed so as to be adhered onto the map (main image). The image of the recommended menu is displayed synchronously with the audio. When such a sub-image exists, the data construction table formation means 50 forms a table regarding the reproduction times of the locus and sub-image and the trigger. Fig. 14 shows a format of transmission data including the sub-image data as mentioned above. The data group of the sub-image includes the contents of the sub-image and the coordinates data indicating at which portion of the main image the sub-image is adhered. Fig. 15 is a flowchart showing the operation on the transmission side of the apparatus when there is a sub-image. It is preferable that the sub-image is previously inputted or formed and is moved or adhered at a desired position as necessary. After the main image was read or formed, whether the audio/locus/subimage are added or not is asked to the user (S21). When none of the audio, locus, and sub-image is added (S21), the normal facsimile transmission is executed as it is (S22). When they are added, the main image is displayed to the screen of the display means 48 (S23). The apparatus waits for an instruction to start the input of the audio/locus/sub-image (S24). After the input was started, the input data of the audio/locus/sub-image is stored (S25). The data group to be transmitted is constructed and the table indicative of the relation between the time and the trigger for synchronous reproduction is formed (S26) and is transmitted together with the main image (FAX image) (S27). Fig. 16 is a flowchart showing the operation on the reception side of the apparatus. After the reception image was displayed by the display means 64 by an instruction of the user, a check is made to see if the audio/locus/sub-image data has been added or not (S31). If NO, the processing routine is finished there. When they are added (S31), the start button indicative of the existence of the audio/locus/subimage data is displayed (S32). The apparatus waits for the depression of the start button (S33). The audio, locus, and sub-image are synchronously reproduced in accordance with the received table (S34). In the embodiment, although the sub-image has been displayed so as to be adhered to the main image, the sub-image can be also displayed by a page feeding system like a picture-card show such that the main image is erased and the sub-image is displayed as a next image. Further, the sub-image can be also moved or erased together with the audio and locus. As mentioned above, since another image is displayed in addition to the audio and locus, an expressing power of the explanation can be further improved. As will be easily understood from the above description, according to the invention, in addition to the ordinary image transmission, audio and locus and, further, another image can be synchronously reproduced. Therefore, an understanding degree of information is improved. Further, since there is no need to make sentences of the additional explanation each time as well, the troublesomeness on the transmission side can be also omitted.

Claims: A data processing apparatus for processing data to be transmitted comprising: means (40) for generating image data representative of an image; means (42) for inputting locus data representative of at least one locus on the image; means (44) for inputting audio data; and means (46, 50) for producing control data for causing the locus data to be reproduced in synchronisation with the audio data by a receiving means arranged to receive the transmitted image data, locus data, audio data and control data. An apparatus according to claim 1, further comprising transmission means for transmitting said image data representative of an image, locus data representative of a locus on the image, audio data and control data. An apparatus according to claim 2, further comprising display means (48) for displaying a representation of said image information input by said image input means and a representation of said locus data input by said locus data input means (42). An apparatus according to claim 3, wherein said locus data is representative of at least two loci on said image. An apparatus according to claim 4, wherein said locus data represents a point on a document where a seal is required. An apparatus according to claim 4, wherein said image information represents a map. An apparatus according to any preceding claim, further comprising: means for generating image data representative of an additional image; and means (46,50) for producing control data for causing said additional image data to be reproduced in synchronisation with the audio data by a receiving means arranged to receive the transmitted image data, locus data, audio data and control data. A data processing apparatus for processing received data comprising: receiving means (62, 66) for receiving transmitted image data, locus data and audio data; and reproduction means (64) for reproducing said image data, locus data and audio data; characterised in that : said receiving means (62,66) is arranged to receive control data transmitted with said image data; and said reproduction means (64) is arranged to respond to said control data to cause said audio data, and locus data to be synchronously reproduced. An apparatus according to claim 8, wherein said locus data is representative of at least two loci on said image. An apparatus according to claim 9, wherein said locus data represents a point on a document where a seal is required. An apparatus according to claim 9, wherein said image information represents a map. An apparatus according to any of claims 8 to 11, further comprising: means for receiving image data representative of an additional image; wherein said control data synchronises said audio data with said additional image data when reproduced. A data processing method comprising the steps of: generating image data representative of an image; inputting locus data representative of at least one locus on the image; inputting audio data; and producing control data (56) for causing the locus data to be reproduced in synchronisation with the audio data by a receiving means arranged to receive the transmitted image data, locus data, audio data and control data. A method according to claim 13, further comprising the step of transmitting said image data representative of an image, locus data representative of a locus on the image, audio data and control data. A method according to claim 14, wherein said locus data is representative of at least two loci on said image. A method according to claim 15, wherein said locus data represents a point on a document where a seal is required. A method according to claim 16, wherein said image information represents a map. A method according to any of claims 13 to 17, further comprising the steps of: generating image data (S21) representative of an additional image; and producing control data (S26) for causing said additional image data to be reproduced in synchronisation with the audio data by a receiving means arranged to receive the transmitted image data, locus data, audio data and control data. A data processing method for processing received data comprising the steps of: receiving transmitted image data, locus data and audio data; and reproducing said image data, locus data and audio data; characterised by the steps of: receiving control data transmitted with said image data, locus data and audio data; and responding to said control data to cause said audio data and locus data to be synchronously reproduced (S34). A method according to claim 19, wherein said locus data is representative of at least two loci on said image. A method according to claim 20, wherein said locus data represents a point on a document where a seal is required. A method according to claim 20, wherein said image information represents a map. A method according to any of claims 19 to 22, further comprising the steps of: receiving image data representative of an additional image; wherein said control data synchronises said audio data with said additional image data when reproduced (S34).

Inventors: SHIMADA KAZUTOSHI, SHIMADA, KAZUTOSHI, SHIMADA, KAZUTOSHI, C/O CANON K.K.
Applicants: CANON KK, CANON KABUSHIKI KAISHA
Application year: 1995
Main classification: H04N1
IPC codes: H04N1, H04M11, H04N7