METHOD AND DEVICES FOR TRANSMITTING POINTING DEVICE CO-ORDINATES

Abstract: For the transmission of co-ordinate data of pointing devices (4), such as a so-called mouse, from a terminal (1) to a server (2), use is made of a communication path (3) comprising a transmission channel (T) and a control channel (C). The transmission channel (T) may comprise a video channel (V), an audio channel (A) and/or a data channel (D). The co-ordinate data (S) is transmitted in the same representation as keyboard data, corresponding to the numeric keyboard, by way of the control channel (C). As a result, there are required no amendments to existing equipment, and a transmission of co-ordinate data may be achieved in a simple and effective manner.

Description: Method and devices for transmitting pointing device co-ordinates BACKGROUND OF THE INVENTION The invention relates to a method for transmitting pointing device co-ordinates. More in particular, the invention relates to a method for transmitting, from a first terminal to a second terminal, screen co-ordinates of pointing devices, such as a so-called mouse, "track ball" or "joy stick". In many telecommunication applications it is desirable or even necessary to transmit selection information from the user terminal to another terminal, or to a server. In the event of a videophone, e.g. , it is customary for choice options to be displayed on-screen, such as "dial connection", "repeat number" and "set options". It is customary for the selection information to be entered on the available numeric keyboard ("key pad") of the videophone, using the digits 0-9 and the characters "*" and "#". The videophone subsequently transmits corresponding DTMF signals (DTMF - Dual Tone Multiple Frequency) in the audio channel. In addition, there is the option of transmitting the digits 0-9 and the characters "*" and "#" as characters within the control channel. For the latter purpose, there have been developed protocols, as described in the ITU-T (International Telecommunication Union) recommendations H.242 and H.245. Other application areas in which selection information is applied are, e.g., video courses which are transmitted to the user terminal (computer) by way of a so-called server. The user may select parts of the course by entering choices on the numerical keyboard of his computer. Although it is possible to enter selection information with a keyboard, many users feel there is the need of being able to make a selection with the help of a pointing device, such as a so-called mouse or a "track ball". Although applications which run on a server are often set to receive selection information and for this purpose may display choice options on a video screen, existing servers are set only to receive selection information in the form of the digits 0-9, possibly extended to the characters "*" and "#" . International patent application W093/08522 discloses a device for transmitting video information. The position of a pointing device may be transmitted to a corresponding remote device. If video information is to be transmitted, the device switches from a speech mode to a data mode. The simultaneous transmission of video and audio information is not possible. SUMMARY OF THE INVENTION The object of the invention is to eliminate the above and other drawbacks of the prior art, and to provide a method which enables the transmission of co-ordinate data in a simple but effective manner. In addition, the object of the invention is to enable the transmission of co-ordinate data with minimal amendments to existing hardware. Another object of the invention is to provide devices with which the method may be implemented. The invention therefore provides a method for transmitting co¬ ordinate data from a first terminal to a second terminal by way of a communication path, in which communication path there have been provided a transmission channel and a control channel, which method according to the invention is characterised by the steps of: converting, in the first terminal, the co-ordinate data into a representation in conformity with keyboard data, transmitting, by way of the control channel, the converted co¬ ordinate data, and - converting back, in the second terminal, the transmitted converted co-ordinate data. In other words, the co-ordinate data is transmitted, in accordance with the invention, in the shape of an existing representation of keyboard data. This has the advantage that there need not be made any amendments for an entirely new representation. In addition, the co¬ ordinate data is transmitted in the control channel, so that in the transmission channel (which may comprise an audio part, a video part and/or a data part) no amendments are required. The said keyboard data, corresponding to, e.g., the digits 0 to 9, extended to "*" and "#" , are usually generated as DTMF signals in the audio channel. In addition, the digits 0-9 and the characters "*" and "#" may be transmitted in the PSTN (Public Switched Telephone Network) by way of so-called "User Input Indication Messages" or UII messages. In the event of an ISDN (Integrated Services Digital Network) , there has been provided for "Single Byte Extension Messages" or SBE messages. According to the invention, said standard representations may be used with minimal amendments for the transmission of co-ordinate data. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 schematically shows a communications system to which the invention is applied. FIG. 2 schematically shows the appearance of the first terminal of the system of FIG. 1 in more detail. FIG. 3 schematically shows the functional structure of the communication path of FIG. 1. FIG. 4 schematically shows the system of FIG. 1 in more detail, with the terminals being built up according to the invention. EMBODIMENTS The communications system shown in FIG. 1 by way of example comprises a first terminal 1 and a second terminal 2, connected by a communication path 3. The first terminal 1 is, e.g., the computer or videophone of a user, while the second terminal 2 is, e.g., a server on which video applications are running. By way of the communication path 3, video information is transmitted from the second to the first terminal, while selection information in the form of co-ordinate data may be transmitted from the first terminal to the second. The communication path 3 may be formed by a single transmission line, or by a communications network. To the first terminal 1 there is connected a pointing device (mouse) 4. Instead of a mouse, there may also be applied a so-called "track ball" or possibly a "joy stick". The second terminal 2 may be formed by a conventional computer (PC). The second terminal 2 preferably is a server, i.e., a so-called "dedicated computer", which is designed for video communication according to ITU-T recommendations H.320, H.323 and H.324. In FIG. 2, the first terminal 1 of FIG. 1 is shown in more detail. On the screen 5 of the terminal 1, there are shown several choices. By moving the mouse 4, the cursor arrow 6 may be placed on one of the selection "buttons" 7, whereafter the button in question is activated with the help of "clicking". Thereupon, the terminal 1 generates selection information in the form of co-ordinate data, i.e., data which corresponds to the co-ordinates of the button in question on the screen. In addition, the terminal 1 may produce auxiliary data, such as the type of action (e.g., single or double clicking) and the window in question within the image. Below, the term co-ordinate data will always be understood to mean the combination of the co¬ ordinates with auxiliary data, if any. According to the prior art, selection information may be transmitted only in the form of keyboard data, i.e., data which is generated by the terminal in response to the operation of, e.g., the numeric keyboard 8. Current servers which are designed, according to the ITU recommendations H.320, H.323 or H.324, for video communications are not capable of receiving co-ordinate data. The invention offers an option of still transmitting co-ordinate data to such servers, so that the user may choose between the use of the (numeric) keyboard 8 and of the mouse 4. This will be further explained later. The communication path 3 schematically shown in FIG. 3 comprises a transmission channel (transmission part) T and a control channel (control part) C. As is shown in FIG. 3 by way of example, the transmission channel T may comprise an audio channel A, a video channel V and a data channel D. In accordance with the invention, selection information S, i.e., co-ordinate data, is transmitted by way of the control channel C. For this purpose, the selection information S is first converted into an appropriate representation R which, in accordance with the invention, corresponds to the representation of the digits 0-9 and the characters "*" and "#" . At the receiving side of the communication path 3, the representation R is converted back into selection information S. In FIG. 4, the system of FIG. 1 is shown in more detail. Just as in FIG. 1, a first terminal 1 is connected, by way of a communication path 3, to a second terminal 2. The first terminal 1 and the second terminal 2 in the example of FIG. 4 are formed by a first processor device 1 and a second processor device 2, respectively. Processor device 1 is provided with an input II for receiving keyboard signals. Said input II is coupled to converter means 18 for converting the keyboard signals into control information (keyboard data) and is provided with an auxiliary input 12 for receiving co- ordinate signals. Said auxiliary input 12 is coupled to auxiliary converter means 19 for converting the co-ordinate signals into control information (co-ordinate data). Both converter means 18 and 19 are coupled, by way of a control interface 15, to (de)multiplexer means 12 for multiplexing information (audio, video and/or control information) to be transmitted by way of input/output 10 and for demultiplexing information (audio, video and/or control information) to be received by way of input/output 10. Furthermore, processor device 1 is provided with an audio unit 16 (comprising, e.g., a loudspeaker and a microphone) which is coupled, by way of an audio interface 13, to the (de)multiplexer means 12, and with a video unit 17 (comprising, e.g., a video screen and a camera) which is coupled, by way of a video interface 14, to the (demultiplexer means 12. The input/output 20 of processor device 2 is coupled to (de)multiplexer means 21 for multiplexing information to be transmitted by way of input/output 20 and for demultiplexing information to be received by way of input/output 20. The (de)multiplexer means 21 are coupled to a control interface 22, which is further coupled to processor means 26 for processing the control information, and to detector means 23 for detecting co-ordinate data and keyboard data. The detector means 23 are further coupled to the processor means 26 and to again auxiliary converter means 24 for converting the co-ordinate data into co-ordinate signals. The again auxiliary converter means 24 are further coupled to the processor means 26. In the event of a PSTN (Public Switched Telephone Network) or a GSTN (General Switched Telephone Network) , each of the (de)multiplexer means 12 and 21 is known, e.g., by the code H.223 as referred to in ITU-T Recommendation H.223, each of the control interfaces 15 and 22 is known, e.g., by the code H.245 as referred to in (FIG. 1 of) ITU-T Recommendation H.324, the audio interface 13 is known, e.g. , by the code G.723.1 as referred to in (FIG. 1 of) ITU-T Recommendation H.324, and the video interface 14 is known, e.g., by the code H.263 as referred to in (FIG. 1 of) ITU-T Recommendation H.324. The invention is not limited, however, to PSTN or GSTN and might very advantageously be applied to, e.g., ISDN. The operation of the telecommunications system shown in FIG. 1 is, e.g., as follows. From the processor device 2 which is implemented, e.g., as a server (although processor device 2 might also be implemented as a terminal) , there is transmitted video information to the processor device 1. The processor device 1 is implemented, e.g., as a terminal. Said video information shows, e.g., an overview of titles of video fragments to be received. A user of the terminal 1 examines said overview and then selects, e.g., one of the titles by generating, by way of a keyboard, so-called keyboard signals (e.g., by keying in the digit one if the first title is desired, the digit two if the second title is desired etc.). Said keyboard signals arrive at the input II and are converted, by way of the converter means 18 and control interface 15, into control information (keyboard data) . Subsequently, the (de)multiplexer means 12 multiplex said control information with available audio information and/or video information, if any, and transmit the multiplexed information by way of the input/output 10 and the communication path 3 which comprises, e.g., an audio, video and control channel. In the server 2, the assembly is demultiplexed by way of the (de)multiplexer means 21, whereafter the control interface 22 and the processor means 26 process the control information, which results in that the film associated with the selected title is transmitted to the user at a specific point in time. The use of a telecommunications system which operates in said known manner, however, has the drawback, inter alia, that it is awkward and cumbersome. By providing the first processor device 1 of the telecommunications system according to the invention with the auxiliary input 12 for receiving co-ordinate signals, and with auxiliary converter means 19 located between the auxiliary input 12 and the (de)multiplexer means 12 for converting the co-ordinate signals into control information, and by providing the second processor device 2 of the telecommunications system with detector means 23 coupled to the (de)multiplexer means 21 for detecting co¬ ordinate data and keyboard data, there is created the option of the user being capable of generating, by way of, e.g., a so-called "pointing device", such as, e.g., a so-called mouse or a so-called joy stick or a so-called track ball, co-ordinate signals which are converted, by way of the auxiliary converter means 19, into control information (co-ordinate data). Thus, the co-ordinate data may be transmitted instead of, or in addition to, the keyboard data. Thereafter, in the server 2 by way of the detector means 23, it should be detected whether a piece of control information represents co¬ ordinate data or keyboard data. The transmission of co-ordinate signals originating from the pointing device instead of the keyboard signals, and therefore in a representation corresponding to the keyboard signals, makes it possible that the number of modifications and/or amendments to the devices be kept to a minimum. Of course the use of pointing devices, such as a mouse or a joy stick, is known per se, and the exchange of co-ordinate signals between terminals is known per se. In this connection, however, there is generally included a separate channel, or there are required sweeping amendments to existing devices. If the control information is subdivided into indication inputs (such as so-called "user indication inputs"), with a keyboard signal comprising a first number of bits which are transmitted by way of one indication input, and with a co-ordinate signal comprising a second number of bits which are transmitted by way of one indication input, which second number of bits is greater than the first number of bits, a co-ordinate signal is converted, by way of the auxiliary converter means 19, into the second number of bits, while a keyboard signal is converted, by way of the converter means 18, into the first number of bits. The second number of bits is greater than the first number of bits, since a co-ordinate signal will generally consist of, e.g., two co-ordinates, one or more click signals and a window signal. It is noted that a so-called indication input is a message which comprises a type indication (UII) and a content (corresponding to the key or button activated by the user) and which is transmitted by way of the control channel. Also, there is the option that a keyboard signal comprises a symbol which is transmitted by way of one indication input, with a co¬ ordinate signal comprising at least two symbols which are transmitted by way of one indication input. In this case, the conversion is carried out very efficiently, since a co-ordinate signal is coded, as it were, into several symbols. As a result, one symbol in an indication input represents a keyboard signal, while several symbols in an indication input represent a co-ordinate signal. In either case, the detector means 23 detect the co-ordinate data and the keyboard data as a function of the number of bits. In this connection, the detector means 23 might be implemented with counter means which have to count only the number of bits (or the number of symbols) to be able to determine whether it concerns a co¬ ordinate signal or a keyboard signal. A co-ordinate signal may consist of, e.g., a signal corresponding to two co-ordinates, "Xrelative" and "Yrelative", each of which is defined as a so-called "four-digit integer", to one or more click signals, each of which is defined as a so-called "two-digit integer", and to a window signal which is defined as a so-called "five-digit integer". In this case, an indication input will consist of the string of said integers, each pair of which is separated, e.g., by an "*" or a "#" or a combination thereof. The auxiliary converter means 19 should then convert the co-ordinate signal into said string. In this connection, Xrelative (the relative X co-ordinate of the pointing device at the moment of selection, i.e., the moment of activating the pointing device, e.g., by clicking) is calculated, e.g., by determining the ratio between Xposition (the X co-ordinate of the pointing device) and Xmaximum, multiplying said ratio by the value 9999 and taking the integer value of the result, and Yrelative is calculated, e.g., by determining the ratio between Yposition and Ymaximum, multiplying said ratio by the value 9999 and taking the integer value of the result. The again auxiliary converter means 24 should then convert back the string in a similar way. When the control information is broken down into at least indication inputs of a first type (e.g., a so-called "user indication input" for the purpose of keyboard signals) and indication inputs of a second type (e.g., a so-called "mouse indication input" for the purpose of co-ordinate signals) , with a keyboard signal therefore being transmitted by way of an indication input of a first type, and a co-ordinate signal being transmitted by way of an indication input of a second type, and since an indication input comprises a number of bits of which a first part indicates what type of indication input is involved and of which a second part indicates what is the content of the indication input, it may thus be indicated by the first part what type of indication input is involved. In this connection, the detector means 23 might be implemented with comparator means which compare the first part of the indication input with a predetermined value, in order to be capable of determining whether it concerns a co- ordinate signal or a keyboard signal. If a conflict between signals originating from the converter means 18 and signals originating from the auxiliary converter means 19 must be avoided, there should be included an arbitrator function known to those skilled in the art between the converter means 18 and the auxiliary converter means 19 on the one hand and control interface 15 on the other. Of course, there also exists the option of using two control interfaces 51 and 52, each of which is placed in series with both converter means 18 and 19, with such arbitrator function then having to be included between both control interfaces 51 and 52 on the one hand and (de)multiplexer means 12 on the other. In practice, the telecommunications system shown in FIG. 1 in most cases will be further supplemented by modems which are then placed between the (de)multiplexer means 12 and 21, respectively, and input/output 10 and 20, respectively. In addition, there is the option of implementing each part of each processor device 1 and 2 completely in the form of hardware as well as completely in the form of software, or partially in the form of hardware and partially in the form of software. Thus, control interfaces 15 and 22 are already available in the form of software packages, while video interface 14 is currently available only as a hardware package. In laboratories, however, video interfaces have already been implemented completely in the form of software. It will be understood by those skilled in the art that the invention is not limited to the embodiments described above, and that many modifications and amendments are possible without departing from the scope of the invention.

Claims: CLAIMS 1. Method for transmitting co-ordinate data (S) from a first terminal (1) to a second terminal (2) by way of a communication path (3) , in which communication path (3) there is provided a transmission channel (T) and a control channel (C) , characterised by the steps of: converting, in the first terminal (1), the co-ordinate data (S) into a representation (R) in conformity with keyboard data, transmitting, by way of the control channel (C) , the representation (R) of the co-ordinate data, and - converting back, in the second terminal (2) , the transmitted converted co-ordinate data. 2. Method according to claim 1, wherein the transmission channel (T) is designed for PSTN and the representation comprises User Input Indications (UIIs) . 3. Method according to claim 1 or 2, wherein the transmission channel (T) is designed for ISDN and the representation comprises Single Byte Extensions (SBEs). 4. Method according to any of the preceding claims, wherein the transmission channel (T) comprises an audio channel (A) , a video channel (V) and/or a data channel (D) . 5. Method according to any of the preceding claims, wherein the second terminal (2) comprises a server. 6. Method according to any of the preceding claims, wherein co¬ ordinate data is transmitted in the form of numbers which represent at least a horizontal position, a vertical position and an identification, with the numbers being separated by characters (*, #) . 7. First processor device (1) for use as a video terminal, comprising: an input (II) for receiving keyboard signals, - converter means (18) coupled to the input (II) for converting the keyboard signals into control information, (de ultiplexer means (12) coupled to the converter means (18) for (de)multiplexing audio information, video information and/or control information, - an input/output (10) coupled to the (demultiplexer means (18) for transmitting/receiving the information by way of a communication path (3) channel, characterised in that the first processor device (1) is further provided with: an auxiliary input (12) for receiving co-ordinate signals, auxiliary converter means (19) located between the auxiliary input (12) and the (de)multiplexer means (12) for converting the co-ordinate signals into control information. 8. First processor device according to claim 7, designed for converting into control information subdivided into indication inputs, with a keyboard signal comprising a first number of bits which are transmitted by way of one indication input, and with a co-ordinate signal comprising a second number of bits which are transmitted by way of one indication input, with the second number of bits being greater than the first number of bits. 9. First processor device according to claim 8, designed in such a manner that a keyboard signal comprises a symbol which is transmitted by way of one indication input, with a co-ordinate signal comprising at least two symbols which are transmitted by way of one indication input. 10. First processor device according to claim 7, designed in such a manner that the control information is broken down into at least indication inputs of a first type and indication inputs of a second type, with a keyboard signal being transmitted by way of an indication input of a first type and a co-ordinate signal being transmitted by way of an indication input of a second type. 11. Second processor device (2) for use as a video terminal, comprising: an input/output (20) for, by way of a communication path (3), transmitting/receiving audio information, video information and/or control information, (de)multiplexer means (21) coupled to the input/output (20) for (de)multiplexing the information, processor means (26) coupled to the (demultiplexer means (21) for processing the control information, characterised in that the second processor device is further provided with: - detector means (23) coupled to the (de)multiplexer means (21) for detecting co-ordinate data and keyboard data. 12. Second processor device according to claim 11, designed for receiving control information which is subdivided into indication inputs, with a keyboard signal comprising a first number of bits which are transmitted by way of one indication input, and with a co-ordinate signal comprising a second number of bits which are transmitted by way of one indication input, with the second number of bits being greater than the first number of bits, and with the detector means (23) being designed for detecting the co-ordinate data and the keyboard data as a function of the number of bits. 13. Second processor device according to claim 12, designed for receiving a keyboard signal comprising a symbol which is transmitted by way of one indication input, with a co-ordinate signal comprising at least two symbols which are transmitted by way of one indication input. 14. Second processor device according to claim 11, designed for receiving control information which is subdivided into at least indication inputs of a first type and indication inputs of a second type, with a keyboard signal being transmitted by way of an indication input of a first type and a co-ordinate signal being transmitted by way of an indication input of a second type, and wherein the detector means (23) are designed for detecting the co-ordinate data and the keyboard data as a function of the type of indication input.

Inventors: LOCHS HENDRIK GERARDUS MARIE, RIJKSE KAREL JAKOB, LOCHS, HENDRIK, GERARDUS, MARIE, RIJKSE, KAREL, JAKOB
Applicants: LOCHS HENDRIK GERARDUS MARIE, NEDERLAND PTT, RIJKSE KAREL JAKOB, KONINKLIJKE PTT NEDERLAND N.V., LOCHS, HENDRIK, GERARDUS, MARIE, RIJKSE, KAREL, JAKOB
Application year: 1997
Main classification: H04L29
IPC codes: H04L29