Apparatus, method, and computer program for performing text-to-speech conversion of instant messages during a conference call

An instant message associated with a conference call is received. At least some of the contents of the instant message are converted into speech, which may be represented as an audio stream. The speech is then provided to at least one participant associated with the conference call. The speech provided to the at least one participant may further include additional speech in addition to the speech corresponding to the contents of the instant message. The additional speech could, among other things, identify a participant who sent the instant message.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to:

U.S. patent application Ser. No. 10/610,511 entitled “DISTRIBUTED CALL SERVER SUPPORTING COMMUNICATION SESSIONS IN A COMMUNICATION SYSTEM AND METHOD,” filed on Jun. 30, 2003;

U.S. patent application Ser. No. 10/610,517 entitled “METHOD AND SYSTEM FOR PROVIDING TEXT-TO-SPEECH INSTANT MESSAGING,” filed on Jun. 30, 2003; and

U.S. patent application Ser. No. 10/675,121 entitled “APPARATUS, METHOD, AND COMPUTER PROGRAM FOR PROVIDING INSTANT MESSAGES RELATED TO A CONFERENCE CALL,” filed on Sep. 30, 2003;

each of which is incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates generally to call conferencing systems and more specifically to an apparatus, method, and computer program for performing text-to-speech conversion of instant messages during a conference call.

BACKGROUND

Audio and video conferencing systems are becoming more popular in the United States and around the world. In a conventional conferencing system, one participant communicates audio signals to other participants (often through a multipoint conferencing server or other unit) and receives audio signals from the other participants (indirectly through the server). The participants may also exchange video images allowing the participants to see one another.

SUMMARY

This disclosure provides an apparatus, method, and computer program for performing text-to-speech conversion of instant messages during a conference call.

In one aspect, an instant message associated with a conference call is received. At least some of the contents in the instant message are converted into speech, which may be represented as an audio stream. The speech is then provided to at least one participant associated with the conference call.

In a particular aspect, the speech generated and provided to the at least one participant includes speech generated using the contents of the instant message and additional speech. The additional speech may, for example, identify a person who sent the instant message.

In another particular aspect, the contents of the instant message are forwarded to at least one participant associated with the conference call.

DETAILED DESCRIPTION

FIG. 1illustrates an example communication system100according to one embodiment of this disclosure. The system100shown inFIG. 1is for illustration only. Other embodiments of the system100may be used without departing from the scope of this disclosure.

A problem with conventional conferencing systems is that it is often difficult or impossible for one person who has not joined a conference call to communicate with other people who have joined the conference call. For example, it may be difficult for a person who is running late for a conference call to convey that fact to others who have already joined the conference call. Moreover, some people who have already joined the conference call may not be using devices that are capable of receiving voice or electronic messages from the person running late.

In the illustrated example, the system100includes one or more communication devices102a-102d, a network104, and a media application server (“MAS”)106.

The communication devices102a-102drepresent devices used by users or subscribers during communication sessions. Communication sessions represent data conversions or conversations between devices or applications over a network. For example, each communication device102a-102drepresents an input/output device that could include a microphone and a speaker to capture and play audio information. A communication device102a-102dcould also include a camera and a display to capture and present video information. A communication device102a-102dcould further represent a portable computing device for sending and receiving text or other messages.

During a communication session, one or more of the devices102communicate with the MAS106over the network104. As an example, a communication device102may transmit audio information to the MAS106and receive audio information from the MAS106. Each communication device102may be constructed or configured from any suitable hardware, software, firmware, or combination thereof for transmitting or receiving audio, video, or other information.

The system100shown inFIG. 1illustrates various embodiments of the communication devices102. For example, the communication device102arepresents a wireless mobile station that communicates with the network104through a mobile switching center (“MSC”)108. The communication device102brepresents a wired Internet Protocol (“IP”) telephone that communicates directly with the network104. An example of a suitable device is an i2004 Internet Telephone, commercially available from Nortel Networks of Brampton, Ontario, Canada. The communication device102crepresents a personal computer, such as a desktop computer or a laptop computer. The communication device102drepresents a wireless device, such as a Blackberry or personal digital assistant. One or more of these devices102a-102dmay include video functionality, such as when the communication device102aincludes a video camera or when the communication device102cis coupled to a web camera110.

While this represents several embodiments of the communication devices102, other or additional communication devices102may be utilized in the system100ofFIG. 1. By way of illustration inFIG. 1, each of the communication devices102a-102dis different. It will be understood, however, that the communication devices102in the system100may include or represent the same or similar type of device or other combination of communication devices.

The network104is coupled to the communication devices102, the MAS106, and the mobile switching center108. In this document, the term “couple” refers to any direct or indirect communication between two or more components, whether or not those components are in physical contact with each other. The network104facilitates communication between components of the system100. For example, the network104may communicate Internet Packets (“IP”), frame relay frames, Asynchronous Transfer Mode (“ATM”) cells, Ethernet, X.25, frame relay, or other suitable information protocols between network addresses or devices. The network104may include one or more local area networks (“LANs”), metropolitan area networks (“MANs”), wide area networks (“WANs”), all or portions of a global network such as the Internet, or any other communication system or systems at one or more locations.

The media application server (“MAS”)106is coupled to the network104. The MAS106supports communication sessions between communication devices102in the system100. For example, the MAS106may receive from one or multiple communication devices102requests to establish or join a conference call. The MAS106may also transmit/receive audio or video information to/from each communication device102involved in the conference call.

The MAS106may be constructed or configured using hardware, software, firmware, or combination thereof for supporting communication sessions in the system100. As an example, the MAS106could include one or more processors112that execute instructions and one or more memories114that store instructions and data used by the processors112. The processor(s)112is generally understood to be a device that drives a general-purpose computer. It is noted, however, that other processor devices such as microcontrollers, Field Programmable Gate Arrays (FPGAs), or Application Specific Integrated Circuits (ASICs) can be used as well and achieve the benefits and advantages described herein. An example MAS106is shown inFIG. 2, which is described below and in co-pending U.S. patent application Ser. No. 10/610,511 entitled “DISTRIBUTED CALL SERVER SUPPORTING COMMUNICATION SESSIONS IN A COMMUNICATION SYSTEM AND METHOD,” filed on Jun. 30, 2003, and which is incorporated by reference herein.

Communication session(s) established and managed by the MAS106can include additional communication devices other than the communication devices102a-102d. For example, a communication device102eis shown coupled to the network104through a public telephone network, such as a public switched telephone network (“PSTN”)116. The communication device102emay include a conventional analog or digital telephone or some other type of communication device. In embodiments where the PSTN116and the network104use different or incompatible protocols to communicate, a gateway118may be used that is coupled to the network104and the PSTN116to facilitate communication between the networks. The gateway118functions to translate between the different protocols used by the network104and the PSTN116. Although one PSTN116is shown inFIG. 1coupled to the network104, other or additional types of public or private networks may be coupled to the network104.

The communication devices102and the MAS106are configured to support suitable standards or protocols used to set up, maintain, and terminate communication sessions between end users. As examples, the communication devices102and the MAS106may be operable to communicate audio, video, or other information using the Realtime Transfer Protocol (“RTP”) over User Datagram Protocol (“UDP”), the International Telecommunication Union—Telecommunications (“ITU-T”) H.263 standard (video CODEC), the G.711 and G.729 audio CODEC standards, and other or additional standards or protocols. Other CODECs, such as Moving Picture Experts Group-4 (“MPEG-4”), Digital Video Express (“DIVX”), and Windows Media Video (“WMV”), can be supported by the MAS106. In addition, signaling messages sent between the communication devices102and the MAS106may include or conform with the Session Initiation Protocol (“SIP”), which is an application layer protocol for the establishment, modification, and termination of conferencing and telephony sessions over IP-based networks. As will be appreciated, other or additional protocols and configurations may be used.

In one aspect of operation, the MAS106includes one or more processes, such as software applications providing an activity, a function, or a systematic sequence of operations that produces a specified result, for handling conference calls, for receiving instant messages, and for performing text-to-speech conversion. In this document, the phrase “conference call” refers to a communication session that can (but need not) involve more than two participants. Also, the term “participant” refers to a person who joins a conference call, whether or not the person actually participates in the conference call (such as by speaking). A participant could represent a person who joins a conference call for the entirety of the call or for just a portion of the call. For a conference call, the MAS106supports a conference bridge that allows multiple participants to dial or otherwise contact the bridge and join a conference call. A “conference bridge” represents a facility or service that allows participants to be connected together for a conference call. In addition, the phrase “instant message” refers to a message that is transmitted from a source to a destination for presentation at the destination at the time it is received by the destination. An instant message may include any suitable contents, such as text, graphic symbols, audio contents, or any other contents.

During operation, the MAS106receives an instant message associated with a conference call. The instant message could, for example, represent a message sent from a participant who is running late for the conference call. As another example, the instant message could represent a message sent from a computing system having stock quotes or a location of a target being tracked. The target could represent a person, object, or any other entity capable of being tracked. The instant messages received by the MAS106could include any other or additional contents without departing from the scope of this disclosure.

After receiving an instant message, the MAS106identifies the conference bridge associated with the instant message. The MAS106converts at least some of the contents of the instant message into speech. The generated speech is then sent to one or more participants who have joined the conference bridge. In this way, information may is be provided to the participants who have joined the conference bridge. Moreover, the information may be easily provided to the participants without requiring those participants to have access to an instant message-capable device.

In addition to converting at least some of the contents of the instant message into speech, the MAS106may optionally provide additional speech. For example, the MAS106could generate speech identifying the name of the participant who sent the instant message. As a particular example, the MAS106could insert text into the instant message, such as the name of the participant who sent the message followed by “sent the following message.” This text is then converted into speech and sent to the participants who have joined the conference bridge. The MAS106could also generate speech identifying the name of the participant followed by “sent the following message” independently of converting the instant message into speech (without inserting text into the instant message), and the MAS106then combines the speech. In this way, the participants hearing the generated speech know the source of the instant message. Any other or additional information could be inserted into the instant message or included as part of the generated speech, such as the date and time of the instant message and the location of the participant sending the message.

In addition to converting the instant message into speech and providing the speech to the participants who have joined the conference bridge, the MAS106could forward the instant message to one, some, or all of the participants. In this way, participants who have access to an instant message-capable device may be able to receive the instant messages received by the MAS106. This may allow, for example, a participant responding to the instant message. As a particular example, after hearing an instant message converted into speech indicating that one participant is running late, another participant could respond with an instant message indicating that the conference call will be delayed for ten minutes.

In some embodiments, an instant message forwarded to a particular participant is sent to a specific communication device102associated with that participant. In other embodiments, an instant message forwarded to a particular participant is sent to some or all communication devices102associated with, that participant. In particular embodiments, such as when the SIP protocol is used in the system100, a communication device102registers with the MAS106and is associated with a participant. When an instant message is to be forwarded to a participant, the instant message is sent to all registered communication devices102for that participant. By communicating the instant message to all registered communication devices102for a participant, the participant may be more likely to receive the instant message on at least one device.

As described above, when an instant message is received by the MAS106, the MAS106identifies the conference bridge associated with the instant message. This may be accomplished in any suitable manner. For example, in some embodiments, a conference bridge may be is associated with a unique instant message user account, and any messages for that particular conference bridge are received through that user account. In this case, the MAS106detects an instant message arriving in a user account and identifies the conference bridge associated with that user account.

In other embodiments, the MAS106has a single instant message user account that receives instant messages for multiple conference bridges, and the MAS106identifies a particular conference bridge using information included in the instant message. For example, an instant message could include a hidden instant message parameter identifying the conference bridge. As a particular example, the hidden instant message parameter could take the form “SIP SIMPLE tag=XYZ identifyxxxxx”, where the XYZ parameter represents the participant sending the message and the identifyxxxxx parameter identifies the conference bridge. While this represents one example of a parameter that can be used, other or additional parameters could be used in the instant message. As another example, an instant message could include an instant message source account name identifying the conference bridge. In addition, an instant message could include explicit text identifying the conference bridge.

In yet another embodiment, the MAS106sends instant messages to one or more participants or future participants identifying various events that occur during a conference call. For example, when a new participant joins a conference bridge, the MAS106may send an instant message to one or more current participants or future participants identifying the new participant. This mechanism is described in U.S. patent application Ser. No. 10/675,121 entitled “APPARATUS, METHOD, AND COMPUTER PROGRAM FOR PROVIDING INSTANT MESSAGES RELATED TO A CONFERENCE CALL.” In these embodiments, in accordance with this disclosure, when a current participant or future participant receives an instant message from the MAS106, that participant may respond to the instant message with another instant message. The MAS106receives the other instant message, converts the message to speech, and conveys the speech to the participants who have already joined the conference call. As a particular example, a future participant who is running late for a conference call may receive instant messages indicating that several participants have already joined the conference bridge. The participant who is running late may respond to one of the instant messages with another instant message indicating that the participant will be late for the call.

This has described several examples of the instant messages that may be communicated to one or more participants in a conference call and the various ways that instant messages can be sent and associated with a conference bridge. Instant messages including other or additional contents could be sent at any suitable time to any or all of the participants without departing from the scope of this disclosure. Also, additional information may be added to the instant messages and/or provided as speech to one or more participants. Further, the instant messages can be sent and associated with a conference bridge in any other suitable manner. In addition, while the MAS106may convert any suitable contents in the instant message into speech, the MAS106may not need to convert other contents into speech. As a particular example, if the instant message contains audio information, the MAS106may provide the audio information without needing to convert the audio information into speech. As another particular example, if the instant message contains an image that cannot be converted into speech, the MAS106may provide speech indicating that the instant message contains unconvertible material.

AlthoughFIG. 1illustrates one example of a communication system100, various changes may be made toFIG. 1. For example, any number of communication devices102, networks104, and servers106could be used in the system100. Also, the functionality of MAS106, described above as being implemented on a server, could be implemented on another computing device or devices, such as a desktop computer or a laptop computer. In addition,FIG. 1illustrates one operational environment in which the various features of the MAS106may be used. These features could be implemented in any other suitable operating environment.

FIG. 2illustrates an example media application server106according to one embodiment of this disclosure. The MAS106illustrated inFIG. 2is for illustration only. Other embodiments of the MAS106could be used without departing from the scope of this disclosure. Also, whileFIG. 2illustrates the MAS106operating in the system100ofFIG. 1, the MAS106may operate in other suitable systems.

In the illustrated example, the MAS106includes a media conductor202, a media controller204, two media processors (“MPs”)206a-206b, and a content store208.

The media conductor202processes signaling messages received by the MAS106. In some embodiments, the communication devices102communicate the signaling messages directly (or via a gateway, which serves as an entrance/exit into a communications network) to the MAS106. In other embodiments, the communication devices102communicate signaling messages indirectly to the MAS106, such as when a Session Initiation Protocol (“SIP”) application server210(that received a request from a communication device102) sends the signaling messages to the media conductor202on behalf of the communication device102. The communication devices102may communicate directly with the SIP application server210or indirectly through a gateway, such as gateway118. The media conductor202processes the signaling messages and communicates the processed messages to the media controller204. As particular examples, the media conductor202may implement SIP call control, parameter encoding, and media event package functionality.

The media controller204manages the operation of the MAS106to provide services to the communication devices102. For example, the media controller204may receive processed SIP requests from the media conductor202, where the requests involve conference or other calls. The controller204may then select the media processor206to handle each of the calls, support audio/video capability negotiations, enforce licenses controlling how the MAS106can be used, and control negotiations based on the licenses. The negotiations could include identifying the CODEC or CODECs to be used to encode and decode audio or video information during a call.

The media processors206a-206bhandle the exchange of audio or video information between communication devices102involved in a conference or other call. For example, a media processor206could receive audio and video information from one communication device102involved in a call, process the information as needed, and forward the information to at least one other communication device102involved in the call. The audio and video information may be received through one or more ports212, which couple the media processors206to the network104. The ports212may represent any suitable structure operable to facilitate communication between the MAS106and the network104. In some embodiments, each of the media processors206represents a software application for specific media processing, such as interactive voice response (“IVR”) media or conference media, which is executed on the MAS106hardware platform via the operating system.

In this example embodiment, each media processor206provides different functionality in the MAS106. For example, in some embodiments, the media processor206aprovides IVR functionality in the MAS106. As particular examples, the media processor206asupports a voice mail function that can record and play messages or an auto-attendant function that provides a menu and directs callers to particular destinations based on their selections. The media processor206bprovides conferencing functionality in the MAS106, such as by facilitating the exchange of audio and video information between communication devices102.

The content store208provides access to content used by the various components of the system100. For example, in some embodiments, the content store208provides access to stored voice mail messages and access codes used to initiate or join conference calls. The content store208may also provide access to custom emoticons provided by participants in the system100. The content store208further provides access to any other or additional information. In other embodiments, the content store208is replaced by a conventional database or other data storage facility.

A Java 2 Enterprise Edition (“J2EE”) platform214is coupled to the MAS106. The J2EE platform214allows the MAS106to retrieve information used to provide subscriber services in the system100. For example, the J2EE platform214may provide audio announcements used by the IVR media processor206a. The J2EE platform214may also provide standard or default emoticons that are available for use during communication sessions. The J2EE platform214represents one possible apparatus used to provide audio or other information to the MAS106. Any other or additional device or apparatus may be used to provide the information to the MAS106.

In addition, the MAS106includes a text-to-speech converter216. The text-to-speech converter216receives text data and generates speech data based on the text data. In particular, the text-to-speech converter216converts the contents of an instant message from text to speech by generating an audio stream based on the contents of the message. The text-to-speech converter216may receive the contents of the instant message in any suitable manner. For example, in some embodiments, the media controller204receives an instant message through the media conductor202, and the media controller204extracts and forwards the contents of the instant message to the conference media processor206band/or the text-to-speech converter216. The text-to-speech converter216represents any suitable hardware, software, firmware, or combination thereof for converting text to speech.

In a particular embodiment, various components of the MAS106represent software processes executed by the processor112of the MAS106. While the components have been described as being executed by a MAS106, the software processes could be executed by other computing devices such as a desktop computer. In other embodiments, the various components of the MAS106may be implemented in other ways, such as in hardware.

In the illustrated example, the conference media processor206bimplements the conferencing functionality described above. For example, the media conductor202receives signaling messages indicating that two or more communication devices102wish to engage in a conference call. The controller204receives the requests and causes the conference media processor206bto establish the conference call over a conference bridge. The conference is media processor206bthen receives audio or video information from each communication device102and forwards the information to the other communication devices102.

In some embodiments, during a conference call, the MAS106receives an instant message to be delivered to the participants who have joined the conference call. The media conductor202receives the instant message and forwards the message to the media controller204, which then forwards at least the contents of the message to the conference media processor206b. The conference media processor206bidentifies the conference bridge associated with the instant message and sends the contents of the message to the text-to-speech converter216. The text-to-speech converter216generates an audio stream using the contents of the instant message. The conference media processor206bthen provides the audio stream to one, some, or all of the participants who have joined the conference call.

Before sending the audio stream to the participants, the conference media processor206band/or the text-to-speech converter216encodes the audio stream using one or more CODECs. The number of CODECs used may depend, for example, on whether the participants who will receive the audio stream are using a common CODEC or different CODECs. In particular embodiments, the conference media processor206bencodes the audio stream once for each CODEC being used by the participants.

The MAS106may further forward the contents of the original instant message to the participants who have already joined the conference call. For example, the media controller204may generate a new instant message for each of the participants who have already joined the conference call, where the contents of the new message include the contents of the original instant message received and converted into speech. The new instant messages are then converted into SIP format by the media conductor202, which communicates the new instant messages to one or more communication devices102. The new instant messages may be sent directly to the communication devices102by the media conductor202or indirectly through the SIP application server210.

AlthoughFIG. 2illustrates one example of a media application server106, various changes may be made toFIG. 2. For example, any number of media processors206could be used in the MAS106. Also, the functional divisions shown inFIG. 2are for illustration only. Various components can be combined or omitted or additional components can be added according to particular functional designations or needs. In addition, while the components202-208have been described as being executed by a server, the components202-208may be executed by other hardware platforms, such as a desktop computer or a laptop computer.

FIG. 3illustrates an example method300for performing text-to-speech conversion of instant messages during a conference call according to one embodiment of this disclosure. For ease of illustration and explanation, the method300is described with respect to the MAS106ofFIG. 2operating in the system100ofFIG. 1. The method300could be used by any other suitable apparatus or device and in any other suitable system.

The MAS106receives an instant message associated with a conference call at step302. Receipt of the instant message may include, for example, the media conductor202receiving the instant message and forwarding at least the contents of the instant message to the media controller204. The media conductor202may receive the instant message directly from a communication device102or indirectly through the SIP application server210. Receipt may also include the media controller204forwarding at least the contents of the instant message to the conference media processor206b.

The MAS106identifies a conference bridge associated with the instant message at step304. Identification of the conference bridge may include, for example, the media controller204or the conference media processor206bidentifying the conference bridge using a user account associated with a specific conference bridge or a hidden instant message parameter, source account name, or text included in the instant message. Identification of the conference bridge may also include the media controller204or the conference media processor206bdetermining that the received instant message was sent in response to a prior instant message, where the prior instant message was associated with a particular conference bridge.

The MAS106converts the contents of the instant message into speech at step306. Conversion may include, for example, the conference media processor206btransferring the instant message or the contents of the instant message to the text-to-speech converter216, the text-to-speech converter216generating an audio stream based on the contents of the instant message, and the text-to-speech converter216providing the audio stream to the conference media processor206b. Conversion into speech may also include the conference media processor206bcausing additional speech to be generated or otherwise received in addition to the speech associated with the contents of the instant message. For example, the conference media processor206bmay add additional text to the contents of the instant message before conversion. As another example, the conference media processor206bmay supply additional text to the text-to-speech converter216separate from the contents of the instant message and receive two separate audio streams that are then combined. As yet another example, the conference media processor206bmay retrieve stored speech to be combined with the speech generated by the text-to-speech converter216.

The MAS106encodes the generated speech using one or more CODECs at step308. Encoding may include, for example, the conference media processor206bidentifying at least one audio CODEC being used by the participant(s) who have joined the conference bridge. Encoding may also include the conference media processor206bencoding the audio stream using the identified CODEC or CODECs.

The MAS106provides the encoded speech to one or more of the participants who have joined the conference bridge at step310. This may include, for example, the conference media processor206bproviding the encoded audio stream to one or more communication devices102associated with each participant.

The MAS106determines whether the original instant message should be forwarded to one or more of the participants at step312. This determination may include, for example, the MAS106using the contents of the original instant message received at step302to determine whether to forward the message. This determination may also include the MAS106using a default or customized setting indicating whether the original instant message should be forwarded.

If the original instant message should not be forwarded, the MAS106allows the conference call to continue until conclusion, which may include a repeat of the method300. Otherwise, if the original instant message may be forwarded, the MAS106forwards the original instant message to one or more of the participants who have joined the conference bridge at step314. This step may include, for example, the conference media processor206bforwarding the original instant message or new instant message(s) including the contents of the original message to the participant(s) through the media controller204and the media conductor202.

AlthoughFIG. 3illustrates one example of a method300for performing text-to-speech conversion of instant messages during a conference call, various changes may be made toFIG. 3. For example, the instant message could always be forwarded to the participants in a conference call, and the MAS106could skip decisional step312. Also, the MAS106need not encode the generated speech.