Method and apparatus for live call text-to-speech

A method and apparatus provide live call text-to-speech. The method can include entering an ongoing voice call at a first communication device with a second communication device. The method can include receiving text input at the first communication device during the ongoing voice call. The method can include converting the text input to speech during the ongoing voice call to generate a text-to-speech audible signal. The method can include sending the text-to-speech audible signal from the first communication device to the second communication device during the ongoing voice call.

BACKGROUND

The present disclosure is directed to a method and apparatus for live call text-to-speech. More particularly, the present disclosure is directed to converting text to speech during an ongoing voice call.

Presently, communication devices allow users to place and receive voice calls between the communication devices. These communication devices include cell phones, flip phones, smartphones, smart watches, and other communication devices that provide for voice calls. For example, a user can use a phone application on a smartphone to place a phone call to another user's remote communication device so the users can speak with each other.

Often times there are scenarios during an ongoing phone call where one user in the call is not able to speak, but can still can hear the audio from the remote communication device. For example, the user may be wearing a wireless headset while in a meeting, where the user can hear the other party in the call, but should not speak to avoid distracting other people in the meeting. The user may also have temporarily or permanently lost their voice. Additionally, the user may be in a public place, but may want to be discreet about the communication. The user may further be at a location with loud ambient noise that makes it difficult for the remote party to hear the user. In such situations, the user may want to keep the call active and continue communicating. Unfortunately, the user cannot continue communicating in a voice call if the user cannot speak while in the voice call.

Thus, there is a need for a method and apparatus for live call text-to-speech.

DETAILED DESCRIPTION

Embodiments provide a method and apparatus for live call text-to-speech. The method can include entering an ongoing voice call at a first communication device with a second communication device. The method can include receiving text input at the first communication device during the ongoing voice call. The method can include converting the text input to speech during the ongoing voice call to generate a text-to-speech audible signal. The method can include sending the text-to-speech audible signal from the first communication device to the second communication device during the ongoing voice call.

FIG. 1is an example block diagram of a system100according to a possible embodiment. The system100can include a first apparatus110, a second apparatus120, and a network160. The first apparatus110can include an audio input130, such as a microphone, a transducer, and/or other audio input. The first apparatus110can also include a text input140, such as a physical keypad, a physical keyboard, an alphanumeric input, a virtual keypad or keyboard on a display, and/or any other text input. The first apparatus110can include a transceiver114and an antenna115. The transceiver114can be a wireless transceiver that uses the antenna115or a wired transceiver that uses a physical connection for communication. The apparatus110can include a live call text-to-speech audio module112and an application116. The text-to-speech audio module112can be hardware, can be software, can be an application, or can be any other means of receiving text and converting it to speech.

The first apparatus110and/or the second apparatus120can be wireless terminals. For example, the apparatus110can be a portable wireless communication device, a smartphone, a cellular telephone, a personal digital assistant, a personal computer, a selective call receiver, a tablet computer, or any other device that is capable of sending and receiving communication signals on a network. The apparatus110can communicate with the network160and other devices using wireless communication signals, such as cellular or wireless local area network communication signals, as well as electrical communication signals, optical communication signals, or other wireless communication signals as well as wired communication signals.

The network160may include a wireless communication network, a cellular telephone network, a Time Division Multiple Access (TDMA)-based network, a Code Division Multiple Access (CDMA)-based network, an Orthogonal Frequency Division Multiple Access (OFDMA)-based network, a Long Term Evolution (LTE) network, a 3rd Generation Partnership Project (3GPP)-based network, a satellite communications network, a packet-based data network, the Internet, an intranet, a wireless wide area network, a wireless local area network, a wired communication network, and other communications systems. The network160may include more than one network and may include a plurality of different types of networks. Thus, the network160may include a plurality of data networks, a plurality of telecommunications networks, a combination of data and telecommunications networks and other communication systems capable of sending and receiving communication signals.

According to a possible embodiment, the first apparatus110can enter an ongoing voice call with a second apparatus120. The first apparatus can receive voice communications from the user132via the audio input130during the ongoing voice call. The first apparatus110can also receive text input during the ongoing voice call. For example, the first apparatus110receive the text input from a user input from a user132on a physical or virtual alphanumeric keypad140. The text input can also be received via a copy and paste function, such as from an application116. The text input can additionally be received from an application116that otherwise shares information with the live call text-to-speech audio module112. For example, the text input can be location data received from a map application or can be translated text from a translator application, such as the application116. A translator application can further receive voice audio, convert the voice audio to text, translate the text, and convert the text to audio. The resulting translated text can then be received as text input from the translator application.

The first apparatus110can convert the text input to speech during the ongoing voice call to generate a text-to-speech audible signal. For example, the live call text-to-speech audio module112can convert the text input to speech signal during the ongoing voice call. The first apparatus110can send the text-to-speech audible signal to the second communication device120during the ongoing voice call. For example, the first apparatus110can communicate with the second apparatus120by connecting to the network160. The connection with the network160can include a voice communication path150and a data communication path115and the first apparatus110can send the text-to-speech audible signal to the second apparatus120over the voice communication path150. The first apparatus110can send the text-to-speech audible signal to the second apparatus120over any other connection, virtual or actual, that sends voice communications to the second apparatus120during an ongoing voice call.

According to a possible embodiment, text-to-speech can be used with audio mixing during a live phone call where one party of the call is not able to speak, can hear the other party, and still wants to keep the call alive and continue communicating. For example, it may be inappropriate for a user to speak while attending a meeting even though the user is on a separate voice call and hearing audio from the other calling party over a headset. As another example, the user may have lost their voice temporarily or permanently, but still may wish to communicate over a voice call. As a further example, the user can be in public place and can want to be discreet about the voice communication. As an additional example, the user may be in an environment, such as at a concert or in a windy location, where ambient noise makes the user's voice difficult to hear over a voice call. In such situations, after a phone call is established, the user can choose to switch a phone application to a live call text-to-speech audio mode, such as a discreet mode. In this mode, the phone application and user interface on the apparatus110can change and allow the user to input text using the keypad140. The apparatus110can automatically convert the inputted text to speech using an in-built text-to-speech engine and can inject the resulting converted text to audio stream into the live phone call. A user at the remote apparatus120can continue responding, such as by talking, as normal while the user132operates the apparatus110in discreet mode.

FIG. 2is an example illustration of an apparatus200, such as the apparatus110, according to a possible embodiment. The apparatus200can include a display210and a text input220. The text input220can be a physical text input, such as a keyboard or keypad. The text input220can also be a virtual text input, such as a virtual keyboard, virtual keypad, handwriting recognition area, and/or other virtual text input on the display210. The display210can display inputted text222, a speak icon224, and exit icon226, and a text input history230.

In operation, during a live call text-to-speech audio mode, such as a Text-To-Speech Speak (TTSSpeak) mode, a user132can input text on the text input220. The display210can display the inputted text222. The display210can also display a previously inputted text history230. After the user132inputs the text222, the user can actuate the speak icon222to convert the text to audio and send the resulting audio over an ongoing voice call. Alternately, the user132can use any other method of entering and sending text, such as by pressing an enter key or by using any other useful text entry completion input. The user132can exit the live call text-to-speech audio mode using the exit icon226or by any other useful way of exiting an application or input mode.

FIG. 3is an example flowchart300illustrating the operation of a first communication device, such as the apparatus110, according to a possible embodiment. At310, the flowchart300can begin. At320, the first communication device can enter an ongoing voice call with a second communication device. The ongoing voice call can include a voice communication path that sends audible signals from the first communication device to the second communication device or can include any other way of sending audible signals between communication devices. At330, audio received from the second communication device can be output at the first communication device while receiving text input at the first communication device during the ongoing voice call.

At340, text input can be received at the first communication device during the ongoing voice call. The text input can be received from a user input on the first communication device. The text input can also be received at a live call text-to-speech and audio application on the first communication device from another application on the first communication device during the ongoing voice call. At350, the text input can be converted to speech during the ongoing voice call to generate a text-to-speech audible signal. At360, the text-to-speech audible signal can be sent from the first communication device to the second communication device during the ongoing voice call. The text-to-speech audible signal can be sent from the first communication device to the second communication device over the voice communication path of the ongoing voice call. At370, the flowchart300can end.

FIG. 4is an example flowchart400illustrating the operation of a first communication device, such as the apparatus110, according to a possible embodiment. At410, the flowchart400can begin. At420, a live call text-to-speech and audio mode selection can be detected at the first communication device. For example, a user can switch between voice communication mode and live call text-to-speech and audio mode during the ongoing voice call. As an alternate example, the user can enter the live call text-to-speech and audio mode selection before the ongoing voice call. The live call text-to-speech and audio mode selection can also be provided automatically by the first communication device, such as by detecting the first communication device is in a location where speech calls are discouraged. Such a location can be a movie theater, a library, a user selected location, or any other location where speech calls are not desired. The location can be detected using a communication device positioning system, such as based on Global Positioning System (GPS), Wireless Local Area Network (WLAN), cellular, and/or other signals that provide positioning information. The location can also be detected by the first communication device receiving a signal from the location that indicates that speech calls should not be made at the location. The live call text-to-speech and audio mode can also be enabled automatically when loud ambient noise, such as wind noise, is detected by the first communication device. In this condition, the first communication device can detect the loud ambient noise, provide a warning to the user that the live call text-to-speech and audio mode is preferred due to the loud ambient noise, and enable the live call text-to-speech and audio mode so information can be heard by the recipient despite the loud ambient noise.

The live call text-to-speech and audio mode can also provide both audio from a microphone simultaneously along with the text-to-speech audio during the ongoing voice call. This can allow a user to speak while also providing text-to-speech audible signals, such as when the user wants some information to be discreet from typing while other information is still public from speaking. This can also allow a user to provide ambient sound while also providing text-to-speech audible signals, such as to allow a user to provide ambient concert music via the microphone while also providing text-to-speech information that may otherwise be difficult for the recipient to hear over loud ambient concert music.

At430, an alphanumeric text keyboard can be provided on a display of the first communication device in response to receiving the live call text-to-speech and audio mode selection. The alphanumeric text keyboard can be a virtual QWERTY keyboard, a virtual phone keypad, or any other keyboard that can provide alphanumeric text input. At440, text input can be received at the first communication device via the alphanumeric text keyboard during the ongoing voice call. Alternately, the text input can be received via any other user interface. At450, a history of previous text inputs can be displayed on the display. At460, the flowchart400can end.

FIG. 5is an example flowchart500illustrating the operation of a first communication device, such as the apparatus110, according to a possible embodiment. At510, the flowchart500can begin. At520, a user selection of a discreet mode can be received at the first communication device. For example, the user selection of a discreet mode can be a dedicated selection of a discrete mode during the ongoing voice call, can be a dedicated selection of a discrete mode before the ongoing voice call, can be a user selection of a mute function while in a live call text-to-speech and audio mode, or can be any other user selection of a discrete mode where the user wants to disable audio input for at least a portion of an ongoing voice call. At530, audio input to the first communication device can be disabled in response to receiving the user selection of the discreet mode at the first communication device. At540, the flowchart500can end.

FIG. 6is an example flowchart600illustrating the operation of a first communication device, such as the apparatus110, according to a possible embodiment. At610, the flowchart600can begin. At620, text input can be converted to speech at the first communication device during an ongoing voice call. At630, input audio can be received at an audio input of the first communication device during the ongoing voice call. For example, a microphone of the first communication device or an audio path for input audio may or may not be disabled while sending the text-to-speech audible signal from the first communication device to the second communication device during the ongoing voice call. This can allow for ambient sounds, such as music or meeting discussion or concert music, as well as user speech to be sent along with converted text.

At640, a sampling rate of the text-to-speech audio stream can be unified with a sampling rate of the input audio stream if the text-to-speech audio stream sampling rate is a different sampling rate from the sampling rate of the input audio stream. For example one sampling rate can be resampled to match the other sampling rate or one or both sampling rates can be resampled to match a desired output sampling rate. At650, an input audio stream from the input audio can be mixed with the text-to-speech audio stream from the text-to-speech audible signal to generate a mixed audio stream. According to a possible embodiment, the unified input audio stream can be mixed with the unified text-to-speech audio stream to generate the mixed audio stream. At660, the mixed audio stream can be sent from the first communication device to the second communication device during the ongoing voice call. According to a possible embodiment, the mixed audio stream can be sent from the first communication device to the second communication device via a first communication device modem during the ongoing voice call. At670, the flowchart600can end.

It should be understood that, notwithstanding the particular steps as shown in the figures, a variety of additional or different steps can be performed depending upon the embodiment, and one or more of the particular steps can be rearranged, repeated or eliminated entirely depending upon the embodiment. Also, some of the steps performed can be repeated on an ongoing or continuous basis simultaneously while other steps are performed. Furthermore, different steps can be performed by different elements or in a single element of the disclosed embodiments.

FIG. 7is an example block diagram of an apparatus700, such as the apparatus110, according to a possible embodiment. The apparatus700can include a housing710, a controller720within the housing710, audio input and output circuitry730coupled to the controller720, a display740coupled to the controller720, a transceiver750coupled to the controller720, an antenna755coupled to the transceiver750, a user interface760coupled to the controller720, a memory770coupled to the controller720, a network interface780coupled to the controller720, and an image capture device790coupled to the controller720. The apparatus700can also include a modem752coupled between the transceiver750and the controller720. The apparatus700can perform the methods described in all the embodiments.

The display740can be a viewfinder, such as a liquid crystal display (LCD), a light emitting diode (LED) display, a plasma display, a projection display, a touch screen, or any other device that displays information. The transceiver750may include a transmitter and/or a receiver. The audio input and output circuitry730can include a microphone, a speaker, a transducer, or any other audio input and output circuitry. The user interface760can include a keypad, a keyboard, buttons, a touch pad, a joystick, a touch screen display, another additional display, or any other device useful for providing an interface between a user and an electronic device. The image capture device790can be a camera, a video camera, a webcam, an electronic image sensor, or any other image capture device. The network interface780can be a universal serial bus port, an Ethernet port, an infrared transmitter/receiver, an IEEE 1394 port, or any other interface that can connect an apparatus to a network or computer and that can transmit and receive data communication signals. The memory770can include a random access memory, a read only memory, an optical memory, a subscriber identity module memory, a flash memory, a removable memory, a hard drive, a cache, or any other memory that can be coupled to a wireless communication device.

The apparatus700or the controller720may implement any operating system, such as Microsoft Windows®, UNIX®, or LINUX®, Android™, or any other operating system. Apparatus operation software may be written in any programming language, such as C, C++, Java or Visual Basic, for example. Apparatus software may also run on an application framework, such as, for example, a Java® framework, a .NET® framework, or any other application framework. The software and/or the operating system may be stored in the memory770or elsewhere on the apparatus700. The apparatus700or the controller720may also use hardware to implement operations. For example, the controller720may be any programmable processor. Disclosed embodiments may also be implemented on a general-purpose or a special purpose computer, a programmed microprocessor or microprocessor, peripheral integrated circuit elements, an application-specific integrated circuit or other integrated circuits, hardware/electronic logic circuits, such as a discrete element circuit, a programmable logic device, such as a programmable logic array, field programmable gate-array, or the like. In general, the controller720may be any controller or processor device or devices capable of operating an electronic device and implementing the disclosed embodiments.

In operation, the transceiver750can connect an ongoing voice call at a first communication device with a second communication device. The first communication device can be the apparatus700or can be coupled to the apparatus700. The ongoing voice call can include a voice communication path that sends audible signals from the first communication device to the second communication device. The user interface760can receive a user selection of a live call text-to-speech and audio mode at the first communication device. The controller720can disable audio input to the first communication device in response to the user interface receiving the user selection of a live call text-to-speech and audio mode at the first communication device or the controller720can maintain operation of audio input.

The controller720can detect a live call text-to-speech and audio mode selection, can provide an alphanumeric text keyboard on the display740in response to receiving the live call text-to-speech and audio mode selection and can receive text input via the alphanumeric text keyboard during the ongoing voice call. The controller720can also operate a first application that receives the text input from a second application operating on the apparatus700during the ongoing voice call. The controller720can receive text input at the first communication device during the ongoing voice call and can convert the text input to speech during the ongoing voice call to generate a text-to-speech audible signal. An audio output of the audio input and output circuitry730can output audio received from the second communication device at the first communication device while receiving text input at the first communication device during the ongoing voice call.

The transceiver750can send the text-to-speech audible signal from the first communication device to the second communication device during the ongoing voice call. For example, the transceiver750can send the text-to-speech audible signal from the first communication device over the voice communication path of the ongoing voice call to the second communication device. An audio input of the audio input and output circuitry730can receive input audio of the first communication device during the ongoing voice call. The controller720can mix an input audio stream from the input audio with the text-to-speech audio stream from the text-to-speech audible signal to generate a mixed audio stream. The transceiver750can send the mixed audio stream from the first communication device to the second communication device during the ongoing voice call. The controller720can also unify a sampling rate of the text-to-speech audio stream with a sampling rate of the input audio stream if the text-to-speech audio stream sampling rate is a different sampling rate from the sampling rate of the input audio stream to generate the mixed audio stream. The transceiver750can then send the mixed audio stream from the first communication device to the second communication device via a first communication device modem750during the ongoing voice call.

FIG. 8is an example block diagram of an apparatus800, such as the apparatus110and/or the apparatus700, according to a possible embodiment. The apparatus800can include an audio input810, a text input820, a text-to-speech engine830, an audio manager840, an audio hardware abstraction layer850, an audio mixer860, and a modem870.

In operation, apparatus800can enter an ongoing voice call with a remote apparatus. The ongoing voice call can be an outgoing call originated by the apparatus800or can be an incoming call originated by the remote apparatus and terminated at the apparatus800. The ongoing voice call can be over a mobile communication system, over a landline, over a wireless wide area network, and/or over any other communication system or combination of communication systems.

The audio input810can receive input audio, such as speech and/or ambient sounds from a microphone, and can send the input audio to the audio manager840. The audio input or any other hardware or software can sample the input audio at a given sampling rate before sending the input audio to the audio manager. For example, the sampling rate can be an 8 kHz sampling rate, a 16 kHz sampling rate, a wideband sampling rate, or any other sampling rate.

During the ongoing call, the text input820can receive user text, such as through an input dialog provided by a live voice call text-to-speech application. The text input820can send the user text to the text-to-speech engine830. The text-to-speech engine830can convert the user text to audible speech and can send the resulting audible speech audio stream to the audio manager840. The text-to-speech engine830can convert the user text to audible speech at any useful sampling rate. The audio manager840can provide an interface between the text input820and the audio input810, and the audio hardware abstraction layer850and the audio mixer860. For example, the audio manager840can render the input audio and audible speech streams for lower level hardware. The audio manager840may also resample the input audio and the audible speech streams to unify their sampling rate to a desired sampling rate, such as to a sampling rate of the ongoing voice call. The input audio and the audible speech can also be resampled anywhere else on the apparatus800. The audio hardware abstraction layer850can provide access to hardware resources for higher level software on the apparatus800. The audio mixer860can be in a kernel driver or elsewhere on the apparatus800. The audio mixer860can mix the input audio and the audible speech streams and provide the mixed audio to hardware on the apparatus800, such as to the modem870. The modem870can send the mixed audio to the ongoing voice call remote apparatus.

In this document, relational terms such as “first,” “second,” and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The phrase “at least one of” followed by a list is defined to mean one, some, or all, but not necessarily all of, the elements in the list. The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “a,” “an,” or the like does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Also, the term “another” is defined as at least a second or more. The terms “including,” “having,” and the like, as used herein, are defined as “comprising.” Furthermore, the background section is written as the inventor's own understanding of the context of some embodiments at the time of filing and includes the inventor's own recognition of any problems with existing technologies and/or problems experienced in the inventor's own work.