Patent Description:
With rapid progress of a machine technology, a communications technology, an Internet and an artificial intelligence technology, various smart home appliances, such as smart televisions, smart refrigerators, smart air conditioners, smart speakers, smart watches, smart bracelets, and smart glasses, constantly enter lives of people. At present, a large number of smart devices of various brands based on speech interaction have been put on the market. A user can interact with the smart device by transmitting a speech instruction, to realize functions of song listening, time reporting, chatting, games, companionship, information query, device control, and the like. However, at present the smart devices are mainly applied to the fields such as family life, leisure and entertainment, children education, or the like, and are rare and difficult to be applied to enterprise-level applications.

As the artificial intelligence technology and the communications technology flourish, telephone robots have been widely applied in all industries, with labor costs of a call center being greatly reduced and efficiency being improved. However, prior speech interactive robots, especially telephone human-machine coordination calling robot systems, include a human-machine coordination calling robot system that is primarily based on the artificial intelligence technology and dialogue system, and a speech communication system that is primarily based on communication networks and the VOIP technology, both of which are tightly bound to each other so that a system is extremely complex, and is more difficult to be developed, deployed, and maintained, having high costs. Moreover, it is very difficult and inflexible to replace any components therein. A telephone robot developed on this basis is too complex and large, and is difficult to be moved once deployed. Such a telephone robot is based on a large-scale cloud server, and has no entity that can be easily cognized by ordinary people, which cannot give people an intuitive and friendly impression.

To implement a mobile human-machine coordination calling robot, there is a need of a three-way calling terminal that supports three-party calling and is convenient and easy to use. A conventional human-machine coordination calling robot generally uses a desktop machine as a calling terminal, which is complex in operation and is inconvenient for moving; while a novel mobile telephone three-way calling terminal does not support three-way calling, and cannot implement a human-machine coordination calling robot.

A patent literature <CIT> is a system for controlling whether an agent in a contact center is exposed to sensitive information, comprising: a three-way bridge connected to a call between a remote party and the agent; and an interactive voice response ("IVR") unit, wherein the IVR unit is bridged via the three-way bridge onto the call as directed by the agent in order to collect a first piece of sensitive information provided by the remote party, wherein the IVR unit controls whether SIP-based information conveying the first piece of sensitive information is provided to the agent by controlling a switch, and wherein the IVR unit is configured to: select a first announcement to play to the remote party, determine whether the first announcement to be played to the remote party conveys a second piece of sensitive information, if the first announcement to be played conveys the second piece of sensitive information, then control the switch so that the second piece of sensitive information is not conveyed to the agent, and if the first announcement does not convey the second piece of sensitive information, then control the switch to convey the first announcement to the agent.

An objective of the present disclosure to provide a mobile three-way calling terminal for a mobile human-machine coordination calling robot according to claim <NUM>. Preferred embodiments are provided by the dependent claims <NUM>-<NUM>.

In view of the above, in the present disclosure, a communications system (the human agent and the communications terminal) is decoupled from the human-machine coordination calling robot system (the back-end processing module) by the three-way calling terminal. In this way, system complexity is reduced, so that the system is easy for deployment, and is convenient for flexible switching, thereby significantly reducing costs of a telephone human-machine coordination calling robot system for development, deployment, and maintenance. Moreover, movability is provided for the robot, so that the three-way calling terminal can be conveniently placed in various places. An intuitive and touchable entity may also be provided for the robot, to make the robot friendlier. The three-way calling terminal is conveniently connected to a personal mobile phone or calling terminal. Accessing manners such as a Bluetooth port and an audio port are provided, so that the three-way calling terminal is widely applied.

The present invention is further described in detail below with reference to the accompanying drawings.

As shown in <FIG>, the present disclosure provides a three-way calling system, including: a three-way calling terminal, a back-end processing module, a human agent, and at least one communications terminal. As shown in <FIG>, the three-way calling system includes one communications terminal. During use, the back-end processing module, the human agent, and the communications terminal may be respectively connected to the three-way calling system, so that speech data is transferred among the back-end processing module, the human agent, and the communications terminal through the three-way calling system. The communications terminal is a terminal device used by a call object. The human agent is a terminal device that monitors a dialogue between the back-end processing module and the communications terminal, is configured to monitor the speech data between the back-end processing module and the communications terminal, and may dialogue directly with the communications terminal by human intervention in place of the back-end processing module.

In the present disclosure, a three-way calling terminal shown in <FIG> may be used. The three-way calling terminal includes a first speech interface, a CODEC1 module, a second speech interface, a CODEC2 module, a call control module, a data processing submodule, and a networking submodule. The three-way calling terminal is configured to transfer the speech data among the back-end processing module, the human agent, and the communications terminal.

Operation processes of the three-way calling terminal, the back-end processing module, and the communications terminal are specifically introduced below.

The operation process of the communications terminal includes:.

Modules in the three-way calling terminal are introduced as below.

The first speech interface is configured to transfer call audio between the call object and the back-end processing module.

The CODEC1 module is configured to encode and decode the call audio between the call object and the back-end processing module.

The second speech interface is configured to transfer call audio between the human agent and the call object.

The CODEC2 module is configured to encode and decode the call audio between the human agent and the call object.

The first speech interface and the second speech interface may be Bluetooth ports or audio ports. During a call, the human agent may receive the call audio through the second speech interface, to determine whether to perform human intervention. When human intervention is required, the speech of the human agent is transmitted to the three-way calling terminal also through the second speech interface.

One or more common microphones or MEMS microphones may be disposed at the audio port. A far-field microphone array may also be used to receive speech of the call object that is transmitted by the communications terminal through a speech connection module. A sound play device such as a speaker or the like may be further provided, to play the robot response speech. If desired, the speech of the call object may also be played through the speaker, only in the case of resulting in no reverberation, howling, and reverberation.

The networking submodule is configured to be connected to the back-end processing module through a line link, WIFI, or a <NUM>/<NUM> network, to transmit/receive speech and other data.

The data processing submodule is configured to schedule and control other modules in the three-way calling terminal, to process and transmit the speech data to the back-end processing module, to process and transmit the speech data from the back-end processing module to the communications terminal, to control a display screen to display a text record of the call, and to receive a user instruction from a touch screen.

The call control module is configured to control a communications system to make batch calls with the three-way calling terminal.

The call control module includes a call object database, a system database, a task management module, and a communication controller submodule.

The call object database is configured to store data related to the call object.

The system database is configured to store a call record and other data related to a call process.

The task management module is configured to manage a call task.

The communication controller submodule is configured to schedule other modules to obtain or store data, and control the communications system to make batch calls.

In addition, the call control module further includes a script editor and a script database. The script editor is used for a script producer to produce and modify a script. The script database is configured to store the script that is produced by the script producer by using the script editor.

A workflow of the call control module is described as follows:.

According to requirements, the foregoing processes may be performed in batches; or may be concurrently performed, provided that there are a plurality of communications terminals and three-way calling terminals, and the back-end processing module supports concurrent tasks.

In addition, the three-way calling terminal further includes a display screen and a keypad submodule. The display screen may display a call record or other call-related information about a human-machine coordination calling robot system and the call object. A touch screen may also be used with implementing a button function at the same time, to allow a user to input a control instruction through touching.

The three-way calling terminal may be further provided with a wireless communications system such as Bluetooth, to communicate with the speech connection module.

The three-way calling terminal may be further provided with an audio circuit, so that audio can be directly input and output in a form of digits.

The three-way calling terminal may be further provided with a power amplification module, to amplify a sound signal from the speech connection module.

The three-way calling terminal may be further provided with a speech denoising module, to perform a denoising process of a received audio signal.

The three-way calling terminal may be further provided with an AD/DA conversion chip, to convert received speech of the call object into a digital signal for transmission, and convert received robot speech into an analog signal to be played through the speaker.

The three-way calling terminal may be further provided with a control interface, including buttons, knobs, and the like, for external control.

The operation process of the three-way calling terminal includes:.

The back-end processing module is configured to perform intention recognition on the speech data transmitted by the three-way calling terminal, and generate replay speech based on a speech intention to be transmitted to the three-way calling terminal. The back-end processing module includes a dialogue management submodule, a speech recognition submodule, an intention recognition submodule, a speech synthesis submodule, a word division submodule, a voice separation submodule, voiceprint recognition, and a session management submodule. The back-end processing module is deployed on a cloud server, and communicates with the three-way calling terminal through a wired or wireless network.

The dialogue management submodule is configured to control procedure and logic of a dialogue, and generate response text.

The speech recognition submodule is configured to recognize received speech of the call object, and convert the speech into text.

The intention recognition submodule is configured to recognize an intention of the call object based on the recognized speech text.

The speech synthesis submodule is configured to synthesize speech from the response text and transmit the speech to the three-way calling terminal.

An operation process of a session system of the back-end processing module is described as follows:.

In the present disclosure, the communications system is decoupled from the human-machine coordination calling robot system, with system complexity reduced, so that the system is easy for deployment, and is convenient for flexible switching, thereby significantly reducing costs of a telephone human-machine coordination calling robot system for development, deployment, and maintenance. Moreover, movability is provided for the robot, where the three-way calling terminal can be conveniently put in various places. An intuitive and touchable entity may also be provided for the robot, to make the robot friendlier. The three-way calling terminal is conveniently connected to a personal mobile phone or calling terminal. Accessing manners such as a Bluetooth port and an audio port are provided, so that the three-way calling terminal is widely applied.

Meanwhile, the three-way calling terminal in the present disclosure keeps functions of a conventional telephone client service human-machine coordination calling robot system: providing screen display; being able to be easily set; accessing a call record and switching; being more convenient for use; supporting external devices such as Headphones; and tracking calls at any time, to facilitate a human agent to intervenes.

Specifically, for a telephone robot in the prior art, a communication system, a human-machine coordination calling system, and a background calling robot system thereof are all bound to one another. Therefore, the telephone robot still adopts a deployment method similar to that of a call center, that is, a corresponding telephone robot system is deployed in advance in a place where the telephone robot system is going to be used, and a human agent or other users need to work in the place deployed with the system, so as to realize a function of human-machine coordination calling. Compared with the foregoing prior art, the deployment method of the three-way calling terminal in the present disclosure is more convenient.

In an implementation, the three-way calling terminal applies the communications system in the prior art to a communications terminal of a user, such as a mobile phone or a fixed-line telephone, so as to unbind the communication system from the human-machine coordination calling system and the background calling robot system. A deployment method of the three-way calling terminal in this implementation is illustrated below through an example.

In an example, an independent salesperson, as a user of the three-way calling terminal, directly connects the three-way calling terminal to a calling terminal (such as a mobile phone) thereof in a wired or wireless manner, to complete deployment, without any other operations. Therefore, the three-way calling terminal in this implementation is significantly improved in deployment efficiency and conveniency as compared with that in the prior art.

In actual use, on one hand, the salesperson in the foregoing example can connect the three-way calling terminal to a mobile phone thereof by means of an audio cable or Bluetooth connection; and on the other hand, the three-way calling terminal further provides audio data for the salesperson through a headset connected to the three-way calling terminal (for example, a three-way calling terminal carrying a speaker/microphone may also directly play the audio data). After the salesperson makes a call to an intending client by a mobile phone (or may automatically make a call to an intending client by the three-way calling terminal), the three-way calling terminal uploads speech input by the intending client to the back-end processing module provided in the cloud. After a telephone robot in the back-end processing module generates a corresponding response according to a rule, the three-way calling terminal further returns a response speech to the intending client, so as to implement an interaction between the back-end processing module and the intending client. In the foregoing process, the salesperson can monitor an interaction process between the back-end processing module and the intending client at any time through the headset, and interact directly with the intending client through the three-way calling terminal when human intervention is required.

It may be learned from the foregoing uses that for the three-way calling terminal in this implementation, during use, it is not required that a human agent is arranged in advance, and the user may access the three-way calling terminal at any time through a communications terminal carried therewith. The human agent may select a suitable type of communications terminal according to actual requirements, which is not limited to a same communications terminal.

In addition, during the use of the three-way calling terminal in this implementation, a communication function is initiated by the communications terminal of the user. Therefore, it is not necessary to upload sensitive information, such as a telephone number of the client, to the telephone robot, thereby avoiding possible information leakage caused by the telephone robot during use.

In another implementation, the three-way calling terminal may be deployed directly in a communications terminal (an audio device) of the user, such as a mobile phone or a Bluetooth headset, where the audio devices include a speaker and a microphone, and the second speech interface in the three-way calling terminal is connected to the speaker and the microphone in the audio device. In this way, the audio device itself may have a function of supporting three-way calling. In this case, the user of the audio device can access the human agent through the audio device, transfer the speech data to the three-way calling terminal through the microphone, and receive the speech data transmitted by the three-way calling terminal through the speaker, to achieve entire movability of the three-way calling terminal. A deployment method of the three-way calling terminal in this implementation is illustrated below by an example.

In an example, a salesperson, as a user of the three-way calling terminal, can directly use an audio device (such as a mobile phone) deployed with a three-way calling terminal, thus saving a process of connecting the three-way calling terminal to a calling terminal of the user. In this way, deployment efficiency and conveniency of the three-way calling terminal may be effectively improved. On this basis, due to relatively strong movability of the audio device, places for using the three-way calling terminal are not restricted, and the three-way calling terminal in this implementation is significantly improved in flexibility of places for using as compared with that in the prior art.

In actual use, the salesperson in the foregoing example may turn on a three-way calling function on the mobile phone. Specifically, the salesperson may click three-way calling application software on the mobile phone to activate the three-way calling terminal, and turn on the three-way calling function. For example, after the three-way calling terminal is activated, the mobile phone enables the three-way calling terminal to be connected to the back-end processing module in the cloud through a networking function, and turns on a function of transferring speech data of the mobile phone interacting with another communications terminal to the three-way calling terminal. The salesperson makes a call to an intending client by the mobile phone, may receive speech data transmitted by the intending client through the mobile phone, and transfers the speech data to the three-way calling terminal, where the speech data is uploaded to the back-end processing module through the three-way calling terminal. After generating a corresponding response according to a rule, a telephone robot in the back-end processing module may transmit response data back to the three-way calling terminal, and the three-way calling terminal transmits the response data to the intending client through the mobile phone of the salesperson, so as to implement an interaction between the back-end processing module and the intending client. In the foregoing process, the salesperson can monitor an interaction process between the back-end processing module and the intending client at any time through the speaker of the audio device, and interact with the intending client through the three-way calling terminal when human intervention is required.

In another example, a salesperson serves as a user of the three-way calling terminal, where the three-way calling terminal is integrated in a Bluetooth headset. A three-way calling function may be achieved after the salesperson wears the headset. In actual use, when the salesperson wears the Bluetooth headset integrated with a three-way calling terminal, the three-way calling function may be turned on through a trigger button set on the Bluetooth headset or through an application installed on the mobile phone. After the three-way calling function is turned on, the salesperson makes a call to an intending client through the mobile phone. After receiving the speech data transmitted by the intending client, the mobile phone transfers the speech data to the Bluetooth headset through Bluetooth, and transfers the speech data to the three-way calling terminal, where the speech data is uploaded to the back-end processing module through the three-way calling terminal. After generating a corresponding response according to a rule, a telephone robot in the back-end processing module transmits response data back to the three-way calling terminal, and the three-way calling terminal transmits the response data to the intending client through the microphone in the Bluetooth headset, so as to implement an interaction between the back-end processing module and the intending client. In the foregoing process, the salesperson can monitor an interaction process between the back-end processing module and the intending client at any time through the speaker of the Bluetooth headset, and interact with the intending client through the three-way calling terminal when human intervention is required.

It may be learned from the foregoing uses that integration between the three-way calling terminal and the communications terminal in this implementation is higher, and the user may carry the three-way calling terminal everywhere and use the same in any scenario; this is not limited to a fixed use scenario.

The present disclosure provides a screen display function through the display module. Specifically, the three-way calling terminal displays session content of the call object and response content of the robot through the display module. When a display screen of the human agent is different from that of the three-way calling terminal, the session content and the response content are pushed to the human agent through the display module, to display the session content of the call object and the response content of the robot through the display screen of the human agent.

Specifically, in an implementation manner, the human agent and the call object simultaneously watch the display screen of the three-way calling terminal. At this time, the session content of the call object and the response content of the robot may be displayed directly through the display module (equivalent to the display screen) of the three-way calling terminal, so that the call object and the human agent simultaneously know the interaction process between a communication object and the phone robot, to facilitate interaction data synchronization between the human agent and the call object. In this way, the human agent may intervene in the call in time, to resolve a problem that cannot be resolved by the telephone robot.

In another implementation, the human agent may connect the three-way calling terminal through the communications terminal carried by the human agent (reference may be made to the first implementation of the deployment method of the three-way calling terminal above). At this time, if the communications terminal used by the human agent itself has a display screen, the human agent may browse the session content of the call object and the response content of the robot through the display screen. In this way, although the human agent uses the own communications terminal, an interaction process between the call object and the robot may also be learnt at any time. Furthermore, the back-end processing module may further process the session content of the call object and the response content of the robot, so that first content displayed on the three-way calling terminal and second content displayed on the display screen of the human agent are different. For example, if the human agent wants to learn detailed session content of the call object and response content of the robot, the back-end processing module processes the session content of the call object and the response content of the robot as the second content, and transmits the same to the human agent through the display module, to display the same on the display screen of the human agent. Meanwhile, if the call object wants to browse only the response content of the robot to quickly obtain required response information, the back-end processing module processes the response content of the robot as the first content, and displays the first content on the three-way calling terminal through the display module. This display manner has stronger pertinence, to meet different requirements of the call object and the human agent.

The present disclosure may be applied to implement a human-machine coordination calling telephone robot, and an operation process of the calling system is described as follows:.

Claim 1:
A mobile three-way calling terminal for a mobile human-machine coordination calling robot, comprising:
a first speech interface, configured be connected to a back-end processing module, and to transfer call audio between a call object and the back-end processing module, the call object representing a user for establishing call connection to the three-way calling terminal by using a user terminal, wherein the back-end processing module is configured to interact with the call object according to a present rule;
a CODEC1 module, configured to encode and/or decode the call audio between the call object and the back-end processing module;
a second speech interface, configured to be connected to a human agent, and transfer call audio between the human agent and the call object, wherein the second speech interface is further configured to transfer the call audio between the call object and the back-end processing module to the human agent;
a CODEC2 module, configured to encode and/or decode the call audio between the human agent and the call object;
a call control module, configured to process a control signal, and automatically make, answer, and hang up a call;
a data processing submodule, configured to process speech data and perform data transfer between the data processing submodule and the back-end processing module; and
a networking submodule, configured to perform network connection to the back-end processing module.